Skip to main content

Perl Assistance

fkatzenb
cPanel Community, I am writing a munin plugin to look at CSF entries. One of things I am trying to do is to read the csf.allow and csf.deny entries and determine the number IP/CIDR entries are loaded. My regex is not working correctly. The regex into the CIDR variable is not pulling in the number. I also have another problem is that this feature is only supported in what I think is perl 5.12 and beyond, so while I can test it out on my local machine with 5.18, it will not work on my cPanel server which is 5.8.8. Suggestions? Thanks, Frank #open ALLOW, "/etc/csf/csf.allow"; open ALLOW, "csf.allow.txt"; my $allow_value = 0; while (my $line=) { next if $line =~ m/^\s*(?:#.*)?$/; $line =~ m#^\s*(?(?:(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}(25[0-5]|2[0-4][0-9]|1[0-9]{2}|[1-9][0-9]|[0-9])(\/|\s)))(?(3[0-2]|[1-2][0-9]|[0-9]))?#; my ($ipaddress,$cidr) = ($+{ipaddress},$+{cidr}); print $ipaddress,$cidr;; if (defined $cidr) { $allow_value += 2**(32 - $cidr); } else { $allow_value += 1; } } close(ALLOW); # # csf.allow example, but ignore the first # as that does not exist # 204.11.102.34 # Manually allowed - Fri Mar 4 05:41:19 2011 # 99.63.50.19/24 # Manually allowed - Thu Mar 10 00:07:47 2011 #
[COLOR="silver">- - - Updated - - - Just to provide more about my project, this is what I plan to have this plugin do: # Configuration 1: CSF_status # - Shows # of IP entries in the following log files # - /etc/csf/csf.allow # - use CIDR format to turn all entires into actual number of IPs allowed. IP address is first in the line # - file has # for comments at the beginning of the line # - /etc/csf/csf.deny # - use CIDR format to turn all entires into actual number of IPs denied. IP address is first in the line # - file has # for comments at the beginning of the line # - Output # - Title "CSF Status: Loaded IP Addresses For IPTABLES" # - CSF_Allow.label "# of IP Addresses Allowed" # - CSF_Allow.value xx (where xx is equal to the number of IP addresses allowed) # - CSF_Deny.label "# of IP Addresses Denied" # - CSF_Deny.value yy (where yy is equal to the number of IP addresses denied) # Configuration 2: CSF_denyCountry # - Shows # of IP entries for each Country # - /etc/csf/csf.deny # - use CIDR format to turn all entires into actual number of IPs denied. IP address is first in the line # - file has # for comments at the beginning of the line # - Country is encapsulated in "...(AA/Abcdefg/FQDN)..." where AA is the country code and Abcdefg is the Long Country Name # - Output # - Title "CSF Status: Denied IP Addresses by Country of Origin" # - CSF_Deny_AA.label Abcdefg (where AA is the country code and Abcdefg is the long country name) # - CSF_Deny_AA.value xx (where AA is the country code and xx is the number of IP CIDR entries for that country) # Configuration 3: CSF_denyType # - Shows reason for denial # - /etc/csf/csf.deny # - use CIDR format to turn all entires into actual number of IPs denied. IP address is first in the line # - file has # for comments at the beginning of the line # - Detected failure has four examples: # - "...# lfd: (type)..." where type equals the service that was attacked and detected # - "...# lfd: *type*..." where type equals a non service detection # - "...# lfd: aa.bb.cc.dd (AA/Abcdefg/FQDN), xx distributed_type..." where distrubted type equals the service and type of attack # - "...# type..." where type is actual a manual entry because lfd: is not listed. # - Output # - Title "CSF Status: Denied IP Addresses by Detected Attack" # - CSF_Deny_type.label type (where type is qual to the value of type where type is equal the above examples of type, distributed_type, or manual) # - CSF_Deny_type.value xx (where type is qual to the value of type where type is equal the above examples of type, distributed_type, or manual. xx is the number of CIDR entires)

Comments

5 comments

Date Votes
  • KostonConsulting
    If you're on Perl before 5.10, there are not named capture buffers for regular expressions. Instead, you'll need to use $1, $2, $3, etc. Those special variables are defined by parentheses groupings within your regex. i.e. /(\s+)(abc)(d(e|f))/ would have $1,$2, and $3 available for the 3 sets of parenthesis (the sub-parentheses don't count).
    0
  • fkatzenb
    [quote="KostonConsulting, post: 1494981">If you're on Perl before 5.10, there are not named capture buffers for regular expressions. Instead, you'll need to use $1, $2, $3, etc. Those special variables are defined by parentheses groupings within your regex. i.e. /(\s+)(abc)(d(e|f))/ would have $1,$2, and $3 available for the 3 sets of parenthesis (the sub-parentheses don't count).
    Thank you for the great help. You launched me well on my way. Currently my code now successfully counts all IPs in the allow and deny. my $DisplayVer=0.1; use strict; use warnings; #"/etc/csf/csf.allow" #"/etc/csf/csf.deny" my $allow = "csf.allow.txt"; my $deny = "csf.deny.txt"; print getIPcount($allow),"\n"; print getIPcount($deny),"\n"; sub getIPcount { my $value = 0; my ($filename) = @_; open FILE, $filename; while (my $line=) { next if $line =~ m/^\s*(?:#.*)?$/; my ($ipaddress) = ($line =~ m#^\s*(((25[0-5]|2[0-4][0-9]|1[0-9]{2}|[1-9][0-9]|[0-9])\.){3}(25[0-5]|2[0-4][0-9]|1[0-9]{2}|[1-9][0-9]|[0-9]))#); my ($cidr) = ($line =~ m#\/(3[0-2]|[1-2][0-9]|[0-9])\s#); if (defined $cidr) { $value += 2**(32 - $cidr); print "cdr?\n"; } else { $value += 1; } } close(FILE); return $value; }
    On to my next step.
    0
  • fkatzenb
    So I finished my alpha version and it worked like a champ (screenshots attached). .vB.vB.vB.vB One of the things that was missing is Port Scan entries for CSF_deny_service.pl call. I changed my regex from \((\b\w*)\)\s(\b\w*\s\w*\s\w*\slogin|\b\w*\s\w*\slogin|\b\w*)
    to \s\W((\b\w*\s\w*)|(\b\w*))\W\s((\w*\s\w*\s\w*\slogin)|(\w*\s\w*\slogin)|(\w*))
    That is when I began to see issues where everything in the brackets immediately following \W\s was no longer detected as my $part2 at line 107 went undefined and my one if statement requires it to be defined to create the label at line 115. Any help would be great. Sorry if some of it is doing things the hard way, this is my first attempt at perl by myself. V/R, Frank #!/usr/bin/perl -w # # Munin plugin for CSF: ConfigServer Firewall # This plugin graphs your csf.allow and csf.deny data # #--------------------- # Examples # Create a symbolic link to CSF__ # ln -s /usr/share/munin/plugins/CSF_ /etc/munin/plugins/CSF_allow_reseller # graph # ln -s /usr/share/munin/plugins/CSF_ /etc/munin/plugins/CSF_deny_reseller # graph # ln -s /usr/share/munin/plugins/CSF_ /etc/munin/plugins/CSF_deny_country # graph # ln -s /usr/share/munin/plugins/CSF_ /etc/munin/plugins/CSF_deny_service # graph # #--------------------- # Log # Revision 0.1 2013/10/28 F. Katzenberger # - First version tested reading csf.deny and countying IP addresses w/ CIDR support # Revision 0.2 2013/10/31 F. Katzenberger # - Added/developed regex for detecting resellers, countries, and services # Revision 0.3 2013/11/02 F. Katzenberger # - developed the param structure for the munin config call # Revision 0.4 2013/11/03 F. Katzenberger # - alpha testing release # my $DisplayVer=0.4; #--------------------- # Known Issues # 1. when a country or service no longer has an entry in csf.deny, it will be removed by munin # from the graph regardless of any datapoints still on the graph. # 2. does not yet support port scan and cluster member entries # 3. distributed attacks are not detected and included in the deny_service in most cases # 4. the SMTP service label is cut short, regex is not showing the forth word. need to consider # a search and replace form SMTP AUTH to smtpauth which is the name for distributed attacks against SMTP # 5. add a graph that shows deny by distributed on accounts. # 6. no ipv6 support yet # 7. does not yet track other lfd features such as excessive resources and ignore or temporary use strict; use warnings; use Data::Dumper; # determine what is to be reported on and hwo to display it from the basefile name my ($report,$type) = ($0 =~ m#_(\b[A-Z]|[a-z]*)_(\b[A-Z]|[a-z]*)#); # load up the allow/deny file # my $filename = "/etc/csf/csf.".$report; my $filename = "csf.".$report.".txt"; # display preferences based on the report and type being passed into this script my %param = ( allow => { label => 'Reported As: ', title => "IP Allowed", vtitle => 'addresses', graph_args => '--base 1000 -l 0', warning => '', critical => '', info_tag => "", }, deny => { label => 'Reported As: ', title => "IP Denied", vtitle => 'addresses', graph_args => '--base 1000 -l 0', warning => '', critical => '', info_tag => "", }, reseller => { lookfor => '\bReseller\s(\b\w*)(\s)', draw1 => 'AREA', draw2 => 'STACK', }, country => { lookfor => '\s\((\w*)\/((\w*)|(\w*\s\w*))\/', draw1 => 'AREA', draw2 => 'STACK', }, service => { lookfor => '\s\W((\b\w*\s\w*)|(\b\w*))\W\s((\w*\s\w*\s\w*\slogin)|(\w*\s\w*\slogin)|(\w*))', draw1 => 'AREA', draw2 => 'STACK', }, ); #Munin Config Options if ($ARGV[0] and $ARGV[0] eq "config"){ print "graph_title $param{$report}->{title} by $type\n"; print "graph_vtitle $param{$report}->{vtitle}\n"; print "graph_args $param{$report}->{graph_args}\n"; print "graph_scale yes\n"; print "graph_category configserver\n"; print "graph_info This graph shows how many IP addresses in $filename are $report and filtered by $type\n"; open FILE, $filename; my @list; # determine all possible line items of data to be displayed my $index = 0; while (my $line = ) { next if $line =~ m/^\s*(?:#.*)?$/; my ($part1,$part2) = ($line =~ m#$param{$type}->{lookfor}#); if ($type eq "reseller") { $part2 = $part1; } if ($type eq "service" and defined $part1 and $part1 eq "Port Scan") { $part1 = "port_scan"; $part2 = "Port Scan Detected"; } if (defined $part1 and defined $part2) { $list[$index] = $part1.",".$part2; $index += 1; } } close(FILE); # reduce those line items for data labels to unique values $index = 0; my %comb; @comb{@list} = (); my @uniq = sort keys %comb; # generate the print statements for each unique label names and how to draw the data for my $item (@uniq){ my ($again1,$again2) = ($item =~ m#(\b\w*)\,((\w*\s\w*\s\w*)|(\w*\s\w*)|(\w*))#); print $again1.".label ".$again2."\n"; if ($index == 0) { print $again1.".draw $param{$type}->{draw1}\n"; } else { print $again1.".draw $param{$type}->{draw2}\n"; } $index += 1; } exit 0; } open FILE, $filename; my @list; # determine all possible line items of data to find values for my $index = 0; while (my $line = ) { next if $line =~ m/^\s*(?:#.*)?$/; my ($part1,$part2) = ($line =~ m#$param{$type}->{lookfor}#); if (defined $part1) { $list[$index] = $part1; $index += 1; } } close(FILE); # reduce those line items for data labels to unique values my %comb; @comb{@list} = (); my @uniq = sort keys %comb; # for each unique value, for my $item (@uniq){ open FILE, $filename; my $value = 0; my $index = 0; while (my $line = ) { next if $line =~ m/^\s*(?:#.*)?$/; # find the instances in the file and parse out the IP address my ($ipaddress) = ($line =~ m#^\s*(((25[0-5]|2[0-4][0-9]|1[0-9]{2}|[1-9][0-9]|[0-9])\.){3}(25[0-5]|2[0-4][0-9]|1[0-9]{2}|[1-9][0-9]|[0-9]))#); my ($cidr) = ($line =~ m#\/(3[0-2]|[1-2][0-9]|[0-9])\s#); my ($again1,$again2) = ($line =~ m#$param{$type}->{lookfor}#); if (defined $again1 and $again1 eq $item) { if (defined $cidr) { $value += 2**(32 - $cidr); } else { $value += 1; } } } # create the print statement for the actual values if ($item eq "Port Scan") { $item = "port_scan"; } print $item.".value ".$value."\n"; close(FILE); }
    Here is some data: ############################################################################### # Copyright 2006-2011, Way to the Web Limited # URL: http://www.configserver.com # Email: sales@waytotheweb.com ############################################################################### # The following IP addresses will be blocked in iptables # One IP address per line # CIDR addressing allowed with a quaded IP (e.g. 192.168.254.0/24) # Only list IP addresses, not domain names (they will be ignored) # # Note: If you add the text "do not delete" to the comments of an entry then # DENY_IP_LIMIT will ignore those entries and not remove them # # Advanced port+ip filtering allowed with the following format # tcp/udp|in/out|s/d=port|s/d=ip # # See readme.txt for more information regarding advanced port filtering # 175.42.82.185 # lfd: (ftpd) Failed FTP login from 175.42.82.185 (CN/China/-): 10 in the last 300 secs - Wed Sep 11 09:32:21 2013 14.222.41.125 # lfd: (smtpauth) Failed SMTP AUTH login from 14.222.41.125 (CN/China/-): 10 in the last 300 secs - Wed Sep 11 13:39:37 2013 187.23.95.249 # lfd: (ftpd) Failed FTP login from 187.23.95.249 (BR/Brazil/bb175ff9.virtua.com.br): 10 in the last 300 secs - Wed Sep 11 22:40:42 2013 220.113.246.65 # lfd: (ftpd) Failed FTP login from 220.113.246.65 (CN/China/-): 10 in the last 300 secs - Wed Sep 11 23:08:00 2013 217.39.146.163 # lfd: (smtpauth) Failed SMTP AUTH login from 217.39.146.163 (GB/United Kingdom/host217-39-146-163.in-addr.btopenworld.com): 10 in the last 300 secs - Thu Sep 12 01:58:32 2013 163.125.230.244 # lfd: (ftpd) Failed FTP login from 163.125.230.244 (CN/China/-): 10 in the last 300 secs - Thu Sep 12 03:28:46 2013 209.21.92.21 # lfd: *Port Scan* detected from 209.21.92.21 (US/United States/-). 16 hits in the last 105 seconds - Thu Sep 12 08:42:14 2013 220.113.241.16 # lfd: (ftpd) Failed FTP login from 220.113.241.16 (CN/China/-): 10 in the last 300 secs - Thu Sep 12 11:18:07 2013 184.22.252.114 # lfd: (pop3d) Failed POP3 login from 184.22.252.114 (US/United States/184-22-252-114.static.hostnoc.net): 10 in the last 300 secs - Thu Sep 12 11:35:38 2013 174.21.233.139 # lfd: *Port Scan* detected from 174.21.233.139 (US/United States/174-21-233-139.tukw.qwest.net). 16 hits in the last 210 seconds - Thu Sep 12 14:14:17 2013 85.112.1.153 # lfd: (pop3d) Failed POP3 login from 85.112.1.153 (ES/Spain/-): 10 in the last 300 secs - Thu Sep 12 18:54:00 2013 50.80.232.125 # lfd: (smtpauth) Failed SMTP AUTH login from 50.80.232.125 (US/United States/50-80-232-125.client.mchsi.com): 10 in the last 300 secs - Thu Sep 12 23:00:27 2013 54.225.82.140 # lfd: (ftpd) Failed FTP login from 54.225.82.140 (US/United States/ns1.safe-net.eu): 10 in the last 300 secs - Fri Sep 13 04:02:38 2013 124.42.255.213 # lfd: (ftpd) Failed FTP login from 124.42.255.213 (CN/China/-): 10 in the last 300 secs - Fri Sep 13 11:00:10 2013 216.245.217.198 # lfd: (ftpd) Failed FTP login from 216.245.217.198 (US/United States/shoppingservers.crabdance.com): 10 in the last 300 secs - Fri Sep 13 11:57:12 2013 208.177.76.15 # lfd: *Port Scan* detected from 208.177.76.15 (US/United States/208.177.76.15.ptr.us.xo.net). 16 hits in the last 110 seconds - Fri Sep 13 13:57:34 2013 58.22.10.93 # lfd: *Port Scan* detected from 58.22.10.93 (CN/China/-). 16 hits in the last 250 seconds - Fri Sep 13 14:24:56 2013 213.229.68.54 # lfd: *Port Scan* detected from 213.229.68.54 (GB/United Kingdom/213-229-68-54.static.as29550.net). 16 hits in the last 115 seconds - Fri Sep 13 17:12:48 2013 112.65.211.238 # lfd: (smtpauth) Failed SMTP AUTH login from 112.65.211.238 (CN/China/-): 10 in the last 300 secs - Fri Sep 13 17:47:00 2013 108.222.151.41 # lfd: *Port Scan* detected from 108.222.151.41 (US/United States/108-222-151-41.lightspeed.irvnca.sbcglobal.net). 16 hits in the last 135 seconds - Fri Sep 13 20:53:19 2013 112.95.176.206 # lfd: (ftpd) Failed FTP login from 112.95.176.206 (CN/China/-): 10 in the last 300 secs - Sat Sep 14 04:31:06 2013 86.108.97.237 # lfd: *Port Scan* detected from 86.108.97.237 (JO/Jordan/86.108.x.237.go.com.jo). 16 hits in the last 165 seconds - Sat Sep 14 05:23:03 2013 192.74.237.220 # lfd: *Port Scan* detected from 192.74.237.220 (US/United States/-). 16 hits in the last 135 seconds - Sat Sep 14 05:32:35 2013 163.125.237.241 # lfd: (ftpd) Failed FTP login from 163.125.237.241 (CN/China/-): 10 in the last 300 secs - Sat Sep 14 05:57:30 2013 199.15.233.134 # lfd: *Port Scan* detected from 199.15.233.134 (US/United States/-). 16 hits in the last 110 seconds - Sat Sep 14 07:32:16 2013 5.133.27.66 # lfd: *Port Scan* detected from 5.133.27.66 (PS/Palestinian Territory/-). 16 hits in the last 255 seconds - Sat Sep 14 08:54:45 2013 188.159.114.56 # lfd: *Port Scan* detected from 188.159.114.56 (IR/Iran, Islamic Republic of/-). 16 hits in the last 40 seconds - Sat Sep 14 11:56:20 2013 120.37.237.40 # lfd: (ftpd) Failed FTP login from 120.37.237.40 (CN/China/-): 10 in the last 300 secs - Sat Sep 14 13:13:59 2013 120.37.237.239 # lfd: (ftpd) Failed FTP login from 120.37.237.239 (CN/China/-): 10 in the last 300 secs - Sat Sep 14 13:14:26 2013 120.37.237.183 # lfd: (ftpd) Failed FTP login from 120.37.237.183 (CN/China/-): 10 in the last 300 secs - Sat Sep 14 13:14:46 2013 213.123.201.4 # lfd: (smtpauth) Failed SMTP AUTH login from 213.123.201.4 (GB/United Kingdom/host213-123-201-4.in-addr.btopenworld.com): 10 in the last 300 secs - Sun Sep 15 07:55:55 2013 112.111.185.226 # lfd: *Port Scan* detected from 112.111.185.226 (CN/China/-). 16 hits in the last 190 seconds - Sun Sep 15 11:32:00 2013 177.32.150.201 # lfd: (pop3d) Failed POP3 login from 177.32.150.201 (BR/Brazil/b12096c9.virtua.com.br): 10 in the last 300 secs - Sun Sep 15 13:36:36 2013
    Debuggex Visual Tool
    0
  • fkatzenb
    Well I solved it. It was my judicial use of parenthesis. I was using them like I do math functions.
    0
  • KostonConsulting
    [quote="fkatzenb, post: 1500722">Well I solved it. It was my judicial use of parenthesis. I was using them like I do math functions.
    Glad to hear you got this sorted and the plugin is looking good. Maybe I'm missing it but is there somewhere in Munin that you can see failed attempts per IP?
    0

Please sign in to leave a comment.

Didn't find what you were looking for?

New post