/var/log/messages analysis with awk

bloatedstoat

• March 03, 2020 02:35

Hello, I'm looking to refine the following one liner to extract IP addresses, sort and rank them. I've got so far but need to get rid of the SRC= bit to leave the IP only. The entry in the logs follows this (IP addresses obfuscated for privacy) Mar 3 18:00:06 myserver kernel: [414786.179376] Firewall: *TCP_IN Blocked* IN=eth0 OUT= MAC=00:00:00:00:00:ec:00:00:00:00:00:00:00:00 SRC=xxx.xxx.xxx.xxx DST=xxx.xx.xxx.xxx LEN=44 TOS=0x00 PREC=0x00 TTL=235 ID=32940 PROTO=TCP SPT=52209 DPT=1433 WINDOW=1024 RES=0x00 SYN URGP=0
The one liner as it stands is: cat /var/log/messages | awk '{ print $13 }' | sort | uniq -c | sort -n
The result is : 45 SRC=xxx.xxx.xxx.xxx 49 SRC=xxx.xxx.xxx.xxx 54 SRC=xxx.xxx.xxx.xxx 63 SRC=xxx.xxx.xxx.xxx ... and so on. Is there a tweak that can be made to get the IP only as well as the hits to the left? Thanks for any help!

• 

  fuzzylogic

  • March 03, 2020 11:27

  cat /var/log/messages | awk '/*TCP_IN Blocked*/ {print}' | awk '{ print $13 }' | awk '{ sub(/SRC=/, ""); print }' | sort | uniq -c | sort -n
  Seems to work. Someone who knows what they are doing may be able to acheive the same while only invoking awk once. Also you could use sort -nr instead of sort -n if you want the major offenders at the top of the list.

  0
• 

  cPanelLauren

  • March 03, 2020 18:00

  I have a different log level than you but using cut with a delimeter of the = and printing only what comes after should work so just add: cat /var/log/messages | awk '{ print $13 }' | cut -d= -f2 |sort | uniq -c | sort -n
  should work but you'll get a lot more than firewall blocks with that. I looked for firewall and on my server the awk field I'm printing for the same thing is 12 instead of 13 so it looks like this: [root@server backups]# grep Firewall /var/log/messages | awk '{ print $12 }' | cut -d= -f2 |sort | uniq -c | sort -nr 474 185.156.73.54 140 185.176.27.254 137 185.156.73.52 134 185.176.27.174 122 194.26.29.114
  You can add multiple fields to print with awk as well rather than piping them each out. For example if I want to see if they're connecting via TCP or UDP [root@server backups]# grep Firewall /var/log/messages | awk '{ print $12,$7 }' | cut -d= -f2 |cut -d_ -f1|sort | uniq -c | sort -nr 474 185.156.73.54 *TCP 140 185.176.27.254 *TCP 137 185.156.73.52 *TCP 134 185.176.27.174 *TCP 123 194.26.29.114 *TCP
  You can also pretty effectively use sed for formatting as well but cut seemed easier to me for this

  0
• 

  bloatedstoat

  • March 03, 2020 21:37

  Thanks @fuzzylogic and @cPanelLauren for your help in such a short space of time. The solution and the free lesson much appreciated.

  0
• 

  fuzzylogic

  • March 05, 2020 03:04

  awk's default is to print all of each line that matches the pattern, so we can remove 2 instances of the print command. Also results can be limited to have a count greater than a chosen number. So here is a revised version... cat /var/log/messages | awk '/*TCP_IN Blocked*/' | awk '{ print $13 }' | awk 'sub(/SRC=/, "")' | sort | uniq -c | sort -nr | awk '$1 >= 100'
  

  0

Please sign in to leave a comment.