GoogleBot Attack? 100+ Connections maxing out Apache

anthvale

• January 08, 2020 18:22

I spent the entire day working on solving this with my host company but can't figure it out. I know NOTHING about connecting with SSH and running command lines but he gave me 1 line to run to see how many connections were being made and it's showing over 100 from Google alone. Which is crashing something. Apache? Mod Security?...It's shutting down my entire server with multiple clients and then starting back up later on. Then staying live for some time, then shutting back down. I upped apache to 350 and unconnected Pinterest for now and it seems to be steady but I need to get this sorted out. If I block Google's IP address as a bandaid, for now, how bad will my rankings in the search results be damaged? Thanks

• 

  anthvale

  • January 08, 2020 23:43

  I'm also getting these error when looking at logs: [Wed Jan 08 11:17:17.825906 2020] [:error] [pid 12393:tid 47092598097664] [client 173.231.244.64:34970] [client 173.231.244.64] ModSecurity: Access denied with code 406 (phase 2). Match of "rx ^apache.*perl" against "REQUEST_HEADERS:User-Agent" required. [file "/etc/apache2/conf.d/imh-modsec/01_base_rules.conf"> [line "64"> [id "990011"> [msg "Request Indicates an automated program explored the site"> [severity "NOTICE"> [hostname "account.domain.com"> [uri "/index.php"> [unique_id "XhYqvd6rHkZbwepYXmENCgAAAAQ"> [Wed Jan 08 12:33:05.393958 2020] [:error] [pid 18173:tid 47223102510848] [client 103.5.150.16:57912] [client 103.5.150.16] ModSecurity: Access denied with code 406 (phase 4). Operator GT matched 0 at RESOURCE:xmlrpc_bf_block. [file "/etc/apache2/conf.d/imh-modsec/post_2.8_rules.conf"> [line "17"> [id "13504"> [msg "xmlrpc.php call failures triggered temporary block"> [hostname "domain.com"> [uri "/xmlrpc.php"> [unique_id "XhY8gXfr-E7HWt2eKfDQKgAAABc"> [Wed Jan 08 12:41:16.908278 2020] [:error] [pid 19818:tid 47822563010304] [client 35.239.243.107:50704] [client 35.239.243.107] ModSecurity: Access denied with code 406 (phase 4). Operator GT matched 0 at RESOURCE:xmlrpc_bf_block. [file "/etc/apache2/conf.d/imh-modsec/post_2.8_rules.conf"> [line "17"> [id "13504"> [msg "xmlrpc.php call failures triggered temporary block"> [hostname "domain.com"> [uri "/xmlrpc.php"> [unique_id "XhY@bIIrphNRluJG9Vi8QwAAANY"> [Wed Jan 08 12:51:45.527915 2020] [:error] [pid 20863:tid 47822518884096] [client 192.99.200.69:38376] [client 192.99.200.69] ModSecurity: Access denied with code 406 (phase 4). Operator GT matched 0 at RESOURCE:xmlrpc_bf_block. [file "/etc/apache2/conf.d/imh-modsec/post_2.8_rules.conf"> [line "17"> [id "13504"> [msg "xmlrpc.php call failures triggered temporary block"> [hostname "domain.com"> [uri "/xmlrpc.php"> [unique_id "XhZA4XuOFoCEREtGAkTHdQAAAYE"> [Wed Jan 08 14:27:35.089615 2020] [:error] [pid 29284:tid 46988184954624] [client 73.61.86.193:61050] [client 73.61.86.193] ModSecurity: Warning. Pattern match "recaptcha" at ARGS_POST_NAMES:g-recaptcha-response. [file "/etc/apache2/conf.d/imh-modsec/40_wordpress.conf"> [line "18"> [id "13504"> [hostname "account.domain.com"> [uri "/admin/dologin.php"> [unique_id "XhZXVz4nUpA9IoSXkPZzOgAAAAU">, referer: WHMCS - Login
  

  0
• 

  cPanelLauren

  • January 09, 2020 21:36

  I spent the entire day working on solving this with my host company but can't figure it out. I know NOTHING about connecting with SSH and running command lines but he gave me 1 line to run to see how many connections were being made and it's showing over 100 from Google alone.

  
  What's the command they gave you to run? Google shouldn't have that many connections at once, no but it is pretty normal for Google's bot to connect to the server

  If I block Google's IP address as a bandaid, for now, how bad will my rankings in the search results be damaged?

  
  I believe blocking Google in this manner would be extremely detrimental and I wouldn't advise moving forward with that plan. You can modify the frequency and what is indexed in their console though

  I'm also getting these error when looking at logs:

  
  Those are ModSecurity notices indicating that it's getting hits on specific rules and the actions it takes in conjunction with that. That number and frequency of requests though wouldn't be enough to crash a server (based on what you're showing me)

  0
• 

  anthvale

  • January 10, 2020 14:14

  @cPanelLauren The code I was given to run was this: [QUOTE]netstat -nt | awk '/^tcp/ {print $5}' | awk -F: '{print $1}' | sort | uniq -c | sort -n
  I set the crawl frequency to mid-low for now on some of my accounts. @rajeevacj [QUOTE] I think you have this problem because someone is DDOSing your server using the GoogleBot useragent.
  I was under this impression as well. What do I do in this case?

  0
• 

  anthvale

  • January 15, 2020 14:33

  InMotion is my Host. I think that is a different IP than the one that was doing the crawling that day. Sorry for the confusion.

  0

Please sign in to leave a comment.