Help with Globally blocking Bad Bots
Hello Everyone,
I am running several c panel servers using CentOS 7 and I am experiencing a lot of bandwidth usage on a lot of websites by bots.
Mainly these ones:
- Unknown robot identified by \*bot
- MJ12bot
-
That above thread is all over the place and I dont know exactly what to use, so can you PLEASE post the solution here so I can see exactly what I need to use and where to use it. I have 8 servers using C Panel and would like your help with this please.. Below is a code I added, please let me know if this would work. If not, please tell me the code I need to add with the example BOTS I have below. 0 -
I have this in Apache Configuration, Include Editor: SetEnvIfNoCase User-Agent "MJ12bot" bad_bots SetEnvIfNoCase User-Agent "AhrefsBot" bad_bots SetEnvIfNoCase User-Agent "SemrushBot" bad_bots SetEnvIfNoCase User-Agent "Baiduspider" bad_bots SetEnvIfNoCase User-Agent "YandexBot" bad_bots SetEnvIfNoCase User-Agent "SeznamBot" bad_bots SetEnvIfNoCase User-Agent "DotBot" bad_bots SetEnvIfNoCase User-Agent "MauiBot" bad_bots Require all granted Require not env bad_bots Does this not work? I really need to stop these BOTS as some are using so much of my bandwidth. 0 -
Use the Mod security rule as I suggested & use it in conjunction with csf to ban the IP's 0 -
The include editor even if it worked would do would still allow the bad bots connect to your server Post 11 if you want to try each one by themselves you should be able to combine them then you can tweak the syntax for each rule 0 -
What I have done is added this rule in the mod security, add rules section. Can you tell me if this would work please. SecRule REQUEST_HEADERS:User-Agent "@rx ^(?:MJ12bot|AhrefsBot|SemrushBo|Baiduspider|YandexBot|SeznamBot|DotBot|MauiBot)$" "msg:'Spiderbot blocked',phase:1,log,id:777777,t:none,block,status:403" Thanks so much for all your help, Steve 0 -
How Would I know if it worked ?? check your apache error logs & the hit list in the mod security section in WHM 0 -
Silly question, but where exactly can I find the Mod Security logs? I was looking at ther raw files in all c panel accounts. 0 -
If you've set ModSecurity to log at WHM>>Security Center>>ModSecurity Configuration - > Audit Log Level it will log to /var/log/modsec_audit.log
0 -
In addition to that or if it not enabled it wil log to /var/log/apache2/error_log 0
Please sign in to leave a comment.
Comments
10 comments