How to block baiduspider?
currently i'm running a site 123.tld. it works fine till yesterday. today it currently stopped and display the 403 Forbidden Error. i go through the access logs, baidu spider was the last to access it. i was searched for this issue over the net, i've added the conditions in robots.txt, also added the conditions to block baiduspider in htaccess, then blocked the suspicious baiduspider ip.. but still the problem exists.........
Robots.txt
User-agent: Baiduspider
User-agent: Baiduspider-video
User-agent: Baiduspider-image
User-agent: Mozilla/5.0 (compatible; Baiduspider/2.0; +????????-????)
User-agent: Mozilla/5.0 (compatible; Baiduspider/3.0; +????????-????)
User-agent: Mozilla/5.0 (compatible; Baiduspider/4.0; +????????-????)
User-agent: Mozilla/5.0 (compatible; Baiduspider/5.0; +????????-????)
User-agent: Baiduspider/2.0
User-agent: Baiduspider/3.0
User-agent: Baiduspider/4.0
User-agent: Baiduspider/5.0
Disallow: /
htaccess
RewriteCond %{HTTP_USER_AGENT} Baiduspider [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Baidu [NC]
RewriteRule ^.*$ - [F,L]
blocked ips
119.63.193.0/24
180.76.0.0/20
180.76.2.0/24
180.76.3.0/24
180.76.5.0/24
180.76.6.0/24
180.76.8.0/24
180.76.9.0/24
180.76.11.0/24
180.76.12.0/24
180.76.15.0/24
185.10.104.0/24
185.10.105.0/24
203.90.238.0/24
-
it works fine till yesterday. today it currently stopped and the 403 Forbidden Error
Hello :) Could you let us know the output under the "Errors" option in cPanel when you notice the "403" error message? The option is documented at: Errors - Documentation - cPanel Documentation Thank you.0
Please sign in to leave a comment.
Comments
1 comment