How to block empty referrer in .htaccess??

virtualreality

• November 07, 2019 15:47

Hi cPanel Friends, More and more no-referrer traffic and requests coming to scrap our website contents. Our website has suffered a lot. Really frustrating. How to block empty referrer requests in .htaccess and at the same time not block several major search engine bots (Googlebot, Bingbot, DuckDuckGo bot, etc.)? Thank you very much!

• 

  cPanelLauren

  • November 07, 2019 20:57

  Something like the following might work for empty referrers: RewriteEngine on RewriteCond %{HTTP_REFERER} ^$ RewriteRule ^ - [F,L]
  

  0
• 

  virtualreality

  • November 07, 2019 21:11

  Thank you very much Lauren for help! This code is good and useful in preventing empty referrer requests from scraping our website contents. But, as now I've found most time the major search engines' bots also come to our website with empty referrer. I'm afraid this Code will also block all search engine bots with empty referrer. How to add something to let major search engine bots with empty referrer pass in the code or in the .htaccess? Is there in the cPanel where I can add some legitimate search engine bots to pass while block empty referrer traffic and requests?

  0

Please sign in to leave a comment.