View Single Post
  #28  
Old 28 Jun 2014, 12:53
Ghostt Ghostt is offline
 
Join Date: Oct 2009
i test it . SetEnvIfNoCase or rewrite Rules better?
both codes from the link dont work i think coz i see in cloudflare baidu still crawling.... first rewrite gives an error.
can i use this bots listed there for your write here?
or any better list. in your plugin ive a very big list but i think it will not work with that list because there are not complete spider names


RewriteCond %{HTTP_USER_AGENT} (googlebot|bingbot|Baiduspider|yandex|bing) [NC]
RewriteRule .* - [R=403,L]

Last edited by Ghostt; 29 Jun 2014 at 02:06.
Reply With Quote