 |
|
|
Article Options |
|
.htaccess for webmasters
by
28 Dec 2009
Description: for many webmasters who log visitors and allow spiders to crawl their site this guide will help you with Bad robots, spiders, crawlers and harvesters Require the www (please note this has been tested on vBulletin 4.0.6 Gold) if this does not work , click remember me before you login or you could try clearing your browser cache before you test it
Replace: www\.ereptalk\.co\.uk and www.ereptalk.co.uk With: Your domain name Replace: http://www.tutorials4you.co.uk/ With: Your domain name Loop Stopping Code Sometimes your rewrites cause infinite loops, stop it with one of these rewrite code snippets.
Fix for infinite loops An error message related to this isRequest exceeded the limit of 10 internal redirects due to probable configuration error. Use 'LimitInternalRecursion' to increase the limit if necessary. Use 'LogLevel debug' to get a backtrace.or you may seeRequest exceeded the limit,probable configuration error,Use 'LogLevel debug' to get a backtrace, orUse 'LimitInternalRecursion' to increase the limit if necessary
Prevent Files image/file hotlinking and bandwidth stealing
Replace: http://(www\.)?askapache.com/ With: Your Domain Name Stop browser prefetching
This module uses a rule-based rewriting engine (based on a regular-expression parser) to rewrite requested URLs on the fly. It supports an unlimited number of rules and an unlimited number of attached rule conditions for each rule, to provide a really flexible and powerful URL manipulation mechanism. The URL manipulations can depend on various tests, of server variables, environment variables, HTTP headers, or time stamps. Even external database lookups in various formats can be used to achieve highly granular URL matching. This module operates on the full URLs (including the path-info part) both in per-server context (httpd.conf) and per-directory context (.htaccess) and can generate query-string parts on result. The rewritten result can lead to internal sub-processing, external request redirection or even to an internal proxy throughput. How to prevent or allow directory listing
The above line enables Directory listing.
The above disables directory listing for your web site. Block Bad robots, Spiders, Crawlers and Harvesters
No members have liked this post.
Last edited by TheSupportForum; 19 Aug 2010 at 11:21.. Reason: change to .htaccess file |
||||||||||||||||||||||||||||||||
|
Views: 8725
|
||||||||||||||||||||||||||||||||
| Comments |
|
#2
01 Jan 2010, 06:59
|
|||
|
|||
|
very nicely done
thanks No members have liked this post.
|
|
#3
02 Jan 2010, 11:13
|
||||
|
||||
|
simonhind, can you help with htacces option?
We can't add site to partner sape.ru if use .htaccess No members have liked this post.
|
|
#4
02 Jan 2010, 12:47
|
|||
|
|||
|
Originally Posted by leodestroy
you need to exlain what you are tring to do what steps are you taking to add site to partner sape.ru
__________________
http://www.multihunters.co.uk - all your coding needs No members have liked this post.
|
|
#5
02 Jan 2010, 14:24
|
||||
|
||||
|
I add my site to sape.ru. But it scans only the pages of forums and blogs. CMS does not recognize. It seems to me that problem in redirect
No members have liked this post.
|
|
#6
02 Jan 2010, 16:43
|
||||||||||||
|
||||||||||||
|
Originally Posted by leodestroy
fist thing you need to check in your .htaccess file is the following
this should only appear once this should be below
end result should be
i hope this helps
__________________
http://www.multihunters.co.uk - all your coding needs No members have liked this post.
|
||||||||||||
|
#7
03 Jan 2010, 03:08
|
||||
|
||||
|
simonhind, sape.ru is not my domain name. This is partner program for web master. Its robots crawl only forum and blogs. A CMS is passed
No members have liked this post.
|
|
#8
25 Feb 2010, 17:14
|
|||
|
|||
|
this sounds like a problem on sape.ru maybe their bot is programed for only blogs and forums
No members have liked this post.
|
|
#9
06 May 2010, 09:28
|
||||
|
||||
|
thanks a lot
really useful No members have liked this post.
|
|
#10
25 Sep 2010, 07:34
|
||||
|
||||
|
thianks 4 good work
No members have liked this post.
|
|
#11
05 Oct 2010, 16:11
|
|||
|
|||
|
thank you
 good work 3.8 No members have liked this post.
|
|
#12
23 Oct 2010, 04:07
|
||||
|
||||
|
Originally Posted by simonhind
Hi.
Question! Sorry for my ignorance, but when you say to replace http://(www\.)?askapache.com/ with my URL, does that mean http://(www\.)?myurlhere.com/ or http://myurlhere.com/ In other words, I would like to add my URL without the "www.", but I do not know if I should remove the "(www\.)?"
__________________
Join The Infinity Program, my den of villains and swashbucklers. My latest article: "The effects of a shoutbox on a forum community." No members have liked this post.
|
||||
|
#13
26 Oct 2010, 23:58
|
|||
|
|||
|
replace
http://(www\.)?askapache.com/ with for example http://(www\.)?me.com/ http://(www\.)?website.com/ etc.... hope this makes it easier
__________________
http://www.multihunters.co.uk - all your coding needs No members have liked this post.
|
|
#14
19 Dec 2010, 21:56
|
||||
|
||||
|
Hi, I am trying to block the 100+ Baiduspiders that are crawling my site all the time. All I need to do is copy/paste the bottom box to my .htaccess file? Where do I add that code? At the bottom of my existing file?
No members have liked this post.
|
|
#15
27 Dec 2010, 17:23
|
|||
|
|||
|
anywhere in the .htaccess file
No members have liked this post.
|
|
«
Previous Article
|
Next Article
»
| Currently Active Users Viewing This Thread: 1 (0 members and 1 guests) | |
| Article Options | |
|
|
| New To Site? | Need Help? |
All times are GMT. The time now is 11:00.