vBulletin Mods

The Official vBulletin Modifications Site
https://www.vbulletin.org/forum/showthread.php?t=231418

.htaccess for webmasters
by TheSupportForum
28 Dec 2009 09:46

Description:

for many webmasters who log visitors and allow spiders to crawl their site this guide will help you with Bad robots, spiders, crawlers and harvesters


Require the www
(please note this has been tested on vBulletin 4.0.6 Gold)
if this does not work , click remember me before you login
or you could try clearing your browser cache before you test it


Block Disabled:      (Update License Status)  
Suspended or Unlicensed Members Cannot View Code.

Replace:
www\.ereptalk\.co\.uk
and
www.ereptalk.co.uk
With:
Your domain name

Replace:
http://www.tutorials4you.co.uk/
With:
Your domain name


Loop Stopping Code
Sometimes your rewrites cause infinite loops, stop it with one of these rewrite code snippets.


Block Disabled:      (Update License Status)  
Suspended or Unlicensed Members Cannot View Code.

Fix for infinite loops

An error message related to this isRequest exceeded the limit of 10 internal redirects due to probable configuration error. Use 'LimitInternalRecursion' to increase the limit if necessary. Use 'LogLevel debug' to get a backtrace.or you may seeRequest exceeded the limit,probable configuration error,Use 'LogLevel debug' to get a backtrace, orUse 'LimitInternalRecursion' to increase the limit if necessary


Block Disabled:      (Update License Status)  
Suspended or Unlicensed Members Cannot View Code.

Prevent Files image/file hotlinking and bandwidth stealing


Block Disabled:      (Update License Status)  
Suspended or Unlicensed Members Cannot View Code.

Replace:
http://(www\.)?askapache.com/
With:
Your Domain Name


Stop browser prefetching


Block Disabled:      (Update License Status)  
Suspended or Unlicensed Members Cannot View Code.

This module uses a rule-based rewriting engine (based on a regular-expression parser) to rewrite requested URLs on the fly. It supports an unlimited number of rules and an unlimited number of attached rule conditions for each rule, to provide a really flexible and powerful URL manipulation mechanism. The URL manipulations can depend on various tests, of server variables, environment variables, HTTP headers, or time stamps. Even external database lookups in various formats can be used to achieve highly granular URL matching.

This module operates on the full URLs (including the path-info part) both in per-server context (httpd.conf) and per-directory context (.htaccess) and can generate query-string parts on result. The rewritten result can lead to internal sub-processing, external request redirection or even to an internal proxy throughput.


How to prevent or allow directory listing


Block Disabled:      (Update License Status)  
Suspended or Unlicensed Members Cannot View Code.

The above line enables Directory listing.


Block Disabled:      (Update License Status)  
Suspended or Unlicensed Members Cannot View Code.

The above disables directory listing for your web site.



Block Bad robots, Spiders, Crawlers and Harvesters


Block Disabled:      (Update License Status)  
Suspended or Unlicensed Members Cannot View Code.


lazydesis 01 Jan 2010 06:59

very nicely done

thanks

leodestroy 02 Jan 2010 11:13

simonhind, can you help with htacces option?
We can't add site to partner sape.ru if use .htaccess

TheSupportForum 02 Jan 2010 12:47

Quote:

Originally Posted by leodestroy (Post 1944780)
simonhind, can you help with htacces option?
We can't add site to partner sape.ru if use .htaccess


you need to exlain what you are tring to do

what steps are you taking to add site to partner sape.ru

leodestroy 02 Jan 2010 14:24

I add my site to sape.ru. But it scans only the pages of forums and blogs. CMS does not recognize. It seems to me that problem in redirect

TheSupportForum 02 Jan 2010 16:43

Quote:

Originally Posted by leodestroy (Post 1944900)
I add my site to sape.ru. But it scans only the pages of forums and blogs. CMS does not recognize. It seems to me that problem in redirect


fist thing you need to check in your .htaccess file is the following


Block Disabled:      (Update License Status)  
Suspended or Unlicensed Members Cannot View Code.

this should only appear once

this should be below


Block Disabled:      (Update License Status)  
Suspended or Unlicensed Members Cannot View Code.

end result should be


Block Disabled:      (Update License Status)  
Suspended or Unlicensed Members Cannot View Code.


i hope this helps

leodestroy 03 Jan 2010 03:08

simonhind, sape.ru is not my domain name. This is partner program for web master. Its robots crawl only forum and blogs. A CMS is passed

darren1981 25 Feb 2010 17:14

this sounds like a problem on sape.ru maybe their bot is programed for only blogs and forums

abdobasha2004 06 May 2010 09:28

thanks a lot
really useful

as7apcool 25 Sep 2010 07:34

thianks 4 good work

avsunforum 05 Oct 2010 16:11

thank you ;)
good work 3.8

Tanapangarap 23 Oct 2010 04:07

Quote:

Originally Posted by simonhind (Post 1940152)
Prevent Files image/file hotlinking and bandwidth stealing


Block Disabled:      (Update License Status)  
Suspended or Unlicensed Members Cannot View Code.

Replace:
http://(www\.)?askapache.com/
With:
Your Domain Name

Hi.

Question!

Sorry for my ignorance, but when you say to replace http://(www\.)?askapache.com/ with my URL, does that mean http://(www\.)?myurlhere.com/ or http://myurlhere.com/ In other words, I would like to add my URL without the "www.", but I do not know if I should remove the "(www\.)?"

TheSupportForum 26 Oct 2010 23:58

replace
http://(www\.)?askapache.com/
with for example

http://(www\.)?me.com/
http://(www\.)?website.com/

etc....

hope this makes it easier

GONUMBER6 19 Dec 2010 21:56

Hi, I am trying to block the 100+ Baiduspiders that are crawling my site all the time. All I need to do is copy/paste the bottom box to my .htaccess file? Where do I add that code? At the bottom of my existing file?

final kaoss 27 Dec 2010 17:23

anywhere in the .htaccess file


All times are GMT. The time now is 04:37.

Powered by vBulletin® Version 3.8.12
Copyright © 2019, MH Sub I, LLC dba vBulletin. All Rights Reserved. vBulletin® is a registered trademark of MH Sub I, LLC
Copyright ©2001 - , vbulletin.org. All rights reserved.