Register Members List Search Today's Posts Mark Forums Read

Reply
 
Mod Options
HTTP Error Response Codes by BOP5 (Soft 404 Solution) Details »
HTTP Error Response Codes by BOP5 (Soft 404 Solution)
Mod Version: 2.2.1, by BirdOPrey5 (IB Staff) BirdOPrey5 is offline
Developer Last Online: Feb 2019 I like it Show Printable Version Email this Page

vB Version: 4.x.x Rating: (14 votes - 5.00 average) Installs: 90
Released: 11 Mar 2014 Last Update: 27 May 2014 Downloads: 338
Supported Uses Plugins Re-usable Code  

Brought to you by BirdOPrey5 / Qapla.com

This issue was brought to my attention by Sherrie at TAZ in this thread.

The issue is that when Google crawls a site and runs into the vBulletin No Permission message or other standard vBulletin error pages vBulletin gives the standard "200" HTTP response code which means everything is OK, even though there is no real content on these pages. Therefore Google considers these "soft errors" and they can damage overall SEO of your site if there are too many of them.

What this does is make vBulletin return standard HTTP "Error" responses- by default 403 (Not Authorized) for No Permission pages and 404 (Not Found) for general error messages. When Google sees these responses (as opposed to 200) it knows not to bother indexing the page and everyone is happy.

As of version 2.2 new options have been added to return custom response codes when the forum is closed or when the forum is busy. It will also send a retry header telling the crawler they can try again later. As of version 2.0 there is also a separate response code for search.php errors.

A limitation of this mod is it works best with PHP 5.4.0 or higher. As of version 2.0 If you are on an older version of PHP it may not work, but it should not cause any errors, so it is worth trying.

There are no instructions, once installed it is enabled by default. In Admin CP -> Settings -> Options -> HTTP Error Responses by BOP5 you can enable/disable the mod and choose the response codes you want for the different error types. You can use any standard HTTP response code available from this list at wikipedia.

The mod works fine on VB 3.x or 4.x including VB 3.8.8 (still in Beta) and VB 4.2.2.

To test that it is working use a free online status checker such as: http://httpstatus.io/
(Note the site will cache your result for a while so test with a unique URL each time.)

Before you install the mod any URL that will show a no-permission or other vBulletin error will show a status code of 200. After installing the same page will show a 403, 404, or other code that you have configured.

Added Boothby as a "co-author" as thanks for his contribution of code and ideas.

------------------------------------------------------

Please "Mark as Installed" if you use this.
Donations always appreciated.
Nominate MOTM if you LOVE it! (You did, Thanks!)

Download Now

Only licensed members can download files, Click Here for more information.

Supporters / CoAuthors

Screenshots

Click image for larger version

Name:	http_codes_ss_settings-v2.jpg
Views:	397
Size:	107.6 KB
ID:	149188  

Show Your Support

  • To receive notifications regarding updates -> Click to Mark as Installed.
  • If you like this modification support the author by donating.
  • This modification may not be copied, reproduced or published elsewhere without author's permission.
  #31  
Old 18 Apr 2014, 23:30
karnevil's Avatar
karnevil karnevil is offline
 
Join Date: Feb 2007
After looking through which template was returning the 404 error

STANDARD_ERROR

I replaced the 404 Not Found text to read 403 Forbidden

I checked that a genuine 404 page still came up as such so I know it hasn't affected that.

Not ideal but will see what results webmaster crawl errors comes back with over the next couple of weeks and let you know.
__________________
www.legalbeagles.info
Reply With Quote
  #32  
Old 19 Apr 2014, 00:36
BirdOPrey5's Avatar
BirdOPrey5 BirdOPrey5 is offline
 
Join Date: Jun 2008
Real name: Joe D.
I think there may be some confusion... Standard Error doesn't return a 404 error... If it did this mod wouldn't be necessary- the response code on standard error is 200 (OK) regardless of the actual text displayed in the browser- the text in the browser doesn't determine the response code Google gets. The response code isn't a visible element.

Am I right you have VBSEO installed? I don't think there is any other means vBulletin would display a 404 error message?
__________________
-Joe
Former vBulletin.org Staff Member

(@BirdOPrey5) Former vb.org Moderator. Fighting for a free & independent vb.org.
BirdOPrey5.com - Exclusive VB Mods! (Formerly Qapla.com) | Joe's Ultimate Off Topic
Note - I do not read my PMs often, do not expect quick replies.
Reply With Quote
  #33  
Old 21 Apr 2014, 10:38
Mickie D Mickie D is offline
 
Join Date: Jun 2002
Thank you

Perfect Mod, should be standard built into vbulletin!!

Mick
__________________
Digitalworldz Satellite Cable TV Forums
Cable, Satellite, Consoles, Mobile Phones, Audio Visual, and general chat
Reply With Quote
  #34  
Old 21 Apr 2014, 14:46
Mecho's Avatar
Mecho Mecho is offline
 
Join Date: Aug 2006
Awesome

it works without any issue on 4.2.2 PL1 - Vbseo installed.

First error number was 200
after installing this, it is 403

thanks BirdOPrey5
Reply With Quote
  #35  
Old 23 Apr 2014, 17:40
synseal's Avatar
synseal synseal is offline
 
Join Date: Apr 2009
Real name: Ben Dover.
Hi and thanks for this!

In my webmasters tools I get a load of errors like this all related to tags



Shouldn't this modification fix this with the 403 no permission message?

Crawlers are getting to these pages (Tags) and because I don't allow guests to use the search function they are getting these type of errors and marking it as a soft 404.





Any idea TIA.
__________________
Tech forum.
Reply With Quote
  #36  
Old 23 Apr 2014, 18:59
BirdOPrey5's Avatar
BirdOPrey5 BirdOPrey5 is offline
 
Join Date: Jun 2008
Real name: Joe D.
I can confirm http://www.techkings.org/tags/worked.html is now getting a REAL 404 response code. So those soft errors should go away soon.
__________________
-Joe
Former vBulletin.org Staff Member

(@BirdOPrey5) Former vb.org Moderator. Fighting for a free & independent vb.org.
BirdOPrey5.com - Exclusive VB Mods! (Formerly Qapla.com) | Joe's Ultimate Off Topic
Note - I do not read my PMs often, do not expect quick replies.
Reply With Quote
  #37  
Old 23 Apr 2014, 20:19
synseal's Avatar
synseal synseal is offline
 
Join Date: Apr 2009
Real name: Ben Dover.
Originally Posted by BirdOPrey5 View Post
I can confirm http://www.techkings.org/tags/worked.html is now getting a REAL 404 response code. So those soft errors should go away soon.
These are errors I am still getting after installing this modification.

I cleared all the errors in webmasters tools (and I had loads) and these tag errors have come back after installing this, see the "detected dates" above in the image
__________________
Tech forum.
Reply With Quote
  #38  
Old 23 Apr 2014, 20:39
BirdOPrey5's Avatar
BirdOPrey5 BirdOPrey5 is offline
 
Join Date: Jun 2008
Real name: Joe D.
if you put any of your "tag" URLs into this site- http://httpstatus.io - it confirms a 404 is being returned. It isn't 403 because by default the mod sends 403 for "No Permission" errors only and 404 for all others.

The dates on the image are all several or more days ago so I assume you won't see any more past today assuming the mod remains active.

However it is pretty clear you should add your entire /tags/ path to your robots.txt file, that would be the right way to prevent the crawl issues.
__________________
-Joe
Former vBulletin.org Staff Member

(@BirdOPrey5) Former vb.org Moderator. Fighting for a free & independent vb.org.
BirdOPrey5.com - Exclusive VB Mods! (Formerly Qapla.com) | Joe's Ultimate Off Topic
Note - I do not read my PMs often, do not expect quick replies.
Reply With Quote
  #39  
Old 24 Apr 2014, 11:51
Mecho's Avatar
Mecho Mecho is offline
 
Join Date: Aug 2006
installed and result is great. check soft.png (attached)

just one question , after installing this Access Denied urls increased (access.png) . All these urls are for the threads which has been moved to Trash and those redirect to Trashcan url. is it ok or bad for SEO?
Attached Images
File Type: png soft.png (7.9 KB, 13 views)
File Type: png access.png (7.6 KB, 18 views)
Reply With Quote
  #40  
Old 24 Apr 2014, 11:57
BirdOPrey5's Avatar
BirdOPrey5 BirdOPrey5 is offline
 
Join Date: Jun 2008
Real name: Joe D.
From what I've read (I'm not claiming to be an expert) a 403/404 won't hurt overall SEO although the actual page causing the error will eventually be de-indexed. (Which is exactly what we want to happen.)
__________________
-Joe
Former vBulletin.org Staff Member

(@BirdOPrey5) Former vb.org Moderator. Fighting for a free & independent vb.org.
BirdOPrey5.com - Exclusive VB Mods! (Formerly Qapla.com) | Joe's Ultimate Off Topic
Note - I do not read my PMs often, do not expect quick replies.
Reply With Quote
  #41  
Old 07 May 2014, 08:52
PattiOz PattiOz is offline
 
Join Date: May 2009
Brilliant. Perfect. Works like a charm! My SEO thanks you.
Reply With Quote
  #42  
Old 17 May 2014, 09:21
Boothby Boothby is offline
 
Join Date: Mar 2002
Real name: Stefan
Hi Joe. Could you add the following codes for search-results?

404 Not Found for no result found

410 Gone for outdated search-ID

When a distinction is not possible then the 410 code should be presented.

Thx
Reply With Quote
  #43  
Old 17 May 2014, 13:09
BirdOPrey5's Avatar
BirdOPrey5 BirdOPrey5 is offline
 
Join Date: Jun 2008
Real name: Joe D.
Originally Posted by Boothby View Post
Hi Joe. Could you add the following codes for search-results?

404 Not Found for no result found

410 Gone for outdated search-ID

When a distinction is not possible then the 410 code should be presented.

Thx
Please "Mark as Installed"


Updated to version 1.1
. I wasn't able to determine whether it was no results found or outdated id however, so I added a new setting that will be the response code for any search.php errors. The default response for this is now 410.

I also added code that will prevent the mod from breaking your site if you install it on PHP 5.4+ but later you or your web host downgrade back to PHP 5.3. (Which happened to me.)

This was a minor code change so I believe it worked but I was unable to test it- if someone could test and confirm it works I'd appreciate it.
__________________
-Joe
Former vBulletin.org Staff Member

(@BirdOPrey5) Former vb.org Moderator. Fighting for a free & independent vb.org.
BirdOPrey5.com - Exclusive VB Mods! (Formerly Qapla.com) | Joe's Ultimate Off Topic
Note - I do not read my PMs often, do not expect quick replies.
Reply With Quote
  #44  
Old 17 May 2014, 23:36
Boothby Boothby is offline
 
Join Date: Mar 2002
Real name: Stefan
Thx Joe. The question came from a member of vb-germany.org who wanted a 404 status code for search. And your mod was the closest. And now it fits perfectly.
Reply With Quote
  #45  
Old 18 May 2014, 03:38
BirdOPrey5's Avatar
BirdOPrey5 BirdOPrey5 is offline
 
Join Date: Jun 2008
Real name: Joe D.
Thanks. I would like to add however the better way would be for search.php to be off limits via a robots.txt file so Google doesn't try to index search links at all.

This is a generic VB robots.txt file for a forum in the /forums/ directory of a website-


Block Disabled:      (Update License Status)  
Suspended or Unlicensed Members Cannot View Code.

__________________
-Joe
Former vBulletin.org Staff Member

(@BirdOPrey5) Former vb.org Moderator. Fighting for a free & independent vb.org.
BirdOPrey5.com - Exclusive VB Mods! (Formerly Qapla.com) | Joe's Ultimate Off Topic
Note - I do not read my PMs often, do not expect quick replies.
Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Mod Options

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


New To Site? Need Help?

All times are GMT. The time now is 17:01.

Layout Options | Width: Wide Color: