Register Members List Search Today's Posts Mark Forums Read

Reply
 
Thread Tools
  #1  
Old 23 Jan 2017, 16:47
Snowhog's Avatar
Snowhog Snowhog is offline
 
Join Date: Oct 2011
Real name: Paul L
Question Search Engine Friendly Archives and store_sitemap directory

How and where does vBulletin create and store Search Engine Friendly Archives?

We have the store_sitemap directory in our robots.txt file as a disallow:

disallow: /store_sitemap/

I'm not sure this should be a disallow. Should the files in this directory be available to spiders/crawlers?
__________________
"It is a capital mistake to theorize in advance of the facts." - Sherlock Holmes
Kubuntu Forums . Net | ZB BLOCK | StopForumSpam
Reply With Quote
  #2  
Old 27 Jan 2017, 00:30
bzcomputers's Avatar
bzcomputers bzcomputers is offline
 
Join Date: Apr 2012
Originally Posted by Snowhog View Post
How and where does vBulletin create and store Search Engine Friendly Archives?

We have the store_sitemap directory in our robots.txt file as a disallow:

disallow: /store_sitemap/

I'm not sure this should be a disallow. Should the files in this directory be available to spiders/crawlers?
You should "allow: /store-sitemap/"; without allowing this directory to be scanned/indexed will stop Google and every other search engine from being able to download and see the site as you wish it to be seen.

Sample Page:
<url>
<loc>http://www.domain.com /</loc>
<lastmod>2017-01-01</lastmod>
<changefreq>weekly</changefreq>
<priority>0.8</priority>
</url>
Reply With Quote
  #3  
Old 27 Jan 2017, 01:05
Snowhog's Avatar
Snowhog Snowhog is offline
 
Join Date: Oct 2011
Real name: Paul L
Thank you. I had already made up my mind that it shouldn't be a Disallow and removed it (as an entry) from our robots.txt file.
__________________
"It is a capital mistake to theorize in advance of the facts." - Sherlock Holmes
Kubuntu Forums . Net | ZB BLOCK | StopForumSpam
Reply With Quote
Reply



Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


New To Site? Need Help?

All times are GMT. The time now is 21:21.

Layout Options | Width: Wide Color: