View Single Post
Old 23 Jan 2017, 15:47
Snowhog's Avatar
Snowhog Snowhog is offline
Join Date: Oct 2011
Real name: Paul L
Question Search Engine Friendly Archives and store_sitemap directory

How and where does vBulletin create and store Search Engine Friendly Archives?

We have the store_sitemap directory in our robots.txt file as a disallow:

disallow: /store_sitemap/

I'm not sure this should be a disallow. Should the files in this directory be available to spiders/crawlers?
"It is a capital mistake to theorize in advance of the facts." - Sherlock Holmes
Kubuntu Forums . Net | ZB BLOCK | StopForumSpam
Reply With Quote