Thread: Google question
View Single Post
Old 05-29-2012, 02:08 AM   #1
eooc.box
Registered User
 
Join Date: Mar 2012
Posts: 87
Google question

I worked on a robots.txt for a customer to disallow certain areas of the site from google crawls and indexing... well, not just google but all bots... However, google is ignoring the robots.txt and doing its own thing anyway. Anybody else have this problem?

Google's been using up about 10GB/month on this particular site. Slurp was the worst culprit though topping out about 13GB. The site is heavily trafficked and the bandwidth is consistently going through the roof so any little bit of reduction would help them.
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
eooc.box is offline   Reply With Quote