Google not indexing all pages? - Site Owners Forums

ncjbhd · 10-10-2011, 01:58 AM

Hi

I have spent some time converting my files over to mod-rewrite. On some of my sections, there are tens of thousands of pages. I have made sitemaps to make this content easier for google to find. However, I seem to hit a ceiling of pages that google can index for each section.

Each section is a different subdomain so it is easy for me to check site:subdomain.mysite.com. But whether a section has 12,000 or 70,000 pages available, only about 6,000 pages get indexed from that section max. Ive left the sections alone for a few months to see if the amount would go up, but it stays about the same.

Has anyone else experienced this? Any ideas on how to improve it?

Commodityman · 10-10-2011, 08:48 AM

Have you check to see the robots.txt file to see how to direct the Googlebot for your website. If your site is being crawled or some of it, then you may have a restriction place in the file unknowingly.

devisrimari · 10-11-2011, 12:19 AM

hi,
is yours xml or html site map?

10-10-2011, 01:58 AM	#1
ncjbhd Registered User Join Date: Jul 2011 Posts: 262	Google not indexing all pages? Hi I have spent some time converting my files over to mod-rewrite. On some of my sections, there are tens of thousands of pages. I have made sitemaps to make this content easier for google to find. However, I seem to hit a ceiling of pages that google can index for each section. Each section is a different subdomain so it is easy for me to check site:subdomain.mysite.com. But whether a section has 12,000 or 70,000 pages available, only about 6,000 pages get indexed from that section max. Ive left the sections alone for a few months to see if the amount would go up, but it stays about the same. Has anyone else experienced this? Any ideas on how to improve it? __________________ *To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.* *To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.*

10-10-2011, 08:48 AM	#2
Commodityman Registered User Join Date: Jul 2011 Posts: 64	Have you check to see the robots.txt file to see how to direct the Googlebot for your website. If your site is being crawled or some of it, then you may have a restriction place in the file unknowingly. __________________ If you ever have the time there are easy and cheap car insurance quotes at To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. and the auto insurance quotes will save you more money than you think?

10-11-2011, 12:19 AM	#3
devisrimari Registered User Join Date: Sep 2011 Posts: 45	hi, is yours xml or html site map? __________________ To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. , To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. , To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. , To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)