Go Back   Site Owners Forums - Webmaster Forums > Search Engine Optimization > Search Engine Optimization

Notices


Reply
 
Thread Tools Rate Thread Display Modes
Old 04-22-2017, 11:41 AM   #1
cultratradseo
Registered User
 
Join Date: Apr 2017
Posts: 74
Crawl time is a function of inbound bandwidth to the crawling machines, the processing time on the crawling machine, the time to send the result of the analysis to another machine, and the crawl delay as specified in each sites robots.txt file. Multiply the number of pages at a site times the site's crawl delay to get the total time to crawl the site politely. If images are being downloaded, processed, and uploaded to another machine all sorts of bottlenecks can slow down the download process. Crawling is typically done using parallel machines
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
cultratradseo is offline   Reply With Quote

Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Secrete for seo alina albert Search Engine Optimization 87 03-11-2018 05:47 AM
Why to Decrease the Website Load Time on Google ogeninfosystem Post your ad here 0 03-16-2017 01:52 AM
Google website promotion techniques zamina Web Promotion 7 09-24-2013 06:00 AM


All times are GMT -7. The time now is 10:53 PM.


Powered by vBulletin Copyright © 2020 vBulletin Solutions, Inc.