What are Spiders, Robots and Crawlers and what are their functions?
 

Hlo Friends,
What are Spiders, Robots and Crawlers and what are their functions?

A Web crawler, sometimes called a spider, is an Internet bot that systematically browses the World Wide Web, typically for the purpose of Web indexing (web spidering). Web search engines and some other sites use Web crawling or spidering software to update their web content or indices of others sites' web content.

Spiders, Robots and Crawlers all are same these are automated software programme search engine use to stay up to date with web activities and finding new links and information to index in their database. Search engines need to keep their database updated so they created some automated programmes which goes from site to site and find the new data for search engine also collects the information about the web page what is the page all about.

Crawling & Indexing are the basic functionalities

Web search engines and some other sites use Web crawling or spidering software to update their web content or indices of others sites' web content.

It is a software program that follows, or “Crawls” different links throughout the internet, and then grabs the content from the sites and adds to the search engine indexes.

Spiders and crawlers are same . They are search engine software to crawl (scan) the website and to store the website information in the data base. The robots are used to block particular webpage in the website.

Spiders, Robots and Crawlers all are same these are automated software programme search engine use to stay up to date with web activities and finding new links and information to index in their database.