What are Spiders, Robots and Crawlers and what are their functions?
 

Spiders, robot and crawler, they are all same and referred by different names. It is a software program that follows, or “Crawls” different links throughout the internet, and then grabs the content from the sites and adds to the search engine indexes.

They are all the same and are automated search engine programs that are responsible to read through webpage source to provide information to search engines.

Spiders (or crawlers, or bots) are the means through which search engines download the content that is on your website's pages.

its a set of computer programs that browses the www in orderly fashion as well as automatically scans the webpage or website and download a copy to its data to index.

Search engines uses these activity during read the site.