Site Owners Forums - Webmaster Forums - View Single Post

sudipbrahma · 01-08-2018, 11:37 PM

A crawler is a program that visits Web sites and reads their pages and other information in order to create entries for a search engine index. The major search engines on the Web all have such a program, which is also known as a "spider" or a "bot."

01-08-2018, 11:37 PM	#1
sudipbrahma Registered User Join Date: Jan 2018 Posts: 7	A crawler is a program that visits Web sites and reads their pages and other information in order to create entries for a search engine index. The major search engines on the Web all have such a program, which is also known as a "spider" or a "bot."