Site Owners Forums - Webmaster Forums - View Single Post

ashok patrick · 09-20-2017, 09:15 AM

The robots exclusion protocol (REP), or robots.txt is a text file webmasters create to instruct robots (typically search engine robots) how to crawl and index pages on their website.

09-20-2017, 09:15 AM	#38
ashok patrick Registered User Join Date: Sep 2017 Posts: 21	The robots exclusion protocol (REP), or robots.txt is a text file webmasters create to instruct robots (typically search engine robots) how to crawl and index pages on their website.