Site Owners Forums - Webmaster Forums - View Single Post

MaitriShah · 05-02-2017, 10:51 PM

Robot.txt is a file which is actually a text file created to instruct robots (search engine robots) that how and which pages to crawl of a website.

05-02-2017, 10:51 PM	#4
MaitriShah Registered User Join Date: Apr 2017 Posts: 87	Robot.txt is a file which is actually a text file created to instruct robots (search engine robots) that how and which pages to crawl of a website.