Thread: What is robots.txt?
View Single Post
Old 03-29-2017, 10:27 PM   #33
Dennydevi
Registered User
 
Join Date: Sep 2016
Posts: 429
robots.txt is a text file webmasters create to instruct robots, how to crawl and index pages on their website.
Dennydevi is offline   Reply With Quote