Thread: What is the purpose of robots.txt file?
View Single Post
Old 11-27-2017, 08:00 AM   #9
infiniumtech
Registered User
 
Join Date: Nov 2017
Location: Surat, Gujarat, India
Posts: 36
Robots.txt is a text file webmasters create to instruct web robots. how to crawl pages on their website.
infiniumtech is offline   Reply With Quote