Thread: What is robots.txt?
View Single Post
Old 10-05-2016, 03:22 AM   #22
Jayasree
Registered User
 
Join Date: Oct 2016
Posts: 252
Bots will use robots.txt to crawl our website and webpages used for crawlers about indexing .
Jayasree is offline   Reply With Quote