Thread: what is a robots.txt file?
View Single Post
Old 10-23-2019, 11:29 PM   #10
Mr.Yoso
Registered User
 
Join Date: Jun 2015
Location: Singapore
Posts: 600
Robots.txt file instructs the web crawlers not to crawl specific page/s of the website.
__________________
The world's trusted
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
Mr.Yoso is offline   Reply With Quote