Site Owners Forums - Webmaster Forums - View Single Post

techguru · 03-29-2018, 06:44 AM

robots.txt is a text file which helps to navigate, search bots to allow or disallow in your files. This file use for giving instruction to crawlers in website to disallow in your confidential files

03-29-2018, 06:44 AM	#26
techguru Registered User Join Date: Mar 2018 Location: london Posts: 8	robots.txt is a text file which helps to navigate, search bots to allow or disallow in your files. This file use for giving instruction to crawlers in website to disallow in your confidential files