Site Owners Forums - Webmaster Forums - View Single Post

blueapple · 09-05-2012, 05:38 AM

A robots.txt file is a simple txt file. robots file on a website wills utility as a appeal that specified robots discount specified files or directories when crawling a site.

09-05-2012, 05:38 AM	#5
blueapple Registered User Join Date: Feb 2012 Posts: 92	A robots.txt file is a simple txt file. robots file on a website wills utility as a appeal that specified robots discount specified files or directories when crawling a site.