Site Owners Forums - Webmaster Forums - View Single Post

rajkiran · 03-11-2019, 12:19 AM

The robots exclusion standard, also known as the robots exclusion protocol or simply robots.txt, is a standard used by websites to communicate with web crawlers and other web robots.

03-11-2019, 12:19 AM	#5
rajkiran Registered User Join Date: Aug 2016 Posts: 961	The robots exclusion standard, also known as the robots exclusion protocol or simply robots.txt, is a standard used by websites to communicate with web crawlers and other web robots.