Site Owners Forums - Webmaster Forums - View Single Post - What is the purpose of using robots.txt file in seo?

emilyoliver · 11-02-2016, 05:07 AM

Before a search engine crawls your site, it will look at your robots.txt file as instructions. it give instruction to search engine to crawl the site

11-02-2016, 05:07 AM	#11
emilyoliver Registered User Join Date: Sep 2016 Posts: 248	Before a search engine crawls your site, it will look at your robots.txt file as instructions. it give instruction to search engine to crawl the site