Site Owners Forums - Webmaster Forums
Page 2 of 2 1 2
Show 40 post(s) from this thread on one page

Site Owners Forums - Webmaster Forums (http://siteownersforums.com/index.php)
-   Search Engine Optimization (http://siteownersforums.com/forumdisplay.php?f=16)
-   -   What is the purpose of using robots.txt file in seo? (http://siteownersforums.com/showthread.php?t=179621)

nathanleo 12-31-2021 03:26 AM
A robots. txt file tells search engine crawlers which URLs the crawler can access on your site. This is used mainly to avoid overloading your site with requests; it is not a mechanism for keeping a web page out of Google.


All times are GMT -7. The time now is 07:26 PM.
Page 2 of 2 1 2
Show 40 post(s) from this thread on one page


Powered by vBulletin Copyright © 2020 vBulletin Solutions, Inc.