Site Owners Forums - Webmaster Forums

Site Owners Forums - Webmaster Forums (http://siteownersforums.com/index.php)

- Social Networks (http://siteownersforums.com/forumdisplay.php?f=43)

- - Robots.txt (http://siteownersforums.com/showthread.php?t=535057)

riprook7

07-06-2018 10:50 PM

Robots.txt

What is Robots.txt file?

wellliving

07-10-2018 03:23 AM

Hi

The robots.txt file, also known as the robots exclusion protocol or standard, is a text file that tells web robots (most often search engines) which pages on your site to crawl. It also tells web robots which pages not to crawl. Let's say a search engine is about to visit a site.

https://www.welllivingshop.com/pillows/organic-cotton/

All times are GMT -7. The time now is 07:40 PM.