Site Owners Forums - Webmaster Forums

Site Owners Forums - Webmaster Forums (http://siteownersforums.com/index.php)
-   General Discussion (http://siteownersforums.com/forumdisplay.php?f=2)
-   -   What is robots.txt? (http://siteownersforums.com/showthread.php?t=211310)

balajidarbar 03-18-2018 11:41 PM
What is robots.txt?
 
What is robots.txt?

divinearcade 03-21-2018 11:14 PM
nice information. thanks for sharing..

jordanleon 03-22-2018 12:30 AM
Hello
 
Robots.txt is a text file that instruct Google webmaster crawler how to index or crawl pages of your site.

Printer Offline | Printer Setup | Printer Installation

samaron 03-22-2018 11:05 PM
The robots exclusion standard, also known as the robots exclusion protocol or simply robots.txt, is a standard used by websites to communicate with web crawlers and other web robots. The standard specifies how to inform the web robot about which areas of the website should not be processed or scanned.

successmantra 03-22-2018 11:48 PM
Robots.txt is for web crawling

stellasweety 03-23-2018 12:01 AM
It is a web crawling instruction.

smith1 03-23-2018 12:36 AM
The robots exclusion standard, also known as the robots exclusion protocol or simply robots.txt, is a standard used by websites to communicate with web crawlers and other web robots. The standard specifies how to inform the web robot about which areas of the website should not be processed or scanned.


All times are GMT -7. The time now is 08:01 AM.


Powered by vBulletin Copyright © 2020 vBulletin Solutions, Inc.