03-18-2018, 11:41 PM | #1 |
Registered User
Join Date: Mar 2018
Posts: 11
|
What is robots.txt?
What is robots.txt?
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
03-21-2018, 11:14 PM | #2 |
Registered User
Join Date: Feb 2018
Posts: 4
|
nice information. thanks for sharing..
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
03-22-2018, 12:30 AM | #3 |
Registered User
Join Date: Mar 2018
Location: United State
Posts: 1
|
Hello
Robots.txt is a text file that instruct Google webmaster crawler how to index or crawl pages of your site.
Printer Offline | Printer Setup | Printer Installation Last edited by jordanleon; 03-22-2018 at 12:59 AM.. |
03-22-2018, 11:05 PM | #4 |
Registered User
Join Date: Sep 2017
Posts: 177
|
The robots exclusion standard, also known as the robots exclusion protocol or simply robots.txt, is a standard used by websites to communicate with web crawlers and other web robots. The standard specifies how to inform the web robot about which areas of the website should not be processed or scanned.
|
03-22-2018, 11:48 PM | #5 |
Registered User
Join Date: Aug 2017
Location: batala
Posts: 127
|
Robots.txt is for web crawling
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
03-23-2018, 12:01 AM | #6 |
Registered User
Join Date: Mar 2017
Posts: 651
|
It is a web crawling instruction.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
03-23-2018, 12:36 AM | #7 |
Registered User
Join Date: Feb 2018
Posts: 462
|
The robots exclusion standard, also known as the robots exclusion protocol or simply robots.txt, is a standard used by websites to communicate with web crawlers and other web robots. The standard specifies how to inform the web robot about which areas of the website should not be processed or scanned.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
Currently Active Users Viewing This Thread: 1 (0 members and 1 guests) | |
Thread Tools | |
Display Modes | Rate This Thread |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
Robots.txt or Meta Robots.txt??? | santoshkumar | Search Engine Optimization | 10 | 02-14-2017 04:03 AM |
Robots.txt | jaysh4922 | Search Engine Optimization | 13 | 07-28-2016 05:10 AM |
Evil bots consuming all my sites traffic even after i disallowed them in robots.txt | Fking | General Discussion | 1 | 12-13-2013 11:27 PM |
How to edit virtual robots.txt file in wordpress? | geniusoptimizer | PHP / mySQL | 1 | 04-12-2013 12:36 PM |
Unable to delete robots.txt file | notfake | Yahoo | 0 | 04-29-2012 02:19 AM |