|
|
07-24-2017, 05:26 AM
|
#1 |
|
Registered User
Join Date: Jul 2017
Posts: 258
|
The robots exclusion standard, also known as the robots exclusion protocol or simply robots.txt, is a standard used by websites to communicate with web crawlers and other web robots. The standard specifies how to inform the web robot about which areas of the website should not be processed or scanned.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. , To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. , To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. , To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. , To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. , To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. , To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
 |
«
Previous Thread
|
Next Thread
»
| Currently Active Users Viewing This Thread: 1 (0 members and 1 guests) | |
| Thread Tools | |
| Display Modes | Rate This Thread |
Hybrid Mode
|
|
Similar Threads
|
||||
| Thread | Thread Starter | Forum | Replies | Last Post |
| Evil bots consuming all my sites traffic even after i disallowed them in robots.txt | Fking | General Discussion | 1 | 12-13-2013 11:27 PM |
| Robots.txt | sameerseo | Search Engine Optimization | 26 | 09-28-2012 04:18 AM |
| robots.txt | wheelbarrow001 | 2 | 07-27-2012 04:10 AM | |
| Unable to delete robots.txt file | notfake | Yahoo | 0 | 04-29-2012 02:19 AM |
| What Is Robots.txt? | samlko | Search Engine Optimization | 13 | 03-09-2012 02:37 PM |
All times are GMT -7. The time now is 05:17 PM.