03-07-2018, 08:36 AM
|
#4 |
|
Registered User
Join Date: Nov 2017
Posts: 253
|
Msnbot, discobot, and Slurp are all called out specifically, so those user-agents will only pay attention to the directives in their sections of the robots.txt file. All other user-agents will follow the directives in the user-agent: * group.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. / To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. / To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. / To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. / To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
«
Previous Thread
|
Next Thread
»
| Currently Active Users Viewing This Thread: 1 (0 members and 1 guests) | |
Threaded Mode
|
|
Similar Threads
|
||||
| Thread | Thread Starter | Forum | Replies | Last Post |
| Which is better Robots.txt or Meta Robots Tag | doorhardwareuk | Search Engine Optimization | 0 | 07-10-2017 11:17 PM |
| Which is better Robots.txt or Meta Robots Tag? | mithucarrentals | Search Engine Optimization | 0 | 01-30-2017 04:04 AM |
| Which is better Robots.txt or Meta Robots Tag? | apranajoshi | Search Engine Optimization | 8 | 08-30-2016 01:26 AM |
| Which is better Robots.txt or Meta Robots Tag? | riyakhan | Search Engine Optimization | 3 | 12-14-2015 01:01 AM |
All times are GMT -7. The time now is 01:16 PM.