07-25-2016, 10:59 AM | #1 |
Registered User
Join Date: Jul 2016
Posts: 41
|
What is the X-Robots tag?
What is the X-Robots tag?
|
07-26-2016, 01:38 AM | #2 |
Registered User
Join Date: Jul 2016
Posts: 142
|
The robots meta tag in the above example instructs all search engines not to show the page in search results. The value of the name attribute (robots) specifies that the directive applies to all crawlers. To address a specific crawler, replace the robots value of the name attribute with the name of the crawler that you are addressing. Specific crawlers are also known as user-agents (a crawler uses its user-agent to request a page.) Google's standard web crawler has the user-agent name Googlebot. To prevent only Googlebot from crawling your page, update the tag as follows:
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
07-26-2016, 04:32 AM | #3 |
Registered User
Join Date: Apr 2016
Posts: 542
|
It is used as an HTTP header for a given URL in meta tag
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
Currently Active Users Viewing This Thread: 1 (0 members and 1 guests) | |
Thread Tools | |
Display Modes | Rate This Thread |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
Check the website robots file | china-jewels | Bing | 4 | 10-11-2015 08:14 AM |
What is Spiders, Robots and Crawlers? | anita_singh98 | Search Engine Optimization | 4 | 10-20-2014 04:39 AM |
Robots exclusion protocols? | jamesranatte | General Discussion | 7 | 12-26-2013 12:01 AM |