robots meta tag - Site Owners Forums

zinavoco · 08-28-2015, 05:40 AM

If robots meta tag is not included in a website page the effect is

Rajan007 · 09-24-2015, 11:06 PM

If you want to restrict search engines from crawling some important pages of your website then you need robots.txt. or otherwise you no need to add robots.txt file.

shanken · 09-25-2015, 12:34 AM

Quote:

Originally Posted by zinavoco

If robots meta tag is not included in a website page the effect is

Nothing will happen, your page will be visited by robots and spiders and they will index your page as normal.

If you do ad robots meta tag with noindex then your page wont be indexed.

shanken · 09-25-2015, 12:38 AM

Quote:

Originally Posted by cpsi

Robot Visit your Website And Collect your Information. If you don't include robot meta tag then google not crawl your website.

Are you sure about that? You use the robots meta tag to stop Goolge from indexing a particular page and to set nofollow link atributes to all links on that page.

You can also use the robots meta tag to only allow certain spiders like Google bot to index your page.

If you dont ad the robots metaa tag then you are basically telling Google to go ahead and index my page.

shanken · 09-25-2015, 12:40 AM

Quote:

Originally Posted by manpreetkaur

The robots meta tag lets you utilize a granular, page-specific approach to controlling how an individual page should be indexed and served to users in search results. Place the robots meta tag in the <head> section of a given page, like this:

<!DOCTYPE html>
<html><head>
<meta name="robots" content="noindex" />
(…)
</head>
<body>(…)</body>
</html>
The robots meta tag in the above example instructs all search engine not to show the page in search results. The value of the name attribute (robots) specifies that the directive applies to all crawlers. To address a specific crawler, replace the robots value of the name attribute with the name of the crawler that you are addressing. Specific crawlers are also known as user-agents (a crawler uses its user-agent to request a page.) Google's standard web crawler has the user-agent name Googlebot. To prevent only Googlebot from crawling your page, update the tag as follows:

<meta name="googlebot" content="noindex" />
This tag now instructs Google (but no other search engines) not to show this page in its web search results. Both the name and the content attributes are non-case sensitive.

Search engines may have different crawlers for different properties or purposes. See the complete list of Google's crawlers. For example, to show a page in Google's web search results, but not in Google News, use the following meta tag:

<meta name="googlebot-news" content="noindex" />
If you need to specify multiple crawlers individually, it's okay to use multiple robots meta tags:

<meta name="googlebot" content="noindex">
<meta name="googlebot-news" content="nosnippet">
If competing directives are encountered by our crawlers we will use the most restrictive directive we find.

Best answer of the whole lot.

rajivdevkk · 09-25-2015, 11:30 PM

meta robots is useful fo crawl and discrawls the site pages.

<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">

day picnic delhi ncr | picnic spots in gurgaon

rajeshhome · 09-26-2015, 12:52 AM

The robots.txt is a text file webmasters create to instruct robots how to crawl and index pages on their website.

Martinricky · 09-26-2015, 02:24 AM

Thanks for sharing informative tips about meta tags, looking forward for more information in this thread

randiv · 09-27-2015, 11:22 PM

Nice Information sharing about robots. Thanks.

ramskl · 09-27-2015, 11:26 PM

You can use a special HTML <META> tag to tell robots not to index the content of a page, and/or not scan it for links to follow.

08-28-2015, 05:40 AM	#1
zinavoco Registered User Join Date: Aug 2015 Posts: 2	robots meta tag If robots meta tag is not included in a website page the effect is __________________ Web Design Company in India \| Web Development Company in India \| Web Designing Company in India \| Web Development Companies in India

09-24-2015, 11:06 PM	#2
Rajan007 Registered User Join Date: Sep 2015 Posts: 75	If you want to restrict search engines from crawling some important pages of your website then you need robots.txt. or otherwise you no need to add robots.txt file. __________________ Excel Training in Hyderabad Excel VBA Training in Chennai

09-26-2015, 12:52 AM	#7
rajeshhome Registered User Join Date: Sep 2015 Posts: 33	The robots.txt is a text file webmasters create to instruct robots how to crawl and index pages on their website. __________________ Medical Billing Outsourcing USA \| Revenue Cycle Management Services \| Outsourcing Services in USA \| Medical Billing Experts Services USA \| Data Entry Outsourcing Company in USA \| Eligibility Verification and Benefits Services \| Account Receivables Management Services

09-26-2015, 02:24 AM	#8
Martinricky Registered User Join Date: Aug 2015 Posts: 434	Thanks for sharing informative tips about meta tags, looking forward for more information in this thread __________________ Dentist in Delhi \| Best Dentist in Ashok Vihar \| Best Dentist in Shalimar Bagh \| Best Prosthodontist in Delhi

09-27-2015, 11:26 PM	#10
ramskl Registered User Join Date: Sep 2015 Posts: 240	You can use a special HTML <META> tag to tell robots not to index the content of a page, and/or not scan it for links to follow. __________________ Link Building Resources \| Free SEO Tools List \| Engravers Chennai \| Engraving Chennai

09-25-2015, 11:30 PM	#6
rajivdevkk Registered User Join Date: Sep 2013 Location: Delhi Posts: 344	meta robots is useful fo crawl and discrawls the site pages. <META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW"> day picnic delhi ncr \| picnic spots in gurgaon

09-27-2015, 11:22 PM	#9
randiv Registered User Join Date: May 2014 Posts: 105	Nice Information sharing about robots. Thanks.

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)

Thread Tools
Show Printable Version Email this Page
Display Modes	Rate This Thread
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode	Rate This Thread:

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
The Sticky for Meta Description, keywords, keyword density, and title tags	sinicline	Search Engine Optimization	2	01-06-2017 10:22 AM
Check the website robots file	china-jewels	Bing	4	10-11-2015 08:14 AM
If the meta robots tag has a value of "no index, no follow" what does it mean?	poojaseo	Search Engine Optimization	0	06-06-2014 03:08 AM
How to add a robots follow meta tag?	jamesranatte	Search Engine Optimization	1	10-11-2012 10:24 PM
What is Robots meta tag?	samlko	Search Engine Optimization	2	04-06-2012 12:10 AM