Importance of robots.txt in SEO - Page 2 - Site Owners Forums

outure11 · 03-22-2013, 03:17 AM

Robots.txt is very important txt file of our website. It is help to us to crawl the specific page of our website to be crawl or not.

moopons95 · 03-25-2013, 02:37 AM

Robot.txt is very important page.

vrfranks · 03-25-2013, 03:30 AM

It BLOCKS permission to crawl certain areas. zyxwvu was right about that

It is a simple text file that goes in the "root" folder of your website, the same folder that contains index.html

robots.txt has no importance in SEO

outure11 · 03-25-2013, 09:50 PM

Robots.txt file is a very important file if you want to have a good ranking on search engines, many websites don't offer this file. A Robots.txt file is helpful to keep out unwanted search engine spiders like email retrievers, image strippers, etc. It defines which paths are off limits for spiders to visit. This is useful if you want to hide some personal information or some secret files.

Martin2013 · 03-27-2013, 10:00 AM

User-Agent: *
Disallow:

Budd · 03-27-2013, 10:12 PM

will it help if your only indexing your home page? thats where all my anchor text points to.

03-22-2013, 03:17 AM	#16
outure11 Registered User Join Date: Oct 2012 Posts: 334	Robots.txt is very important txt file of our website. It is help to us to crawl the specific page of our website to be crawl or not. __________________ plus size bra

03-25-2013, 03:30 AM	#18
vrfranks Registered User Join Date: Mar 2013 Posts: 5	It BLOCKS permission to crawl certain areas. zyxwvu was right about that It is a simple text file that goes in the "root" folder of your website, the same folder that contains index.html robots.txt has no importance in SEO __________________ SMS Messages // Pakistani Songs

03-25-2013, 09:50 PM	#19
outure11 Registered User Join Date: Oct 2012 Posts: 334	Robots.txt file is a very important file if you want to have a good ranking on search engines, many websites don't offer this file. A Robots.txt file is helpful to keep out unwanted search engine spiders like email retrievers, image strippers, etc. It defines which paths are off limits for spiders to visit. This is useful if you want to hide some personal information or some secret files. __________________ plus size bra

03-27-2013, 10:00 AM	#20
Martin2013 Registered User Join Date: Feb 2013 Posts: 40	User-Agent: * Disallow: __________________ Indian social networking site \| social media sites in delhi \| top social dating sites in delhi \| online Indian friendship sites \| indian dating websites \| social media sites contest

03-27-2013, 10:12 PM	#21
Budd Registered User Join Date: Mar 2002 Location: Las Vegas, Nevada Posts: 211	will it help if your only indexing your home page? thats where all my anchor text points to. __________________ Will Carpet Cleaning Las Vegas Carpet Cleaning Henderson Upholstery Cleaning Las Vegas

03-25-2013, 02:37 AM	#17
moopons95 Registered User Join Date: Mar 2013 Posts: 39	Robot.txt is very important page.

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Evil bots consuming all my sites traffic even after i disallowed them in robots.txt	Fking	General Discussion	1	12-13-2013 11:27 PM
How to edit virtual robots.txt file in wordpress?	geniusoptimizer	PHP / mySQL	1	04-12-2013 12:36 PM
The robots.txt file	rubberfender	Yahoo	0	12-02-2012 07:32 PM
Does Anyone Know About The Importance Of SEO?	brettlee585	Search Engine Optimization	5	11-26-2012 05:15 AM
Unable to delete robots.txt file	notfake	Yahoo	0	04-29-2012 02:19 AM