What is robots.txt? - Site Owners Forums

danish00 · 03-28-2013, 10:54 PM

Hi friend !
What is robots.txt. is it necessary for website and how to make robots file.
camp kanatal

peterthomas · 03-28-2013, 11:29 PM

Robot.txt file is a simple notepad file which process use for crawler.

festivlbuts · 03-29-2013, 12:34 AM

A robots.txt file on a website will function as a request that specified robots ignore specified files or directories when crawling a site.

williamrichard · 03-29-2013, 04:04 AM

Robots.txt is a text file you put it on your web site to tell search robots which pages you would like them not to visit. It is important to clarify that robots.txt is not a way from preventing search engines from crawling your site.

maria.methews · 03-29-2013, 02:32 PM

robot.txt file is for bot, tell him what is allowed and disallow for indexation

laylamichelle · 03-29-2013, 10:36 PM

robot.txt is a simple text file which is used to instruct the search engine which page have to be cache or not.

ShivJaiswaL · 05-04-2013, 11:34 PM

hello friend, Robots.txt is a text file resides in root DIR of host website. It permits us to block complete or partial website portion. It has three parts i,e. user agent, disallow section and sitemaps declarations section. Almost all major search engines like Google, Bing, http://www.oko69.com ...... respect the presence of robots.txt.

montyalexa · 05-07-2013, 12:32 AM

Robots.txt is text file, when give permission which URL crawling or n't

reddykalpana87 · 05-07-2013, 03:04 AM

Robots.txt is a plain text ﬁle that you upload to the root directory of your site.

Cameonouveau · 05-08-2013, 04:26 AM

robots.txt is text file placed in a websites root directory and linked in the html code.

kim85 · 05-08-2013, 08:34 AM

It helps boost the crawling rate

delhicourses · 05-09-2013, 06:00 AM

a robots.txt is a file in which you can block any pages or files, like you do not want google will read abouts page then you can block that page.

Matt Hutchings · 05-09-2013, 12:00 PM

I was wondering what that was. Thanks guys.

mitsay · 05-09-2013, 07:41 PM

Robot.txt is the notepad file you have a crawl you page which one that mostly not use so you add that one page in robot.txt so that page is online so client see very well but Google is not crawl that one page....

zumanto · 05-12-2013, 07:54 PM

Robots text file are those which forced to search engine how to crawl and index your site or web pages.
It is great when search engines frequently visit your site and index your content but often there are cases when indexing parts of your online content is not what you want.

03-28-2013, 10:54 PM	#1
danish00 Registered User Join Date: Feb 2013 Location: delhi Posts: 103	What is robots.txt? Hi friend ! What is robots.txt. is it necessary for website and how to make robots file. camp kanatal __________________ camp in kanatal

03-28-2013, 11:29 PM	#2
peterthomas Registered User Join Date: Feb 2013 Posts: 35	Robot.txt file is a simple notepad file which process use for crawler. __________________ To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

03-29-2013, 04:04 AM	#4
williamrichard Registered User Join Date: Feb 2013 Posts: 29	Robots.txt is a text file you put it on your web site to tell search robots which pages you would like them not to visit. It is important to clarify that robots.txt is not a way from preventing search engines from crawling your site. __________________ To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

03-29-2013, 02:32 PM	#5
maria.methews Registered User Join Date: Aug 2012 Location: Rochester, New York Posts: 47	robot.txt file is for bot, tell him what is allowed and disallow for indexation __________________ To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

05-04-2013, 11:34 PM	#7
ShivJaiswaL Registered User Join Date: Feb 2012 Posts: 94	hello friend, Robots.txt is a text file resides in root DIR of host website. It permits us to block complete or partial website portion. It has three parts i,e. user agent, disallow section and sitemaps declarations section. Almost all major search engines like Google, Bing, http://www.oko69.com ...... respect the presence of robots.txt. __________________ To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

03-29-2013, 12:34 AM	#3
festivlbuts Registered User Join Date: Mar 2013 Posts: 3	A robots.txt file on a website will function as a request that specified robots ignore specified files or directories when crawling a site.

03-29-2013, 10:36 PM	#6
laylamichelle Registered User Join Date: Mar 2013 Posts: 15	robot.txt is a simple text file which is used to instruct the search engine which page have to be cache or not.

05-07-2013, 12:32 AM	#8
montyalexa Registered User Join Date: Jan 2011 Posts: 33	Robots.txt is text file, when give permission which URL crawling or n't

05-07-2013, 03:04 AM	#9
reddykalpana87 Registered User Join Date: Feb 2013 Location: India Posts: 40	Robots.txt is a plain text ﬁle that you upload to the root directory of your site. __________________ To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

05-08-2013, 04:26 AM	#10
Cameonouveau Registered User Join Date: May 2013 Location: New Jersey Posts: 2	robots.txt is text file placed in a websites root directory and linked in the html code. __________________ To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

05-08-2013, 08:34 AM	#11
kim85 Registered User Join Date: Apr 2013 Posts: 15	It helps boost the crawling rate __________________ To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

05-09-2013, 06:00 AM	#12
delhicourses Registered User Join Date: Dec 2012 Location: delhi Posts: 56	a robots.txt is a file in which you can block any pages or files, like you do not want google will read abouts page then you can block that page. __________________ hi i am a webmaster who helps all peoples to get rank well.

05-09-2013, 12:00 PM	#13
Matt Hutchings Registered User Join Date: Apr 2013 Posts: 97	I was wondering what that was. Thanks guys. __________________ To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

05-09-2013, 07:41 PM	#14
mitsay Registered User Join Date: Apr 2013 Posts: 64	Robot.txt is the notepad file you have a crawl you page which one that mostly not use so you add that one page in robot.txt so that page is online so client see very well but Google is not crawl that one page.... __________________ To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. - To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. - To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

05-12-2013, 07:54 PM	#15
zumanto Registered User Join Date: Mar 2013 Posts: 74	Robots text file are those which forced to search engine how to crawl and index your site or web pages. It is great when search engines frequently visit your site and index your content but often there are cases when indexing parts of your online content is not what you want. __________________ To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. - To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. - To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
What is robots.txt?	seoinheritx	Search Engine Optimization	29	09-21-2022 11:57 PM
Evil bots consuming all my sites traffic even after i disallowed them in robots.txt	Fking	General Discussion	1	12-13-2013 11:27 PM
How to edit virtual robots.txt file in wordpress?	geniusoptimizer	PHP / mySQL	1	04-12-2013 12:36 PM
The robots.txt file	rubberfender	Yahoo	0	12-02-2012 07:32 PM
Unable to delete robots.txt file	notfake	Yahoo	0	04-29-2012 02:19 AM