Register on the forum now to remove ALL ads + popups + get access to tons of hidden content for members only!

Go Back   Site Owners Forums - Webmaster Forums > Search Engine Optimization > Search Engine Optimization

Notices


Reply
 
Thread Tools Rate Thread Display Modes
Old 03-18-2016, 01:21 AM   #1
europeanoversea
Registered User
 
Join Date: Jan 2016
Posts: 25
what is the use of robot.txt?

what is the use of robot.txt?
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
europeanoversea is offline   Reply With Quote

Old 03-18-2016, 01:52 AM   #2
alicemark15
Registered User
 
Join Date: Oct 2015
Posts: 104
Web site owners use the /robots.txt file to give instructions about their site to web robots; this is called The Robots Exclusion Protocol.
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
alicemark15 is offline   Reply With Quote
Old 03-18-2016, 01:58 AM   #3
arianagrand
Registered User
 
Join Date: Feb 2014
Posts: 205
robots.txt is a text file webmasters create to instruct robots (typically search engine robots) how to crawl and index pages on their website.
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
arianagrand is offline   Reply With Quote
Old 03-18-2016, 03:00 AM   #4
daviddakarai
Registered User
 
Join Date: Jan 2015
Posts: 759
Robots.txt is a simple text file that is placed on your site’s root directory. It is that file on your website that tells these search engine robots what to crawl and what not to crawl on your site. To facilitate auto-discovery of your sitemap file through your robots.txt, all you have to do is place a directive with the URL in your robots.txt
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
,
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
,
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
,
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
daviddakarai is offline   Reply With Quote
Old 03-18-2016, 03:17 AM   #5
krify
Registered User
 
Join Date: Feb 2016
Posts: 97
The main use of Robot text file is to make search engines know what is are the URLs need to cowl around the websites !

robot dot text is used to upload at the time of site creation then site with sitemap upload using ftp publications.
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
KRIFY DOT CO
krify is offline   Reply With Quote
Old 03-18-2016, 06:00 AM   #6
AshokDixit89
Registered User
 
Join Date: Jan 2016
Location: Mumbai, India
Posts: 1,161
Use of Robots.txt - The most common usage of Robots.txt is to ban crawlers from visiting private folders or content that gives them no additional information.

Robots.txt Allowing Access to Specific Crawlers.
Allow everything apart from certain patterns of URLs.
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
AshokDixit89 is offline   Reply With Quote
Old 03-18-2016, 11:20 PM   #7
indianlawwatch
Registered User
 
Join Date: Feb 2016
Posts: 48
Web site owners use the /robots.txt file to give instructions about their site to web robots; this is called The Robots Exclusion Protocol.
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
indianlawwatch is offline   Reply With Quote
Old 03-19-2016, 01:24 AM   #8
Sunsite
Registered User
 
Join Date: Mar 2016
Posts: 10
It is a instruction to robots in a text how to crawl and index pages on their website
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
business online
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
Sunsite is offline   Reply With Quote
Old 03-19-2016, 04:14 AM   #9
Sangeetakashyap
Registered User
 
Join Date: Sep 2015
Location: Delhi
Posts: 168
Robot.txt is a text file that has been used by website owners to give instructions to the web robots about the website. It helps the robots of Google to crawl each page of website so that it can index it properly & can rank it well in the Google search engine easily. It is an important factor of On Page of any website.
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.



To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.



To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
Sangeetakashyap is offline   Reply With Quote
Old 03-22-2016, 02:58 AM   #10
ketul
Registered User
 
Join Date: Aug 2014
Location: Bangalore
Posts: 3,143
The robots exclusion protocol (REP), or robots.txt is a text file webmasters create to instruct robots (typically search engine robots) how to crawl and index pages on their website.
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
ketul is offline   Reply With Quote
Old 03-22-2016, 04:37 AM   #11
accutranskpo
Registered User
 
Join Date: Mar 2016
Posts: 16
robots.txt is a text file which is used by web-master to give crawling instructions to the search engine crawlers which page is to be crawl and which page is not.
accutranskpo is offline   Reply With Quote
Old 03-22-2016, 04:54 AM   #12
vashikaransutra
Registered User
 
Join Date: Aug 2015
Posts: 173
It is must file for every website if webmasters want to do SEO. This file tells to Robots about which files should crawl or which not. It must place on root of the website.
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
vashikaransutra is offline   Reply With Quote
Old 03-22-2016, 05:31 AM   #13
sivakumar1
Registered User
 
Join Date: Jul 2013
Posts: 948
robots in a text how to crawl and index pages on their website you can access
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
sivakumar1 is offline   Reply With Quote
Old 03-22-2016, 05:49 AM   #14
Nexevo Technolo
Registered User
 
Join Date: Sep 2015
Location: Bangalore
Posts: 618
When a search engine crawler comes to your site, it will look for a special file on your site. That file is called robots.txt and it tells the search engine spider, which Web pages of your site should be indexed and which Web pages should be ignored
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
Nexevo Technolo is offline   Reply With Quote
Old 03-22-2016, 04:40 PM   #15
aajokhio
Registered User
 
Join Date: May 2015
Location: Hyderabad/Pakistan
Posts: 28
Basically, Robots Txt File Allow or Disallow with Google Robots for Crawl your Post, Pages etc. You set it as your requirement after that Google Robots crawl your post pages you have set up in Robots Txt File.
aajokhio is offline   Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Why we use robot.txt in SEO ? sabeerakhan00 Google 13 02-26-2016 04:36 AM
what is robot.txt? rubysethi Search Engine Optimization 0 12-19-2014 10:27 PM
How to know about robot.txt ? vanaja Search Engine Optimization 11 10-08-2014 04:39 AM
how can Create the Robot.txt pankaj Search Engine Optimization 3 10-08-2014 03:43 AM
how can create the Robot.txt file eagleeye dental Search Engine Optimization 0 09-01-2014 09:40 PM


All times are GMT -7. The time now is 06:29 PM.


Powered by vBulletin Copyright © 2013 vBulletin Solutions, Inc.