10-26-2016, 05:40 AM | #1 |
Registered User
Join Date: Oct 2015
Posts: 36
|
What is the use of Robots.txt ?
Hello friends,
I want to know that What is the use of Robots.txt ? tell me ..
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
10-26-2016, 11:02 PM | #2 |
Registered User
Join Date: Aug 2016
Posts: 195
|
Web site owners use the /robots.txt file to give instructions about their site to web robots; this is called The Robots Exclusion Protocol. The "User-agent: *" means this section applies to all robots. The "Disallow: /" tells the robot that it should not visit any pages on the site.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
10-27-2016, 02:14 AM | #3 |
Registered User
Join Date: Sep 2016
Posts: 326
|
Web site owners use the /robots.txt file to give instructions about their site to web robots.The "Disallow: /" tells the robot that it should not visit any pages on the site.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
10-27-2016, 02:15 AM | #4 |
Registered User
Join Date: Jul 2016
Posts: 755
|
Web site owners use the /robots.txt file to give instructions about their site to web robots.The "Disallow: /" tells the robot that it should not visit any pages on the site.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
10-27-2016, 03:34 AM | #5 |
Registered User
Join Date: Oct 2016
Posts: 71
|
Robot.txt files are used to give information to the spiders to visit the website for crawling and indexing.We can use Disallow option which will tell the robot not to visit the website.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
10-27-2016, 05:45 AM | #6 | |
Registered User
Join Date: Sep 2016
Posts: 85
|
Quote:
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
10-27-2016, 06:06 AM | #7 |
Registered User
Join Date: Sep 2016
Location: chennai
Posts: 93
|
Robots.txt is nothing hide the some pages of our site because of privacy.for example you have an e-commerce site like shopclues, flipkart in festival time we post some offer in particular page after u no need but it can use future purpose in that situation also we can use.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. logo designers chennai To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. brochure designers chennai To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. Love marriage procedure in Chennai, How to register marriage in Chennai, Court marriage procedure in Chennai To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. HRD Attestation in Chennai, Embassy Attestation, Apostalle Services To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. UAE Embassy attestation in bangalore |
10-27-2016, 06:39 AM | #8 |
Registered User
Join Date: Oct 2016
Posts: 37
|
It's a file used by the webmasters to give instructions to web robots. It helps if we don't want any page to be crawled by any web robot.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
10-27-2016, 06:50 AM | #9 |
Registered User
Join Date: May 2016
Posts: 105
|
Robots.txt is a text file,we can write in word and notepad. Robots.txt you put on your website to tell the search engine which pages you would like them not to visit.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
10-27-2016, 07:08 PM | #10 |
Registered User
Join Date: Oct 2016
Posts: 33
|
It is great when search engines frequently visit your site and index your content but often there are cases when indexing parts of your online content is not what you want. For instance, if you have two versions of a page (one for viewing in the browser and one for printing), you'd rather have the printing version excluded from crawling, otherwise you risk being imposed a duplicate content penalty. Also, if you happen to have sensitive data on your site that you do not want the world to see, you will also prefer that search engines do not index these pages (although in this case the only sure way for not indexing sensitive data is to keep it offline on a separate machine). Additionally, if you want to save some bandwidth by excluding images, stylesheets and javascript from indexing, you also need a way to tell spiders to keep away from these items.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
11-01-2016, 11:04 PM | #11 |
Registered User
Join Date: Jun 2016
Posts: 219
|
Web site owners use the /robots.txt file to give instructions about their site to web robots.The "Disallow: /" tells the robot that it should not visit any pages on the site that is called for use robot.txt
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
11-03-2016, 02:35 AM | #12 |
Registered User
Join Date: Oct 2016
Posts: 81
|
Web site owners use the /robots.txt file to give instructions about their site to web robots; this is called The Robots Exclusion Protocol. The "User-agent: *" means this section applies to all robots. The "Disallow: /" tells the robot that it should not visit any pages on the site.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
11-03-2016, 05:17 AM | #13 |
Registered User
Join Date: Aug 2016
Posts: 175
|
Web site owners use the /robots.txt file to give instructions about their site to web robots; this is called The Robots Exclusion Protocol. The "User-agent: *" means this section applies to all robots. The "Disallow: /" tells the robot that it should not visit any pages on the site.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
11-03-2016, 05:39 AM | #14 |
Registered User
Join Date: Nov 2016
Posts: 24
|
robots.txt is a ON PAGE SEO TECHNIQUE is used to hide your data during the google reading on your website.
it has two choice Allow and Disallow if give ur data Allow site google read your data otherwise it can't read your data |
11-03-2016, 06:16 AM | #15 |
Registered User
Join Date: Aug 2015
Location: London
Posts: 47
|
When a search engine crawls (visits) your website, the first thing it looks for is your robots.txt file. This file tells search engines what they should and should not index.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
Currently Active Users Viewing This Thread: 1 (0 members and 1 guests) | |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
Robots.txt | jaysh4922 | Search Engine Optimization | 13 | 07-28-2016 05:10 AM |
Crawlers, spider & robots.txt? | jackthomas087 | Search Engine Optimization | 0 | 04-22-2014 03:29 AM |
Evil bots consuming all my sites traffic even after i disallowed them in robots.txt | Fking | General Discussion | 1 | 12-13-2013 11:27 PM |
How to edit virtual robots.txt file in wordpress? | geniusoptimizer | PHP / mySQL | 1 | 04-12-2013 12:36 PM |
Unable to delete robots.txt file | notfake | Yahoo | 0 | 04-29-2012 02:19 AM |