05-25-2016, 11:52 PM | #1 |
Registered User
Join Date: Mar 2016
Location: bangalore
Posts: 75
|
What is a robots.txt file?
What is a robots.txt file?
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
05-26-2016, 12:23 AM | #2 |
Registered User
Join Date: May 2016
Posts: 115
|
Robots.txt is a text file that is uploaded to a Website's root directory and linked in the html code of the website. The robots.txt file used to provide instructions about the website to webroots and spiders.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
05-26-2016, 05:02 AM | #3 |
Registered User
Join Date: May 2016
Location: Florida, United State
Posts: 36
|
Robot.txt is a text file.
It gives instruction to search engine crawlers about indexing and caching of a web page, file of website or directory, domain.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
05-26-2016, 05:38 AM | #4 |
Registered User
Join Date: May 2016
Posts: 551
|
robots.txt is a text file webmasters create to instruct robots (typically search engine robots) how to crawl and index pages on their website.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
05-26-2016, 06:59 AM | #5 |
Registered User
Join Date: Sep 2015
Location: Bangalore
Posts: 548
|
Robots.txt is a text (not html) file you put on your site to tell search robots which pages you would like them not to visit.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
05-26-2016, 07:14 AM | #6 |
Registered User
Join Date: May 2016
Location: Dhaka
Posts: 55
|
What is Robots.txt File?
A Robots.txt file is an automatically generated by
Blogger for giving details information to search Robot about Blog posts. As a result when Search Robots from various search engines like Google, Bing, Yahoo, Yandex etc. visits your site then Robots.txt tells search crawler what to crawl and what not to crawl. |
05-26-2016, 07:41 AM | #7 |
Registered User
Join Date: May 2016
Location: New York City, NY, USA
Posts: 37
|
Site proprietors utilize the/robots.txt document to give guidelines about their webpage to web robots; this is known as The Robots Exclusion Protocol.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
05-26-2016, 09:48 PM | #8 |
Registered User
Join Date: May 2016
Posts: 22
|
Very Useful information on Robots.txt, thanks for sharing
|
05-26-2016, 11:14 PM | #9 |
Registered User
Join Date: Dec 2012
Posts: 657
|
Robots.txt is a text file that is inserted into your website and contains information for search engine robots. The file lists webpages that are allowed and disallowed from search engine crawling.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
Currently Active Users Viewing This Thread: 1 (0 members and 1 guests) | |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
What is the purpose of robots.txt file? | sahithya | Search Engine Optimization | 23 | 12-28-2021 03:48 AM |
How to edit virtual robots.txt file in wordpress? | geniusoptimizer | PHP / mySQL | 1 | 04-12-2013 12:36 PM |
The robots.txt file | rubberfender | Yahoo | 0 | 12-02-2012 07:32 PM |
Unable to delete robots.txt file | notfake | Yahoo | 0 | 04-29-2012 02:19 AM |