04-13-2014, 10:36 PM
|
#1 |
|
Registered User
Join Date: Apr 2014
Posts: 118
|
What is robots.txt?
This is a page that gives search engines information about the pages a company wants indexed or crawled. You can find this page by doing to YOURDOMAIN/robots.txt.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
|
04-13-2014, 10:51 PM
|
#2 |
|
Registered User
Join Date: Sep 2013
Location: Delhi
Posts: 344
|
Robots.txt file is very helpful to disallow the pages which you don't want to index in the search engine.
picnic spots near delhi | water park in delhi Last edited by rajivdevkk; 04-13-2014 at 10:53 PM.. |
|
|
|
|
04-13-2014, 10:55 PM
|
#3 |
|
Registered User
Join Date: Apr 2014
Posts: 17
|
Its a file to prevent the crawlers to not to crawl the restricted pages in your site.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
|
|
04-13-2014, 11:08 PM
|
#4 |
|
Registered User
Join Date: Dec 2013
Posts: 104
|
Robot.txt file is use for hide web page from search engine, if you don't want to index any page of website you can use this robot.txt and disallow that page....
|
|
|
|
|
04-13-2014, 11:12 PM
|
#5 |
|
Registered User
Join Date: Apr 2014
Posts: 33
|
with help to robots.txt Google find website internal page easily
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
|
|
04-13-2014, 11:12 PM
|
#6 |
|
Registered User
Join Date: Feb 2014
Posts: 541
|
its restrict the robots not crawler your all page only index page they can craw.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
|
|
04-14-2014, 03:07 AM
|
#7 |
|
Registered User
Join Date: Apr 2014
Posts: 118
|
thanx guys, for giving that nice suggestion or answer regarding robot.txt.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
|
|
04-14-2014, 03:49 AM
|
#8 |
|
Registered User
Join Date: Apr 2014
Posts: 5
|
The Robot Different Frequent, additionally known as the Robots Exclusion Protocol or robots.txt method, is a personalized to training working net elephants as well as added net spiders about acquiring all or portion of a site that's otherwise readily readable. Bots are usually employed by yahoo to truly save and classify websites, or by web masters to estimate source-code. The normal differs from, but could be used together with, Site Maps, a robot intro common for web sites.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
|
|
04-14-2014, 04:24 AM
|
#9 |
|
Registered User
Join Date: Apr 2014
Posts: 110
|
The robots.txt is a simple text file in your web site that inform search engine bots how to crawl and index website or web pages.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
|
|
04-14-2014, 04:36 AM
|
#10 |
|
Registered User
Join Date: Apr 2014
Posts: 18
|
Robots.txt file is a system file. If you still want to common things it by your own, create a text file using notepad or any other text editor and name it robots.txt. Then upload it to the root folder of your site by means File Manager or FTP.
|
|
|
|
|
04-14-2014, 05:19 AM
|
#11 |
|
Registered User
Join Date: Mar 2014
Posts: 357
|
Robots.txt is a text (not html) file you put on your site to tell search robots which pages you would like them not to visit. Robots.txt is by no means mandatory for search engines but generally search engines obey what they are asked not to do.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. - 100% Working and Valid hostgator Web Hosting Coupons |
|
|
|
|
05-14-2017, 10:56 PM
|
#12 |
|
Registered User
Join Date: Apr 2017
Posts: 355
|
Robots.txt is a text file you put on your site to tell search robots which pages you would like them not to visit. Robots.txt is by no means mandatory for search engines but generally search engines obey what they are asked not to do.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
|
 |
«
Previous Thread
|
Next Thread
»
| Currently Active Users Viewing This Thread: 1 (0 members and 1 guests) | |
Linear Mode
|
|
Similar Threads
|
||||
| Thread | Thread Starter | Forum | Replies | Last Post |
| Evil bots consuming all my sites traffic even after i disallowed them in robots.txt | Fking | General Discussion | 1 | 12-13-2013 11:27 PM |
| How to edit virtual robots.txt file in wordpress? | geniusoptimizer | PHP / mySQL | 1 | 04-12-2013 12:36 PM |
| The robots.txt file | rubberfender | Yahoo | 0 | 12-02-2012 07:32 PM |
| robots.txt | wheelbarrow001 | 2 | 07-27-2012 04:10 AM | |
| Unable to delete robots.txt file | notfake | Yahoo | 0 | 04-29-2012 02:19 AM |
All times are GMT -7. The time now is 11:11 PM.