03-22-2013, 03:17 AM
|
#16 |
|
Registered User
Join Date: Oct 2012
Posts: 334
|
Robots.txt is very important txt file of our website. It is help to us to crawl the specific page of our website to be crawl or not.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
|
03-25-2013, 02:37 AM
|
#17 |
|
Registered User
Join Date: Mar 2013
Posts: 39
|
Robot.txt is very important page.
|
|
|
|
|
03-25-2013, 03:30 AM
|
#18 |
|
Registered User
Join Date: Mar 2013
Posts: 5
|
It BLOCKS permission to crawl certain areas. zyxwvu was right about that
It is a simple text file that goes in the "root" folder of your website, the same folder that contains index.html robots.txt has no importance in SEO
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. // To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
|
|
03-25-2013, 09:50 PM
|
#19 |
|
Registered User
Join Date: Oct 2012
Posts: 334
|
Robots.txt file is a very important file if you want to have a good ranking on search engines, many websites don't offer this file. A Robots.txt file is helpful to keep out unwanted search engine spiders like email retrievers, image strippers, etc. It defines which paths are off limits for spiders to visit. This is useful if you want to hide some personal information or some secret files.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
|
|
03-27-2013, 10:00 AM
|
#20 |
|
Registered User
Join Date: Feb 2013
Posts: 40
|
User-Agent: *
Disallow:
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
|
|
03-27-2013, 10:12 PM
|
#21 |
|
Registered User
Join Date: Mar 2002
Location: Las Vegas, Nevada
Posts: 211
|
will it help if your only indexing your home page? thats where all my anchor text points to.
__________________
Will To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
|
 |
«
Previous Thread
|
Next Thread
»
| Currently Active Users Viewing This Thread: 1 (0 members and 1 guests) | |
Linear Mode
|
|
Similar Threads
|
||||
| Thread | Thread Starter | Forum | Replies | Last Post |
| Evil bots consuming all my sites traffic even after i disallowed them in robots.txt | Fking | General Discussion | 1 | 12-13-2013 11:27 PM |
| How to edit virtual robots.txt file in wordpress? | geniusoptimizer | PHP / mySQL | 1 | 04-12-2013 12:36 PM |
| The robots.txt file | rubberfender | Yahoo | 0 | 12-02-2012 07:32 PM |
| Does Anyone Know About The Importance Of SEO? | brettlee585 | Search Engine Optimization | 5 | 11-26-2012 05:15 AM |
| Unable to delete robots.txt file | notfake | Yahoo | 0 | 04-29-2012 02:19 AM |
All times are GMT -7. The time now is 01:47 PM.