04-19-2017, 10:49 PM | #1 |
Registered User
Join Date: Aug 2013
Posts: 332
|
Why We Use robot.txt file In Seo..?
Hello friends,
Why We Use robot.txt file In Seo..?
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
04-19-2017, 10:55 PM | #2 |
Registered User
Join Date: Mar 2017
Posts: 224
|
When a search engine crawler comes to your site, it will look for a special file on your site. That file is called robots.txt and it tells the search engine spider, which Web pages of your site should be indexed and which Web pages should be ignored.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
04-19-2017, 11:31 PM | #3 |
Registered User
Join Date: Oct 2016
Posts: 259
|
The robots exclusion protocol (REP), or robots.txt is a text file webmasters create to instruct robots (typically search engine robots) how to crawl and index pages on their website.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
04-19-2017, 11:34 PM | #4 |
Registered User
Join Date: Nov 2016
Posts: 515
|
Robots.txt is a text file you put on your site to tell search robots which pages you wouls like them not to visit
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
04-20-2017, 04:18 AM | #5 |
Registered User
Join Date: Jan 2017
Posts: 108
|
Robots.txt is a text file you put on your site to tell search robots which pages you wouls like them not to visit
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
04-20-2017, 08:59 AM | #6 |
Registered User
Join Date: Apr 2017
Posts: 74
|
In a nutshell. Web site owners use the /robots.txt file to give instructions about their site to web robots; this is called The Robots Exclusion Protocol. The "User-agent: *" means this section applies to all robots. The "Disallow: /" tells the robot that it should not visit any pages on the site.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
04-20-2017, 10:11 AM | #7 |
Registered User
Join Date: Mar 2017
Posts: 264
|
Robots.txt is a text file you put on your site to tell search robots which pages you would like them not to visit.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
04-20-2017, 12:59 PM | #8 |
Registered User
Join Date: Apr 2017
Location: Arizona, USA
Posts: 16
|
You can use Google Search Consol to test your sites robots.txt. I definitely recommend doing this to make sure it is working properly
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. // To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. // To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. // To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
Currently Active Users Viewing This Thread: 1 (0 members and 1 guests) | |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
What is robot.txt file and what are the benefits of using it..? | AshokDixit89 | Search Engine Optimization | 8 | 05-05-2017 04:30 AM |
What is robot.txt file and what are the benefits of using it? | Martinricky | Social Networks | 31 | 12-21-2016 02:46 AM |
Which pages should be put in robot.txt file? | Martinricky | Search Engine Optimization | 1 | 07-24-2016 11:15 PM |
how can create the Robot.txt file | eagleeye dental | Search Engine Optimization | 0 | 09-01-2014 09:40 PM |