Go Back   Site Owners Forums - Webmaster Forums > General Arena > General Discussion

Notices


Reply
 
Thread Tools Rate Thread Display Modes
Old 03-23-2017, 10:06 PM   #1
kumar1_16
Registered User
 
Join Date: Mar 2017
Posts: 29
What is Robot.txt?

Does any one know about robot.txt and how it is useful for SEO?
kumar1_16 is offline   Reply With Quote

Old 03-23-2017, 10:29 PM   #2
eFilingPortal
Registered User
 
Join Date: Sep 2014
Location: Gurgaon
Posts: 856
The robots exclusion standard, also known as the robots exclusion protocol or simply robots.txt, is a standard used by websites to communicate with web crawlers and other web robots. The standard specifies how to inform the web robot about which areas of the website should not be processed or scanned.

Thanks
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
eFilingPortal is offline   Reply With Quote
Old 03-23-2017, 11:43 PM   #3
gracedervishi
Registered User
 
Join Date: Mar 2017
Posts: 6
Robots.txt is a text file which is used for allow or disallow to any search engine crawlers to visit on their website or any particular page.
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

Last edited by gracedervishi; 03-23-2017 at 11:51 PM..
gracedervishi is offline   Reply With Quote
Old 03-23-2017, 11:55 PM   #4
Syn Gates
Registered User
 
Join Date: Mar 2017
Posts: 9
Robot.txt should be used by web developers, you can request which sites that search engine cannot crawl and index.
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
Syn Gates is offline   Reply With Quote
Old 03-24-2017, 03:11 AM   #5
Williamjohn02
Registered User
 
Join Date: Mar 2017
Posts: 13
Robots.txt is a text file which is used for allow or disallow to any search engine crawlers to visit on their website or any particular page.
Williamjohn02 is offline   Reply With Quote
Old 03-24-2017, 04:56 AM   #6
ankur111
Registered User
 
Join Date: Jun 2016
Location: Van Nuys
Posts: 34
Robots.txt is a text file which is used to give instructions to search engine crawler to visit on their website or any particular page.
ankur111 is offline   Reply With Quote
Old 03-24-2017, 07:28 AM   #7
robinmym
Registered User
 
Join Date: Feb 2017
Location: Dhaka
Posts: 486
The robots exclusion standard, also known as the robots exclusion protocol or simply robots.txt, is a standard used by websites to communicate with web crawlers and other web robots. The standard specifies how to inform the web robot about which areas of the website should not be processed or scanned.

The robots exclusion protocol (REP), or robots.txt is a text file webmasters create to instruct robots (typically search engine robots) how to crawl and index pages on their website.
robinmym is offline   Reply With Quote
Old 03-24-2017, 07:29 AM   #8
robinmym
Registered User
 
Join Date: Feb 2017
Location: Dhaka
Posts: 486
The robots exclusion standard, also known as the robots exclusion protocol or simply robots.txt, is a standard used by websites to communicate with web crawlers and other web robots. The standard specifies how to inform the web robot about which areas of the website should not be processed or scanned.

The robots exclusion protocol (REP), or robots.txt is a text file webmasters create to instruct robots (typically search engine robots) how to crawl and index pages on their website.
robinmym is offline   Reply With Quote
Old 03-27-2017, 02:17 AM   #9
sharmaroshni012
Registered User
 
Join Date: Nov 2016
Posts: 412
The robots exclusion protocol (REP), or robots.txt is a text file webmasters create to instruct robots (typically search engine robots) how to crawl and index pages on their website.
You might be surprised to hear that one small text file, known as robots.txt, could be the downfall of your website. If you get the file wrong you could end up telling search engine robots not to crawl your site, meaning your web pages won’t appear in the search results. Therefore, it’s important that you understand the purpose of a robots.txt file and learn how to check you’re using it correctly.
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
sharmaroshni012 is offline   Reply With Quote
Old 03-27-2017, 02:30 AM   #10
sin123
Registered User
 
Join Date: Jan 2017
Posts: 674
Robots.txt is a text file you put on your site to tell search robots which pages you would like them not to visit.
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
sin123 is offline   Reply With Quote
Old 03-28-2017, 02:04 AM   #11
maria5
Registered User
 
Join Date: Nov 2016
Location: Augusta, Georgia
Posts: 15
Robot.txt is a text file via which web crawler or web robots gets a notification regarding which page of the entire site should be indexed and which shouldn’t be.
__________________
Accounting in a leading
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
provider company
maria5 is offline   Reply With Quote
Old 03-29-2017, 03:15 AM   #12
stella
Registered User
 
Join Date: Mar 2017
Posts: 134
Robots.text is a text file which is used for allow or disallow to any search engine crawlers to visit on their website or any particular page.
stella is offline   Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
What is robot.txt file and what are the benefits of using it? Martinricky Social Networks 31 12-21-2016 02:46 AM
Which pages should be put in robot.txt file? Martinricky Search Engine Optimization 1 07-24-2016 11:15 PM
Robot.txt? siva SEO Search Engine Optimization 17 01-09-2015 07:02 PM
how to use robot.txt? dean1122 Search Engine Optimization 11 01-05-2013 02:10 AM


All times are GMT -7. The time now is 07:06 AM.


Powered by vBulletin Copyright © 2020 vBulletin Solutions, Inc.