08-07-2018, 03:05 AM | #1 |
Registered User
Join Date: Jun 2018
Location: India
Posts: 59
|
what is a robots.txt file?
Hi, my name is Sonam, please anyone explain what is a robots.txt file? and why it is an important role in SEO?
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
11-18-2018, 08:29 AM | #2 |
Registered User
Join Date: Nov 2016
Posts: 24
|
His knowledge was useful in his success
Hi, my name is Sonam, please anyone explain what is a robots.txt file? and why it is an important role in SEO?
Yes, thank you
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
11-18-2018, 08:32 AM | #3 |
Registered User
Join Date: Nov 2016
Posts: 24
|
His knowledge
Is a special webmaster and file that can give special commands to crawlers to search for private pages of your site and not to archive other pages
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
12-16-2018, 10:11 PM | #4 |
Registered User
Join Date: May 2015
Location: Forum
Posts: 2,517
|
Robots.txt is a text file that lists webpages which contain instructions for search engines robots. The file lists webpages that are allowed and disallowed from search engine crawling.
__________________
█ To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | $1 VPS Hosting █ To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | Windows Server with Remote Desktop Access █ To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | Servers with Free IPMI Setup |
12-18-2018, 05:05 AM | #5 |
Registered User
Join Date: Mar 2015
Location: Bangalore
Posts: 288
|
Robots.txt is a text file webmasters create to instruct robots (typically search engine robots) how to crawl & index pages on their website.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | |
06-09-2019, 11:46 PM | #6 |
Registered User
Join Date: Oct 2016
Posts: 655
|
Robots.txt is a text file that lists webpages which contain instructions for search engines robots.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
09-06-2019, 01:32 AM | #7 |
Registered User
Join Date: Aug 2016
Posts: 961
|
Robots.txt is a text file that lists webpages which contain instructions for search engines robots.
|
10-03-2019, 06:49 AM | #8 |
Registered User
Join Date: Oct 2015
Posts: 383
|
Robots.txt is a text file webmasters create to instruct web robots (typically search engine robots) how to crawl pages on their website.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
10-10-2019, 10:25 PM | #9 |
Registered User
Join Date: Feb 2016
Posts: 454
|
It helps to crawl all web crawler from all web pages or specif web crawler from specif web pages.
|
10-23-2019, 11:29 PM | #10 |
Registered User
Join Date: Jun 2015
Location: Singapore
Posts: 600
|
Robots.txt file instructs the web crawlers not to crawl specific page/s of the website.
__________________
The world's trusted To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
11-08-2019, 11:11 PM | #11 |
Registered User
Join Date: Jan 2013
Location: Delhi
Posts: 30
|
Robot.txt file is a part of on-page SEO ,in this text type file,we can allow or disallow to crawler that,which page will be crawled .or not
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
12-10-2019, 02:35 AM | #12 |
Registered User
Join Date: Jul 2015
Location: India, Delhi
Posts: 190
|
robots. txt file (also known as a robot exclusion protocol or standard) is a text file that tells web robots (the most common search engine) which pages on your site to crawl. It also tells web bots which pages to not crawl. Suppose a search engine is about to visit a website.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. , To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. , To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. , To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. , To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. , To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. , To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. , To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. , To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
Currently Active Users Viewing This Thread: 1 (0 members and 1 guests) | |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
How to edit virtual robots.txt file in wordpress? | geniusoptimizer | PHP / mySQL | 1 | 04-12-2013 12:36 PM |
The robots.txt file | rubberfender | Yahoo | 0 | 12-02-2012 07:32 PM |
Unable to delete robots.txt file | notfake | Yahoo | 0 | 04-29-2012 02:19 AM |