02-24-2013, 02:18 AM | #1 |
Registered User
Join Date: Jan 2013
Posts: 73
|
What is robot text
Do you know about robot text? please say and discuss about it.
__________________
AMAZON PROMOTIONAL CODES To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
02-24-2013, 10:20 PM | #2 |
Registered User
Join Date: Feb 2013
Posts: 21
|
Robots.txt is a file through which you can guide search engines to crawl or not to crawl certain sections of your website.Robot.txt is used to prevent crawlers crawl on a certain page.
|
02-25-2013, 12:19 AM | #3 |
Registered User
Join Date: Feb 2013
Location: delhi
Posts: 103
|
By the help of robots.txt file, search engines crawls your webpages or not crawls your pages.
day picnic around delhi
__________________
camp in kanatal |
02-25-2013, 03:18 AM | #4 |
Registered User
Join Date: Dec 2012
Posts: 657
|
Robot.txt is an on-page SEO technique and it is basically used to allow for the web robots also known as the web wanderers, crawlers or spiders. It is a program that traverses the website automatically and this helps the popular search engine like Google to index the website and its content.
Just create a file named robots.txt, put it on your root directory of your website(where index.htm/index.php is), edit with a text editor and add these: User-agent: * Disallow: *something*
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
02-25-2013, 05:31 AM | #5 |
Registered User
Join Date: Feb 2013
Posts: 165
|
robots.txt is a text file which instructs the search engine what are the pages should be crawled.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
02-25-2013, 05:33 AM | #6 |
Registered User
Join Date: Nov 2012
Location: India
Posts: 276
|
Robots.txt is the text file that is mostly used to instruct search engine which page should be crawled and which shouldn't be crawled.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. - Shopping Cart- PrestaShop, Zencart; CMS - WordPress, Joomla; Custom Development To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. - SEO, Pay Per Click (PPC), SEM, Link Popularity Building, SEO Copywriting, SMO, SMM |
02-25-2013, 06:10 AM | #7 |
Registered User
Join Date: Feb 2013
Posts: 2
|
Robots.txt is a file that you can add to your website which instructs the search engine crawler which pages to crawl. You can also instruct the crawlers not to crawl some pages through robots.txt.
|
02-25-2013, 06:23 AM | #8 |
Registered User
Join Date: Feb 2013
Location: Ahmedabad
Posts: 39
|
Robots.txt is a text (not html) file you put on your site to tell search robots which pages you would like them not to visit.
The structure of a robots.txt is pretty simple (and barely flexible) – it is an endless list of user agents and disallowed files and directories. Basically, the syntax is as follows: User-agent: Disallow: “User-agent” are search engines' crawlers and disallow: lists the files and directories to be excluded from indexing. In addition to “user-agent:” and “disallow:” entries, you can include comment lines – just put the # sign at the beginning of the line: # All user agents are disallowed to see the /temp directory. User-agent: * Disallow: /temp/ |
02-25-2013, 07:08 AM | #9 |
Registered User
Join Date: May 2012
Location: India
Posts: 64
|
Robots.txt file is in the root directory of a website which has been created to direct
the activity of search engine crawlers or spiders.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. , To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. , To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
02-25-2013, 11:00 PM | #10 |
Registered User
Join Date: Feb 2013
Posts: 44
|
The concept and structure of robots.txt has been developed more than a decade ago. Robots.txt is a text (not html) file you put on your site to tell search robots which pages you would like them not to visit. Robots.txt is by no means mandatory for search engines but generally search engines obey what they are asked not to do. The location of robots.txt is very important. It must be in the main directory because otherwise user agents (search engines) will not be able to find it – they do not search the whole site for a file named robots.txt.
|
02-26-2013, 03:07 PM | #11 |
Registered User
Join Date: Jan 2013
Posts: 73
|
Go on......Thanks all..
__________________
AMAZON PROMOTIONAL CODES To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
02-27-2013, 03:30 AM | #12 |
Registered User
Join Date: Feb 2013
Location: Philadelphia USA
Posts: 76
|
as you can easily judge by its name, robot txt means message or txt for robot (search engine spiders). In you webmaster tell bots which page to crawl or which not.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
02-28-2013, 04:12 AM | #13 |
Registered User
Join Date: May 2012
Location: India
Posts: 64
|
I do not see anymore effects of Robot.text file. Search Engine also index those one which comes under no follow. I tried that. No effect of this file now.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. , To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. , To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
02-28-2013, 05:16 AM | #14 |
Registered User
Join Date: May 2012
Posts: 65
|
Robots.txt is a text (not html) file you put on your site to tell search robots which pages you would like them not to visit. Robots.txt is by no means mandatory for search engines but generally search engines obey what they are asked not to do.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
02-28-2013, 09:12 AM | #15 |
Registered User
Join Date: Dec 2012
Location: delhi
Posts: 56
|
robots text is normal file in which you can give instruction which page you crawl or not to all robots of search engines.
__________________
hi i am a webmaster who helps all peoples to get rank well. |
Currently Active Users Viewing This Thread: 1 (0 members and 1 guests) | |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
telepresence robot | hemnchy | General Discussion | 1 | 01-02-2014 04:35 AM |
Robot text. | terrijhon | Search Engine Optimization | 18 | 06-08-2013 03:35 PM |
Hidden text and links | farm gate | 0 | 11-11-2012 08:08 PM | |
Anchor Text Vs Google Algorithm | Jimerson12 | Search Engine Optimization | 0 | 08-16-2012 03:26 AM |
Text Areas | ncjbhd | HTML / DHTML | 1 | 11-15-2011 04:09 AM |