01-09-2015, 01:47 AM
|
#16 |
|
Registered User
Join Date: Dec 2014
Posts: 23
|
Robots.txt are supposed to give the web crawlers instructions
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
|
01-09-2015, 11:05 AM
|
#17 |
|
Registered User
Join Date: Sep 2013
Posts: 887
|
Robots.txt file is a text file. That stops web crawler software such as Googlebot. from crawling certain pages of your site. The file is essentially a list of commands, such Allow and Disallow , that tell web crawlers which URLs they can or cannot retrieve.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. , To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. , To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. , To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. , To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
|
|
01-09-2015, 07:02 PM
|
#18 |
|
Registered User
Join Date: Sep 2014
Posts: 2
|
Cái này hay thật đó, cảm ơn vì chia sẻ của bạn
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
|
 |
«
Previous Thread
|
Next Thread
»
| Currently Active Users Viewing This Thread: 1 (0 members and 1 guests) | |
Linear Mode
|
|
Similar Threads
|
||||
| Thread | Thread Starter | Forum | Replies | Last Post |
| what is robot.txt? | rubysethi | Search Engine Optimization | 0 | 12-19-2014 11:27 PM |
| How to know about robot.txt ? | vanaja | Search Engine Optimization | 10 | 10-08-2014 04:39 AM |
| how can Create the Robot.txt | pankaj | Search Engine Optimization | 3 | 10-08-2014 03:43 AM |
| how can create the Robot.txt file | eagleeye dental | Search Engine Optimization | 0 | 09-01-2014 09:40 PM |
| how to use robot.txt? | dean1122 | Search Engine Optimization | 11 | 01-05-2013 02:10 AM |
All times are GMT -7. The time now is 10:20 AM.