03-28-2018, 11:37 PM
|
#7 |
|
Registered User
Join Date: Feb 2018
Posts: 190
|
Robots.txt file is used to information to the web crawler whether it crawls particular page or not. For example,the webpage consists of information about login and payment details we can make disallow that page.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
«
Previous Thread
|
Next Thread
»
| Currently Active Users Viewing This Thread: 1 (0 members and 1 guests) | |
| Thread Tools | |
| Display Modes | Rate This Thread |
Threaded Mode
|
|
Similar Threads
|
||||
| Thread | Thread Starter | Forum | Replies | Last Post |
| What is the limit of a robot.txt file? | ninadordev | Search Engine Optimization | 4 | 12-24-2021 12:29 AM |
| What is the limit of a robot.txt file? | ruhikhan | Search Engine Optimization | 7 | 01-22-2018 04:28 AM |
| What is robot.txt file and what are the benefits of using it? | Martinricky | Social Networks | 31 | 12-21-2016 02:46 AM |
All times are GMT -7. The time now is 01:44 PM.