07-12-2022, 11:56 PM
|
#16 |
|
Registered User
Join Date: Jan 2016
Location: Hyderabad
Posts: 4,287
|
What is the limit of a robot.txt file?
Because the robots. txt file contains information about how the search engine should crawl, the information found there will instruct further crawler action on this particular site. If the robots. txt file does not contain any directives that disallow a user-agent's activity (or if the site doesn't have a robots.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
 |
«
Previous Thread
|
Next Thread
»
| Currently Active Users Viewing This Thread: 1 (0 members and 1 guests) | |
Linear Mode
|
|
Similar Threads
|
||||
| Thread | Thread Starter | Forum | Replies | Last Post |
| What is the limit of a robot.txt file? | artijain1900 | Search Engine Optimization | 11 | 05-17-2022 12:48 AM |
| What is the limit of a robot.txt file? | ninadordev | Search Engine Optimization | 4 | 12-24-2021 12:29 AM |
| What is the limit of a robot.txt file? | partee | Review My Website | 6 | 09-01-2019 10:27 PM |
| What is the limit of a robot.txt file? | NandiniShinde | Search Engine Optimization | 23 | 07-25-2018 11:49 PM |
| What is the limit of a robot.txt file? | ruhikhan | Search Engine Optimization | 7 | 01-22-2018 04:28 AM |
All times are GMT -7. The time now is 01:08 PM.