Register on the forum now to remove ALL ads + popups + get access to tons of hidden content for members only!

Go Back   Site Owners Forums - Webmaster Forums > Search Engine Optimization > Search Engine Optimization

Notices


Reply
 
Thread Tools Rate Thread Display Modes
Old 02-15-2018, 11:27 PM   #1
ninadordev
Registered User
 
Join Date: Sep 2014
Posts: 263
What is the limit of a robot.txt file?

Hello friends,

What is the limit of a robot.txt file?
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
ninadordev is offline   Reply With Quote

Old 02-15-2018, 11:39 PM   #2
lilianalilly
Registered User
 
Join Date: Apr 2017
Posts: 387
there is a limit to robots.txt file in terms of filesize 500 KB as Vipin points out. > A maximum file size may be enforced per crawler. Content which is after the maximum file size may be ignored. Google currently enforces a size limit .
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
lilianalilly is offline   Reply With Quote
Old 02-15-2018, 11:56 PM   #3
Carlbrewster
Registered User
 
Join Date: Aug 2016
Posts: 145
Despite of implementing Robots.txt, there are chances that the blocked page is crawled and indexed by Google.
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
Carlbrewster is offline   Reply With Quote
Old 02-16-2018, 05:27 AM   #4
Yashmin
Registered User
 
Join Date: Mar 2017
Location: Dubai
Posts: 153
Yes, there is a limit to robots.txt file in terms of filesize 500 KB as Vipin points out. > A maximum file size may be enforced per crawler. Content which is after the maximum file size may be ignored.
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
Yashmin is offline   Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
What is the limit of a robot.txt file? ruhikhan Search Engine Optimization 7 01-22-2018 04:28 AM
What is robot.txt file and what are the benefits of using it? Martinricky Social Networks 31 12-21-2016 02:46 AM
Is there a limit to robots.txt file? sparkleoffice Search Engine Optimization 0 12-05-2016 02:58 AM
how can create the Robot.txt file eagleeye dental Search Engine Optimization 0 09-01-2014 10:40 PM


All times are GMT -7. The time now is 09:01 AM.


Powered by vBulletin Copyright © 2013 vBulletin Solutions, Inc.