|
Crawlers, spider & robots.txt?
By default robots.txt is in port 80. There is any way to verify in case robots.txt it's not in port 80 to get it from any other port it may be. Thanks.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|