|
Robots.txt file contains set of instruction, which tells google what to crawl on that particular site.
sample code
User Agent: *
Allow: /
The above code give permission to robots to crawl the entire website.
write this code in a notepad and save it as robots.txt and upload it in your server.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|