View Single Post
Old 02-25-2013, 03:18 AM   #4
jaysh4922
Registered User
 
Join Date: Dec 2012
Posts: 657
Robot.txt is an on-page SEO technique and it is basically used to allow for the web robots also known as the web wanderers, crawlers or spiders. It is a program that traverses the website automatically and this helps the popular search engine like Google to index the website and its content.

Just create a file named robots.txt, put it on your root directory of your website(where index.htm/index.php is), edit with a text editor and add these:

User-agent: *
Disallow: *something*
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
jaysh4922 is offline   Reply With Quote