How to Create the Robots txt file.
 

How to Create the Robots txt file.

Check this link it might help you
http://tools.seobook.com/robots-txt/generator/

First you can use any webmaster tool on the internet to create it. Website owner uses the robot.txt file to give instruction about their to site to web robots.

All you need to create the file in the high-level directory to your web server.
Robot looks for the " example.com/robot.txt" file
robots.txt to exclude all robots from the entire server, add next.

Simply used webmaster tools on the internet to create a file

Steps for creating Robots.txt:
You must save your robots.txt code as a text file,
You must place the file in the highest-level directory of your site (or the root of your domain), and
The robots.txt file must be named robots.txt.

1. Identify which directories and files on your web server you want to block from being rawled by the crawler
2. Identify whether or not you need to specify additional instructions for a particular search engine bot beyond a generic set of crawling directives
3. Use a text editor to create the robots.txt file and directives to block content
4. Optional: Add a reference to your sitemap file (if you have one)
5. Check for errors by validating your robots.txt file
6. Upload the robots.txt file to the root directory of your site

Use webmaster tool to generate robots.txt

A Web Crawler is a computer program that browses the World Wide Web in a methodical, automated manner or in an orderly fashion.
Other terms for Web crawlers are ants, automatic indexers, bots,Web...

Steps for creating Robots.txt:
You must save your robots.txt code as a text file,
You must place the file in the highest-level directory of your site (or the root of your domain), and
The robots.txt file must be named robots.txt

Robot.txt file is an html code that used to inform the crawler that the file content some it can be created with web master tools

Robot.txt file is an HTML code that used to inform the crawler that the file content some file that should not shows in search results

1.You must save your robots.txt code as a text file,
2.You must place the file in the highest-level directory of your site (or the root of your domain)

Tips to create robot .txt file
Create the file in the high-level directory to your web server.
Robot looks for the " example.com/robot.txt" file
robots.txt to exclude all robots from the entire server, add next

Steps for creating Robots.txt:
- save your robots.txt code as a text file,
- place the file in the highest-level directory of your site (or the root of your domain), and
- The robots.txt file must be named robots.txt.

Check this link

https://www.youtube.com/watch?v=Hi41dBV3BxM

HourGlass IT Solutions - Seo Companies in Chennai & Digital Marketing Services Chennai