how to use robot.txt?
 

hi,
What is robot.txt in SEO and how to install .
Please friends share your view.
Thanks

robots.txt is a set of instructions to let the search engines know where and what to look at on your site.

you out it in the top-level directory of your web server.

read a couple articles before doing it though because there is much more detail then this

If you don't have a robot.txt the search engines just crawl every page of your website. There's a lot of articles on how to use the file.

Robot.txt is a text file which have set of instructions for crawlers ot crawl the website or a specific page. If you have it then it must be placed in root directory.....and for more help you can read lots Articles related to this...thanks

The robots.txt is a simple text file in your web site that informs search engine bots how to crawl and index website or web pages. It is great when search engines frequently visit your site and index your content but often there are cases when indexing parts of your online content is not what you want.

Robots.txt is a file that is used to exclude content from the crawling process of search engine spiders / bots. Robots.txt is also called the Robots Exclusion Protocol.

type this in
User-agent: * Disallow:

this is an example... open notepad, and just save a robots, then upload to your root.

Robots.txt is a file that is used to exclude content from the crawling process of search engine spiders / bots. Robots.txt.

Only this code is enough?
By adding this code which pages search engines will not visit?

robots.txt is a set of instructions to tell the search engines where and what to look at on your site.

robots.txt is a set of instructions to let the search engines know where and what to look for on your site.

Web site owners use the /robots.txt file to give instructions about their site to web robots; this is called The Robots Exclusion Protocol.

It works likes this: a robot wants to vists a Web site URL, say http://www.example.com/welcome.html. Before it does so, it firsts checks for http://www.example.com/robots.txt, and finds:
User-agent: *
Disallow: /

The "User-agent: *" means this section applies to all robots. The "Disallow: /" tells the robot that it should not visit any pages on the site.

There are two important considerations when using /robots.txt:

robots can ignore your /robots.txt. Especially malware robots that scan the web for security vulnerabilities, and email address harvesters used by spammers will pay no attention.
the /robots.txt file is a publicly available file. Anyone can see what sections of your server you don't want robots to use.