Why is robert.txt file is important to Website ?
 

Hey!!!


I am an seo executive and i want to know that Why is robert.txt file is important to Website ? tell me ..

It is great when google regularly view your site and catalog your material but often there are situations when listing areas of your online submissions are not what you want.

Yes it is very important. Specially for robots who all need to crawl your site.

Yes, it is important.

Your robot txt file is what tells the search engines which pages to accesses and index on your website on which pages not to for example if you specify in your robots txt file that you don't want the search engines to be able to accesses your thank you page.

Yes it is very important.

it is very important, i think so

Yes i agree with above it is important

The robots.txt file as instructions on where they are allowed to crawl (visit) and index (save) on the search engine results. Robots.txt files are useful: If you want search engines to ignore any duplicate pages on your website.