Site Owners Forums - Webmaster Forums - Which pages should be put in robot.txt file?

Site Owners Forums - Webmaster Forums (http://siteownersforums.com/index.php)

- Search Engine Optimization (http://siteownersforums.com/forumdisplay.php?f=16)

- - Which pages should be put in robot.txt file? (http://siteownersforums.com/showthread.php?t=174939)

Martinricky

07-24-2016 10:16 PM

Which pages should be put in robot.txt file?

Hello friends,

I have a new website in most of the web pages have no content so I am confused how many web pages URL should I put in robot.txt file?

buy pin likes

07-24-2016 11:15 PM

Web site owners use the /robots.txt file to give instructions about their site to web robots; this is called The Robots Exclusion Protocol.

It works likes this: a robot wants to vists a Web site URL

All times are GMT -7. The time now is 04:42 PM.