Which pages should be put in robot.txt file? - Site Owners Forums

Martinricky · 07-24-2016, 10:16 PM

Hello friends,

I have a new website in most of the web pages have no content so I am confused how many web pages URL should I put in robot.txt file?

buy pin likes · 07-24-2016, 11:15 PM

Web site owners use the /robots.txt file to give instructions about their site to web robots; this is called The Robots Exclusion Protocol.

It works likes this: a robot wants to vists a Web site URL

07-24-2016, 10:16 PM	#1
Martinricky Registered User Join Date: Aug 2015 Posts: 434	Which pages should be put in robot.txt file? Hello friends, I have a new website in most of the web pages have no content so I am confused how many web pages URL should I put in robot.txt file? __________________ To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

07-24-2016, 11:15 PM	#2
buy pin likes Registered User Join Date: Jul 2016 Posts: 80	Web site owners use the /robots.txt file to give instructions about their site to web robots; this is called The Robots Exclusion Protocol. It works likes this: a robot wants to vists a Web site URL __________________ *To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.* \|\| *To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.* \|\| *To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.* \|\| *To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.* *To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.* *To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.*

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
What is robot.txt file and what are the benefits of using it..?	AshokDixit89	Search Engine Optimization	8	05-05-2017 04:30 AM
What is robot.txt file and what are the benefits of using it?	Martinricky	Social Networks	31	12-21-2016 02:46 AM
how can create the Robot.txt file	eagleeye dental	Search Engine Optimization	0	09-01-2014 09:40 PM
RSS Feed	rajivwebads	Search Engine Optimization	12	08-31-2012 07:15 AM

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)