How to create Robots.txt file for my website - Site Owners Forums

rahul47 · 07-25-2016, 06:04 PM

hi,
How to create Robots.txt file for my website

sadia mahfud · 07-26-2016, 02:48 AM

there are many tools available to generate Robot.txt ....just search on google search engine ( create Robot.txt) the go to any robot.txt creating sites which come on search engine ,type your website domain and follow further step....your file of Robot.txt will generate for your website

Edwardgeorge · 07-26-2016, 02:54 AM

Use a text editor to create the robots.txt file and add REP directives to block content from being visited by the bots. The text file should be saved in ASCII or UTF-8 encoding. Create a list of Disallow directives listing the content you want blocked.

marryjoe · 07-26-2016, 04:14 AM

Google have tool to put your website that tool

SEONinja · 07-26-2016, 05:06 AM

A Simple way to create robots.txt file. You can also create robot.txt file for own website to follow some simple steps:
1. Open Notepad/Text Pad or other editor tools.
2. Write these code
User-agent: *
Disallow:
Allow:
Sitemap: your URL/sitemap.xml
3. Save this file "robots.txt
Your robots.txt file ready, use it.

Thanks

Nexevo Technolo · 07-26-2016, 05:26 AM

Dont use the any tools you should create the robots text manually. If you want learn about robots text click this link -- www.robotstxt.org/

PixelW3Tech · 07-26-2016, 08:31 AM

The syntax for using the keywords is as follows:

User-agent: * [the name of the robot the following rule applies to]

Disallow: [the URL path you want to block]

Allow: [the URL path in of a subdirectory, within a blocked parent directory, that you want to unblock]

jordanangel · 11-21-2016, 12:18 AM

Use a text editor to create the robots.txt file and add REP directives to block content from being visited by the bots. The text file should be saved in ASCII or UTF-8 encoding. Create a list of Disallow directives listing the content you want blocked.

07-25-2016, 06:04 PM	#1
rahul47 Registered User Join Date: May 2016 Posts: 22	How to create Robots.txt file for my website hi, How to create Robots.txt file for my website __________________ Rahul To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. - a To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

07-26-2016, 02:54 AM	#3
Edwardgeorge Registered User Join Date: Jun 2016 Location: London Posts: 78	Use a text editor to create the robots.txt file and add REP directives to block content from being visited by the bots. The text file should be saved in ASCII or UTF-8 encoding. Create a list of Disallow directives listing the content you want blocked. __________________ To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

07-26-2016, 04:14 AM	#4
marryjoe Registered User Join Date: Jul 2016 Posts: 113	Google have tool to put your website that tool __________________ To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

07-26-2016, 05:06 AM	#5
SEONinja Registered User Join Date: May 2016 Location: Florida, United State Posts: 36	A Simple way to create robots.txt file. You can also create robot.txt file for own website to follow some simple steps: 1. Open Notepad/Text Pad or other editor tools. 2. Write these code User-agent: * Disallow: Allow: Sitemap: your URL/sitemap.xml 3. Save this file "robots.txt Your robots.txt file ready, use it. Thanks __________________ To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

07-26-2016, 05:26 AM	#6
Nexevo Technolo Registered User Join Date: Sep 2015 Location: Bangalore Posts: 548	Dont use the any tools you should create the robots text manually. If you want learn about robots text click this link -- www.robotstxt.org/ __________________ To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

07-26-2016, 02:48 AM	#2
sadia mahfud Registered User Join Date: Jul 2016 Posts: 11	there are many tools available to generate Robot.txt ....just search on google search engine ( create Robot.txt) the go to any robot.txt creating sites which come on search engine ,type your website domain and follow further step....your file of Robot.txt will generate for your website

07-26-2016, 08:31 AM	#7
PixelW3Tech Registered User Join Date: May 2016 Location: Hyderabad, India Posts: 15	The syntax for using the keywords is as follows: User-agent: * [the name of the robot the following rule applies to] Disallow: [the URL path you want to block] Allow: [the URL path in of a subdirectory, within a blocked parent directory, that you want to unblock] __________________ To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

11-21-2016, 12:18 AM	#8
jordanangel Registered User Join Date: Aug 2016 Posts: 387	Use a text editor to create the robots.txt file and add REP directives to block content from being visited by the bots. The text file should be saved in ASCII or UTF-8 encoding. Create a list of Disallow directives listing the content you want blocked. __________________ To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. *To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.* *To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.*

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
How to find dofollow website to create backlink	James12smith	Search Engine Optimization	6	04-05-2017 02:18 AM
What is a robots.txt file?	orientation	Search Engine Optimization	8	05-26-2016 11:14 PM
How to Create the Dynamic Static Website?	jamesranatte	Search Engine Optimization	0	06-25-2012 04:02 PM