Register on the forum now to remove ALL ads + popups + get access to tons of hidden content for members only!

Go Back   Site Owners Forums - Webmaster Forums > Search Engine Optimization > Search Engine Optimization > Google

Notices


Reply
 
Thread Tools Rate Thread Display Modes
Old 02-23-2016, 07:32 AM   #1
Nexevo Technolo
Registered User
 
Join Date: Sep 2015
Location: Bangalore
Posts: 594
How to Create the Robots txt file.

How to Create the Robots txt file.
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
Nexevo Technolo is offline   Reply With Quote

Old 02-23-2016, 10:02 PM   #2
quantumleap
Registered User
 
Join Date: Jan 2016
Location: Hyderabad
Posts: 730
Check this link it might help you
http://tools.seobook.com/robots-txt/generator/
quantumleap is offline   Reply With Quote
Old 03-04-2016, 12:52 AM   #3
sudeepkhana
Registered User
 
Join Date: Jan 2016
Posts: 874
First you can use any webmaster tool on the internet to create it. Website owner uses the robot.txt file to give instruction about their to site to web robots.

All you need to create the file in the high-level directory to your web server.
Robot looks for the " example.com/robot.txt" file
robots.txt to exclude all robots from the entire server, add next.
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
sudeepkhana is offline   Reply With Quote
Old 11-05-2016, 03:47 AM   #4
marvinsam
Registered User
 
Join Date: Apr 2016
Posts: 568
Simply used webmaster tools on the internet to create a file
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
marvinsam is offline   Reply With Quote
Old 11-05-2016, 03:50 AM   #5
jamesvincent21
Registered User
 
Join Date: Sep 2016
Posts: 365
Steps for creating Robots.txt:
You must save your robots.txt code as a text file,
You must place the file in the highest-level directory of your site (or the root of your domain), and
The robots.txt file must be named robots.txt.
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
jamesvincent21 is offline   Reply With Quote
Old 11-06-2016, 11:14 PM   #6
Ajaysharma
Registered User
 
Join Date: Nov 2016
Posts: 412
1. Identify which directories and files on your web server you want to block from being rawled by the crawler
2. Identify whether or not you need to specify additional instructions for a particular search engine bot beyond a generic set of crawling directives
3. Use a text editor to create the robots.txt file and directives to block content
4. Optional: Add a reference to your sitemap file (if you have one)
5. Check for errors by validating your robots.txt file
6. Upload the robots.txt file to the root directory of your site
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
Ajaysharma is offline   Reply With Quote
Old 11-15-2016, 03:06 AM   #7
juliagrace
Registered User
 
Join Date: Nov 2016
Posts: 107
Use webmaster tool to generate robots.txt
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
juliagrace is offline   Reply With Quote
Old 11-28-2016, 12:15 AM   #8
jackparrow
Registered User
 
Join Date: Oct 2016
Posts: 150
A Web Crawler is a computer program that browses the World Wide Web in a methodical, automated manner or in an orderly fashion.
Other terms for Web crawlers are ants, automatic indexers, bots,Web...
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
jackparrow is offline   Reply With Quote
Old 12-29-2016, 03:58 AM   #9
jasonroy21
Registered User
 
Join Date: Jul 2016
Posts: 864
Steps for creating Robots.txt:
You must save your robots.txt code as a text file,
You must place the file in the highest-level directory of your site (or the root of your domain), and
The robots.txt file must be named robots.txt
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
jasonroy21 is offline   Reply With Quote
Old 01-05-2017, 04:09 AM   #10
marvinsam
Registered User
 
Join Date: Apr 2016
Posts: 568
Robot.txt file is an html code that used to inform the crawler that the file content some it can be created with web master tools
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
marvinsam is offline   Reply With Quote
Old 02-18-2017, 01:00 AM   #11
sharmaroshni012
Registered User
 
Join Date: Nov 2016
Posts: 510
Robot.txt file is an HTML code that used to inform the crawler that the file content some file that should not shows in search results
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
sharmaroshni012 is offline   Reply With Quote
Old 02-18-2017, 01:33 AM   #12
jenerou
Registered User
 
Join Date: Jan 2017
Posts: 56
1.You must save your robots.txt code as a text file,
2.You must place the file in the highest-level directory of your site (or the root of your domain)
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
jenerou is offline   Reply With Quote
Old 03-20-2017, 01:18 AM   #13
AshokDixit89
Registered User
 
Join Date: Jan 2016
Location: Mumbai, India
Posts: 1,161
Tips to create robot .txt file
Create the file in the high-level directory to your web server.
Robot looks for the " example.com/robot.txt" file
robots.txt to exclude all robots from the entire server, add next
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
AshokDixit89 is offline   Reply With Quote
Old 04-03-2017, 04:18 AM   #14
v.shukla
Registered User
 
Join Date: Jul 2016
Posts: 163
Steps for creating Robots.txt:
- save your robots.txt code as a text file,
- place the file in the highest-level directory of your site (or the root of your domain), and
- The robots.txt file must be named robots.txt.
__________________

  1. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
v.shukla is offline   Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
How to add the robots txt file in weebly blog Nexevo Technolo Search Engine Optimization 4 10-24-2017 11:04 PM
Check the website robots file china-jewels Bing 4 10-11-2015 09:14 AM
How to Create Robots file? hookahb2c Search Engine Optimization 1 07-10-2015 12:42 AM
How to create the Sitemap file manually? sahithya Search Engine Optimization 4 11-26-2014 06:23 AM
How to create user friendly URL in FTP file? sahithya Search Engine Optimization 2 08-08-2014 04:14 AM


All times are GMT -7. The time now is 05:01 PM.


Powered by vBulletin Copyright © 2013 vBulletin Solutions, Inc.