Go Back   Site Owners Forums - Webmaster Forums > Search Engine Optimization > Search Engine Optimization

Notices


Reply
 
Thread Tools Rate Thread Display Modes
Old 01-02-2016, 03:16 AM   #1
JeanMackMicky
Registered User
 
Join Date: Feb 2013
Location: Kanpur
Posts: 29
Send a message via Yahoo to JeanMackMicky
How to add robots.txt?

Hello all, how to add robots.txt to my website http://www.enerco.co.uk
Is it necessary to add robots.txt in my website?
Please suggest me, thanks
JeanMackMicky is offline   Reply With Quote

Old 01-02-2016, 10:23 AM   #2
ShivJaiswaL
Registered User
 
Join Date: Feb 2012
Posts: 94
robots.txt file instructs spiders about blocked folders of website
ShivJaiswaL is offline   Reply With Quote
Old 01-04-2016, 12:10 AM   #3
Rajan007
Registered User
 
Join Date: Sep 2015
Posts: 75
Robots.txt file contains set of instruction, which tells google what to crawl on that particular site.

sample code

User Agent: *
Allow: /

The above code give permission to robots to crawl the entire website.
write this code in a notepad and save it as robots.txt and upload it in your server.
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.



To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
Rajan007 is offline   Reply With Quote
Old 01-04-2016, 12:47 AM   #4
harish9
Registered User
 
Join Date: Dec 2015
Posts: 15
robots.txt is very important in seo to give instructions about the website for the Google.

Code:
User-agent: *
Disallow: /
harish9 is offline   Reply With Quote
Old 01-04-2016, 12:54 AM   #5
heoabcd
Registered User
 
Join Date: Dec 2015
Posts: 83
i think should update go host @@
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
heoabcd is offline   Reply With Quote
Old 01-04-2016, 02:10 AM   #6
ipack
Registered User
 
Join Date: Oct 2015
Posts: 116
Quote:
Originally Posted by harish9 View Post
robots.txt is very important in seo to give instructions about the website for the Google.

Code:
User-agent: *
Disallow: /
Hi, I'm also looking for this information so thanks for sharing it.
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
ipack is offline   Reply With Quote
Old 01-04-2016, 06:53 AM   #7
Nexevo Technolo
Registered User
 
Join Date: Sep 2015
Location: Bangalore
Posts: 548
robots.txt is very important in seo to give instructions about the website for the Google.
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
Nexevo Technolo is offline   Reply With Quote
Old 01-04-2016, 12:02 PM   #8
bellacooper
Registered User
 
Join Date: Jan 2016
Posts: 2
Save the robots.txt code as a text file. Place the file in the highest-level directory of your site or the root of your domain.
__________________
Hire a professional
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
.
bellacooper is offline   Reply With Quote
Old 01-04-2016, 06:39 PM   #9
tungkolia
Registered User
 
Join Date: Dec 2015
Posts: 10
You can add code to world with file robots.txt:
Quote:
User Agent: *
Allow: /
and you can sign in webmastertool add code:
Quote:
User Agent: *
Allow: /
disallow:/folder =>you want banned robots index
__________________
Chia sẻ bí quyết điều trị bệnh
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
ở phụ nữ đang mang thai
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
để ngủ được, địa chỉ
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
an toàn hiệu quả
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
là gì và
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

tốt nhất giúp bạn có thể điều trị được
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
hiệu quả, giúp lưu thông máu tốt
tungkolia is offline   Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Crawlers, spider & robots.txt? jackthomas087 Search Engine Optimization 0 04-22-2014 03:29 AM
Evil bots consuming all my sites traffic even after i disallowed them in robots.txt Fking General Discussion 1 12-13-2013 11:27 PM
How to edit virtual robots.txt file in wordpress? geniusoptimizer PHP / mySQL 1 04-12-2013 12:36 PM
The robots.txt file rubberfender Yahoo 0 12-02-2012 07:32 PM
Unable to delete robots.txt file notfake Yahoo 0 04-29-2012 02:19 AM


All times are GMT -7. The time now is 03:04 PM.


Powered by vBulletin Copyright © 2020 vBulletin Solutions, Inc.