Register on the forum now to remove ALL ads + popups + get access to tons of hidden content for members only!

Go Back   Site Owners Forums - Webmaster Forums > Search Engine Optimization > Search Engine Optimization

Notices


Reply
 
Thread Tools Rate Thread Display Modes
Old 11-29-2017, 12:20 AM   #31
scrapcarsuk
Registered User
 
Join Date: Jun 2017
Posts: 421
Web site owners use the /robots.txt file to give instructions about their site to web robots; this is called The Robots Exclusion Protocol. The "User-agent: *" means this section applies to all robots. The "Disallow: /" tells the robot that it should not visit any pages on the site.
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
scrapcarsuk is offline   Reply With Quote
Old 11-29-2017, 12:20 AM   #32
scrapcarsuk1234
Registered User
 
Join Date: Jun 2017
Posts: 353
Web site owners use the /robots.txt file to give instructions about their site to web robots; this is called The Robots Exclusion Protocol. The "User-agent: *" means this section applies to all robots. The "Disallow: /" tells the robot that it should not visit any pages on the site.
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
| immigration solicitor in birmingham|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
scrapcarsuk1234 is offline   Reply With Quote
Old 12-09-2017, 11:34 PM   #33
cuong25081995
Registered User
 
Join Date: Nov 2017
Posts: 500
Web site owners use the /robots.txt file to give instructions about their site to web robots; this is called The Robots Exclusion Protocol. The "User-agent: *" means this section applies to all robots. The "Disallow: /" tells the robot that it should not visit any pages on the site.
cuong25081995 is offline   Reply With Quote
Old 12-10-2017, 11:15 PM   #34
leenahahim
Registered User
 
Join Date: Feb 2016
Posts: 227
robots.txt is text file it helps to block all web crawler from all pages or specified web crawler from specific web page.
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
leenahahim is offline   Reply With Quote
Old 12-10-2017, 11:16 PM   #35
rajapriya
Registered User
 
Join Date: Nov 2017
Posts: 54
Web site owners use the /robots.txt file to give instructions about their site to web robots; this is called The Robots Exclusion Protocol. The "User-agent: *" means this section applies to all robots. The "Disallow: /" tells the robot that it should not visit any pages on the site.
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
rajapriya is offline   Reply With Quote
Old 02-12-2018, 01:59 AM   #36
Jessianut
Registered User
 
Join Date: Jul 2017
Posts: 443
Web site owners use the /robots.txt file to give instructions about their site to web robots; this is called The Robots Exclusion Protocol. ... The "Disallow: /" tells the robot that it should not visit any pages on the site.
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
Jessianut is offline   Reply With Quote
Old 02-13-2018, 08:29 AM   #37
ROYCPO
Registered User
 
Join Date: Dec 2015
Location: Bangladesh
Posts: 359
Robots.txt is a text file that is inserted into your website to contain instructions for search engine spiders. The file lists web pages that are allowed and disallowed from search engine crawling.
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
Outsource
,
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
providing
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
services all over the world. Expert in
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
Service.
ROYCPO is offline   Reply With Quote
Old 02-14-2018, 02:10 AM   #38
k2bseo
Registered User
 
Join Date: Dec 2017
Posts: 76
Whenever we need certain pages not to be crawled by search engine, we can block those pages in robots file
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
k2bseo is offline   Reply With Quote
Old 02-14-2018, 04:04 AM   #39
triarch
Registered User
 
Join Date: Feb 2018
Posts: 21
Robots.txt is a text file webmasters create to instruct web robots (typically search engine robots) how to crawl pages on their website.
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
triarch is offline   Reply With Quote
Old 02-14-2018, 04:09 AM   #40
Elizabethezhil
Registered User
 
Join Date: Aug 2017
Posts: 341
Website owners use the /robots.txt file to give instructions about their site to web robots; this is called The Robots Exclusion Protocol.
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
Elizabethezhil is offline   Reply With Quote
Old 02-14-2018, 04:54 AM   #41
nathanleo
Registered User
 
Join Date: Jul 2017
Posts: 344
Robots.txt is a text (not html) file you put on your site to tell search robots which pages you would like them not to visit.
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
nathanleo is offline   Reply With Quote
Old 02-14-2018, 05:11 AM   #42
Navjot Singh
Registered User
 
Join Date: Aug 2017
Posts: 104
The purpose of a robots.txt file, also known as the robots exclusion protocol, is to give webmasters control over what pages robots can crawl and index on their site.
__________________
We are one of the leading
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
with NABL accredited & ISO certified lab.
Navjot Singh is offline   Reply With Quote
Old 02-14-2018, 05:17 AM   #43
irsgroup
Registered User
 
Join Date: Dec 2017
Posts: 20
Robots.txt is a text file webmasters create to instruct web robots how to crawl pages on their website.
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
irsgroup is offline   Reply With Quote
Old 02-14-2018, 05:21 AM   #44
Adya
Registered User
 
Join Date: Jul 2017
Posts: 504
Robot.txt is the file included in the site which indicate Crawler about which pages of the site will be crawled and which not.
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
Adya is offline   Reply With Quote
Old 02-15-2018, 12:28 AM   #45
lilianalilly
Registered User
 
Join Date: Apr 2017
Posts: 578
Robots.txt is a text (not html) file you put on your site to tell search robots which pages you would like them not to visit.
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
lilianalilly is offline   Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
What is the purpose of robots.txt file? sahithya Search Engine Optimization 12 Yesterday 08:51 AM
What is the purpose of using robots.txt file in seo? SunFineHanger Search Engine Optimization 13 Yesterday 08:44 AM
Apple: a new file system encrypted by nature bolhaw General Discussion 0 06-17-2016 08:25 AM
How to edit virtual robots.txt file in wordpress? geniusoptimizer PHP / mySQL 1 04-12-2013 01:36 PM
Unable to delete robots.txt file notfake Yahoo 0 04-29-2012 03:19 AM


All times are GMT -7. The time now is 02:59 AM.


Powered by vBulletin Copyright © 2013 vBulletin Solutions, Inc.