Go Back   Site Owners Forums - Webmaster Forums > Internet Promotion > Web Promotion

Notices


Reply
 
Thread Tools Rate Thread Display Modes
Old 04-30-2018, 02:04 AM   #1
Shivam Tripathi
Registered User
 
Join Date: Feb 2018
Location: Pune
Posts: 64
robot.txt

Hey Friends,

Please anyone can tell me what is main purpose of robot.txt and why we use robot.txt in SEO ?

Because till now i am struggling for this question.so,please share with me asap..
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
Shivam Tripathi is offline   Reply With Quote

Old 04-30-2018, 04:08 AM   #2
pradeep01
Registered User
 
Join Date: Apr 2018
Posts: 11
The robots exclusion normal, additionally called the robots exclusion protocol or just robots.txt, may be a normal utilized by netsites to speak with net crawlers and different web robots. the quality specifies the way to inform the online automaton regarding that areas of the web site shouldn't be processed or scanned.
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
pradeep01 is offline   Reply With Quote
Old 04-30-2018, 04:29 AM   #3
drugrehabpoint
Registered User
 
Join Date: Sep 2017
Posts: 134
Thanks for sharing
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
drugrehabpoint is offline   Reply With Quote
Old 05-01-2018, 04:11 AM   #4
wellliving
Registered User
 
Join Date: Dec 2015
Location: USA
Posts: 744
Hi

Web site owners use the /robots.txt file to give instructions about their site to web robots; this is called The Robots Exclusion Protocol. The "User-agent: *" means this section applies to all robots. The "Disallow: /" tells the robot that it should not visit any pages on the site.

https://www.welllivingshop.com/baby/
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
wellliving is offline   Reply With Quote
Old 05-02-2018, 03:37 AM   #5
multai
Registered User
 
Join Date: Apr 2018
Posts: 59
It is the text file it gives the instructions to the robots about how to crawl the web page...

http://www.multaisolutions.com/ecommerce.html
multai is offline   Reply With Quote
Old 12-16-2018, 10:11 PM   #6
RH-Calvin
Registered User
 
Join Date: May 2015
Location: Forum
Posts: 2,517
Robots.txt is a text file that lists webpages which contain instructions for search engines robots. The file lists webpages that are allowed and disallowed from search engine crawling.
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
| $1 VPS Hosting

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
| Windows Server with Remote Desktop Access

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
| Servers with Free IPMI Setup
RH-Calvin is offline   Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
What is Robot.txt? kumar1_16 General Discussion 11 03-29-2017 03:15 AM
Robot.txt? siva SEO Search Engine Optimization 17 01-09-2015 07:02 PM
how can Create the Robot.txt pankaj Search Engine Optimization 3 10-08-2014 03:43 AM
how can create the Robot.txt file eagleeye dental Search Engine Optimization 0 09-01-2014 09:40 PM


All times are GMT -7. The time now is 04:21 AM.


Powered by vBulletin Copyright © 2020 vBulletin Solutions, Inc.