Register on the forum now to remove ALL ads + popups + get access to tons of hidden content for members only!

Go Back   Site Owners Forums - Webmaster Forums > General Arena > Social Networks

Notices


Reply
 
Thread Tools Rating: Thread Rating: 5 votes, 2.00 average. Display Modes
Old 07-06-2018, 11:50 PM   #1
riprook7
Registered User
 
Join Date: Nov 2017
Location: Oregon
Posts: 40
Robots.txt

What is Robots.txt file?
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
riprook7 is offline   Reply With Quote

Old 07-10-2018, 04:23 AM   #2
wellliving
Registered User
 
Join Date: Dec 2015
Location: USA
Posts: 797
Hi

The robots.txt file, also known as the robots exclusion protocol or standard, is a text file that tells web robots (most often search engines) which pages on your site to crawl. It also tells web robots which pages not to crawl. Let's say a search engine is about to visit a site.

https://www.welllivingshop.com/pillows/organic-cotton/
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
wellliving is offline   Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Robots.txt or Meta Robots.txt??? santoshkumar Search Engine Optimization 11 02-14-2017 04:03 AM
Robots.txt jaysh4922 Search Engine Optimization 13 07-28-2016 06:10 AM
Evil bots consuming all my sites traffic even after i disallowed them in robots.txt Fking General Discussion 1 12-13-2013 11:27 PM
How to edit virtual robots.txt file in wordpress? geniusoptimizer PHP / mySQL 1 04-12-2013 01:36 PM
Unable to delete robots.txt file notfake Yahoo 0 04-29-2012 03:19 AM


All times are GMT -7. The time now is 11:28 AM.


Powered by vBulletin Copyright © 2013 vBulletin Solutions, Inc.