what is robot.txt in blogger..? - Site Owners Forums

sonamsharma · 04-02-2019, 11:23 PM

Hello friends,

what is robot.txt in blogger..?

vinukum · 04-05-2019, 12:03 AM

Robots.txt is a text file webmasters create to instruct web robots how to crawl pages on their website. The robots.txt file is part of the the robots exclusion protocol (REP), a group of web standards that regulate how robots crawl the web, access and index content, and serve that content up to users.
Basic format of the robot.txt file.
User-agent: [user-agent name]
Disallow: [URL string not to be crawled]

albertocosta · 04-05-2019, 02:25 AM

Creating a robots.txt file to position a blog is a simple task and if you thought that this file will not help you at all you are wrong. Since this file is very important for search engines such as Google search or yahoo search, since to be able to index your website and that it can be shown in the search results the Robots of these companies look in your file Robots.txt for the different established parameters where we will place which parts of our website are available to the public or not, plus we will give you the exact location of our site map.

Sojan Babu · 04-05-2019, 03:44 AM

Robots.txt is a text file webmasters create to instruct web robots (typically search engine robots) how to crawl pages on their website. The robots.txt file is part of the robots exclusion protocol (REP), a group of web standards that regulate how robots crawl the web, access and index content, and serve that content up to users.

Allowing all web crawlers access to all content

User-agent: *
Disallow:

Using this syntax in a robots.txt file tells web crawlers to crawl all pages on www.example.com, including the homepage.

rajkiran · 04-08-2019, 01:34 AM

Robots.txt is a text file on the server in that you can write which directories and web pages or links should not be included for search results. It means you can restrict search engine bots to crawl some directories and web pages or links of your website or blog. Now custom robots.txt is available for Blogger.

RH-Calvin · 04-08-2019, 01:55 AM

Robots.txt is a text file that lists webpages which contain instructions for search engines robots. The file lists webpages that are allowed and disallowed from search engine crawling.

sonamg · 04-09-2019, 02:52 AM

Robots.txt it’s a text file, it's instructed to the search engine which pages on your site to crawl.

04-02-2019, 11:23 PM	#1
sonamsharma Registered User Join Date: Jul 2013 Posts: 373	what is robot.txt in blogger..? Hello friends, what is robot.txt in blogger..? __________________ To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

04-05-2019, 12:03 AM	#2
vinukum Registered User Join Date: Feb 2018 Location: Bangalore Posts: 283	Robots.txt is a text file webmasters create to instruct web robots how to crawl pages on their website. The robots.txt file is part of the the robots exclusion protocol (REP), a group of web standards that regulate how robots crawl the web, access and index content, and serve that content up to users. Basic format of the robot.txt file. User-agent: [user-agent name] Disallow: [URL string not to be crawled] __________________ To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

04-05-2019, 02:25 AM	#3
albertocosta Registered User Join Date: Apr 2016 Posts: 83	Creating a robots.txt file to position a blog is a simple task and if you thought that this file will not help you at all you are wrong. Since this file is very important for search engines such as Google search or yahoo search, since to be able to index your website and that it can be shown in the search results the Robots of these companies look in your file Robots.txt for the different established parameters where we will place which parts of our website are available to the public or not, plus we will give you the exact location of our site map. __________________ *To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.* , *To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.* , *To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.* , *To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.* , *To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.* , *To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.* , *To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.*

04-05-2019, 03:44 AM	#4
Sojan Babu Registered User Join Date: May 2017 Posts: 245	Robots.txt is a text file webmasters create to instruct web robots (typically search engine robots) how to crawl pages on their website. The robots.txt file is part of the robots exclusion protocol (REP), a group of web standards that regulate how robots crawl the web, access and index content, and serve that content up to users. Allowing all web crawlers access to all content User-agent: * Disallow: Using this syntax in a robots.txt file tells web crawlers to crawl all pages on www.example.com, including the homepage. __________________ To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

04-08-2019, 01:55 AM	#6
RH-Calvin Registered User Join Date: May 2015 Location: Forum Posts: 2,517	Robots.txt is a text file that lists webpages which contain instructions for search engines robots. The file lists webpages that are allowed and disallowed from search engine crawling. __________________ █ To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| $1 VPS Hosting █ To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| Windows Server with Remote Desktop Access █ To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| Servers with Free IPMI Setup

04-08-2019, 01:34 AM	#5
rajkiran Registered User Join Date: Aug 2016 Posts: 961	Robots.txt is a text file on the server in that you can write which directories and web pages or links should not be included for search results. It means you can restrict search engine bots to crawl some directories and web pages or links of your website or blog. Now custom robots.txt is available for Blogger.

04-09-2019, 02:52 AM	#7
sonamg Registered User Join Date: Jun 2018 Location: India Posts: 59	Robots.txt it’s a text file, it's instructed to the search engine which pages on your site to crawl. __________________ To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
robot.txt	Shivam Tripathi	Web Promotion	5	12-16-2018 10:11 PM
Brandon Brice is top rated rated Social Media Blogger	deepakrajput	Post your ad here	0	04-24-2015 04:48 AM
how to eligible for adsense in blogger	ifte111	Making money on the web	2	11-01-2014 08:35 AM
Per post keywords in Blogger	Shamatae	Search Engine Optimization	3	10-10-2014 01:42 AM
Blogger Sitemap through Google Webmaster Tools	Shamatae	Google	1	12-06-2013 01:00 AM