robots.txt - Site Owners Forums

anuverma1 · 06-03-2014, 05:21 AM

why we use robots.txt file??

geneliabravo · 06-03-2014, 05:47 AM

Content of this file tells search engine do not crawl that specific area and don't want indexed by any search engine.

ashiselma · 06-03-2014, 10:13 AM

Robots.txt file tells search engine crawlers how they should visit your site. If there are files and directories you do not want indexed by search engines, you can use a robots.txt file to define where the robots should not go. These files are very simple text files that are placed on your web server.

ramgopal · 06-04-2014, 04:59 AM

Hello,
Really this is nice answer and i am satisfied with you but one more question is find in my mind that, I want to crawled page on search engine only some time.
Please give me reply thanks.

irisphotobooth · 06-05-2014, 12:50 AM

to permit google crawl to spesific page...

anuverma1 · 06-05-2014, 03:21 AM

thanks for sharing your view, and make clear to me use of this file.

goldiehearn · 06-05-2014, 05:31 AM

Robots.txt is a text (not html) file you put on your site to tell search robots which pages you would like them not to visit. Robots.txt is by no means mandatory for search engines but generally search engines obey what they are asked not to do. It is important to clarify that robots.txt is not a way from preventing search engines from crawling your site .

QC34020Xo9 · 06-05-2014, 08:29 AM

Nhận tăng like SLL cho các Shop, Cửa Hàng, Doanh Nghiệp, các Fanpage đã có sẵn

Nhận tăng like SLL cho các Shop, Cửa Hàng, Doanh Nghiệp, các Fanpage đã có sẵn....

Nhận Tăng theo số lượng yêu cầu ( Uy Tín - Trách Nhiệm )

Giá cả phải chăng cạnh tranh với thị trường

Nhận Từ 5.000 Like Đến 200.000 Like

Đảm bảo 100% like là like thật, tự nguyện và like chất lượng, người like từ độ tuổi 15>>40 Tuổi

Mọi Chi tiết Thắc Mắc Liên Hệ

ĐT: 0937 300 O81

Đối với doanh nghiệp, cửa hàng,... Fanpage là nơi giao lưu, tương tác giữa doanh nghiệp và khách hàng (gọi là fan) hoặc các sự kiện (event) được doanh nghiệp đưa lên Fanpage để Fan tham gia… Mọi thông tin đưa lên Fanpage sẽ được đăng lên Tường(Wall) của các trang cá nhân của Fan, từ Wall của Fans bạn bè của Fans cũng có thể thấy được thông tin, qua đó thông tin cũng như hình ảnh công ty được lan truyền.

Hãy nhanh chóng tạo dựng 1 trang Facebook FanPage. Đó là 1 mạng xã hội tốt nhất tại thời điểm này,việc duy trì 1 trang FanPage giúp cho công việc kinh doanh và thương hiệu dễ được quảng bá tới nhiều khách hàng biết đến.

06-03-2014, 05:21 AM	#1
anuverma1 Registered User Join Date: Feb 2014 Posts: 541	robots.txt why we use robots.txt file?? __________________ To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

06-03-2014, 05:47 AM	#2
geneliabravo Registered User Join Date: May 2014 Posts: 189	Content of this file tells search engine do not crawl that specific area and don't want indexed by any search engine. __________________ To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

06-05-2014, 12:50 AM	#5
irisphotobooth Registered User Join Date: Jun 2014 Posts: 8	to permit google crawl to spesific page... __________________ Jasa To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. melayani To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

06-05-2014, 03:21 AM	#6
anuverma1 Registered User Join Date: Feb 2014 Posts: 541	thanks for sharing your view, and make clear to me use of this file. __________________ To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

06-05-2014, 05:31 AM	#7
goldiehearn Registered User Join Date: May 2014 Posts: 26	Robots.txt is a text (not html) file you put on your site to tell search robots which pages you would like them not to visit. Robots.txt is by no means mandatory for search engines but generally search engines obey what they are asked not to do. It is important to clarify that robots.txt is not a way from preventing search engines from crawling your site . __________________ To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

06-03-2014, 10:13 AM	#3
ashiselma Registered User Join Date: Apr 2014 Location: United States Posts: 55	Robots.txt file tells search engine crawlers how they should visit your site. If there are files and directories you do not want indexed by search engines, you can use a robots.txt file to define where the robots should not go. These files are very simple text files that are placed on your web server.

06-04-2014, 04:59 AM	#4
ramgopal Registered User Join Date: Jun 2014 Posts: 12	Hello, Really this is nice answer and i am satisfied with you but one more question is find in my mind that, I want to crawled page on search engine only some time. Please give me reply thanks.

06-05-2014, 08:29 AM	#8
QC34020Xo9 Registered User Join Date: May 2014 Posts: 41	Nhận tăng like SLL cho các Shop, Cửa Hàng, Doanh Nghiệp, các Fanpage đã có sẵn Nhận tăng like SLL cho các Shop, Cửa Hàng, Doanh Nghiệp, các Fanpage đã có sẵn.... Nhận Tăng theo số lượng yêu cầu ( Uy Tín - Trách Nhiệm ) Giá cả phải chăng cạnh tranh với thị trường Nhận Từ 5.000 Like Đến 200.000 Like Đảm bảo 100% like là like thật, tự nguyện và like chất lượng, người like từ độ tuổi 15>>40 Tuổi Mọi Chi tiết Thắc Mắc Liên Hệ ĐT: 0937 300 O81 Đối với doanh nghiệp, cửa hàng,... Fanpage là nơi giao lưu, tương tác giữa doanh nghiệp và khách hàng (gọi là fan) hoặc các sự kiện (event) được doanh nghiệp đưa lên Fanpage để Fan tham gia… Mọi thông tin đưa lên Fanpage sẽ được đăng lên Tường(Wall) của các trang cá nhân của Fan, từ Wall của Fans bạn bè của Fans cũng có thể thấy được thông tin, qua đó thông tin cũng như hình ảnh công ty được lan truyền. Hãy nhanh chóng tạo dựng 1 trang Facebook FanPage. Đó là 1 mạng xã hội tốt nhất tại thời điểm này,việc duy trì 1 trang FanPage giúp cho công việc kinh doanh và thương hiệu dễ được quảng bá tới nhiều khách hàng biết đến.

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Crawlers, spider & robots.txt?	jackthomas087	Search Engine Optimization	0	04-22-2014 03:29 AM
Evil bots consuming all my sites traffic even after i disallowed them in robots.txt	Fking	General Discussion	1	12-13-2013 11:27 PM
How to edit virtual robots.txt file in wordpress?	geniusoptimizer	PHP / mySQL	1	04-12-2013 12:36 PM
The robots.txt file	rubberfender	Yahoo	0	12-02-2012 07:32 PM
Unable to delete robots.txt file	notfake	Yahoo	0	04-29-2012 02:19 AM