Go Back   Site Owners Forums - Webmaster Forums > Search Engine Optimization > Search Engine Optimization > Yahoo

Notices


Reply
 
Thread Tools Rate Thread Display Modes
Old 01-08-2012, 02:06 AM   #1
billmat
Registered User
 
Join Date: Jan 2012
Posts: 17
How to slow the crawl rate of yahoo?

Hello members,

How do I slow the crawl rate of yahoo?
its killing my bandwidth 700 bots a day and putting extra strain on my server.

Please tell me How to do this?
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
billmat is offline   Reply With Quote

Old 02-05-2017, 02:52 AM   #2
Avinash kumar
Registered User
 
Join Date: Feb 2017
Posts: 149
Slurp is the Yahoo search bot for the crawl and index your web page.
It obeys the Robot Exclusion standard. If you don not want to crawl frequently your web page, create a robots.txt file in the root directory (Home folder of site and add a rule for "User-agent: Slurp"

Code example in a robot.txt file:

User-agent: Slurp<br>disallow:/cgi-bin/
Avinash kumar is offline   Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Yahoo Questions and Answers jaxondavis37 Yahoo 11 05-22-2017 09:16 PM
Yahoo optimization – keywords by relevance loganallen48 Yahoo 2 10-09-2014 11:57 PM
New mobile service by Yahoo! gavindmello053 Yahoo 3 01-02-2012 10:14 PM
How can earn money through Yahoo! haroldmacmillan Yahoo 3 12-08-2011 05:46 PM
Optimizing Yahoo andysmith476 Yahoo 0 11-16-2011 02:04 AM


All times are GMT -7. The time now is 02:10 PM.


Powered by vBulletin Copyright © 2020 vBulletin Solutions, Inc.