|
One of the more common uses of Crawlbot and our article extraction API: monitoring news sites to identify the latest articles, and then extracting clean article text (and all other data) automatically. In this post we’ll discuss the most straightforward way to do that. Which sometimes leads to crawler to be not crawling in particular articles or blogs for which we follow the way of blog pinging. Which helps the crawler to get the the location signal from this blog pingers.
|