How to Filter Millions of Articles Daily to Curate the Content you Need


It is often said there is no such thing as content overload but filter failure. There is certainly a lot of content. There are millions of articles published every day and the volume of content is growing. Google indexed 1 trillion articles in 2007, today it has indexed over 30 trillion. Many newspapers now publish over 1,000 articles a day. The challenge we all face is to find effective ways of filtering this content.

I explore below how you can use the free Anders Pink content curation tool to filter millions of articles every day to curate just the content you need. So you can stay smarter and updated in less time.

The Limitations of RSS feeds and Social Networks as Filters

It is not practical to visit tens of websites each morning to stay updated and like many people used an RSS reader to pull together feeds from different sites. However, there are limitations to RSS feeds. These are:

  • many sites no longer offer RSS feeds
  • you only see a narrow selection of content ie only from the feeds or sources you select
  • the publisher decides what is included in the feed so you may get promotional releases as well as articles

Social networks have become an alternative means of content discovery, and many of us now discover new articles from what our connections are sharing on LinkedIn, Twitter and Facebook. These are helpful but the usefulness depends upon the quality of your network, how much they search and share. There is also a danger of echo chambers on social, where the same content and same ideas recirculate.

Using Anders Pink’s Free Content Curation Tool

Our Anders Pink daily content curation tool is a new and efficient way to stay updated. In essence you set your content curation rules, the content you want to see, and we crawl millions of articles each day and filter out the ones that match your requirements. There are a number of filtering options.

1. Filter All Sites by Keywords

One of the restrictions of RSS feeds is they only provide content from specific sources. With Anders Pink you can specify keywords and we will crawl all the articles published each day and bring back those where the title or the topic matches your keywords. At a simple level this could be all articles on elearning or elearning and social learning.

There is a danger that your keywords will bring back a lot of content as we crawl millions of articles daily. Thus you can add ‘must include’ words to filter your results further. For example, you might be interested in trends so you can add ‘must include’ words such as trends, forecast, etc. This will refine your search to only articles that match your keywords plus these additional keywords. For example, ‘elearning’ or ‘social learning’ plus ‘trends’ or ‘forecast’.

You can also exclude words and domains. For example, don’t show me any articles with certain keywords in the title. You can also exclude domains, for example don’t show me any content from YouTube or Tumblr or any specific domains.

The beauty of using keywords is there is no source restriction as you get with RSS feeds. Thus you can pick up articles that match your keywords from anywhere as they are published each day.


2. Filter by Domains and Keywords

You may want to track content each day published by your clients or your competitors. With Anders Pink you can simply enter their domains and we will bring back for you the content they publish each day. You no longer need to visit their sites to check. You can add multiple domains to a single briefing.

You can also further filter this content with keywords, for example only show me articles they publish about a specific topic. You can use must include words and exclude words as you would do with keyword searches.

You can also use this approach to see articles that match your keyword from a specific set of domains. For example, the latest articles on leadership from Harvard and McKinsey or the latest articles on a specific topic from say 10 major news sites.


3. Filter what Experts are Sharing

Another way you can filter and curate content is by seeing what experts are sharing on Twitter. Just enter their Twitter handles or names and Anders Pink will filter and show you all the latest articles they are sharing. You can enter multiple Twitter handles for say a group of ten elearning experts.

Again you can also further filter using keywords. So you can only see say articles they are sharing on a particular set of topics.


4. Use RSS feeds and Keywords

You can also use RSS feeds as filters in Anders Pink. You can import these directly from your existing RSS reader such as Feedly and we will display these articles in a single briefing for you or multiple briefings.

With Anders Pink you can also filter your RSS feeds so you only see articles that match a specific set of keywords.


5. Combine All of the Above Filters to Curate Content

You can curate content by using a mix of all of the above filters. For example you can filter articles about competitors using keywords, articles published by their domains, articles they share via Twitter and any RSS feeds they have. You can also filter all of these by specific keywords including must include words and exclude words. Anders Pink’s free curation tool allows you to create sophisticated and fine grained rules.

Add a Human Curation Filter

The automatically curated briefings above are a great way to streamline your content curation. However, you can also add a human layer. You can do this is a number of ways.

You can review the briefing content each day and save the most relevant articles to a saved board. You can share this board with your team or publicly.

Your team can also upvote, save and comment on articles in the briefings. You can then sort by most upvoted content for example. Your team can also flag any must read content they come across.

These human curation filters add significantly more value as your team is likely to know what content is particularly relevant.

Share and Display Your Curated Content Anywhere

You can share the briefings you create on Anders Pink simply by ticking the option to make them public and then sharing the url.

You can also use the Anders Pink API to display your curated content wherever your users are.

For example, we have built a plugin for the popular open source learning platform Moodle using the Anders Pink API. This makes it easy to display a briefing or a saved content curation in any course or on any page of the site. You can display the content as you want, we have provided three different example layouts as you can see below.


You can see more on how our Moodle plugin works in the video below.

You can build your widgets to display your Anders Pink briefings on any platform using our API.  See more information on our Anders Pink API.

If you haven’t yet tried using Anders Pink for your content curation give it a go, its free to sign up and create briefings.