You don’t need to be a software developer or hardcore techie to use APIs. In fact, some web APIs, like the Twitter API, are relatively easy to use, and you can use them get access to useful and otherwise hidden data. When you start looking into APIs, you’ll also see how simple some of those web applications really are when you discover the underlying API calls used to build them.

Before you can start to play with Twitter’s API, you’ll need to know a few API basics. You can find information about constructing URLs, rate limiting, keys and more in my previous post about using APIs. Now, let’s look at a few fun things you can do with Twitter’s API.

Show User Data

The various web applications that do simple things like showing when a user joined Twitter are probably using data from the users/show API call. While most of this data can be found by looking at a user’s profile page, there are a few interesting tidbits that are only in the API, like account creation date (created_at), number of favorites (favourites_count), time zone (time_zone), ID number (id) and more.

http://api.twitter.com/1/users/show.xml?screen_name=geekygirldawn

In this example, you should replace geekygirldawn with the username that you want to learn more about.

Friend Data

The statuses/friends API call is a fun way to get the data we got using at in the users/show call, but for all of the people a particular user is following. The data is displayed for 100 users at a time with the most recently followed users at the top.

http://api.twitter.com/1/statuses/friends.xml?screen_name=geekygirldawn&cursor=-1

Again, replace geekygirldawn with the username that you want to learn more about.

Unfortunately, it doesn’t seem to have a way to sort by oldest friends first (or at least no documented way in the API) without some additional programming required. However, adding the cursor=-1 option, shown above, does let you page through the results. To get the next page of results, scroll to the bottom of the results to find the next_cursor attribute value, and replace the -1 in the above call with the value found in next_cursor. It should look something like this:

http://api.twitter.com/1/statuses/friends.xml?screen_name=geekygirldawn&cursor=1320699204947613644

Follower Data

The statuses/followers API call is very similar to the statuses/friends API call, but it returns data about the people who have followed a certain account. It’s an interesting way to get all kinds of relevant data on one page about the people who have recently followed you. Like the statuses/friends call above, the data is displayed for 100 users at a time with the most recently followed users at the top.

http://api.twitter.com/1/statuses/followers.xml?screen_name=geekygirldawn&cursor=-1

Like the other examples, you should replace geekygirldawn with the username that you want to learn about. You can also page through the results using the next_cursor attribute described in the above example.

Important Information You Should Know

• Rate Limiting. Using this technique to get data from the Twitter API results in a limit of 150 calls per IP address per hour, which is fine for most users. However, those of us who work at large corporations where most of the company’s Internet traffic goes out under a single or small number of IP addresses will find ourselves almost immediately rate limited, so doing this from home is probably a better option. Developers get around this by authenticating with the Twitter API, but that’s a little more complicated.
• Viewing XML. Some browsers do a better job than others in displaying XML from the Twitter API in a format easily read by humans. While I use Chrome for most of my web browsing, I like to view XML / RSS using Firefox, since it displays a nicely formatted structure for the data. You can also save the XML as a file and view it in your favorite text / code editor.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Big Data Marketplaces Put a Price on Finding Patterns
• Predicting Twitter’s Best Business Opportunities
• Defining Hadoop: the Players, Technologies and Challenges of 2011

There is more information available in the world than any one person could hope to consume (hundreds of exabytes of data), but most of that information is uninteresting, out of date, inaccurate, or not relevant for you. The key to reducing information overload is to more efficiently find the data you want among the information that you don’t care about. The tools that I talked about in my SXSW session are focused on discarding or de-emphasizing the data you don’t need, while highlighting the data that’s relevant for you. I wanted to share some of what I talked about during my presentation.

Individual RSS feeds from blogs, news and other sources are a great starting point for your information overload reduction efforts. Some individual RSS feeds from friends’ blogs or the top people in your field might almost always be relevant and won’t need any other work. But what about the blogs where one in five or one in 10 posts are relevant for you? How do you narrow them down to a manageable flow of information that allows you to keep up with at least the most important content?

While there are some simple ways to make better use of your RSS reader to manage information overload, the real magic is in filtering. My favorite filtering tool is Yahoo Pipes, which lets me filter an RSS feed using various criteria: URL, author, date, content and more. Some examples of filtered feeds in my reader right now include industry analyst blogs filtered to only find posts about online community; searches across social websites where my projects are mentioned; and my some blogs filtered for just the best posts using PostRank. The image on the right contains a simple Yahoo Pipes filtering example from my SXSW presentation.

PostRank is a great service that allows you to get the best posts from any feed based on an “engagement” ranking score that incorporates measures like comments, Diggs, sharing on social sites like Twitter, and more. The best thing about PostRank is that you can get an RSS feed of just the best posts from a particular publisher, and that feed then includes the PostRank score, which means that you can do even more hacking on the PostRank RSS feed using Yahoo Pipes. One useful way to use PostRank and Yahoo Pipes is to take several feeds containing only the best posts from a few of your favorite blogs, and filter those top posts to find only the articles mentioning a specific group of keywords using Yahoo Pipes. Because the PostRank feed includes the rank, you can even sort the results so that the highest ranked posts appear at the top of your feed. The image to the right shows an example of how you might do this.

Another technique that helps me to consume information more efficiently is to modify the format of many of my RSS feeds; I bring relevant information into the headlines of the feed to make it easier to quickly scan it to determine which posts are important enough for me to click on them for more details. By bringing more details into the title, I can avoid spending time clicking to get more information. There’s an example of reformatting a Twitter RSS feed in the image to the right.

The final trick is to use Web APIs to gather additional data that can’t be found in an RSS feed. I’ve written about using APIs before, so I won’t go into much detail here, but you can see an example of how I’ve used several APIs together with Yahoo Pipes to build an RSS feed of people posting links from Twitter to my blog posts on slides 17 – 23 of my presentation.

You can listen to the audio from my session and download the slides here.

What are your favorite RSS hacking tools and techniques to manage information overload?

Photo courtesy Flickr user SparkCBC.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Big Data Marketplaces Put a Price on Finding Patterns
• The Near-Term Evolution of Social Commerce
• Are Comments Facebook’s Next Big Service?

Construct the URL

This first thing you need to know is how to construct the URL. Each API is slightly different, so you need to review the documentation for the API you’re interested in. Most of the time, the documentation will include examples that you can tweak to get the output that you’re after.

For example, here is the documentation for the MediaWiki API’s backlink query. It can return a list of pages that link back to a specific page that you are interested in. It includes an example URL that looks for all the pages that link to Wikipedia’s home page. I’m going to tweak that so that it finds 10 links to the Star Wars Wikipedia page.

http://en.wikipedia.org/w/api.php?action=query&list=backlinks&bltitle=Star%20Wars&bllimit=10

Let’s break down that URL and see how it’s made up:

• http://en.wikipedia.org/w/api.php — This is the place on the web where the API lives. Here, we’re using Wikipedia as an example, but you can replace this first part with the URL of any MediaWiki installation.
• action=query — fetch the data requested in the rest of the URL.
• list=backlinks — list pages that link to a specific page.
• bltitle=Star%20Wars — the specific page with the title “Star Wars” (note that %20 represents a space).
• bllimit=10 — limit to 10 results.

I simply took one of the default examples listed in the API documentation and made only minor changes to gather the information that I was interested in seeing.

Available Formats

API calls can return a variety of formats depending on the service. I usually start out by returning the data in HTML or XML that I can easily view in a web browser to make sure that my query is correct I’m getting the results from the API that I expect. I’ll then sometimes switch to another format, like JSON, if I want to download the data to use for some other purpose later.

Let’s take a look at another example.  Here is the documentation for Twitter’s GET users/show function, which returns all of the available information about a user, including description, URL, link to profile image, last tweet, count of friends / followers, and much more. Again, I can simply modify the provided example query to return the information I’m interested in:

http://api.twitter.com/1/users/show.xml?screen_name=geekygirldawn

• http://api.twitter.com/1 — Version 1 of the Twitter API.
• users — users section of the API to gather information on a user.
• show.xml — display the output as XML.
• screen_name=geekygirldawn — the user that you want information about (geekygirldawn)

Rate Limiting

Keep in mind that most APIs have some kind of rate limiting, which means that you can only make so many calls to the API from a given IP address or account in a given amount of time before they cut you off. For Twitter this limit is 150 API calls per hour. This is to prevent people from abusing the API and putting too heavy a load on the servers, but it also means that you might be playing with some API calls to get the parameters just right when you are suddenly cut off. Don’t panic. Go have a drink or a snack, and come back in an hour or so to try again.

API Keys

Some APIs require that you sign up for an API key. This is usually to keep track of your requests, and you should think of it as a little like a password that shouldn’t be shared with other people. In many cases, an API key is what the API uses to rate limit your requests.

Combination for More Power

With some additional programming expertise, or using a tool like Yahoo Pipes, you can start to combine these APIs to get some really interesting information that can’t easily be gathered in other ways. For example, I have one Yahoo Pipe that:

• starts with an RSS feed of all of my WebWorkerDaily posts
• takes each link and runs it through the BackType API to see who has posted the link to Twitter
• then uses the Twitter API to see how many followers each person who tweeted the link has
• and finally it formats all of this information into a new RSS feed in the form of “username (num of followers): Tweet text” that I can view in my RSS reader

What are your favorite tricks for using APIs to gather interesting data?

Related content from GigaOM Pro (sub. req.):

• Enabling the Web Work Revolution
• Report: The Real-Time Enterprise
• Who Owns Your Data in the Cloud?

We all want to keep up with the latest news and trends in our industry along with learning more about other areas of interest, like our hobbies. With only so many hours in the day, people often struggle with finding the information they need to be successful while not spending too much time sorting through massive quantities of data. Here are a few tips to help you whittle the information down from a fire hose to a trickle of only the most relevant content.

RSS Readers

If you don’t already use an RSS reader, you should find one, since this is the best way to get only the information that you want pushed to you while being able to easily see which posts you have read and which you have not. Google Reader and Netvibes are good places to start. For the rest of you who already use RSS, there are many ways to make it more effective.

Firstly, change the way you think about RSS; it’s more like a newspaper than email. If you fall behind on the daily newspaper, you aren’t going to keep trying to catch up and read papers that are months old. RSS is just like a newspaper, so don’t worry if you don’t get to everything, and don’t feel like you need to catch up. Right now, I have thousands of unread items in Google Reader, but I am OK with it. If that big number of unread items bothers you, simply take advantage of the “mark all as read” feature once a week if it makes you feel better or, better yet, use an RSS readers that lets you hide the number of unread items.

Secondly, do some pruning and get rid of the dead wood. Spend a couple of hours looking at which feeds give you the most value and get rid of the rest. If you are feeling overwhelmed and overloaded, you are probably oversubscribed. Some RSS readers even have tools to help you find feeds that you rarely read or are rarely updated.

Third, spend as little time as possible in your RSS reader by prioritizing your feeds. I use folders in Google Reader to group my feeds, and I put the most critical feeds right at the top. I make sure that I read through those high priority feeds first to spend more time on what I need to know while hopefully having some time left over to read a little extra. I also encourage you to learn the keyboard shortcuts for your RSS reader, since this can shorten your RSS reading time.

News Aggregation

While RSS readers are great for the things you know you want to read, they are not the best way to find new sources of information or news from unexpected sources; this is where news aggregators really excel. My favorite aggregator is Twitter Tim.es, since it takes the links from the people that I follow on Twitter and displays them in newspaper-like format, with the links that have been posted by more of my friends appearing as headlines on my Twitter Tim.es page. I also use Techmeme to find the topics that people are discussing online. Depending on your interests, you might be able to find other news aggregation sources focused on your areas of expertise.

Filtering

Filtering RSS feeds takes a little work, but it is worth it in the long run if it helps you find only the information on the topics that you want to see. While there are many filtering tools, my tool of choice is Yahoo Pipes. I use Yahoo Pipes to find out when people are talking about me or the topics that I am most interested in. For example, I have Pipes that comb through industry analyst feeds looking for a few specific keywords, which allows me to find the reports from analysts on those topics while ignoring the rest.

The best thing you can do to avoid overload is to stay focused on the most important information while not worrying about what you might be missing. If you can become more efficient at finding and consuming the right information for your needs, you can easily stay informed while minimizing the feeling of being overwhelmed. Read what you can and don’t stress about what you don’t have time to read.

What are your tips for managing content information overload?

Related GigaOM Pro content (sub. req.): Are You Empowering Your Mobile Workforce?

Photo by Flickr user See-ming Lee 李思明 SML, licensed under a Creative Commons Attribution-ShareAlike 2.0 Generic license

2.0

Photo by Flickr User Orange_Beard under Creative Commons

Josh Bernoff wrote about the modern listening problem and compares it to those speaker phones where you can’t talk and listen at the same time. On those not-full-duplex speaker phones, you are either talking or listening, but not doing both at the same time. In the social media age, we need to be both listening and talking, but many people are only doing one or the other. On Twitter and other social web sites, we talk about what we are doing and listen to other people talk, and we do it simultaneously.

Listening

I tend to automate as much of my listening as I can using various monitoring tools. These are two of my favorite ways to monitor and listen to what people are saying, and we’ve covered both of them in more depth in previous blog posts.

• TweetDeck: I have several different searches that are set up in TweetDeck, and I get real-time notifications when someone mentions my name or several of the projects that I’m involved in.
• Yahoo Pipes: More sophisticated monitoring with Yahoo Pipes looks for mentions of a list of keywords across many different social media sites, blogs, Twitter and more.

Responding

Now that you can find the conversations and have started listening, the hard part begins: finding the time to respond. Depending on the volume, this could be a small task or a huge effort. I do a pretty good job of finding and listening to feedback, but when I get busy, I sometimes find it difficult to carve out the time to respond. If I respond right away, I won’t forget to respond, but responding immediately can really disrupt my work flow. If I put the responses off and do them in batches, I am probably more productive, but I run the risk of missing opportunities or forgetting to respond. The key for me is finding the right balance to respond quickly, but without disrupting my ability to be productive and efficient in my other work.

How well do you listen and respond?

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Big Data Marketplaces Put a Price on Finding Patterns
• In Q3, NewNet Focus Turns to Business Models and Search
• The Near-Term Evolution of Social Commerce

Twitter Search

Twitter’s built-in search engine is probably my favorite way to find information in Twitter. It is great for quick searches to find specific pieces of information, watching trending topics, and persistent vanity searches for your name or company. While I do use the search engine to type in queries, the real power is in using RSS feeds for searches and running them through Yahoo Pipes for additional filtering. In many cases, I use Yahoo Pipes to loop through a series of keywords from a CSV file to search Twitter for each of those keywords and monitor the results as an RSS feed in my reader. I recently did a 2 minute video explaining exactly how to search twitter using Yahoo Pipes and a CSV file of keywords, so I won’t cover it in any more detail here.

Twitter Stream Graphs

I’m a big fan of getting data via RSS, but sometimes you just need a picture. Twitter Stream Graphs are a great way to show your boss how much buzz you generated on Twitter with a particular tweet or a keyword. In this example, I searched for legionoftech, an organization here in Portland than organizes our local BarCampPortland event, and you can see that we did a call for volunteers over Twitter on April 21 and had some conversation leading up to the event on May 1 and 2.

TweetVolume

TweetVolume is a great tool for comparing the number of mentions across several keywords. With the recent Mother’s Day holiday, I’m happy to report that mothers are more important than vacations, the beach and Star Trek. However, sadly, bacon seems to be a more popular topic on Twitter than mothers (shame on you!)

What are your favorite tools for mining Twitter for information?

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Big Data Marketplaces Put a Price on Finding Patterns
• Why Google Should Fear the Social Web
• The Near-Term Evolution of Social Commerce

When I talk about monitoring “dashboards,” I use the term very loosely. In some cases, I set clients up with RSS readers that have a typical dashboard look and feel for monitoring feeds, while in other cases the “dashboard” is really a monitoring section in an existing RSS reader with the feeds delivered as an OPML file.

In my experience, people who are new to RSS readers tend to do better with a reader that looks more like a dashboard than the a tree or folder structure. This is particularly true for monitoring because a dashboard lets you see more information at a glance. Netvibes and iGoogle are both good choices for new users. However, I think that the Netvibes layout tends to work slightly better for this purpose. Here’s an example of a monitoring dashboard built using Netvibes:

However, I personally use NetNewsWire for my RSS reader. I have my monitoring dashboards set up in folders that don’t in any way resemble a typical dashboard. Try out a few different readers to find the one that works best for your style and usage. The tool that you select isn’t the critical element. The real magic is in the content that you are monitoring.

I monitor three primary types of content in my dashboards: vanity mentions, competition and industry analysis.

Vanity mentions are the conversations that people are having about you, your company, your products and your employees. Keep a close eye on these mentions so you can respond quickly to questions and concerns. A proactive approach to monitoring and responding to discussions can help you avoid potential issues before they get out of hand and can show people that you are responsive to your customers. I track vanity mentions for companies that I am involved with across Twitter, blog searches, Flickr, various video sites, FriendFeed and more.

You can get interesting insights about your competition and their activities by proactively monitoring their communications and what other people say about them online. I often monitor competitors blogs, press releases, support forums, job postings and personal blogs or social media accounts of key employees, in addition to monitoring mentions of the competition on various sources.

Industry analysis should also be part of your monitoring dashboard. Monitor blogs written by thought leaders within your industry along with tracking for mentions of keywords that are important to your organization or your interests. I often use this section of the monitoring dashboard as a way to find content for blog posts. It can be a great way to see what other people are talking about in your industry and give you an opportunity to respond to, disagree with or build on interesting ideas from other people.

In addition to the dashboard technology, I use tools like Yahoo Pipes and PostRank to help me find relevant content and filter it down to the pieces that are the most important for my purposes. If you have never used Yahoo Pipes, I have several two-minute Yahoo Pipes video demos that can help you learn what you need to get started.

How do you monitor online conversations? What kind of monitoring dashboards do you use?

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Big Data Marketplaces Put a Price on Finding Patterns
• Why iPad 2 Will Lead Consumers Into the Post-PC Era
• The Near-Term Evolution of Social Commerce

Interesting conversations are happening all over the web, on blogs, Twitter, FriendFeed and many other sites. People are talking about you, your company and your industry, and revealing many tips and tricks that you should know. I am a self-confessed data junkie, so I have a few tips to help you make sense of the massive amounts of data available and to focus on monitoring just what really matters.

Use a Dashboard

An RSS-based monitoring dashboard is a great way to collect everything you’re monitoring into one place. I’ve set these up for clients in a number of different ways depending on the preference of the person responsible for monitoring. For people new to RSS, I generally encourage them to use something like Netvibes, which has a visual layout with multiple tabs and columns where you can see several key conversations at a glance. I have a couple of sections in my RSS reader where I keep everything that I’m monitoring, and I make sure that they are the first things I read.

Filter Your Feeds

Filtering RSS feeds through Yahoo Pipes or other tools is a good way to make sure that your monitoring dashboard contains relevant content and not just a list of blogs to read. I use Yahoo Pipes to filter for mentions of my name, my company and efforts that I am involved with across various sites (Twitter, FriendFeed, blogs, Flickr, video sites, etc.) Yahoo Pipes can also be used to combine many feeds and filter those high-volume RSS feeds for only relevant content that you need to know. Tools like PostRank are a good way to find the posts within a feed that are generating the most buzz.

Choose the Right Twitter Client

Use a smart Twitter client that lets you group the important people that you want to monitor and provides a way to get real-time notifications for mentions of certain keywords. TweetDeck is a good choice. I have a couple of different groups set up to help make sure that I see the posts from people who have important ideas and who provide me with the greatest value. I also have keyword searches in TweetDeck for companies or events that I am involved with to make sure that I don’t miss any important conversations about these efforts on Twitter.

Find Hot Topics With FriendFeed

FriendFeed is a great way to find the hot topics of the day, to make sure that you’re keeping up with industry trends and new tools or techniques that you can apply to your work. I like to group people by topic, like community management, and use the “best of” feature to find the best posts of the day, week or month from those subgroups of people. The example below shows the best post of the month from my “News Makers” group. It’s an easy way for me to filter the flood of content in FriendFeed down to something much more manageable.

FriendFeed Best of Month

While monitoring is important, you should also be responding to these conversations. People are more likely to engage with you in future if they get a response back, rather than feeling like their feedback went into a black hole. Respond to as many @replies on Twitter as you can, and also use your blog for longer responses or to post reactions to relevant conversations that are happening across the web.

What are your tips and tricks for monitoring online conversations?

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Millennials in the enterprise, part 1: strategies for supporting the new digital workforce
• The Near-Term Evolution of Social Commerce
• A 2011 NewNet Forecast

I love information and wish I could spend more time reading and consuming it, to learn more about a variety of topics. However, the harsh reality is that there are only so many hours in the day that I can spend reading and learning. I could take the easy way out and just read less, but my goal is to become more efficient at finding the content that I want to read the most.

Pruning your feeds is a good way to start. Fellow WWD blogger Celine Roque wrote a great article about how to fine-tune your RSS subscriptions a few months ago. Dead feeds and feeds that you rarely read only slow down the process of finding the content that you really need. I have a tendency to add blogs to my reader based on one or two great posts, but sometimes those blogs just don’t keep my interest and must be pruned later. I try to go through my RSS reader a few times a year and focus on getting rid of, or de-prioritizing, feeds that add less value or aren’t being regularly updated any more. Some feed readers even have tools for finding those old blogs that aren’t being updated regularly (dinosaurs) and the blogs that currently get the least amount of your attention.

I also make extensive use of filtering to reduce the amount of content in my feeds. Yahoo Pipes is a great filtering tool, since you can use it to combine many RSS feeds and only display posts that match certain key words that are the most interesting to you. I use Yahoo Pipes to filter groups of blogs that typically have low relevancy for me but have occasional great nuggets of information that I don’t want to miss.  With Pipes, I can make sure that I only see the nuggets, and not the posts that contain less interesting topics.

Use an RSS reader that gives you a way to organize your feeds and use it as a way to prioritize the things that you should read more often. I use NetNewsWire, which gives me the ability to create folders and move them up or down the list. The things that I need to read every day are at the top, while the blogs that I read only occasionally are near the bottom.

When working with clients who don’t already use a reader, I usually set them up with Netvibes, which has more of a dashboard feel. In this case, I organize their feeds with the top priorities in the upper left, lower priorities in the lower right and multiple tabs to group topics. It doesn’t really matter how you organize the information as long as it helps you prioritize your reading.

Take the time to learn the keyboard shortcuts for your feed reader. I can get through the material more quickly when I use a few keys to navigate through the content. In my case, NetNewsWire lets you use arrow keys, the space bar and other keys to navigate through the content, mark items as read/unread, open in a browser, collapse/expand folders, and more. Navigating the menu items to find these functions can sometimes take more time than you realize.

It also helps me to think of feeds as a stream that flows through my RSS reader. I don’t try to catch up by reading everything, and I don’t worry about leaving unread items in my reader. I read what I can with the time I have available and don’t spend any time worrying about the rest. There will always be more interesting content than I can ever read in a lifetime.

What are your tips for getting through your RSS feeds more quickly and efficiently?

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Big Data Marketplaces Put a Price on Finding Patterns
• Why iPad 2 Will Lead Consumers Into the Post-PC Era
• The Near-Term Evolution of Social Commerce

My favorite filtering techniques involve Yahoo Pipes, which looks and sounds much more complicated than it is. Jackson West described Yahoo Pipes pretty well when he called them “hard to grok, but snazzy“; however, Yahoo Pipes doesn’t have to be quite so difficult for people to understand. The first time I looked at Pipes, the interface scared me away until a friend of mine gave me a very quick demo that showed how easy it was to use. After using Pipes for while, I gave similar demos to help other friends get started and even recorded a 2 minute introduction to Yahoo Pipes that shows how to use Pipes to filter RSS feeds. How complicated could it be if I can explain it in a 2 minute screencast?

Here are a few of my favorite filtering techniques that I use to find the most relevant content.

Filter high volume feeds. Many of the big technology news blogs have a huge volume of stories in the feed. The aggregated feed for the entire GigaOM network, for example, contains dozens of posts every day. I have a hard time finding the content relevant to me when I have to sort through a large number of posts, so I use Pipes to search these stories to find the ones that matter to me. The example pipe below shows how to filter the entire GigaOM network feed down to the few posts containing certain keywords.

Example: Filtering high volume feeds

Aggregate and filter less relevant content. There are certain categories of blogs that have outstanding content, but they rarely publish posts that I want to read. For me, this is true of industry analyst blogs. While I really want to hear about the latest research in online communities and social media, most of the reports published are on topics that are not interesting to me. The Pipe below is one that I use to aggregate 17 industry analyst blogs together into one feed while filtering the posts for a few keywords relevant to my interests.

Example: Analyst research blogs filtered for social media

Find the posts everyone else is reading. For news junkies like me, it helps to be able to find the posts that are generating the most buzz. PostRank, a service that used to be called AideRSS, simplifies this process by finding the most talked about posts from any RSS feed. I often use PostRank in conjunction with Pipes to rank and filter posts at the same time. The example below ranks the posts here on WebWorkerDaily with a feed containing only the posts that are generating the most conversation.

Example: PostRank for WebWorkerDaily and feed for the top posts

WebWorkerDaily PostRank

Both Yahoo Pipes and PostRank have an option to get the output as RSS, so I make sure that the results from these tools make it into my RSS reader. With these more efficient filtered feeds, I can remove some of the aggregated and filtered source feeds from my reader. By using these filtering techniques to further prune my feeds, I plan to spend less time catching up on feed reading and more time working, exercising, or spending time with friends and family.

What are your favorite filtering techniques?

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Big Data Marketplaces Put a Price on Finding Patterns
• Why iPad 2 Will Lead Consumers Into the Post-PC Era
• The Near-Term Evolution of Social Commerce