Blog Post

Twitter’s big problem: It still needs better filters

Things couldn’t be better at Twitter, CEO Dick Costolo told attendees at a conference on Tuesday, saying the business is going “phenomenally well,” and the company is “confident we have a hit on our hands” with its Promoted Tweets feature for advertisers. So why launch yet another new version of its Discover tab? Because even Costolo admits that Twitter needs to get better at figuring out a user’s “interest graph” and recommending topics and content to them, something the CEO says he plans to devote a lot of time to this year. And that makes sense, because services like, Zite and Prismatic are also busy trying to fill that gap.

The original launch of the Discover tab, for both the web version of Twitter and its various iOS and other apps, came last fall amid a revamp of the website — one that founder and product lead Jack Dorsey said was designed to make it easier for new users to find things that might interest them. Now, the algorithms that power the Discover tab have been upgraded with the addition of new personalization features, which look at links shared by your network and try to recommend related content. It’s also easier to see who posted links, and to reshare them from the Discover tab.

Will the new discovery features bring more users?

I was interested in this news because, like many users — including those described by Google staffer Hunter Walk in a recent blog post on discovery tools within Flickr and other networks — I rarely use the Discover tab, mostly because it always seems to be filled with spammy-looking trending topics and hashtags that I have no interest in. In my initial use of the upgraded one (which is being rolled out to all users over the next few weeks), I found things somewhat improved, but only in the sense that the obvious spam was gone. The recommended topics still were somewhat hit and miss:

To take just one example, the top link on the tab at one point on Tuesday was an article from Diner’s Journal, in which a writer answered questions about Mexican food. The link was shared by Julia Moskin, the dining reporter for the New York Times — who was described by Twitter as “people who share your interests.” I don’t follow Ms. Moskin, and I hardly ever tweet about food of any kind, Mexican or otherwise. So why is she someone whose link might interest me? Is it because I follow a lot of NYT staff, and they follow her? I don’t know. In any case, I wasn’t just somewhat uninterested in her link, I had zero interest — possibly even negative interest.

Obviously, any system based on algorithms is going to be hit and miss — especially one that must sift through the half a billion or so tweets that go streaming through Twitter every day, according to Costolo. And figuring out a user’s broader “interest graph” is no easy task at the best of times, especially when the only thing Twitter has to go on is 140 characters of text and perhaps an image now and then. Recommendation services are a little like voice-recognition, in that no one notices when you get it right but everyone hates you when you get it wrong. But more than anything else, that is what Twitter has to figure out — and soon, before someone else does it better.

Curation and filtering are the holy grail for media

As we noted at the time, the company’s acquisition of Summify was an attempt to get a leg up in this particular race — the race to curate the never-ending stream of information coming from social networks. from Betaworks is trying to be one aspect of that filter, and to create an Instagram-style social network around the news as well, a niche that Twitter would like to fill (and already does, in some ways). The CNN-owned iPad app Zite is also pursuing the curation-and-recommendation market, as is Flipboard.

Some users, like Hunter Walk, have said they’d rather see one of those services, or even the page-saving app Read It Later, instead of the Discover tab. I’ve been using a new service called Prismatic a lot to filter Twitter, and so far its algorithms have been doing a pretty good of recommending links I might want to read — substantially better than Twitter’s own filters, even though Twitter should have more info about me. And like Zite, you can help Prismatic learn and improve by voting on the stories you see, while Twitter’s recommendation engine remains somewhat of a black box.

Getting the “interest graph” right is about more than just users. Twitter needs to solve this problem for its advertisers as well, because if their promoted tweets don’t go to the right people then they will be ineffective. As I’ve tried to argue before, Twitter is a new-age media company, and as a new breed of media player it has to be the best at what new-media companies need to do to succeed — and that is curate and filter better than anyone else. The Discover improvements are nice, but there is still a long road ahead.

Post and thumbnail images courtesy of Flickr users Rosaura Ochoa and Luc Legay

18 Responses to “Twitter’s big problem: It still needs better filters”

  1. William Mougayar

    Being a general-purpose platform and trying to appeal to a broad-base of consumers might make it difficult for Twitter to meet everyone’s needs in terms of filtering and recommendations. I don’t think we can expect them to have a solution for all. That’s why peripheral tools like Pragmatic, Zite or others are needed. Twitter will be like a mainstream channel, where you’ll be responsible for your personalized channels that rely on Twitter, but not exclusively.

  2. Blase Ciabaton

    There’s definitely room for improvement in this area for Twitter. For me personally, I use and and they seem to do a very good job of filtering my Twitter stream and locating relevant content for me-in fact, I found this post in my TweetedTimes this morning. Wouldn’t be surprised if Twitter decides to button up this problem with more acquisitions. Thanks much for the post Matt!

  3. Drew Laughlin

    Frankly, I’m quite amazed at just how ‘perfect’ twitter works in its current format. Sure, the “Discover” feature could be more useful. But the whole point of twitter is to curate your list of users and what they feed you. I find retweets a great way to find new users.

  4. Steve Ardire

    >Because even Costolo admits that Twitter needs to get better at figuring out a user’s “interest graph” and recommending topics and content to them, something the CEO says he plans to devote a lot of time to this year. And that makes sense, because services like, Zite and Prismatic are also busy trying to fill that gap.

    Matt Ingram yes these are a few of the players who have a partial solution but you have barely framed the issue because it’s the fusion of Social graph + Interest ( Content ) Graph + serendipity + Knowledge Graphs that is the REAL opportunity.


  5. Rex Hammock

    I am impressed with the “interest graph” suggestions that are emailed me throughout the day from Summify. I think I got to this story via it. So why is Costello not talking about it. As there are no new sign-ups for summify, I figured it would become the kind of filter you’re describing.

  6. Kevin Marshall

    I see Twitter’s latest effort as an improvement in the space, but still just a basic start…the thing that they are missing (and, frankly, it’s the same thing most of the players are missing) is true personalization. They are basically relying on popularity counts (mostly determined by follows and shares)…it needs to be more about what I’m actually saying, sharing, and engaging with…help me dig deeper into those things, help me learn something new about those things (btw – this was the core difference/concept behind our attempts with the service)

    Additionally Twitter has never quite understood the subtleties in the follow model they introduced…first with recommended follows and now with content discovery…it’s not always that I don’t know about someone, it’s often that I *do* know about them and I still choose not to follow them (or pay attention to what they say).

    Anyway – I could ramble on about this forever…the truth is, it’s still early days in the personalization space…and by definition, personalization means something different to each individual…it’s going to take time, money, and passion to get it right (which is why Twitter has a great chance)…

  7. ronald

    One could simple say data finds data, and that twitter doesn’t use “your” data to find data. It seems to make predictions on similarity, prediction is not understanding. Most predictions are based on “focusing”.
    In voice recognition “weather” and “whether” is a close call depending on dialect and the noise in the environment and then depends on context, something twitter seems to struggle with. They seem to think filter is focusing by ever more rules, but good filters expand to general context and very good filters expand to personalized context. In other words the opposite what most people think about filters.
    As we can see with Google results (SEO). You can write rules all day long, people will game the system. Better to get an understanding what Information and context is and how people process it and take it from there. Which would lead to the interest graph to well be a graph instead of a simple list of none relevant points. In above example “weather” would be defined by data/points “around” it, picking “whether” because it’s first in an unrelated list is just silly.

  8. Dave Copps

    Is it really wise for Twitter to place a focus on apps? It seems the bigger play is rather to focus on the platform and perhaps even create tools and access that allow others to build the goodness. The platform is where the value is me thinks.

    • The platform was definitely Twitter’s focus in the past, but over the past year or so it has made it clear that it wants to control most of the apps and the service end as well, so that it can monetize the network more easily.

  9. Steve Mallett

    there were people recommending in the wiredbizcon stream this morning. works good for that application.