4 Comments

Summary:

Twitter is rolling out some new features for its search, including the ability to see tweets ranked by relevance, but the bad news is that there is still much more the company needs to offer if it is really going to do search properly.

1431384410_db38f8a58f_z (1)

If you were to list the things that Twitter has badly needed for some time now — including support for photo sharing, which it launched Wednesday in partnership with Photobucket — comprehensive search would have to be pretty high on that list. With hundreds of millions of people using the real-time information network every day, the inability to find information has become more and more obvious. The good news is that Twitter has rolled out some enhancements to its search that are going to help somewhat; the bad news is that there is still much more the company needs to offer if it is really going to do search properly.

Twitter’s blog post on the new features — written by co-founder Jack Dorsey, who recently rejoined the company as the director of product development, in addition to being CEO and co-founder of mobile payment company Square — says that the new search is being rolled out gradually, and pulls in not just tweets but also photos and videos. Twitter also launched a plug-in for the Firefox browser that allows users to search hashtags and @ mentions directly from the search window in the browser (so far there’s no similar support for Google’s Chrome though).

The biggest difference when you search on the new Twitter is that you get “Top Tweets” that are sorted by relevance, along with a drop-down menu that lets you see all tweets or only tweets that contain links. And how is the relevance of these top tweets determined? According to a discussion that Danny Sullivan of Search Engine Land had with a Twitter engineer, the answer is that the service judges relevance “using a combination of signals, your follower graph, who you follow [and] who’s following you” as well as the content itself and what Twitter calls the “resonance” of that content.

It’s not clear what “resonance” consists of either — Twitter hasn’t said much about this particular metric since it first introduced the idea last year. But judging by comments from the company, it is a proprietary measurement similar to Google’s PageRank, except it looks at activity within the tweet-stream, and presumably signals of authority such as retweets and so on (Twitter has said that it has internal ways of ranking users, but apparently has no plans to offer this as an outward-facing service). There’s more about the engineering behind the new search in a separate Twitter blog post.

There’s no question that the new features are an improvement on the previous Twitter search, which the company added when it acquired Summize in 2008. It has showed Top Tweets — i.e. those that have been retweeted a lot — but otherwise had no real ranking, and a fairly limited ability to filter search results. For a service that handles over 140 million tweets a day about important subjects like revolutions in Egypt and the death of Osama bin Laden, Twitter’s search has been largely useless. Instead, most people probably use Google, which added “real time” results (consisting largely of Twitter) in 2009.

While the ability to see relevant and/or “resonant” tweets is great, it is really just a small step forward in terms of what Twitter needs to offer for truly comprehensive search. One issue that came up in former Engadget editor Joshua Topolsky’s interview with Twitter CEO Dick Costolo at the All Things Digital conference, for example — where the new features were announced — was the lack of a search function that can pull up any tweets older than about a week.

It’s one thing to embrace the idea that the Twitter “stream” is just a river of content that flows by and searching into the past is philosophically irrelevant, but for a major consumer-focused information network to not have archival search is a fairly gaping hole, I would argue. It’s true that older tweets can be searched via Topsy, and Twitter has also done a deal with Gnip to offer access (for a fee) to past tweets, but for Twitter not to offer a full-fledged search for users — even of their own tweetstream — is a big flaw.

Costolo’s response to Topolsky’s question suggested that Twitter doesn’t have the resources to offer this kind of search — and others suggested that the company might not even want to do this for various reasons, including the ability to sell its archives to others. The Twitter CEO later clarified his comments, however, and said it was a question of priorities for the company and that relevance came first. Now that that’s out of the way, hopefully Twitter can start filling some of those other holes.

Post and thumbnail photos courtesy of Flickr user Woodley Wonderworks

You’re subscribed! If you like, you can update your settings

  1. A move in the right direction…

  2. William Mougayar Thursday, June 2, 2011

    Agreed that it’s a timid step into improved search. It’s still mind boggling to me that you can’t search beyond 2 weeks of content via the Twitter search. Gnip is for high end developers and that doesn’t help the consumer directly. Topsy is doing some interesting things in search display, things that Twitter should be doing or buying :)

    It has been a pain for Twitter to manage both growth and innovation.

  3. Alex Schleber Thursday, June 2, 2011

    Great analysis, agreed that there are several more search functions that represent huge missed potential. Among them:

    1) Search over our own complete tweet archive (making Twitter a sort of free-from Delicious.com), as you mention.

    2) Search over other people’s “following”. At least you now can see the full “following” tweet stream the way another user sees it again, since a few days ago.

    Now if I could search that as well, e.g. search only Scoble’s inbound stream that way, it would be extremely valuable.

    3) Search over individual (hopefully well curated) Twitter Lists. That to me ist still the Holy Grail of Twitter Search. We’ll see if Twitter will eventually give it to us, and actually make Twitter truly useful. So far, mainstream users are mostly not getting the use cases, which leads to huge abandonment rates…

    But this is a start. Possible problems could arise from the new (hidden) algorithm used to determine relevance for us. What could we be missing b/c of this? In most cases we’ll never know…

    See here: http://alexschleber.amplify.com/2011/05/22/must-read-interview-about-how-personalization-is-blinding-you/

  4. Susanna Hall Friday, June 24, 2011

    The BIGGEST drawback as I see it is the inability to search Tweets older than 3 days. This is a major flaw, and Google realtime search allows one to see tweets all the way back to Jan 1st of current year. If Twitter search wants my “business” they’d better fix this.

Comments have been disabled for this post