Twitter is a stream, but it’s also a reservoir

The idea of a stream of constantly changing real-time information wasn’t invented by Twitter, but it’s probably one of the closest approximations of that idea for many users. To some extent, the appeal is the same as any stream: to watch it flow by, to step into it now and then. But there’s also a lot of potential value in that stream if you could capture it and analyze it, and that ‘s where a number of startups and services are focusing their gaze, including ThinkUp — which just launched a 1.0 version — and DataSift, which also just launched. And Gnip plans to sell data from the Twitter firehose to analysts and traders who want to tap into the zeitgeist of the Twitter-sphere.

ThinkUp is an app designed by former Lifehacker editor Gina Trapani, and comes from Expert Labs, the non-profit technology incubator and advisory group run by Anil Dash that has helped the Obama administration (among others) figure out how to use digital technology for social purposes (Dash also runs a consulting firm called Activate media). When installed by an individual user or company, it takes in data from Twitter accounts — as well as Facebook pages and Google+ accounts — then makes it easy to see patterns or to analyze the effectiveness of certain tweets or updates.

ThinkUp tracks and analyzes, but also archives

I’ve been using the app for several months now, and it’s quite fascinating to see the amount of data available. (You can see some of my data here if you are interested.) The dashboard shows the most retweeted content, the times of day when tweets get the most activity, the individual users who respond most, and provides all kinds of charts and graphs for presenting that information visually. For me it’s mostly curiosity, but if you’re running a corporate account, I can see how this kind of data would be hugely useful (the White House has been using it for some time) — and you can export and even embed it easily, since ThinkUp has an open API.

Although the information and insights are important, both Dash and Trapani also mentioned another potential benefit of ThinkUp: It acts as an ongoing archive of all your content, whether it’s tweets or retweets or status updates or even photos and videos. As Dash points out, with any content you upload to a social network of some kind, there’s a very real risk it will one day disappear. And while Facebook and Google let you download your data, Twitter doesn’t provide any way of getting tweets you might have posted in the past. The company’s search only goes back about a week, and even services like Topsy are less than perfect if you want to find a specific message. Says Dash:

The companies behind these networks can, and someday will, destroy all of those moments. Delete them from the record. Forever. With no advance notice… history shows us that it happens. Over and over and over. The clips uploaded to Google Videos, the sites published to Geocities, the entire relationships that began and ended on Friendster: They’re all gone.

In addition to the archiving aspect, however, the really powerful feature that services like ThinkUp can offer is the ability to spot patterns in your Twitter or other social-networking activity. Looking at this data over time could help identify all kinds of different things, particularly for corporate users — what content produced a response and when, who the influencers with respect to your content are. (In what seems to be a veiled reference to Klout’s ranking of users, Dash also said ThinkUp “does not, and will never display some arbitrary score to your profile.”)

DataSift and Gnip also filter and resell the firehose

That’s also the idea behind more ambitious data-filtering services such as DataSift, which was started by British programmer Nick Halstead, which got access to the Twitter firehose as part of a deal with Twitter when it took over the retweet button he created as part of his startup Tweetmeme. DataSift uses semantic filtering to detect patterns in the Twitter stream and is aiming its service at corporations and brands that want to see what kind of reaction their content is getting in social networks (DataSift currently filters just Twitter, but plans to add Facebook and Google+). DataSift launched on Thursday, and is backed by GRP Partners and IA Ventures.

The other company that has access to the full Twitter firehose of data is Gnip, which signed a deal with Twitter to resell that data to companies last year. And Gnip on Thursday announced a partnership with StockTwits (see disclosure below) to provide a feed of both Twitter data and related information from the StockTwits investment community to traders and venture funds who are trying to find patterns in that social activity around stocks or companies. At least one hedge fund has an entire trading strategy that is based on an analysis of Twitter and sentiment related to stocks and the markets.

Not that long ago, Twitter was seen as mostly a social plaything, but as it has become a crucial player in everything from the revolutions of the Arab Spring to the planning and scheduling of television events like the Oscars, it has become more obvious that there is value in that stream, provided you have the right filters.

Disclosure: StockTwits is backed by True Ventures, a venture capital firm that is an investor in the parent company of this blog, Giga Omni Media. Om Malik, founder of Giga Omni Media, is also a venture partner at True.

Post and thumbnail photos courtesy of Flickr user Luc Legay