2 Comments

Summary:

Twitter may be an ever-flowing stream of information, but as it becomes a more mainstream source of news and commentary it also becomes a huge reservoir of data that can be analyzed, and that’s what startups like ThinkUp and DataSift and Gnip are trying to do

The idea of a stream of constantly changing real-time information wasn’t invented by Twitter, but it’s probably one of the closest approximations of that idea for many users. To some extent, the appeal is the same as any stream: to watch it flow by, to step into it now and then. But there’s also a lot of potential value in that stream if you could capture it and analyze it, and that ‘s where a number of startups and services are focusing their gaze, including ThinkUp — which just launched a 1.0 version — and DataSift, which also just launched. And Gnip plans to sell data from the Twitter firehose to analysts and traders who want to tap into the zeitgeist of the Twitter-sphere.

ThinkUp is an app designed by former Lifehacker editor Gina Trapani, and comes from Expert Labs, the non-profit technology incubator and advisory group run by Anil Dash that has helped the Obama administration (among others) figure out how to use digital technology for social purposes (Dash also runs a consulting firm called Activate media). When installed by an individual user or company, it takes in data from Twitter accounts — as well as Facebook pages and Google+ accounts — then makes it easy to see patterns or to analyze the effectiveness of certain tweets or updates.

ThinkUp tracks and analyzes, but also archives

I’ve been using the app for several months now, and it’s quite fascinating to see the amount of data available. (You can see some of my data here if you are interested.) The dashboard shows the most retweeted content, the times of day when tweets get the most activity, the individual users who respond most, and provides all kinds of charts and graphs for presenting that information visually. For me it’s mostly curiosity, but if you’re running a corporate account, I can see how this kind of data would be hugely useful (the White House has been using it for some time) — and you can export and even embed it easily, since ThinkUp has an open API.

Although the information and insights are important, both Dash and Trapani also mentioned another potential benefit of ThinkUp: It acts as an ongoing archive of all your content, whether it’s tweets or retweets or status updates or even photos and videos. As Dash points out, with any content you upload to a social network of some kind, there’s a very real risk it will one day disappear. And while Facebook and Google let you download your data, Twitter doesn’t provide any way of getting tweets you might have posted in the past. The company’s search only goes back about a week, and even services like Topsy are less than perfect if you want to find a specific message. Says Dash:

The companies behind these networks can, and someday will, destroy all of those moments. Delete them from the record. Forever. With no advance notice… history shows us that it happens. Over and over and over. The clips uploaded to Google Videos, the sites published to Geocities, the entire relationships that began and ended on Friendster: They’re all gone.

In addition to the archiving aspect, however, the really powerful feature that services like ThinkUp can offer is the ability to spot patterns in your Twitter or other social-networking activity. Looking at this data over time could help identify all kinds of different things, particularly for corporate users — what content produced a response and when, who the influencers with respect to your content are. (In what seems to be a veiled reference to Klout’s ranking of users, Dash also said ThinkUp “does not, and will never display some arbitrary score to your profile.”)

DataSift and Gnip also filter and resell the firehose

That’s also the idea behind more ambitious data-filtering services such as DataSift, which was started by British programmer Nick Halstead, which got access to the Twitter firehose as part of a deal with Twitter when it took over the retweet button he created as part of his startup Tweetmeme. DataSift uses semantic filtering to detect patterns in the Twitter stream and is aiming its service at corporations and brands that want to see what kind of reaction their content is getting in social networks (DataSift currently filters just Twitter, but plans to add Facebook and Google+). DataSift launched on Thursday, and is backed by GRP Partners and IA Ventures.

The other company that has access to the full Twitter firehose of data is Gnip, which signed a deal with Twitter to resell that data to companies last year. And Gnip on Thursday announced a partnership with StockTwits (see disclosure below) to provide a feed of both Twitter data and related information from the StockTwits investment community to traders and venture funds who are trying to find patterns in that social activity around stocks or companies. At least one hedge fund has an entire trading strategy that is based on an analysis of Twitter and sentiment related to stocks and the markets.

Not that long ago, Twitter was seen as mostly a social plaything, but as it has become a crucial player in everything from the revolutions of the Arab Spring to the planning and scheduling of television events like the Oscars, it has become more obvious that there is value in that stream, provided you have the right filters.

Disclosure: StockTwits is backed by True Ventures, a venture capital firm that is an investor in the parent company of this blog, Giga Omni Media. Om Malik, founder of Giga Omni Media, is also a venture partner at True.

Post and thumbnail photos courtesy of Flickr user Luc Legay

  1. Interesting article but I still need to start seeing some more real world applications of this data. Yes, its valuable data and there’s lots of it but where are the tools that actually solve my problems? Do like the direction that http://flum.es seems to be heading in this respect

    Share
    1. I’m inclined to agree with you John Day.

      I glanced at http://flum.es after reading your comment. Not bad at all! Curation is a huge problem, as all this data is meaningless unless there is a way to extract more value from it, one way or another, than I’ve seen evidence of to-date. There are higher-end, for-fee services e.g. Research.ly, Omniture’s social media analytics and of course Data Sift. Maybe they do something quite useful and impressive.

      Otherwise, as far as curation, that I’ve seen to date, and this is sad but true, http://twitter.com/!#/PopURLs was the most useful yet, for general purpose filtering and aggregation. Of course, that was without any custom tailoring.

      Share
  2. Twitter is a stream, but it’s also a reservoir http://t.co/vhGeCVnV

    Share
  3. “#Twitter is a #stream, but it’s also a reservoir” http://t.co/Z6pv2opI @gigaom

    Share
  4. A good perspective on Twitter – “Stream, but also a reservoir” http://t.co/G2LNI17W #Twitter #data #social

    Share
  5. Twitter is a stream, but it’s also a reservoir — Tech News and Analysis http://t.co/DesjP6b5

    Share
  6. Twitter is a stream, but it’s also a reservoir http://t.co/5bGOB7bv ll RT @jaycbee

    Share
  7. RT @AnnTran_: Twitter is a stream, but it’s also a reservoir http://t.co/5bGOB7bv ll RT @jaycbee

    Share
  8. Twitter is a stream, but it’s also a reservoir http://t.co/ZXjTmQ8H ll RT @jaycbee @AnnTran_

    Share
  9. Twitter is a stream, but it’s also a reservoir — Tech News and Analysis http://t.co/69ync3VA | RT @twitt_erfolg_de

    Share
  10. I think of it as The Fountain of Youth RT @DrVes: Twitter is a stream, but it’s also a reservoir http://t.co/WI9ElBYh

    Share
  11. Twitter is a stream, but it’s also a reservoir http://t.co/PWkisi40

    Share
  12. RT @mpesce: Twitter is a stream, but it’s also a reservoir http://t.co/PWkisi40

    Share
  13. “@mpesce: Twitter is a stream, but it’s also a reservoir http://t.co/L7eblQkK” fascinating

    Share
  14. #mktg250d Twitter is a stream, but it’s also a reservoir http://t.co/juDLRFNR

    Share
  15. RT @mpesce: Twitter is a stream, but it’s also a reservoir http://t.co/PWkisi40

    Share
  16. RT @mpesce: Twitter is a stream, but it’s also a reservoir http://t.co/PWkisi40

    Share
  17. RT @mpesce: Twitter is a stream, but it’s also a reservoir http://t.co/PWkisi40

    Share
  18. Twitter is a stream, but it’s also a reservoir – Track and Analyze Your Activity with @thinkupapp http://t.co/jru9LvJj via @gigaom

    Share
  19. Interesting Twitter insight: Twitter is a stream, but it’s also a reservoir – http://t.co/bwE6iCI2 via @gigaom

    Share
  20. RT @mpesce: Twitter is a stream, but it’s also a reservoir http://t.co/PWkisi40

    Share
  21. Twitter Is A Stream, But It’s Also A Reservoir http://t.co/8Ue88jUz (via @GigaOM)

    Share
  22. “@AttentionUSA: Twitter Is A Stream, But It’s Also A Reservoir http://t.co/WYqv1WCj (via @GigaOM)” #socialmedia

    Share
  23. Twitter is a stream, but it’s also a reservoir http://t.co/3adqtod9

    Share
  24. Twitter is a stream, but it’s also a reservoir | #twittersphere #askTwitter – http://t.co/NQhsZiDO

    Share
  25. Twitter is a stream, but it’s also a reservoir http://t.co/1DAXG57R #SocialMedia

    Share
  26. RT @mclaughlinchris: Twitter is a stream, but it’s also a reservoir http://t.co/1DAXG57R #SocialMedia

    Share
  27. And it can be a stagnant backwater with quicksand border! MT @mclaughlinchris #SocialMedia is a stream AND a reservoir http://t.co/nts2Ldw8

    Share
  28. Twitter is a stream, but it’s also a reservoir http://t.co/n2yfJHv2 via @zite

    Share
  29. Twitter is a stream, but it’s also a reservoir http://t.co/QSZrKM70

    Share
  30. RT @mpesce: Twitter is a stream, but it’s also a reservoir http://t.co/PWkisi40

    Share
  31. http://t.co/mpz3rDDX Twitter is a stream, but it’s also a reservoir – GigaOm

    Share
  32. #Twitter is a stream, but it’s also a reservoir http://t.co/5DN3JlJ9 /@mathewi @Gigaom *great read!*

    Share
  33. RT @2morrowknight #Twitter is a stream, but it’s also a reservoir http://t.co/bRhcGxQ9 /@mathewi @Gigaom *great read!*

    Share
  34. Twitter is a stream, but it’s also a reservoir, and there is value in mining and filtering it. Same for FB & G+: http://t.co/PxHZZxLT

    Share
  35. Twitter is a stream, but it’s also a reservoir http://t.co/Uxl9SiEQ

    Share
  36. Twitter is a stream, but it’s also a reservoir: 17, 2011, 3:07pm PT 1 Comment The idea of a stream of constantly… http://t.co/to6qQfvg

    Share
  37. RT @2morrowknight: #Twitter is a stream, but it’s also a reservoir http://t.co/p1NVuduO /@mathewi @Gigaom *great read!*

    Share
  38. Twitter is a stream, but it’s also a reservoir – http://t.co/uFhdTBm9 http://t.co/KKEdJqMf #twitter

    Share
  39. Twitter is a stream, but it’s also a reservoir – http://t.co/YE43EAoh: http://t.co/ZKtD8JCi

    Share
  40. RT @gigaom: Twitter is a stream, but it’s also a reservoir http://t.co/iY4URdu1

    Share
  41. RT @2morrowknight: #Twitter is a stream, but it’s also a reservoir http://t.co/5DN3JlJ9 /@mathewi @Gigaom *great read!*

    Share
  42. #Twitter is a stream, but it’s also a reservoir http://t.co/ZXo7dG2S /@mathewi @Gigaom *great read!* via @2morrowknight @silverstar22b

    Share
  43. RT @2morrowknight #Twitter is a stream, but it’s also a reservoir http://t.co/z7ixMPVR /@mathewi @Gigaom *great read!*

    Share
  44. RT @2morrowknight: #Twitter is a stream, but it’s also a reservoir http://t.co/5DN3JlJ9 /@mathewi @Gigaom *great read!*

    Share
  45. RT @2morrowknight: #Twitter is a stream, but it’s also a reservoir http://t.co/5DN3JlJ9 /@mathewi @Gigaom *great read!*

    Share
  46. Twitter is a stream, but it’s also a reservoir — Tech News and Analysis http://t.co/AXPl8DxX

    Share
  47. Twitter is a stream, but it’s also a reservoir http://t.co/LEF4gWwF

    Share
  48. Twitter is a stream, but it’s also a reservoir – GigaOm http://t.co/j8zAcsMg

    Share
  49. Twitter is a stream, but it’s also a reservoir — Tech News and Analysis http://t.co/te5yQD1f

    Share
  50. Excellent! RT @2morrowknight #Twitter is a stream, but it’s also a reservoir http://t.co/ssL1GHHJ /@mathewi @Gigaom *great read!*

    Share
  51. “@AnnTran_: Twitter is a stream, but it’s also a reservoir http://t.co/0gm3aoxj ll RT @jaycbee”

    Share
  52. Twitter is a stream, but it’s also a reservoir – http://t.co/nCJExCsQ via @Gigaom RT @2morrowknight

    Share
  53. RT @2morrowknight: #Twitter is a stream, but it’s also a reservoir http://t.co/5DN3JlJ9 /@mathewi @Gigaom *great read!*

    Share
  54. Twitter = constant flow of info, leaving yesterday’s tweets in the dust. Startups record and analyze ALL the data: http://t.co/koDFiMQS

    Share
  55. #Twitter is a stream, but it’s also a reservoir http://t.co/rAFSm26c /@mathewi @Gigaom *great read!* @2morrowknight

    Share
  56. Twitter is a stream, but it’s also a reservoir http://t.co/MTnxo3Aj

    Share
  57. http://t.co/j8zAcsMg Twitter is a stream, but it’s also a reservoir – GigaOm

    Share
  58. Twitter is a stream, but it’s also a reservoir http://t.co/P3RgkBlm

    Share
  59. RT @2morrowknight: #Twitter is a stream, but it’s also a reservoir http://t.co/5DN3JlJ9 /@mathewi @Gigaom *great read!*

    Share
  60. RT @2morrowknight: #Twitter is a stream, but it’s also a reservoir http://t.co/5DN3JlJ9 /@mathewi @Gigaom *great read!*

    Share
  61. RT @2morrowknight #Twitter is a stream, but it’s also a reservoir http://t.co/XrUyxbsV /@mathewi @Gigaom *great read!*

    Share
  62. Twitter is a stream, but it’s also a reservoir — Tech News and Analysis http://t.co/cDgpJF1A

    Share
  63. RT @jeffreypjacobs: Twitter is a stream, but it’s also a reservoir — Tech News and Analysis http://t.co/cDgpJF1A

    Share
  64. #Twitter is a stream, but it’s also a reservoir http://t.co/qFzHngBQ /@mathewi @Gigaom *great read!* RT @2morrowknight

    Share
  65. RT @2cre8: #Twitter is a stream, but it’s also a reservoir http://t.co/qFzHngBQ /@mathewi @Gigaom *great read!* RT @2morrowknight

    Share
  66. Twitter is a stream, but it’s also a reservoir- http://t.co/1DAXG57R #SocialMedia

    Share
  67. #Twitter is a stream, but it’s also a reservoir http://t.co/5YnfvEch /@mathewi @Gigaom @2morrowknight

    Share
  68. “@2morrowknight: #Twitter is a stream, but it’s also a reservoir http://t.co/M6u39V8G /@mathewi @Gigaom *great read!*”

    Share
  69. Twitter is a stream, but it’s also a reservoir http://t.co/5DNtF903 nvia @mathewi

    Share
  70. RT @Digitaltonto: Twitter is a stream, but it’s also a reservoir http://t.co/fCV0KdjG nvia @mathewi

    Share
  71. RT @Digitaltonto: Twitter is a stream, but it’s also a reservoir http://t.co/5DNtF903 nvia @mathewi

    Share
  72. RT @gigaom: Twitter is a stream, but it’s also a reservoir http://t.co/iY4URdu1

    Share
  73. Twitter Activity Stream & Dwindling Attention Reservoir http://t.co/RmE2GM1O Twitter is a Stream & Reservoir http://t.co/wfDmvu8B @mathewi

    Share
  74. RT @gigaom: Twitter is a stream, but it’s also a reservoir http://t.co/iY4URdu1

    Share
  75. Twitter is a stream, but it’s also a reservoir – GigaOm http://t.co/nTlaUdCc

    Share
  76. Twitter is a stream, but it’s also a reservoir– http://t.co/1DAXG57R #SocialMedia

    Share
  77. RT @mclaughlinchris: Twitter is a stream, but it’s also a reservoir– http://t.co/1DAXG57R #SocialMedia

    Share
  78. Interesting piece about the emerging community of startups that do analysis on big #twitter data http://t.co/T1d8RiP5 #bigdata #ecosystem

    Share
  79. RT @dombeveridge: Interesting piece about the emerging community of startups that do analysis on big #twitter data http://t.co/T1d8RiP5 #bigdata #ecosystem

    Share
  80. RT @dombeveridge: Interesting piece about the emerging community of startups that do analysis on big #twitter data http://t.co/T1d8RiP5 #bigdata #ecosystem

    Share
  81. Tools for capturing and archiving the Twitter stream. Could make Twitter even more valuable. http://t.co/qWcg9ONT

    Share
  82. Twitter is a stream, but it’s also a reservoir http://t.co/6pDMf14B

    Share
  83. @Twitter is a stream, but it’s also a reservoir #data #revolution http://t.co/0WBJnB6B

    Share
  84. Twitter is a stream, but it’s also a reservoir (via @swee06840) – http://t.co/xsdBcn2j

    Share
  85. Twitter is a stream, but it’s also a reservoir http://t.co/6XCC6tWA

    Share
  86. #Twitter is a stream, but it’s also a reservoir http://t.co/UwKkZNPg rt @mathewi @Gigaom @2morrowknight

    Share
  87. Twitter is a stream, but it’s also a reservoir http://t.co/5bpZFXCn

    Share
  88. http://t.co/nYCpaWTc Twitter is a stream, but it’s also a reservoir – GigaOm

    Share
  89. http://t.co/nTlaUdCc Twitter is a stream, but it’s also a reservoir – GigaOm

    Share
  90. Twitter is a stream, but it’s also a reservoir – GigaOm http://t.co/nYCpaWTc

    Share
  91. Twitter is a stream, but it’s also a reservoir http://t.co/kYS0NMUI RT @boalt #SocialMedia #patterns

    Share
  92. Twitter is a stream, but it’s also a reservoir http://t.co/J9EFXTfl via @younaz #UOKM

    Share
  93. RT @plevy: Twitter is a stream, but it’s also a reservoir http://t.co/J9EFXTfl via @younaz #UOKM

    Share
  94. RT @plevy: Twitter is a stream, but it’s also a reservoir http://t.co/J9EFXTfl via @younaz #UOKM

    Share
  95. RT @plevy: Twitter is a stream, but it’s also a reservoir http://t.co/J9EFXTfl via @younaz #UOKM

    Share
  96. RT @plevy: Twitter is a stream, but it’s also a reservoir http://t.co/J9EFXTfl via @younaz #UOKM

    Share
  97. RT @plevy: Twitter is a stream, but it’s also a reservoir http://t.co/J9EFXTfl via @younaz #UOKM

    Share
  98. RT @plevy: Twitter is a stream, but it’s also a reservoir http://t.co/J9EFXTfl via @younaz #UOKM

    Share
  99. RT @plevy: Twitter is a stream, but it’s also a reservoir http://t.co/J9EFXTfl via @younaz #UOKM

    Share
  100. RT @plevy: Twitter is a stream, but it’s also a reservoir http://t.co/J9EFXTfl via @younaz #UOKM

    Share
  101. RT @plevy: Twitter is a stream, but it’s also a reservoir http://t.co/J9EFXTfl via @younaz #UOKM

    Share
  102. RT @plevy: Twitter is a stream, but it’s also a reservoir http://t.co/J9EFXTfl via @younaz #UOKM

    Share
  103. RT @plevy: Twitter is a stream, but it’s also a reservoir http://t.co/J9EFXTfl via @younaz #UOKM

    Share
  104. Twitter is a stream, but it’s also a reservoir – GigaOm http://t.co/nYCpaWTc

    Share
  105. RT @plevy: Twitter is a stream, but it’s also a reservoir http://t.co/J9EFXTfl via @younaz #UOKM

    Share
  106. Twitter is a stream, but it’s also a reservoir – GigaOm http://t.co/Nifkdq9q

    Share
  107. http://t.co/nYCpaWTc Twitter is a stream, but it’s also a reservoir – GigaOm

    Share
  108. RT @mathewi: in case you missed it yesterday: “Twitter is a stream, but it’s also a reservoir” http://t.co/Ax9c0E0L

    Share

Comments have been disabled for this post