With an estimated 250 million tweets posted daily, Twitter is a huge trove of data about things as mundane as celebrity wardrobe malfunctions or as weighty as rebellions in the Middle East. What some have called the “Twitter firehose” – DataSift says there were 250 billion tweets in 2010 — can be an invaluable information source, provided the right data can be found and filtered. That’s what DataSift brings to the table.

The company will say more about that other gold mine of information — Facebook — and how it can tap into that in a few weeks, a spokeswoman said.

DataSift’s current service handles only very recent tweets. Gnip, the only other company Twitter has authorized to work with its data, can go back 30 days; DataSift Historics will dial back to Jan. 2010.

While some might question how valuable months- or years-old information really is, DataSift CEO Rob Bailey said companies planning new products, promotions or price changes would do well to study the impact of their past actions before proceeding and that Twitter is the perfect venue for that.

“Companies spend a lot of money designing new products and social network reaction can be a huge accelerant or a countervailing force,” he said.

A phone maker working on a new model would be well served to go back to see what customers thought of its last release — what they liked and didn’t like about the screen size, resolution and features, for example. If a price change is under consideration, a smart company might go back to see how past price actions were positioned and received.

In addition, it is helpful for businesses to know which Twitter users have the most influence and followers and to study their tweets carefully.

Bailey said DataSift works closely with Twitter to make sure the data is handled properly. “Users are extremely sensitive to privacy so we spent four to five months checking out the data . . . if people have deleted tweets, we make sure they stay deleted,” Bailey said.

DataSift Historics will be broadly available in April.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Going social: Recommendations engines need to factor in consumer reviews
• Finding the Value in Social Media Data
• Data markets: in search of new business models

We have written a number of times about how social media and the “democratization of distribution” has compressed the news cycle to the point where the half-life of a scoop is measured in minutes rather than hours or days. And judging by a survey of media attitudes that Craigslist founder Craig Newmark has just released, the number of people who care about who reported something first is rapidly diminishing — if it was ever that big to begin with. Instead, what matters most to readers and listeners and viewers is the trustworthiness of the source, whether it’s a TV program or a newspaper. Trust, as Newmark likes to say, is “the new black.”

The survey, which was done by a polling firm in January, came out of Newmark’s new venture Craigconnects, which he has said is an effort to help nonprofit entities of all kinds connect with supporters. But the Craigslist founder also has an often-stated passion for the issue of trust and the media — he has called a trustworthy press the “immune system of democracy” — and the survey was designed to look at consumer perceptions of both social media and mainstream media sources such as television, radio, Internet news sites and newspapers, and it was focused specifically on news coverage of the upcoming U.S. election.

When it comes to “perceived credibility,” traditional news outlets can take some comfort from the fact that the survey showed newspapers, cable news and network news sources have the highest levels of credibility, much higher than blogs and social media sources. But the bad news is that only about 22 percent of those surveyed said they found traditional sources to be credible (blogs and social media were seen as credible by just 6 percent). And in a surprising ray of hope for newspapers, those between the ages of 18 and 35 were the most likely to see newspapers as credible, at 33 percent.

The survey also asked what the most valuable quality was for a news source when it came to reporting on election news, and gave respondents a choice between “first to report a story” as well as “free of charge,” “in-depth analysis” and “trustworthy.” Close to 50 percent of those who responded (the survey posed the questions to 1,000 people nationwide) chose trustworthiness as the most important quality, and almost a quarter said that in-depth analysis was the most important. Just 6 percent of those who were asked said it was important that a news outlet be the first to report something.

It’s dangerous to read too much into any survey, if only because people often tell researchers what they want to hear. TV surveys used to find huge numbers of people reporting that they watched PBS or nature shows, when in reality most of them were watching sitcoms. So it is possible that many of the people who filled out Newmark’s survey just said they were interested in in-depth analysis because they wanted to look smart or said “trustworthiness” because it sounded like the right thing to say. But the results definitely fit with the evolving nature of the news business online, one in which the trust of readers (or viewers) is far more important than whether an outlet was the first to report something.

News outlets of all kinds continue to fight over bragging rights to news, like dogs fighting over an old bone: Supporters of the New York Times criticize bloggers like Kashmir Hill for “stealing” a story by summarizing it in a blog post, and news services like Sky News and the Associated Press block their journalists from breaking news on Twitter or other social networks because they think they can somehow “save” it for the wire or for their existing platforms. The reality of the news ecosystem now is that news can be broken by just about anyone, including non-journalists who happen to be close to an event, who often wind up committing what NPR’s Andy Carvin has called “random acts of journalism.”

As the old “bloggers vs. journalists” war continues to rage in different forms and traditional journalists like Dan Lyons at the Daily Beast and Michael Hiltzik at the Los Angeles Times criticize what they see as a lack of objectivity in new-media sources, it is worth remembering that trust is the benchmark for any news outlet or media source — regardless of what medium it publishes through or whether those producing the content have degrees from a journalism school or ink beneath their fingers. And Newmark’s survey seems to confirm that the trust game is still wide open for anyone to win.

Post and thumbnail photos courtesy of Flickr user Lars Plougmann

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• How to navigate the new world of digital advertising
• NewNet 2012: companies and technologies set to disrupt
• Social 2013: The enterprise strikes back

According to Rapportive founder Rahul Vohra, the company will continue to support its Gmail plugin , which enables users to get more detailed information about people who have emailed them. That includes social connections such as LinkedIn, Facebook and Twitter, as well as their address books. The acquisition will no doubt lead to more integration with LinkedIn’s core properties, which could bring even more information about your work contacts into the fold.

AllThingsD reports that the price came in at the low teens of millions of dollars, with LinkedIn beating out other potential buyers, including Twitter. Rapportive had raised about $1 million in 2010, with investors that included Charles River Ventures, Dave McClure, Paul Buchheit, Jason Calacanis, Gary Vaynerchuk, Shervin Pishevar and Venture Hacks.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• NewNet 2012: companies and technologies set to disrupt
• Social media in Q1: commerce and discovery dominated
• Startup growth and the new recruiting ecosystem

Tiny Review works because, like Twitter, it causes the user to really think about what he or she is sharing — to work within the constraints not just of three short lines of text, but also around the photo in the background. And brevity is the soul of wit, after all. The results are mostly irreverent and oftentimes hilarious:

Once someone has added text to the photo, they can share it to social networks like Facebook and Twitter, with connections to Foursquare and Tumblr coming soon. There’s a location component, which is meant to allow users to “review” various locations, sort of like Yelp, but shorter.

But the way that its users have utilized the app sort of flies in the face of the original premise, however. While there are some who use it like a more humorous version of Foodspotting, most seem to embrace it as a way to add witty taglines to pretty much anything. Tiny Review has embraced the remix nature of the community it’s built by letting users build on top of others’ pictures by adding their own text. That’s created increased engagement from those who wish to create or build on memes within the community.

For now, Tiny Review is trying to grow its user base by word of mouth and by tapping into other social networks. It’s got a Facebook page and a Pinterest page to highlight some of the more creative uses of the app. It’s also working to drum up celebrity interest in the app, which can help draw some viral interest. That said, the team is convinced there’s something there that “regular” users will love: co-founder Dick Brouwer told me in an interview, “If there’s one thing that excites me, it’s that my mom uses this.”

Tiny Review was founded by Brouwer and Melissa Miranda, who have worked together on gift registry ListCharming. Now that they’ve graduated from the most recent 500 Startups class, the Tiny Review team has moved from Mountain View to Palo Alto, Calif., with $300,000 it’s raised in angel funding.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Going social: Recommendations engines need to factor in consumer reviews
• Google doesn’t like walled gardens — except its own
• Flash analysis: future opportunities for Pinterest

The deal gives Yandex access to Twitter’s firehose, and allows users to search it at twitter.yandex.ru, giving them almost instant access to vast troves of data streaming off the messaging service.

The move comes just hours after the Moscow-based service launched a program to incorporate more social networking data into its search results.

And it also comes after Google, which trails Yandex in the Russian market, famously ended its own deal with Twitter — a move that killed off its own real-time search product.

That decision was just one part of an ongoing spat between the two companies, but it appears that Yandex wants to try and take advantage of the situation to further extend its lead over Google in its home market. Currently Yandex is responsible for around 60 percent of Russian searches, but its rival recently stole a few points and moved up to control around a quarter of the market.

Although the terms of the deal aren’t known, it’s likely that Yandex is paying a substantial fee to access Twitter’s data: Microsoft is thought to pay around $30 million each year for the firehose.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• NewNet 2012: companies and technologies set to disrupt
• Google and the Ghost of Silicon Valley Past
• Google doesn’t like walled gardens — except its own

Is Twitter a publisher and distributor of information like a newspaper, or is it just a dumb pipe like a telephone network? Lawyers in Australia seem to believe that a case could be made that Twitter is a publisher, like a newspaper, and therefore it can be sued for defamation as a result of a single tweet. That may be a stretch — especially in the United States, which has legislation that protects online commentary from such lawsuits — but it highlights the difficulties that Twitter could have as it tries to expand around the globe and into different legal environments.

As reported by the Sydney Morning Herald, a tweet by writer and TV personality Marieke Hardy — in which she falsely accused a man of setting up a blog to attack her — led to a lawsuit by the aggrieved party for defamation, and Hardy wound up having to pay a settlement said to be in the $15,000 range. But the victim apparently wasn’t satisfied with this judgment, and is now suing Twitter because the offending message was retweeted by her followers, and also appeared on the Twitter home page.

Does Twitter edit content? That could be important

This might seem absurd at first. After all, isn’t Twitter just a network like the telephone system, which relays messages sent by users? Perhaps. But the real-time information network is unlike the phone system in many other ways, some of which could be very relevant to such a case. For example, the phone company doesn’t normally listen to conversations and delete or block the ones that it doesn’t like — but Twitter has explained that it routinely blocks users and also deletes tweets for all sorts of reasons, including illegal behavior of various kinds.

Doing this, some lawyers argue, makes the company a lot more like a newspaper publisher than a phone network or dumb pipe. According to media lawyer David Poulton:

There’s not a lot of difference conceptually between Twitter or other internet publishing and an airmail copy of a newspaper; it’s just quicker.

This argument is based on the idea that a publisher who makes choices about what to publish and what not to publish needs to be held to a higher standard of legal behavior than one who just transmits messages without editing them — that’s why phone companies don’t get sued when someone says something defamatory on the phone. But Twitter exerts much more control over its network and the content that flows over it, which is why the British government tried to make the case that it was to some extent liable for some of the violence during the London riots and argued that it should be blocked.

Providers in the U.S. are partially protected from such lawsuits

Such a case isn’t as likely to fly in the United States because of Section 230 of the Communications Decency Act (also called the “Good Samaritan” provision), which — as my paidContent colleague Jeff Roberts has explained — states that a provider of “interactive computer services” is not liable for the content transmitted over that service. This is what protects bloggers and websites from defamation cases in the U.S., and also protects ISPs (to some extent) from having to monitor everything that happens on their networks, something SOPA threatened to change.

That said, however, someone could argue that Twitter isn’t just an electronic information service, but is much closer to being a publisher — in part because it is the only one who controls the data and content that appear on the service, rather than being part of a larger, open web ecosystem with many players.

In countries like Australia and Canada, meanwhile, there is very little U.S.-style protection for any website or service that hosts defamatory content, which leaves Twitter open to actions like the one that was recently filed in Sydney. And other nations have even more rigid and draconian laws around defamation and other crimes related to publishing — including Britain, where the courts routinely use “super-injunctions” not only to block newspapers and even social media from reporting on specific legal cases, but from reporting that any such case even exists.

The Australian case may be settled quickly and/or easily, but what about the next one? The reality is that most of the laws that are currently being enforced weren’t designed to deal with entities that are part publisher and part pipe — with content that is partway between text and speech. Welcome to the challenges of being global, Twitter.

Thumbnail photo courtesy of Flickr user See-ming Lee

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• NewNet 2012: companies and technologies set to disrupt
• The Internet of things: creating tomorrow’s health care
• Going social: Recommendations engines need to factor in consumer reviews

While the addictive properties of social media are totally obvious, so is the truth that introducing social tools in an enterprise context is a tricky business, and driving adoption is sometimes a painfully slow process. As David Lavenda, VP of marketing at social email company harmon.ie, recently pointed out here on WebWorkerDaily, recent Forrester research found widespread under-utilization of the social tools that organizations have invested in, with 64 percent of companies reporting they realized few, if any, benefits from the investment.

And that’s a paradox. Why do we love social tools in our personal lives but often shun them in a professional context? Author and prominent business thinker Tammy Erickson recently pondered this question on the HBR Blog Network, outlining the key differences between the consumer social experience and the enterprise one. In our personal lives, she argues, social media have these characteristics:

• We’re usually invited to participate by people we know and trust.
• There are specific things we want to do with the other people involved, such as share photos, stay up-to-date on a club’s activities, or develop a personal reputation.
• We get something back from participation: advice, practical information we need, a network to tap when times are rough, or the emotional pleasure of seeing others’ photos or hearing their news.
• We have control over who sees our information.
• The applications are intuitive — there’s no training required.
• The applications are well-tuned to support the specific tasks we want to perform, and their features are regularly rated and refined.

Meanwhile, social at work is very different:

• Often we’re instructed to use it by someone in authority, rather than invited by friends.
• Little of what we actually get paid to do (or believe we get paid to do) requires information or input from the vast majority of other people on the network.
• Participation feels like dropping pearls into a black hole — there’s often no sense of getting something in return for sharing an idea or suggestion.
• We have no control over who sees our information and little idea what “they” are doing with it.
• The site is unattractive and requires a manual to get started.
• The software is generic and requires a work-around to do the specific things we would really like to do.

In the thoroughly interesting post she goes on to make suggestions on how organizations can make enterprise social more appealing to speed uptake of the tools, including offering a clear purpose for the initiative and tailoring offerings to existing user behavior. Have a read for her complete recommendations.

Do you think Erickson has nailed the essential differences between personal and professional social tools?

Image courtesy of Flickr user See-ming Lee.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Priorities for Yahoo’s new CEO
• NewNet 2012: companies and technologies set to disrupt
• What’s driving the next phase of the e-commerce evolution

Email is the bane of many people’s lives, but at least most users only have a single inbox to worry about. If you are trying to keep up with comments and other social input from blogs and Facebook and Twitter and LinkedIn, not to mention other social sites you use, it can be overwhelming. That’s the problem a startup called Engagio is trying to solve: The company’s “social dashboard” service, which has been in private beta for several months, launched officially on Wednesday, backed by seed funding from Union Square Ventures partner Fred Wilson and a number of other investors.

One of the interesting things about Engagio is that the service probably wouldn’t exist if it wasn’t for blog comments — specifically, the comments section on Fred Wilson’s blog, where the Union Square partner spends a lot of time responding to questions and interacting with his readers, many of whom are startup founders or venture investors. Engagio founder and CEO William Mougayar is one of Wilson’s regular commenters, and the service was born last fall after a discussion about the difficulty of keeping track of comments across multiple blogs and other social services like Facebook.

“The idea came out of me participating on Fred’s blog and other blogs, and how the experience was very fragmented — I had to go to multiple places to check for replies, and I wanted to be able to track that kind of thing so I could see who I was interacting with,” Mougayar said in an interview before the Engagio launch. The founder, who at the time was involved in another startup called Eqentia, said Wilson suggested he do something about it. “He said to make it like Gmail, but for the social web,” Mougayar recalls. “A social inbox.”

Eight weeks after the idea came up, the startup had a product ready for testing, and Wilson announced the beta on his blog in Dec. 2011. The service has gone from just 20 users to more than 3,000 before the official launch, Mougayar said, and has raised $540,000 in seed funding to help scale the company to the next level — which includes adding servers and staff, but also adding support for more social services. In addition to Wilson and Rho Canada, the seed funding came from venture funds such as Extreme Venture Partners and Bullpen Ventures, as well as individual angels including Mike Yavonditte, the CEO of New York–based startup Hashable.

Engagio currently allows users to connect to their accounts on Facebook, Twitter, LinkedIn and Tumblr, as well as Google+, the Disqus blog-commenting service (which Union Square Ventures is an investor in) and other services such as Foursquare, as well as web forums, including Hacker News. Engagio pulls in any comments, replies, messages and other updates and displays them in a single inbox — and can post any responses back to those services as well. Mougayar said the company is planning to add support for other social services, including the Q&A site Quora and forums run by StackExchange.

Mougayar said that while there has been a lot of backlash recently about blog comments and what some see as a lack of value in them — with some bloggers and sites defending their decision to turn off comments altogether — he believes that commenting, when done well, is “probably the most significant social gesture you can make today, more valuable than sharing or liking or linking.” Relationships that develop with bloggers and even other commenters around discussions on a site can be very valuable, the Engagio founder said, and his connection to Wilson is a perfect example.

After playing around with Engagio during the beta, I can say that it is definitely useful to see all of my social interactions in a single place, and it is much better to have them organized together instead of getting email notifications from Facebook and Twitter in my regular email inbox. The only concern I can see is that Engagio’s inbox could become just as overwhelming as my traditional email, but that could be because I belong to so many different social platforms — and the service does allow users to mute specific contacts or hide services if they become too noisy.

Post and thumbnail photos courtesy of Flickr user Randy Robertson

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Facebook’s IPO filing: ideas and implications
• NewNet 2012: companies and technologies set to disrupt
• Work media tools in 2012 and beyond

The biggest takeaways from the study, which is a must-read for developers and a must-skim for entrepreneurs building consumer-facing startups (seriously, the section on lessons learned and how to hire coders applies to y’all), are how it moved from a traditional open source Linux Apache, MySQL and PHP base to using a thin veneer of PHP code on top of bleeding-edge languages such as Scala and Finagle. The post notes that, while newer noSQL data stores like HBase and Redis are used, the bulk of the data is currently stored in a heavily partitioned MySQL set up and there is no plan to replace MySQL with HBase.

The story is familiar to those who have followed the infrastructure progressions of Facebook and Twitter, and many of the tools Tumblr uses are open-source contributions from those grandaddies of webscale. But the post digs into a problem that is somewhat unique to Tumblr, in that it runs two different types of services, one that looks more like a constantly updating network of user statuses such as Twitter and another that is more like a Facebook page with a huge social graph for each writer that Tumblr must track. The implications for Tumblr’s architecture are huge. From the post:

Users form a connection with other users so they will go hundreds of pages back into the dashboard to read content. Other social networks are just a stream that you sample. …

• Public Tumblelog is what the public deals with in terms of a blog. Easy to cache as it’s not that dynamic.
• Dashboard is similar to the Twitter timeline. Users follow real-time updates from all the users they follow.
• Very different scaling characteristics than the blogs. Caching isn’t as useful because every request is different, especially with active followers.
• Needs to be real-time and consistent. Should not show stale data. And it’s a lot of data to deal with. Posts are only about 50GB a day. Follower-list updates are 2.7TB a day.

Aside from the technical issues, entrepreneurs can learn from this case study because Tumblr is trying to solve more general problems around hiring talent in NYC (it has high hopes for Mayor Bloomberg’s efforts to bolster the tech community), understanding how to conduct job interviews to find good coders and how to slowly try out new technologies at small scale before moving them over to the entire production environment. Really, the post is long, but it’s good. Go read it.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• A near-term outlook for big data
• Dissecting the data: 5 issues for our digital future
• Defining Hadoop: the Players, Technologies and Challenges of 2011

In the not-so-distant past, news generally tended to travel in a few well-worn paths. It was reported by a newspaper, it appeared on television at noon or 6 p.m. or it was mentioned on a drive-time radio show — and those involved usually had plenty of time to report it and produce it. The arrival of CNN and 24-hour news changed all of that, however, and Twitter and Facebook have changed it again: Now the news is just as likely to appear in a tweet or to be posted as a status update by someone who is directly involved in the event. In a nutshell, this means that the value of a simple “scoop” or breaking news report is declining rapidly — and that might just be a good thing.

In one of the most recent examples, the news of singer Whitney Houston’s death at the age of 48 was broadcast far and wide via Twitter and Facebook long before it showed up on most mainstream media outlets. Not only that, but what seemed to be one of the first reports about her death — posted about an hour before the news was reported by a traditional news outlet – included potentially important details about the circumstances that were not revealed until hours later, such as the fact that she was found in a hotel bathtub. The source of the tweet appeared to be someone whose aunt worked for the singer:

omgg , my aunt tiffany who work for whitney houston just found whitney houston dead in the tub . such ashame & sad :-(—
Aja Dior M. (@AjaDiorNavy) February 12, 2012

Whenever there is a news event like Houston’s death, someone inevitably points out that Twitter also routinely reports things that aren’t true, including the deaths of innumerable celebrities who later turn out to be perfectly healthy. And they note that new-media sources — such as the student-run blog network that mistakenly reported Penn State football coach Joe Paterno’s death — can wind up publishing inaccuracies in a rush to be first. But this ignores the fact that mainstream media outlets also routinely report things that aren’t true and have to correct them later.

The news is now happening all around us, and we are part of it

Much of that journalistic process of verifying and making sense of news reports used to happen behind the scenes, inside newsrooms and TV studios and newswire offices. Increasingly, however, it is happening out in the open, where anyone can see it — and where anyone can take part in it by committing what Andy Carvin of NPR has called “random acts of journalism.” German chancellor Otto von Bismarck allegedly warned that anyone who enjoys either the law or sausages should not watch either one being made, and the same is true of the news. But we have no choice but to watch now, because it is happening all around us.

The result of this for journalistic entities of all kinds is that the “news cycle” is being accelerated, like a train whose brakes have failed (and in some cases it becomes distorted as well, as media writer Michael Calderone recently noted about Twitter’s effect on the political news process). In a recent interview, retiring Associated Press newswire president and chief executive Tom Curley said that instead of having days to report on a breaking story, journalists now have hours and in some cases only minutes:

I would say until about 11 September 2001 it was three hours. Now it’s 30 minutes. You might say if you are a certain age – with Twitter and Facebook and all that type of stuff – it’s three minutes.

Curley said the focus for the Associated Press is on speeding up its ability to post scoops and news reports, which presumably is one of the reasons why the service has a prohibition on reporting news on Twitter before it appears on the wire, something the BBC also lectured its journalists about last week. The AP head told the Guardian that if the service (which is owned by a group of member newspapers) can “win by two minutes, on just about every story we can charge a premium.” Curley also said the newswire is working on developing a mobile news app that will feed breaking news reports to users, Twitter-style.

The half-life — and value — of a scoop continues to decline

But is this the right response to the shrinking news cycle, to simply speed up the attempt to beat Twitter or the social web to the news by one or two minutes? Perhaps for the AP it is, but that solution arguably contains the seeds of its own demise. As I have argued before, the half-life of a scoop or a breaking-news report grows shorter every day, in part because sources like Whitney Houston’s hairstylist or the former chief of staff to the Secretary of Defense (who was the first to report that Osama bin Laden had been killed) can “go direct” rather than waiting to have their news reported by a traditional outlet.

Instead, it might be worth more — particularly in the long term — to spend the time trying to confirm the reports that emerge through social media (was that tweet really from the niece of Whitney Houston’s hairstylist?) or to push the story beyond the simple report that something has happened and figure out what it means or why it matters. That kind of analysis and context has always been the most long-lasting aspect of journalism, but mainstream media outlets continually get distracted by the need for another scoop or another “exclusive,” something very few non-journalists care about.

This mentality isn’t confined to mainstream media, of course. The blogosphere is just as guilty of obsessing over scoops and exclusives or trying to pump out as much content as quickly as possible. But there have been some encouraging signs from sources like the Atlantic and Salon that quality content — longer pieces, more thoughtful analysis, etc. — can generate the kinds of results that matter for media entities. Better that than chasing a news cycle whose value continues to diminish with every passing day, and in the end news consumers are better served as well.

Post and thumbnail photos courtesy of Flickr users Waldo Jaquith and Rosaura Ochoa

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• How Media Companies Can Compete Online
• Beyond social: the crowd-based enterprise
• NewNet 2012: companies and technologies set to disrupt