GigaOM » Social graph

Predicting Twitter popularity is all about probability

Amanda Alvarez — Sat, 27 Apr 2013 01:56:38 +0000

Tweets have the power to decimate markets, but they also have users and companies seeing dollar signs. With huge marketing, political, and social mobilization potential, how can you predict which tweets will get more views, and which retweets will go viral? A new study developed a statistical model that attempts to estimate the popularity of tweets, and thus how memes spread.

Starting with 52 “root” tweets from users both famous and obscure, the researchers first analyzed the dynamics of retweeting, like the speed and spread of a tweet from a user to followers and then their followers. The researchers, from the University of Washington, MIT, and Penn, used the Twitter API to collect all the retweet information and found that most retweets occurred within one hour of the original tweet. Not surprisingly, they also found that root tweets are retweeted more than the retweets themselves.

They then plugged the important variables –- number of followers, retweet speed, retweets of other tweets –- into a Bayesian model, a statistical approach that uses prior evidence (the root tweets) to calculate how the retweet graph evolves. They experimented with feeding the model different amounts of prior evidence to see how much was needed to make an accurate prediction. Using only 10 percent of the retweets to guide the model, they were able to reasonably accurately predict retweet time and volume, and the error decreased the more retweet data they included. The average retweet time was only 4.4 minutes.

Throwing more information into the prediction engine (like whether a particular follower has a large numbers of followers of his or her own) could improve the accuracy. Their model was thrown off, it seems, by a few anomalous tweets with a very rapid onset and termination of retweets that didn’t follow the same pattern as the other tweets. (Though they don’t identify who sent those tweets, my bet is on @KimKardashian, whose followers’ actual and predicted retweet timecourse is pictured above.) The researchers didn’t even consider the time of day a tweet was posted, nor its content; there is likely huge potential to mine in those domains for what, and when, leads to trending.

With the abundance of the Twitterverse open to developers via API, this study represents just the tip of the iceberg in predicting tweeting behavior, something that startups like Blab are busily pursuing. It also shows that robust methods like Bayesian statistics can predict if a tweet has any retweet life left, and thus whether it can gather more eyeballs and clicks, something that is sure to prove very lucrative.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

Sector RoadMap: Content personalization in 2013

Mark Mulligan — Wed, 17 Apr 2013 14:54:59 +0000

Content owners, whether they are publishers, retailers, or marketers, are always looking for new ways to deliver a unique experience to their customers. We call this content personalization. Key trends in this area are led by a collection of technologies that we call post-programming curation. These technologies use the best of behavioral tracking, collaborative filtering, audience targeting, and dynamic content presentation.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

Netflix goes social, turns on Facebook integration for U.S. subscribers

Janko Roettgers — Wed, 13 Mar 2013 13:00:47 +0000

Ever wanted to know what your former college roommates are watching on Netflix? Now you’ve got your chance: Netflix plans to turn on its Facebook integration for U.S. subscribers Wednesday, allowing them to share their viewing behavior with their Facebook friends and get social recommendations for what to watch next. The integration followed some political wrangling about a little-known 1980s privacy law.

Subscribers who opt in will see what their friends watch and like (click to enlarge).

U.S.-based Netflix subscribers can now connect their Facebook account to the video service, and then be able to view dedicated categories called “Friends’ Favorites” and “Watched by your friends” on the Netflix website as well as through the company’s apps on mobile and connected devices. Viewers can also opt into sharing all of their viewing behavior on Facebook’s website.

The company said on its blog Wednesday morning that it will eventually offer additional social sharing functionality:

There are some privacy provisions that are supposed to prevent over-sharing: Subscribers who opt into sharing their viewing data with Facebook can prevent the sharing of a title “by clicking Don’t Share This during the first few minutes of playback on most devices,” according to a Netflix help page. Titles that have already been shared can also be unshared — but cautious users may just want to turn off sharing altogether before they embark on some late-night B-movie binge viewing.

Netflix first rolled out its Facebook integration in Canada and Latin America in late 2011, but bringing the feature to the U.S. was complicated by a 15 year-old privacy law called the Video Privacy Protection Act. That law was meant to prevent video rental stores from releasing data about the VHS tapes a customer rented, but it also prevented Netflix from sharing data with one’s Facebook friends.

Netflix lobbied heavily to change the law, and Congress eventually amended it earlier this year, giving Netflix an opportunity to add Facebook integration for its U.S. customers as well.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

It’s not you Facebook, it’s me — okay, it’s partly you: Why I unfriended almost everyone

Mathew Ingram — Sat, 23 Feb 2013 16:00:28 +0000

There have been a rash of posts of late from people who have quit Facebook or decided to unfriend everyone they know on the network. I haven’t gone that far, but I recently went through what I like to call “The Great Unfriending,” in which I unfollowed or disconnected from almost 80 percent of the people in my Facebook social graph. Doing so has changed the way I use the network, and I think that change — and the reason why I felt compelled to do so — says a lot about some of the challenges Facebook is facing.

Unlike Julia Angwin, who says she unfriended everyone she was connected to because Facebook “cannot provide me the level of privacy that I need,” I don’t really have any issues with privacy on Facebook. Angwin said that she was troubled by the fact that “when I share information with a certain group or friend on Facebook, I am often surprised by where the data ends up,” and I respect her decision. But that’s not what bothered me about using the social network.

It’s not the privacy, it’s the overload

For better or worse, I made a deliberate decision when I joined the service (and Twitter, and almost every other social network) to be as open as possible, and to share almost everything about myself, within reason. I would never say that everyone should do this, and there are plenty of reasons why people keep certain things off the web — information about their children, for example — but for the most part I agree with Jeff Jarvis that the benefits of “publicness” outweigh the disadvantages.

So if privacy wasn’t the problem, what was it? In a nutshell, information overload. In the same way I’ve had to struggle with my addiction to real-time connectedness on a mobile device (something I wrote about recently that many readers disagreed with), I started to find that Facebook was a painful experience. And the more I thought about it, the more I thought that the problem was partly me — and the way I was using it — and partly the way Facebook was changing.

I started to think about how some people I admire, including Union Square Ventures founder Fred Wilson, had pared back their use of Facebook by unfriending a lot of people. And such thoughts don’t seem to be unique: a recent survey by the Pew Center showed that two-thirds of users had taken an extended break, and close to 30 percent were planning to use Facebook less.

Partly Facebook and partly me

The part of this that I think was my fault stems from the way I set up my account when I first joined Facebook in 2006: in keeping with my desire to push the limits of openness, I accepted friend requests from almost everyone who sent them, even if they weren’t actually “friends.” And yes, I knew at the time that doing this carried some risk, but I didn’t fully appreciate what it would be like, or how it would eventually ruin the experience for me.

What I wound up with was almost a thousand “friends,” many of whom were people I had met at conferences, or people who were connected to me through others, or some who were just fans of my writing (who can still use the “subscribe” feature). To these people — all of whom I have since unfriended — I would just like to say that you are all wonderful, but I couldn’t take it any more. My stream became a sea of information I had little or no interest in, with only a few scattered pieces of flotsam and jetsam from the people who I am actually close to.

The part of this that I see as Facebook’s fault has to do with how cluttered my stream became, especially with all of the “sponsored stories” and “liked” pages that began to show up more and more — when a “friend” liked a page about Coca-Cola or Ford, for example. And yes, just like the notifications I complained about on the iPhone, I know that Facebook has knobs and dials that you can tweak so that you don’t see certain things. But who has the time to spend twiddling all those dials all the time? I certainly don’t.

Facebook has just become less relevant

So what happened after The Great Unfriending? Facebook became a whole lot more usable as a particular kind of network — the one that lets me see what actual friends and family are doing, including those who are far away (the kind of “ambient intimacy” that researcher Leisa Reichelt talks about). Except for my teenaged daughters, of course, who don’t even use Facebook any more, preferring to spend all their time on Tumblr and Twitter. That’s just one of the things that should worry Mark Zuckerberg, I think.

What I am left with is a more useful network, but also one that I only use for very specific things, and don’t really spend much time on. If I want to connect with people related to work, I do it through LinkedIn; if I want to connect to people through photos, I do it on Instagram or Flickr (which is why Instagram was such a smart acquisition for Facebook to make); and if I want to connect to people I don’t really know, I use Twitter. If I could get more of my friends to use Path, I might use that for friends and family, in which case I wouldn’t need Facebook at all.

Facebook has a whole series of challenges as it tries to grow and justify its $65 billion market value. But its biggest problem — bigger than the shift to mobile or the need to generate ad revenue — is that it has to not only remain relevant in people’s lives, but offer them more and more things that will keep them engaged. For me at least, and it seems for others as well, they are losing that battle.

Post and thumbnail images courtesy of Shutterstock / Stuart Jenner and Flickr user Pew Center

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

Facebook’s Graph Search mastermind shares a few tech secrets

Derrick Harris — Fri, 15 Feb 2013 20:32:08 +0000

A month after launching its vaunted Graph Search feature to much fanfare, Facebook is finally opening up a bit about how, exactly, it works. The product’s primary architect, Lars Rasmussen, took to Reddit yesterday in an Ask Me Anything session during which he elaborated (beyond what Om reported last month) on how Graph Search is built.

Of course, this being a Reddit discussion, Rasmussen answers a bunch of questions about the history of graph search, its privacy issues, his role with building Google Maps (and Wave) and his walks with Mark Zuckerberg. But here are some of the more-informative excerpts about the architecture itself.

I should point out, too, that we’ll be talking a bit about graph processing and graph databases at our Structure: Data conference next month, too. Graphs, as it turns out, are a great way to storing, processing and presenting a lot of data that has nothing to do with social connections.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

Facebook’s graph search and the end of privacy by obscurity

Mathew Ingram — Thu, 24 Jan 2013 00:15:08 +0000

It often seems as though Facebook’s main purpose is to continually remind us of how much we have chosen to share with the world about our online behavior — whether we realize it or not. The latest lesson along those lines comes from the social network’s new “graph search,” which sounds at first like a fairly boring feature of interest only to marketers. Like much of what Facebook does, however, it is also a warning sign: if you were counting on certain things about yourself staying not so much private as obscure or hidden from view, those days are effectively over.

For an example of what this means in practice, look no further than a new Tumblr blog started by London-based programmer Tom Scott, entitled “Actual Facebook Graph Searches.” This also sounds somewhat dry and academic, until you take a closer look at some of the things that Facebook makes it trivially easy to search for — things like “Islamic men interested in men who live in Tehran, Iran” (where homosexuality is a crime punishable by death) or “family members of people who live in China and like Falun Gong,” the latter being a religious group whose members are routinely persecuted.

Many of these searches may be figments of Tom Scott’s over-active imagination (at least for now) but the fact is that Facebook’s graph search makes them relatively easy to conduct. Dave Morin, a former Facebook executive who left to start the mobile social network Path, has pointed out that the company has had this kind of interest-graph powered search for some time — and this kind of targeting based on “likes” and interests, friends and followed pages, etc. has been available in a different form for advertisers. But it is far more robust and more public now.

It’s not about privacy, but the end of obscurity

In a FAQ on the blog, Scott says he’s not trying to make any deep arguments about privacy, and his Tumblr blog is subtitled: “Don’t worry, we’ll all be used to this in a few weeks’ time.” But it’s still worth thinking about the implications of Facebook’s graph search, especially given the fact that many people don’t seem to appreciate the nuances of the network’s privacy settings — something that Facebook doesn’t really make simple or easy to figure out. Even CEO Mark Zuckerberg’s sister Randi was recently blindsided by them, so what chance do the rest of us have?

In a recent piece in The Atlantic, philosophy professor Evan Selinger and his co-author Woodrow Hartzog argued that in many ways it’s not really helpful to talk about privacy, which is a vague concept in a world of real-time information, “frictionless sharing” and “data exhaust” (the information we give off as we move around the internet, often without realizing it). Instead, they argue that what we are really losing is the protection of obscurity — in the sense that information which was technically public before but difficult to find provided a form of privacy through obscurity:

“Obscurity is created through a combination of factors. Being invisible to search engines increases obscurity. So does using privacy settings and pseudonyms. Disclosing information in coded ways that only a limited audience will grasp enhances obscurity, too. Since few online disclosures are truly confidential or highly publicized, the lion’s share of communication on the social web falls along the expansive continuum of obscurity: a range that runs from completely hidden to totally obvious.”

As Selinger notes, the recent publication of a map of New York state’s registered gun owners made much the same point as Scott’s examples of disturbing Facebook graph searches. The information about who has a gun permit was public by default in New York when the map was published (although a new law has been proposed that would make it private) but it was difficult to collect and so hardly anyone bothered. Other kinds of information are also technically public — government databases and so on — but difficult or impossible to extract useful data from.

Clicks you may not even remember are being analyzed

Of course, the same thing was the case with much of the world’s information before Google came along, and we had to learn to adjust to the fact “the internet never forgets,” and that the information you posted online years ago (or information that was posted about you by others) without really thinking of the consequences can come back to haunt you. But Facebook has taken that to a whole new level of intimacy because much of what occurs there seems so ephemeral: a “like,” a follow, a click — things you might not even remember doing.

While they may seem ephemeral, each of these can be as permanent as anything else on the internet, and just as public, unless you can master the intricacies of what Facebook lets you hide and what it doesn’t (and as Scott notes in his FAQ, you probably shouldn’t rely on that anyway). As Megan Garber of The Atlantic points out in a post, the social network is essentially constructing a virtual version of you out of all those signals — a version that is categorized by all of your activities and interests, some of which may be harmless and some of which may not.

It may seem absurd that someone might say they “like” racism, or that anyone would actually search for that behavior and make use of it somehow. But if we have learned anything from the era of big data, it is that if the information is available — as dirty or questionable as it might be — someone is going to make use of it, and not always in the way you would like them to.

Post and thumbnail images courtesy of Flickr user Alan Cleaver and Tom Scott

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

How to compete with Facebook in 2013

Mark Mulligan — Fri, 04 Jan 2013 17:47:31 +0000

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

Social-TV apps and consumer behavior

Nicole — Tue, 17 Jul 2012 06:55:18 +0000

A new category of digital media has emerged in the living room: social TV. This relatively new concept can be defined as any application, website or software that allows viewers to interact with television programming and share that interaction with others. Currently social TV occurs in three ways: organically, as pure play or through TV- or set-top-enabled communication. This report will focus on the pure-play aspect of social TV as it relates to content providers, television networks and advertisers. It answers key questions relating to the segment’s growth potential, important companies, their competitors and likely business models for the future.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

Fan service Moviepilot steams into Hollywood with $7m round

David Meyer — Thu, 10 May 2012 13:00:59 +0000

When you want to follow the development of a new movie, who do you turn to for news and rumours? And if you’re a film studio, how do you find out about the potential fans who are interested in your project?

Berlin’s Moviepilot.com, which is making its big push into the U.S., hopes to kill both birds with one stone. And, judging from the $7 million Series B round it revealed on Thursday, it’s armed with a boulder.

The round was led by new investor DFJ Esprit, along with Deutsche Telekom’s T-Venture, Grazia Equity and Creative Industries Berlin — all of which have already been backing the site’s German incarnation, moviepilot.de.

That German version is quite a different beast, however. It’s a Google-driven rumor and recommendation site that in turn powers Telekom’s T-Entertain IPTV package, and it’s very successful, with two million monthly uniques and 1.4 million Facebook fans. In fact, that makes it Germany’s most popular Facebook page, bar that for soccer team Bayern Munich.

But, as CEO and co-founder Tobi Bauckhage told me, it was only a first step:

“Based on the recommendations and based on this Facebook understanding, we said we had a chance from Berlin to create a new generation movie site that is concentrating on early fans, using the Social Graph and the Open Graph to learn about the fans, and kickstart recommendations based on that.”

So in October last year the startup opened Moviepilot.com, an English-language second iteration that’s plugged into a separate Facebook page that’s already amassed four million fans. Combined, the two services constitute the social network’s largest movie fan community.

Fans can use the service to discover new projects and follow them — one of Moviepilot’s many facets is that of a curated, personalised news site. And for the studios, the community can provide some sorely-needed marketing insight.

“We’ve always thought the real power of tech is not the distribution but discovery, not even of films, but discovery of audiences,” Bauckhage, a film-maker himself, said. “Discovery and recommendation is still supply-driven, but we’re moving to a model where you have to learn about your audience and build audiences.”

Audience insights

The problem, Bauckhage explained, is that the whole industry revolves around the opening weekend.

“It’s a supply-driven industry with a very limited shelfspace, with only a limited number of cinemas and screening rooms,” he said. “The anomaly of this industry is that you have a market entry phase of only 48 hours – this predetermines the money you’ll make in the whole lifecycle of the movie.”

So what do studios do? Generally, they launch a barrage of poorly-targeted advertising just before the all-important weekend. Moviepilot.com is in a position to tell them who’s following their projects, right from the first rumour to the release.

“For example, we had 15,000 fans for the Avengers before the film came out,” Bauckhage explained. “They use Facebook Connect to follow a project, and we get information back from the fans and understand what the fanbase is made of. We can see that, hypothetically speaking, they are 15 times more likely to be Buffy fans – we believe that the real dimension of how you understand audiences is not demographics, but taste.”

And the industry already seems to be lapping it up. Twentieth Century Fox is currently paying Moviepilot to promote the upcoming Abraham Lincoln: Vampire Hunter to early fans, and the studio knows it will get very granular data in return. Universal, Disney and Paramount are also clients.

So there you have it: Moviepilot is a pretty unique combination of fansite, rumour and news aggregator, and social marketing agency. Its new funding will allow it to open an office in Los Angeles this year, with a dedicated sales team in tow.

The likes of IMDB and Flixster should be watching this development with a concerned eye.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

Potential Facebook-Instagram impact

David Card — Mon, 16 Apr 2012 12:02:11 +0000

Don’t be too quick to think that Facebook is abandoning its HTML5 mobile strategy in favor of apps. As a defensive movie, acquiring Instagram would lock down Facebook’s strong position in photo-sharing, leaving little room for would-be competitors, but it gives Facebook few new weapons and no new revenue opportunities.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.