The company is launching a set of tools aimed at brands that want to measure social influence, and in many ways, this makes a lot of sense. While an average user might be curious about his or her reach on Twitter or Facebook, understanding this data is much more valuable to a business than a consumer, and likely something Klout will better be able to monetize.

The company explained in a blog post that it’s already begun measuring the social influence of brands through its Klout Perks program, but now it will target those businesses directly:

“Today, we are taking the data-driven intelligence we’ve developed over the years to begin a more measured march towardKlout for Business. Initially, Klout for Business will give businesses a complimentary set of analytics with pointed insights into how and where influencers are engaging with their brands in social media.

Businesses will be able to look at an easy-to-read dashboard that tells you, at-a-glance, whether you are engaging your influencers on the networks where they are most actively exerting their influence and on which Klout Score ranges you could stand to amp up your efforts. Most importantly, Klout can tell you which topics your audience influences others on, helping you maximize your content efforts to drive consideration for your brand.”

The company last tweaked the formula for its Klout scores in August 2012, and will now launch a specific page for businesses on Klout. Customers will gain access to a dashboard where they can monitor social interactions and communicate with customers.

Klout has been in business for almost five years, and it’s unclear that the company has converted average consumers — or even tech-savvy social-media addicts — into caring about their Klout scores or achieved real traction as a measuring stick for social influence. The company wrote in the blog post that it has acquired a vast amount of social data that it’s now ready to use for other purposes, but it’s possible the company has simply reached the limit of its own influence with consumers.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Listening platforms: finding the value in social media data
• Pinterest: signs of staying power
• Facebook’s IPO filing: ideas and implications

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Controversy, courtrooms and the cloud in Q1
• Takeaways from mobile’s second quarter
• Social media in Q1: commerce and discovery dominated

The most recent example comes by way of an outfit called Unmetric that sent me a “full report” that shows the Dallas Cowboys have the most Twitter and Facebook followers while the Chicago Bears are the “most engaged” on social media. This feat of research also included a nifty infographic with bar charts and more:

Unmetric, of course, is far from the only company cooking up rankings from online media. There’s Klout to rank people like Biebs and Obama as well as rivals like PeekYou which offers more rarified fare like the rank of the Supreme Court justices:

At this rate, it won’t be long until a company comes along to offer multi-media rankings for the neighborhood cats.

All of this raises the question of whether there is anything useful about all these studies and metrics. Does it matter that Bears fans are more “engaged” than Giants fans? Is it helpful to know that Clarence Thomas ranks #3 for web presence even though he’s regarded as sub-par jurist?

According to PeekYou CEO Michael Hussey, the point of the company is to “explore the web from a people perspective” and to show their media influence.

While some of these stats are no doubt useful to marketers, it remains to be seen if media/social media rankings will grow into a useful science or if they’re just a fad to feed our insatiable craving for rankings.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• GigaOM Research highs and lows from CES 2013
• How HR can make the case for workforce analytics
• The 2013 task management tools market

Aspiring heirs to the Klout throne, for example, might look to a project called STINGER currently under development at Georgia Tech University. STINGER, which stands for Spatio-Temporal Interaction Networks and Graphs Extensible Representation, is a graph-processing engine that project lead David Bader says is bigger, faster and more flexible than anything currently in use for analyzing social media connections. You provide a shared-memory computing system, and it provides an open-source tool that can help detect relationships between billions of people, places and things as those relationships change over time — even in real time.

Someone using Facebook data, for example, might write an algorithm using where people or pages would be the vertices and actions (likes, shares, wall posts, etc.) would be the graph’s edges. One relatively easy application, Bader explained, would be to analyze how activity around particular people is increasing, decreasing or changing, therefore indicating changes in their importance or the growth of new communities.

We’ll do the hard work

Writing an algorithm to perform that kind of analysis isn’t really the problem, though — it’s writing one that can scale into the billions of vertices and edges and still perform quickly enough to be useful. An algorithm that generates one false positive in a million isn’t so bad when you’re dealing with tens of thousands of items, Bader explained, but it gets to be a big problem when you’re talking about billions of items against which it’s running.

There are dozens of open source graph databases available, including popular offerings such as Neo4j and InfiniteGraph, but he said, “Our lab focuses on algorithms that run fast on massive data sets and that are more accurate than what is traditionally done in social media.”

David Bader

Bader’s team recently presented a paper detailing a social media algorithm running atop STINGER that ran 100 times faster than some previous approaches because the system stores the graph’s previous state and only performs the minimal amount of processing necessary as new edges are inserted. This is in contrast to traditional approaches that re-process the entire graph every time there’s a change.

That being said, Georgia Tech isn’t entirely alone analyzing massive amounts of social data with graph databases. Google’s Pregel had already scaled to billions of vertices and edges as of 2009, and Facebook is currently analyzing more than a billion edges using Apache Giraph (an open source, Hadoop-based Pregel implementation). But those cases — both companies are loaded with smart engineers, data scientists and powerful infrastructure — just underscore the importance of what researchers like Bader are building and releasing as open source.

Forget social media, solve real problems

But social data isn’t just useful for figuring out who’s influential on Twitter or Facebook — it also can be used to solve some real problems. Bader said he’s already used graph processing with Twitter data to determine who was leading resistance units during Egypt’s recent revolution. ”Anywhere I can look at connections between entities,” he said, “these approaches are available.”

Indeed. On Wednesday afternoon, for example, a group of researchers from the University of Alberta, University of Connecticut and University of California-Merced unveiled a new data-based method that could make it faster, easier and less expensive to root out culprits in fraud cases.

The technique uses a method called the Steiner tree to analyze the connections –social, business, familial, etc. — between the people involved in a given case of fraud. The algorithm is able to determine the shortest path between two objects, which the researchers posit is especially applicable to fraud investigations — the person with the shortest path between himself and the crime is probably the culprit (or at least a solid suspect).

The fraud researchers’ paper follows the publication in August of a method for determining the source of everything from a disease outbreak to a Twitter rumor by tracking its spread across a complex network over time. Their algorithm, the paper’s authors claim, could be particularly effective for combating cybercrime by tracking computer viruses back to their sources. The more connections (in the case of social data), or observers, a particular point has, the fewer that are needed to track down the source point.

However, all the algorithms and data frameworks in the world probably won’t make too big a difference until they’re turned into products that actually work on real-world situations. As the University of Alberta’s Ray Patterson pointed out in a press release detailing the fraudster-detection algorithm, ”It might take several years or many years before anyone picks it up. But it’s a good thing if we can point people towards what’s useful.”

Georgia Tech’s Bader said DARPA, Intel, Sandia National Laboratory and other research institutions have already used STINGER to tackle some complex data sets, and he suspects a strong commercial interest, as well. If a company is willing to take STINGER from a project into a product, it could bring the project’s scale and speed to everything from analyzing customer interactions to monitoring the changing nature of criminal networks, Bader said. Considering the desire from companies of all types to extract some meaning from social data, I have to think someone will give it a shot.

Feature image courtesy of Shutterstock user 3DProfi.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Connected world: the consumer technology revolution
• 12 tech leaders’ resolutions for 2012
• NewNet Q4: Platform mania and social commerce shakeout

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Who and what to watch in the new era of the living room
• 2012: Data, spectrum and the race to LTE
• CES 2012: a recap and analysis

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Who and what to watch in the new era of the living room
• Connected world: the consumer technology revolution
• What the shift to the cloud means for the future EPG

Not quite.

While such accounts are a pain — and Twitter spends significant resources trying to hunt them down — a new study says that the most egregious promoters of Twitter spam could actually be the likes of Lady Gaga or President Obama … and even you.

The study, from researchers at the Max Planck Institute for Software Systems in Germany, examined more than 40,000 identified spam accounts, and looked into how they tried to achieve success.

“Pretty much everyone on Twitter is targeted by spammers,” says Krishna Gummadi, head of the networked systems research group at MPI-SWS. “But the key question for us was ‘why would someone follow a spammer?’. So rather than focus on spammers themselves, we looked at their support network — the people who helped them.”

And what they discovered was that instead of being supported by other spam accounts — as is common with web-based spam — the egg people were reliant on real Twitter users for their success.

How it works

At the heart of Twitter’s spam problem are two things: the way the social messaging service works, and the way search engines think about the site.

First, Twitter prominently displays follower counts as a crucial measurement of popularity — something Google and others use that number to help determine somebody’s authority. Then, there’s a secondary effect: if you are followed by somebody who is followed by lots of people (what the researchers term somebody’s “indegree”) then your influence increases even more.

This leads to a kind of Twitter-based link farming, where spammers don’t simply target small-time users with just a few friends, but larger accounts with thousands — or sometimes millions — of followers. And unlike the web, where people with high reputation tend not to link to nefarious, spammy sites, many Twitter users automatically follow back anyone who comes their way.

“At the high end, politicians like Barack Obama’s campaign staff, or the UK Prime Minister’s office, these guys tend to follow back because they want to increase social engagement. But they’re following spammers back,” says Gummadi.

That top tier of users is topped by Yoko Ono, who boasts more than 775,000 people on her following list, but includes Gaga (currently following 139,000 people), Britney Spears (416,331).

But perhaps more dangerously, the study found that it was those in the middle of the scale who were causing the most damage by handing spammers credibility.

“The people with very low indegrees of just a few people were not responsible for spammers,” he says. “It was really people with 1,000 to 5,000 followers who were worst — that’s the range where there are a lot of people who follow back anybody. These are the social marketers.”

It’s a well-known grey hat tactic that following more people will increase your own follower count, and many people who try to profit will inflate their figures by following as many people as they can. But those individuals who are just trying to bump up their numbers are not just hurting their clients — they’re ruining search results for the rest of us.

“They want to increase their social capital and they have very similar incentives to spammers,” says Gummadi. “Spammers want to collect links, social marketers want influence. So they end up colluding with each other, even if not intentionally.”

You ain’t no follow back girl

Having identified the most dangerous Twitter users, Gummadi’s team — which included researchers from IIT Kharagpur, India, and UFOP in Brazil — have started exploring ways to reduce the impact of indiscriminate follow-backs.

That’s complicated, because it involves trying to change people’s behavior. But the researchers suggest it may be possible to implement some sort of “collusion rank” to sift out those people who are giving spammers oxygen and demote them. Just like Google’s pagerank can change depending on a site’s behavior, so too could their Twitter ranking if they link to too many spam accounts.

The team recognizes, however, that this may be tricky — because the line between helping a spammer and trying to be a good social network user can be very, very fine. After all, imagine how much time it would take Barack Obama’s team to sift each potential follower for spamminess — and the outcry if there was some sort of incorrect line drawn.

Whatever the case, it is clear to the research team that fighting Twitter spam needs to be about more than just shutting down accounts.

“We’re only beginning to understand spam in Twitter, and we’re sketching a solution,” says Gummadi. “But it does not lie in just banning spammers, because it doesn’t stop the social marketers.”

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Web startups: How to guard against security breaches
• The 2013 task management tools market
• How consumer media will change in 2013

Matt Thomson, Klout’s VP of platform, says that a number of major companies—airlines, big-box retailers, hospitality brands—are discussing how best to use Klout scores. Soon, he predicts, people with formidable Klout will board planes earlier, get free access to VIP airport lounges, stay in better hotel rooms, and receive deep discounts from retail stores and flash-sale outlets. ‘We say to brands that these are the people they should pay attention to most,’ Thomson says. ‘How they want to do it is up to them.’

Also, the only person with a perfect-100 Klout score is Justin Bieber.

Photo courtesy of Flickr user [kossy@finedays]

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Takeaways from mobile’s second quarter
• Social-TV apps and consumer behavior
• Is Quora Worth the Hype?

It’s easy to categorize friends and co-workers who are concerned with their Klout scores or number of Twitter followers as vain, but the truth is that this stuff matters. If you’re a marketer looking to get your message in front of the right people, or a politician trying to stoke the campaign fire, figuring out which voice will best help spread your message is a lot easier than trying to connect directly with the entire tweeting world.

I was reminded of the importance of social influence this morning when reading a post by Greg Finn on SearchEngineLand about the Microsoft Bing search engine, and how its algorithms factor in users’ social media activity “to merge a little bit of the search and browsing intent into one.” What Bing is trying to do is figure out what users are interested in and, therefore, what they’re actually trying to find with their search. (This is something Google is trying to do as well, despite surveys showing people really don’t like the idea of personalized search.)

More interesting on the analytics front is how Microsoft determines how social media will influence search results. Facebook friends, of course, get special treatment (friends’ endorsements have been compared to celebrity endorsements in court), but Microsoft tries to determine author influence for Twitter using the following factors (quoting Finn):

• Ratio of followers to following
• Retweets
• Sharing (as a strong[er] indicator than just a like)
• Having good followers
• Following good accounts
• Tweeting about relevant topics
• Authority and Relevance of Retweeters

Some of these factors no doubt go into the Klout algorithm as well. There, business partners pay Klout to put their “perks” in front of the right set of people. The theory is that these influencers will try the products or services and, if they like them, express their feelings via social media.

I assume many companies are already taking similar approaches to using Twitter as a marketing campaign. Step 1 might be finding out how people feel about a particular product, show, etc., by analyzing the Twitter firehose. But Step 2 should be finding out which Twitter users are influential in that space and trying to make them happy. Or maybe part of Step 1 is weighting sentiment based on who expressed it — an influential voice coming out in support of or against something might be worth more than someone with relatively low influence in determining how something will play out.

But social media influence has import beyond just selling stuff: It also could help decide elections. It’s no secret that presidential campaigns are deeply engaged in social media and are mining the data those efforts generate. And, as my colleague Stacey Higginbotham recently reported, the Obama campaign is already using tools like Hadoop to analyze unstructured-data sources that include social media. As their methods get more sophisticated, it seems very likely that campaigns could start trying to identify and target social media influencers as well.

The whole tech world doesn’t need to follow you, support you and retweet your tech policies if @Scobleizer does.

As just about every discussion at our Structure:Data conference next week will address, big data ultimately boils down to efficiency. Companies and organizations want to learn as much as they can so they can get the most bang for their buck. There’s nothing more efficient than letting someone else do the work for you.

Image courtesy of Ronald S. Woan.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• How search can unlock the power of big data
• A near-term outlook for big data
• 12 tech leaders’ resolutions for 2012

Considering the affinity most web companies have for open source software, the heavy use of Microsoft technology within Klout is a bit surprising. The rest of the Klout stack reads like a who’s who of hot open source technologies — Hadoop, Hive, HBase, MongoDB, MySQL, Node.js. Even on the administrative side, where open source isn’t always an option, Klout uses the newer and less-expensive Google Apps instead of Microsoft Exchange.

“I would rather go open source, that’s my first choice always,” Klout VP of Engineering Dave Mariani told me during a recent phone call. “But when it comes to open source, scalable analysis tools, they just don’t exist yet.”

Klout’s stack is telling of the state of big data analytics, where tools for analyzing data processed by and stored in Hadoop can be hard to use effectively, and where Microsoft has actually been turning a lot of heads lately.

How Klout does big data

“Data is the chief asset that drives our services,” said Mariani, and being able to understand what that data means is critical. Hadoop alone might be fine if the company were just interested in analyzing and scoring users’ social media activity, but it actually has to satisfy a customer set that includes users, platform partners (e.g., The Palms hotel in Las Vegas, which ties into the Klout API and uses scores to decide whether to upgrade guests’ rooms) and brand partners (the ones who target influencers with Klout Perks).

As it stands today, Hadoop stores all the raw data Klout collects — about a billion signals a day on users, Mariani said — and also stores it in an Apache Hive data warehouse. When Klout’s analysts need to query the data set, they use the Analysis Services tools within SQL Server. But because SQL Server can’t yet talk directly to Hive (or Hadoop, generally), Klout has married Hive to MySQL, which serves as the middleman between the two platforms. Klout loads about 600 million rows a day into SQL Server from Hive.

It’s possible to use Hive for querying data in a SQL-like manner (that’s what it was designed for), but Mariani said it can be slow, difficult and not super-flexible. With SQL Servers, he said, queries across the entire data set usually take less than 10 seconds, and the product helps Klout figure out if its algorithms are putting the right offers in front of the right users and whether those campaigns are having the desired effects.

Analysis Services also functions as a sort of monitoring system, Mariani explained. It lets Klout keep a close eye on moving averages of scores so it can help spot potential problems with its algorithms or problems in the data-collection process that affect  users’ scores.

Elsewhere, Klout uses HBase to serve user profiles and scores, and MongoDB for serving interactions between users (e.g., who tweeted what, who saw it, and how it affected everyone’s profiles).

Why Microsoft is turning heads in the Hadoop world

Although Klout is using SQL Server in part because Mariani brought it along with him from Yahoo and Blue Lithium before that, Microsoft’s recent commitment to Hadoop has only helped ensure its continued existence at Klout. Since October, Microsoft has been working with Yahoo spinoff Hortonworks on building distributions of Hadoop for both Windows Server and Windows Azure. It’s also working on connectors for Excel and SQL Server that will help business users access Hadoop data straight from their favorite tools.

Mariani said Klout is working with Microsoft on the SQL Server connector, as his team is anxious to eliminate the extra MySQL hop it currently must take between the two environments.

Microsoft’s work on Hadoop on Windows Azure is actually moving at an impressive pace. The company opened a preview of the service to 400 developers in December and on Tuesday, coinciding with the release of SQL Server 2012, opened it up to 2,000 developers. According to Doug Leland, GM of product management for SQL Server, Hadoop on Windows Azure is expanding its features, too, adding support for the Apache Mahout machine-learning libraries and new failover and disaster recovery capabilities for the problematic NameNode within the Hadoop Distributed File System.

Leland said Microsoft is trying ”to provide a service that is very easy to consume for customers of any size,” which means an intuitive interface and methods for analyzing data. Already, he said, Webtrends and the University of Dundee are among the early testers of Hadoop on Windows Azure, with the latter using it for genome analysis.

We’re just three weeks out from Structure: Data in New York, and it looks as if our panel on the future of Hadoop will have a lot to talk about, as will every at the show. As more big companies get involved with Hadoop and the technology gets more accessible, it opens up new possibilities who can leverage big data and how, as well as for an entirely new class of applications that use Hadoop like their predecessors used relational databases.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• 12 tech leaders’ resolutions for 2012
• How search can unlock the power of big data
• The importance of putting the U and I in visualization