Word that Lucky Sort is shutting down and that several of its team are joining Twitter’s revenue engineering department suggests this is exactly what the acquisition aims to accomplish.

As it stands, companies use Twitter as a way to track how people are talking about them and maybe, if they’re really advanced, do some sentiment analysis. If they’re willing to pay a third party, Datasift and Gnip are more than happy to broaden marketers’ views to encompass the entirety of Twitter’s data, both real-time and historical. What companies really can’t do, though, is run their own advanced analytics about topics straight from the Twitter platform.

One view of the Lucky Sort dashboard

The value proposition from such a product should be obvious at this point. Facebook, Google and Yahoo all collect a lot of data about how people are using their platforms and what topics are trending, and they all offer it up via a variety of products targeting marketing types and the public at large. If Twitter wants to be taken seriously as a venue for advertising budgets and a platform for measuring the pulse of the nation, people need to be able to ask questions of its data without relying on an intermediary or the occasional Twitter blog post.

As a journalist, I’d love to have access to this type of tool to track trending topics in real time and spot possible stories as they’re happening. The appeal to marketers should be obvious. As IBM’s Erick Brethenoux told me recently, “[Marketers] talk a good game about social data. Very few actually leverage it effectively today.”

At Twitter, though, data is a slightly different beast than at other web companies. Twitter’s value lies largely in real-time data — topics can be peak, crest and all but vanish within a 48-hour window. This situation has hampered some of Twitter’s efforts to surface optimal search results, and it has spurred the decision to buy companies such as Backtype (for its streaming-processing Storm technology) and parallel-processing startup Ubalo.

The latter move, which happened last week, should help Twitter’s development team create new features without worrying about the intricacies of making them run — and run fast — across a cluster of machines. (You can learn a lot more about how companies such as Google, Facebook and Box are rethinking infrastructure to handle their unique data needs at our Structure conference next month in San Francisco.)

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Why the next front in big data might be psychological
• Finding the Value in Social Media Data
• Listening platforms: finding the value in social media data

Recently, researchers at the Children’s Hospital in Boston analyzed aggregated data on users’ Facebook activity and interests to examine the connection between online social environments and obesity prevalence. They found that areas with higher percentages of people with interests related to healthy activities and fitness had lower obesity rates, while populations with a greater percentage of people who had liked or commented on television was an indicator of higher obesity rates.

Interestingly, the study found that social data about sports in general was not correlated with obesity because people may be merely watching sports or following it, not taking an active role in it.

The researchers, who published their findings in the journal PLOS ONE, not only determined that Facebook data could track obesity prevalence, they suggested that social networks could be used to explore additional conditions and deliver health interventions and public health campaigns.

“I’d hope that people would look to this data source to understand how it can improve our understanding of chronic diseases and population-level conditions,” Rumi Chunara, an instructor at Harvard Medical School and an author of the study, told me.

Chunara’s study isn’t the only one to test Facebook’s value as a tool for public health research. A study published earlier this year found that hospitals with more Facebook “likes” have lower mortality rates and higher patient satisfaction scores. And, citing a search for the word “Facebook” on PubMed (a public database of life sciences research), a recent Wired article reported that there have been about 400 academic papers published in the last four years that include the social networking giant.

Traditional public health research often consists of phone surveys that can require considerable amounts of money and time, but Chunara said a major advantage of using Facebook data is that you can reach a wide swath of people quickly and at a low cost. Facebook also enables researchers to drill down to specific neighborhoods and that kind of fine-grained data can be difficult to come by, she added.

Additionally, Facebook can provide real-time data, as well as the opportunity to explore how interactions with friends and contacts and health messaging could influence user behavior, the study said.

Still, despite Facebook’s advantages when it comes to public health research, it’s important to bear in mind the limitations of web and social data. As a February Nature article on the flaws in Google’s flu-tracking techniques highlighted, social data doesn’t always mean what we think it does. For example, some researchers think that media hype about the flu this past season could have led to a volume of web searches for flu-related terms that was disproportionate to the actual threat.

But Chunara pointed out that there can be biases and issues with more traditional data sets as well. “Every data set has challenges and you have to definitely approach [them] carefully [so as] not be misled,” she said.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Social 2013: The enterprise strikes back
• Examining the rise of crowd labor platforms in 2012
• Social third-quarter 2012: analysis and outlook

Plenty of other social media monitoring tools “listen” to digital chatter by keyword to help companies track the sentiment of online conversations about their brand.  But Treato CEO Ido Hadari said his company goes further. Over four years and with $7.5 million in venture funding, Treato developed technology that collects and processes billions of natural language patient conversation in real-time, as well as qualifies the sources of those conversations.

“What Treato is doing is the next-generation of patient intelligence,” he said. “It’s about an in-depth understanding of patients and caregivers and their real-life personal experiences.”

To start, it analyzes health-oriented web sites to make sure that they include quality health content and personal experiences. To date, Treato has qualified 2,000 sites, including Topix.com, Drugs.com and smaller condition-specific health sites. Interestingly, while it includes public posts from Facebook, Hadari said, Treato does not include comments on Twitter because they found that 90 percent of its data is spam.

Once the company aggregates the commentary, it runs an analysis with its own medical ontology that organizes the unstructured social data into knowledge about patients’ personal experiences. To date, Hadari said, it’s analyzed 1.3 billion conversations from millions of patients about 11,000 drugs and 13,000 conditions.

Since launching in 2011, the company has focused on patient comments related to drugs — on Treato.com, anyone can look up thousands of different medications to learn about side effects, effectiveness, whether patients switched to alternatives and more. And a service launched in October enables pharma companies to monitor patient responses to their products.

Treato declined to share the number of clients that had signed on to the service but said, anecdotally, that its service has helped customers uncover the reasons behind a product’s poor adoption, uncover new uses for a medication and reposition a drug’s marketing plan. Treato could also serve as an early warning system for a drug with harmful side effects to a large population.

With a new $14.5 million round of financing from OrbiMed Israel Partners, New Leaf Ventures and Reed Elsevier Ventures, announced this week  (see disclosure), Hadari said the company plans to build up a presence in the U.S. and expand its “Facebook meets WebMD” concept to unearth patient insights about broader medical conditions or clinical trials.

It also plans to roll out a set of features that would let patients view a more personalized set of data. For example, instead of just seeing how thousands of patients have responded to a drug, a user could create a profile of their medical history and then filter data to see the results most relevant to his condition.

As patients go online to seek out others experiencing similar conditions and symptoms, they’re leaving a valuable trail of data that could lead to advancements in care. Lucine Biotechnology, a Las Vegas-based startup, is attempting to improve women’s healthcare by drawing connections between social comments online and information gathered through salivary hormone monitoring. Alliance Health Networks uses algorithms to analyze conversations in its online communities to better understand how patients influence each other and enable pharma companies and clinicians understand how patients make medical decision.

Disclosure: Reed Elsevier, an investor in Treato, is also an investor in GigaOM through its Reed Ventures arm.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• How consumer media will change in 2013
• Examining the rise of crowd labor platforms in 2012
• The state of cross-platform media measurement

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Connected world: the consumer technology revolution
• NewNet Q4: Platform mania and social commerce shakeout
• IT spending update, fourth quarter 2012

Of course, this being a Reddit discussion, Rasmussen answers a bunch of questions about the history of graph search, its privacy issues, his role with building Google Maps (and Wave) and his walks with Mark Zuckerberg. But here are some of the more-informative excerpts about the architecture itself.

I should point out, too, that we’ll be talking a bit about graph processing and graph databases at our Structure: Data conference next month, too. Graphs, as it turns out, are a great way to storing, processing and presenting a lot of data that has nothing to do with social connections.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• 10 ways big data changes everything
• 9 Companies that Pushed the Infrastructure Discussion in 2010
• How search can unlock the power of big data

The round was led by Battery Ventures and included Wellington Partners. In June, we reported that SumAll had raised a $1.5 million seed round from both of those investors, as well as Matrix Partners and General Catalyst Partners.

SumAll, which reports that it has 15,000 users around the world, helps businesses analyze and visualize a comprehensive set of their most relevant data. For example, an e-commerce company could use SumAll to analyze their sales data from Shopify, PayPal or Magenta against traffic data or Facebook or Twitter social data. At the moment, the company can integrate with more than a dozen sources. But founder and CEO Dane Atkinson said that, with the new funding, they plan to add new features and data streams to the platform, as well as potentially double its staff of 25 over the next year.

“Our customers dig what we’ve been doing so far and they want to see more of their data in the same place,” he said.

Other companies, including startups Custora and GoodData or Google Analytics and Adobe’s Omniture, may compete with SumAll on parts of its business. But Atkinson emphasized that SumAll focuses on helping smaller businesses aggregate disparate data sets and overlay them to reveal insights and patterns.

Despite its focus on small- to medium-sized businesses, he said the company has also attracted Fortune 500 companies. At the moment, SumAll tracks more than $1.5 billion in ecommerce data, 20 billion visits and 2 billion social actions.

The new funding also includes a $500,000 contribution to the SumAll.org foundation, a charitable trust which has a 10 percent ownership stake in the company and was established by SumAll employees.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Listening platforms: finding the value in social media data
• The importance of putting the U and I in visualization
• Facebook’s IPO filing: ideas and implications

Marketers spend a good chunk of their money on content marketing but when it comes to measuring how well that content is doing, it often comes down to counting tweets, retweets, posts, comments and likes. By tabulating friends and followers, marketers can estimate how much reach a message had. But it’s still harder to know what content is really getting clicked on and read the most.

By partnering with Bitly, which is used to share 80 million new links a day, DataSift can help its clients look at social conversations and click data together. That allows them to see what content actually drives the most interaction. This gives brands a way to create content that’s really engaging for users and also have a better way to measure their “return on influence.”

“Now, with Bitly, you don’t just look at links but you look at content. Now you can see who is engaging with stories and you can see which bits of news are resonating. Just because something was retweeted doesn’t mean someone clicked on it,” DataSift’s founder and CTO Nick Halstead told me in an interview. “This will change the perception of success.”

DataSift customers can now pull up live data on what stories are being shared and how many clicks they’re actually getting. They can filter by geography and see where the referrals are coming from and what keywords and titles are working. A company can now apply some SEO techniques to come up with new content that matches what people are clicking on or rework existing content and headlines to maximize engagement.

Halstead says the partnership makes sense for both companies. DataSift has a critical way to provide link analytics on social content while Bitly has a way to get more use out of its data and it can sell DataSift customers on its data stream. DataSift now has 300 big enterprise clients and just raised $15 million earlier this week led by Scale Ventures.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• What’s driving the next phase of the e-commerce evolution
• Going social: Recommendations engines need to factor in consumer reviews
• The future of mobile: a segment analysis by GigaOM Pro

In time for Halloween and the presidential election, the TechStars-backed startup, which enables companies to track and reward consumers for their engagement across social, mobile and online platforms, mined millions of social data points (primarily from Facebook) to look for connections between brands of chocolate and political parties.

They found that people who like non-premium chocolates, such as Hersheys, Reese’s and M&Ms are more likely to identify themselves as Democrats than those who prefer premium chocolates, such as Nestle, Dove, Ghirardelli and Lindt.

For example, consumers with an expressed preference for Lindt chocolate are 40 percent more likely to be a Republican and those who say they like Hersheys are 70 percent more likely to be a Democrat, CrowdTwist says.

Clearly, this isn’t the kind of data with election-changing implications and the connection between chocolate preferences and political affiliation likely just reflects differences in age. Younger people tend to skew Democrat and CrowdTwist’s data shows that younger consumers seem to prefer less premium (cheaper) brands while older consumers gravitate toward more expensive brands.

But it still provides an amusing reminder for brands that seemingly unrelated streams of social data can be combined to generate potentially helpful insights.

“[We] aggregate data from various disparate sources and give clients the ability to look at the correlations in data that they have not thought of in the past that they can use to not only find new audiences but understand audiences better,” said Adam Trisk, CrowdTwist’s head of marketing.

For example, he said, this sort of information could help brands determine where and when to buy media: Lindt may think about buying spots on Fox News while Hershey may seek ad buys during the  Democratic National Convention.

CrowdTwist, which raised $6 million in Series A funding last September, said it’s doubled its headcount to 25 since the end of 2011 and surpassed its 2011 revenue by August of this year.  Its clients range from television networks, such as Fox (for a campaign for the “X Factor” show), to sports franchises, like the Miami Dolphins and the NFL, to top brands in consumer packaged goods, like Pepsi, to VPG, a maker of wheelchair-accessible vehicles.

While brands have long valued loyalty, CrowdTwist really takes advantage of new platforms, such as social and mobile, to help brands comprehensively track engagement and reward consumers with real experiences and tangible items.

The startup not only gives data-hungry companies a broad picture of their consumers, it helps brands present themselves as a unified whole to consumers who interact with them in physical locations, online, on television and through other mediums.

“Companies think of their various channels in potentially more siloed, cordoned off verticals,” said co-founder and CEO Irving Fain. “But when a customer looks at that brand it’s all the “X Factor,” the Dolphins, etc. We provide the ability to unify customer interactions back to an individual, which brands can use to build better relationships and experiences with customers and fans – that’s what leads to loyalty.”

Image by Kesu via Shutterstock.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Social media in Q1: commerce and discovery dominated
• Facebook’s IPO filing: ideas and implications
• The future of mobile: a segment analysis by GigaOM Pro

Using her medical history and experience with the first birth control pill, the specialist recommended a more appropriate alternative and, since then, Winn said she’s been fine. But, having figured out the best medication for herself, the 23-year-old Harvard graduate is building a startup that leverages social data and natural language processing to help other women discover what could work better for them.

“The larger goal is to change the way doctors think about patients and change the way patients think about themselves,” she told me at the Health 2.0 conference this week in San Francisco. “It’s not one size fits all.”

Social data uncovers insights for a range of drugs

Part of TechStars Boston’s fall class, MyBetterFit aggregates comments from health forums and then parses out relevant information about treatments and side effects to create patient profiles against which users can compare their experiences.

Winn and her team are still building out the site, but the goal is for women to be able to answer a few questions about their current experiences with birth control, and ultimately their medical history, and then receive social-data-driven recommendations. A Las Vegas biotech startup, called Lucine Biotechnology, which my colleague Derrick Harris has written about, is somewhat similarly attempting to improve women’s healthcare by drawing connections between social comments online and information gathered through salivary hormone monitoring. And Alliance Health Networks, a Salt Lake City, Utah-based company recently covered by our colleague Stacey Higginbotham, is using machine learning models that have been taught using public medical research to uncover insights in online healthcare discussions.

So far, MyBetterFit has analyzed 100,000 comments from just two forums, eHealth and Topix, and already, Winn said, they’ve observed that for each drug there are about half a dozen cohorts of patients who respond similarly.

Down the road, Winn said, the plan is to expand the platform to patients taking less-understood medication for other conditions, such as autoimmune disorders like psoriasis, depression, ADHD and menopause.

Treato, a company based in Israel, similarly applies big data analytics to social data pulled from comments on thousands of blogs and health forums. But instead of giving patients an opportunity to personalize recommendations, it algorithmically calculates how many patients respond positively to a certain drug relative to others, as well as surfaces patients’ top concerns.

To date, the Reed Elsevier-backed company (see disclosure below) says it’s analyzed more than one billion online patient discussions and, earlier this month, it rolled out a new insights platform for pharmaceutical companies.  Similar to social listening platforms that let marketers observe how consumers are talking about their brands on Twitter and Facebook, Treato’s tool lets drug companies monitor patient experiences with their medications.

As genomic research develops, social data could be a complement

Even though Treato doesn’t personalize results, it still gives patients an important and quick window into the range of side effects caused by a specific drug.

As I wrote about earlier this week, sophisticated genomics research is working to personalize drug prescriptions and enable physicians to recommend medication most appropriate for a patient’s genetic profile.

As that technology develops, Winn said, phenotypic data collected by her startup will still be able to provide a valuable complement, as well as provide insight into reactions that aren’t related to genetic factors.

“We’re excited to be providing a service that can help everybody, right now,” Winn said. “But it would be natural in the future to pair our phenotype findings with genetic data to get an even more complete patient profile.

Disclosure: Reed Elsevier is an investor in GigaOM through its Reed Ventures arm.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Why the next front in big data might be psychological
• GigaOM Research highs and lows from CES 2013
• Listening platforms: finding the value in social media data

The Series C round, led by new investor Pelion Ventures and joined by current investors Flybridge Capital, Greycroft Partners, First Round Capital, iNovia Capital, Panorama Capital, QED Investors, Metamorphic Ventures, and Great Oaks Ventures, brings the company’s total funding to more than $26 million.

Since launching in 2008, the company said it’s been doubling and tripling revenues each year, for a compound annual growth rate over the past three years of 468 percent. Earlier this year, 33Across significantly expanded its data footprint with the acquisition of Tynt, which tracks when users cut and paste content for sharing via email and other social platforms.

Over the past few years, venture capital firms have poured billions into ad tech companies, but some say interest in parts of the marketplace is softening. In a recent Wall Street Journal article, Terence Kawaja, founder and CEO of boutique advisory firm and investment bank LUMA Partners said “this is the year of capitulation” for many ad tech startups, as the winners and losers become more clear.

Companies targeting users of social networks are continuing to get venture capital support, the WSJ noted. And, across the board, demand for nearly anything related to big data has soared.

“Everyone in this ecosystem – marketers, publishers and tech companies – they all have a huge appetite for big data right now,” said 33Across founder and CEO Eric Wheeler. “We’re at the epicenter of that.”

With the new funding, the company plans to accelerate its growth and engineering capabilities, he said. It intends to build out new and better products, as well as hire additional staff across the company. The New York-based company, which has offices in 11 cities, currently has 86 employees, but Wheeler said that figure could double in the next year. The company said its client roster includes upwards of 600,000 publishers and more than 375 Fortune 1000 marketers.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• NewNet Q4: Platform mania and social commerce shakeout
• NewNet Q4: Platform mania and social commerce shakeout
• Google doesn’t like walled gardens — except its own