In the eight years since John Ioannidis dropped the bomb that “most published research findings are false,” pockets of activist scientists from both academia and industry have been forming to address this problem, and it seems this year that some of those efforts are finally bearing fruit.

The research auditors

One interesting development is that a group of scientists is threatening to topple the impact factor, which ranks studies based on the journals in which they appear. This filter for quality research is based on journal prestige, but some scientists and startups are beginning to use alternative metrics in an effort to refocus on the science itself (rather than the publishing journal).

Taking a cue from the internet, they are citing the number of clicks, downloads, and page views that the research gets as better measures of “impact.” One group leading that charge is the Reproducibility Initiative, an alliance that includes an open-access journal (the Public Library of Science’s PLOS ONE) and three startups (data repository Figshare, experiment marketplace Science Exchange, and reference manager Mendeley). The Initiative isn’t trying to solve fraud, says Mendeley’s head of academic outreach William Gunn. Rather, it wants to address the rest of the dodgy data iceberg: the selective reporting of data, the vague methods for performing experiments, and the culture that contributes to so many scientific studies being irreproducible.

The Initiative will leverage Science Exchange’s network of outside labs and contract research organizations to do what its name says: try to reproduce published scientific studies. They have 50 studies lined up for their first batch. The authors of these studies have opted in for the additional scrutiny, so there is a good chance much of their research will turn out to be solid.

Whatever the outcome, though, the Initiative wants to use this first test batch to show the scientific community and funders that this kind of exercise is value-adding despite the costs, which are estimated to be $20,000 per study (about 10% of the original research price tag, depending on the study).

Gunn likens the process to a tax audit: not all studies can or should be tested for reproducibility, but the likely offenders may be among those that have high “impact factors,” much like high-income earners with many deductions warrant suspicion.

A stumbling block may be the researchers themselves, who like many successful people have egos to protect; no one wants to be branded “irreproducible.” The Initiative stresses that the replication effort is about setting a standard for what counts as a good method, and finding predictors of research quality that supersede journal, institution or individual.

The plumbers and librarians of big data

While the Reproducibility Initiative is trying to accelerate science’s natural self-correction process, another nascent group is working on improving the plumbing that serves data. The Research Data Alliance (RDA), which is partially funded by the National Science Foundation, is barely a few months old, but it is already uniting global researchers who are passionate about improving infrastructure for data-driven innovation. “The superwoman of supercomputing” Francine Berman, a professor at Rensselaer Polytechnic Institute, heads up the U.S. division of RDA.

The RDA is structured like the World Wide Web Consortium, with working groups that produce code, policies for data interoperability, and data infrastructure solutions. As of yet there is no working group for data integrity, but it is within RDA’s scope, says Berman. While the effort is still in its infancy, the broad goals would be to come up with a way to make sure that the data contained in a study is more accessible to more people, and also that it doesn’t simply disappear at a certain point because of, say, storage issues.  She says with data it’s like we’re back in the  Industrial Revolution, when we had to create a new social contract to guide how we do research and commerce.

The men who stare at data

You can build places for data to live and spot-check it once it’s published, but there are also things researchers can do earlier, while they’re “interrogating” the data. After all, says Berman, you’re careful around strangers in real life, so why jump into bed with your data before you’re familiar with it?

Visualization is one of the most effective ways of inspecting the quality of your data, and getting different views of its potential. Automated processing is fast, but it can also produce spurious results if you don’t sanity-check your data first with visual and statistical techniques.

Stanford University computer scientist Jeff Heer, who also co-founded the data munging startup Trifacta, says visualization can help spot errors or extreme values. It can also test the user’s domain expertise (do you know what you’re doing and can you tell what a complete or faulty data set looks like?) and prior hypotheses about the data. “Skilled people are at the heart of the process of making sense of data,” says Heer. Someone with domain expertise who brings their memories and skills to the data can spot new insights, and in this way combat the determinism of blindly collected and reported data sets. Context, in the form of metadata, is rich and omni-present, Heer argues, as long as we’ve collected the right data the right way. Context can aid in interpretation and combat the determinism of blindly collected and reported data sets.

The three-pronged approach — better auditing, preservation and visualization — will help steer science away from the iceberg of unreliable data.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• How big data analytics drives competitive advantage
• How data warehousing is now a cost-effective solution for businesses
• Cloud and data first-quarter 2013: analysis and outlook

Yet that’s exactly what a group of researchers at the University of Illinois have created — a cradle that wraps around an iPhone and turns it into a biosensor that can detect, according to a university press release, “toxins, proteins, bacteria, viruses and other molecules.” Inside that cradle are about $200 worth of mirrors, lenses and a photonic crystal that the researchers claim can identify these substances as accurately as a $50,000 spectrophotometer in the lab.

The cradle is essentially there for support, though, while the phone’s camera and processor do the real work. With everything firmly aligned in front of the camera, a scientist would simply snaps a photo and the CPU processes the result. What it’s processing is the difference in wavelength that the photonic crystal, primed to react to a specific molecule, reflects. The team demonstrates the device and app in the video embedded below.

And if you’re into this type of mobile data collection, another group of University of Illinois researchers actually created a smartphone-powered water-pollution device called MoboSens

Like all things mobile or sensor, though — from SkinScan (now SkinVision) to health care apps like Ginger.io — the biggest value might come from data that has nothing to do with what the app is primarily measuring. Rather, when data about a certain condition, air quality or what have you is tagged with time and geodata, for example, it becomes the basis for mapping how situations are spreading or where there might be safe haven.

Imagine a team of scientists with iPhones dispersed throughout a city after a disaster, painting a real-time picture of what areas are most affected by a particular toxin (or maybe radiation). Taking a longer term approach, researchers could track how situations are evolving over time. Throw in even more data that smartphones are capable of detecting — temperature, ambient noise, vibration, etc. — and we might unlock entirely new ways to think about how diseases spread through the air or what conditions tend to favor the spread of foodborne bacteria.

In some ways, though, this is more than another cool thing you can do with a smartphone. It’s the furtherance of something we’ll discussing in depth at our Structure conference next month, which is how we rethink IT when computation and data are no longer bound within a single server or even the corporate network somewhere. The biological data this app will collect isn’t much use locked inside the phone; it needs a way to reliably and securely connect with other datasets and other services, likely distributed across the country or even the world. That’s where the real opportunity lies.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Connected world: the consumer technology revolution
• 12 tech leaders’ resolutions for 2012
• Dissecting the data: 5 issues for our digital future

Kidding aside, that’s actually quite an ambitious goal in a Hadoop market that’s big and growing, but that’s exemplified by expensive consulting arrangements and purpose-built applications. Even more so for companies that want to do something other than transforming unstructured data into structured data (often called ETL) or run back-office analytics jobs. In fact, WibiData has spent the last 18 months doing just this type of deal, and Bisciglia says every single customer has already engaged with one of the big three Hadoop vendors (Cloudera, Hortonworks and MapR).

Home energy-management startup Opower is a good example of this process. It’s actually one of Cloudera’s banner customers, but “when they wanted to take [their software-as-a-service tool] beyond batch analysis and ETL workloads,” Bisciglia said, Opower came to WibiData. So whereas the Opower service was originally focused on nightly data analysis comparing users’ energy usage against that of other users, it’s now working on dynamic recommendations for users and letting them engage with the application in new ways.

The WibiData architecture

During these engagements, WibiData has been building up its core technology for connecting those brawny back-office Hadoop environments to predictive customer-facing applications – a collection of HBase, data-formatting tools and machine learning algorithms that the company has been slowly open-sourcing under the Kiji banner. It has also been learning the similarities among the applications it’s building for customers in the same field, figuring out what’s repeatable. What does any given company in the retail space, for example, need to get started on its own recommendation engine?

And now, Bisciglia says, WibiData is going to double down on building application software based on what it has learned. The first two industries it targets will likely be financial services and retail, two areas where the company has seen a lot of traction. He envisions the finished product including some pre-defined schema for formatting data and some pre-built predictive models, both broadly applicable across that industry rather than specific to a single user.

There will also be different interfaces that allow different types of users (e.g., data scientists, systems engineers and business users) to interact with the data in the ways they need to.

Time will tell if WibiData can actually accomplish its goal of turning Hadoop into a collection of somewhat specialized software packages, but someone has to. Even industry heavyweights like Cloudera see the need, but their hands are full just getting Hadoop integrated into existing environments and getting those early uses up and running. As Cloudera CEO Mike Olson said at Structure: Data in 2012 to anyone ambitious enough to tackle the Hadoop-application gap, “Call me, I’ll connect you with funding. The money is out there.”

If you want to hear more about the need for Hadoop applications, check out this panel from Structure: Data 2013, where I speak with WibiData’s Omer Trajman, Continuuity’s Jonathan Gray and Pivotal’s Muddu Sudhakar.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• A near-term outlook for big data
• Takeaways from the second quarter in cloud and data
• Why the big data startup boom will likely be short-lived

The reason for creating Pattern is pretty simple, according to Concurrent founder and CTO Chris Wensel: “Hadoop is never used alone.” It’s always part of a data environment that also includes databases, visualization tools, analytics software and/or statistical analysis tools that arguably do the really valuable work. Hadoop’s real value is an integration platform that can feed data into these other systems and, ideally, put their outputs to work across much larger datasets.

Developers can use the Pattern Java API to create machine learning jobs, but they can also simply export a Predictive Model Markup Language (PMML) file from software like R, SAS and MicroStrategy that Pattern will read and run them as a Cascading workflow. Models are useless unless you can run them in production, Wensel said, and Pattern lets them run across more data, stored in Hadoop, than you can use to build them with those other tools.

However, Wensel noted, “The real takeaway isn’t Pattern itself.”

From his perspective, the real story is Pattern plus Cascading plus Lingual, the open source SQL-to-Hadoop tool that Concurrent recently developed and released. Lingual is the tie that binds everything together, creating a sort of assembly line for data as it works its way from generation to delivering some value. For example, someone might create a Cascading job that adds structure to incoming data, and then pull some of the data into R using Lingual. Once a model is created in R and exported to the Hadoop cluster using Pattern, Lingual can feed the MapReduce output file back to R so a data scientist can test the model’s accuracy.

And actually, Wensel said, Lingual could have a positive effect on companies’ bottom lines. Airbnb recently replaced a departed engineer with Lingual for monthly migrations of data from Hadoop and into SQL environments. Climate Corporation, a massive Hadoop and Cascading user, could use Lingual to let its crop-and-weather insurance customers access their data from the company’s Hadoop store.

Lingual and Pattern should help Concurrent finally make some money, too. Both of them, as well as the Cascading framework that underpins them, will always be open source, Wensel said, but it plans to create “a suite of products that will make your life much better if … you standardize on Cascading.”

For example, the company has the ability to monitor jobs at the application level rather than the cluster level, meaning it can tell you the details of that job that’s locking up all the resources and whether you really want to kill it (it might be an important report for the CFO …). “We can do some really interesting things,” Wensel said.

Disclosure: Concurrent is backed by True Ventures, a venture capital firm that is an investor in the parent company of this blog, Giga Omni Media. Om Malik, the founder of Giga Omni Media, is also a venture partner at True.

This post was updated at 2:48pm PT to correct Chris Wensel’s title. He is CTO.

Feature image courtesy of Shutterstock user PENGYOU91.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Sector RoadMap: SQL-on-Hadoop platforms in 2013
• Cloud and data first-quarter 2013: analysis and outlook
• How to use big data to make better business decisions

That’s why a group of researchers from Columbia University and Stanford University developed an algorithm that can do just that, turning those cells into something that resembles your social graph. This lets researchers see how the various cells are related to each other so they know , for example, where to focus cancer treatment and what to track as that treatment progresses.

The idea of representing large or complex data as a graph is nothing new, but it has taken on more prominence thanks to the rise of social media and those ubiquitous social graphs that map out who’s connected to whom. As we highlighted recently, however, graph analysis is becoming more popular outside the realm of social networks, and is being applied to problems that are more complex than just figuring out simple relationships within a network. In cases such as medical research, especially, graphs can provide a very effective way of seeing how potentially hundreds of thousands of data points spanning perhaps hundreds of variables are similar to each other.

That’s exactly what the team at Columbia and Stanford has done with a new algorithm that they’ve demonstrated within the realm of mass cytometry. According to a press release announcing the research (which is available via paid download at Nature Biotechnology):

“The method, called viSNE (visual interactive Stochastic Neighbor Embedding), is based on a sophisticated algorithm that translates high-dimensional data (e.g., a dataset that includes many different simultaneous measurements from single cells) into visual representations similar to two-dimensional ‘scatter plots’ ….

“The viSNE software can analyze measurements of dozens of molecular markers. In the two-dimensional maps that result, the distance between points represents the degree of similarity between single cells. The maps can reveal clearly defined groups of cells with distinct behaviors (e.g., drug resistance) even if they are only a tiny fraction of the total population. This should enable the design of ways to physically isolate and study these cell subpopulations in the laboratory.”

I assume they say similar to scatter plots because the algorithm is analyzing data across more than two dimensions, although the resulting chart is essentially the same (i.e., data points with similar characteristics will form clusters).

The results of viSNE, showing cell densities in diagnosis and relapse samples.

Whether or not they’re technically similar, this research seems similar to what Ayasdi is doing with its new data-analysis software based on a technique called topological data analysis. In both cases, though, the algorithms aren’t necessarily concerned with how data points interact with one another (like in network graphs), but rather what similar characteristics the points share. Ayasdi’s software has been used in cancer research, too, including on datasets spanning hundreds of patients and tens of thousands of variables.

In theory — although not likely in practice considering the complexity of the datasets medical researchers are dealing with — these approaches are similar to clustering approaches that are also popular among data scientists working with web companies. In areas such as e-commerce or email management, for example, where there isn’t a strong social element, companies can broadly break customers into distinct groups based on their behavior or interests.

A sample cluster of MailChimp subscribers.

Of course, curing cancer is a slightly more compelling — and difficult — goal than targeted advertising. The algorithms have to be precise so as not to miss similarities hidden within the mass of data. In the case of viSNE, the researchers say they’ve been able to spot small groups of cells (like 20 out of tens of thousands) that might be able to survive chemotherapy and increase the likelihood of a recurring tumor.

But we probably shouldn’t bee too quick to discount the work that web companies do as somehow less valuable than that of cancers researchers, for example. The big data era arguably started with the web, and web companies have generated some of the most important data-analysis techniques and technologies around today (see, for example, Google’s Jeff Dean, with whom I’ll be speaking at our Structure conference next month). As medical researchers start generating more and more data via cytometry, genome sequencing and even electronic medical records, it will be critical for individuals in all fields to keep track of what data scientists in other fields are doing and figure out how that might apply to their own work.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Connected world: the consumer technology revolution
• A near-term outlook for big data
• NewNet Q4: Platform mania and social commerce shakeout

President and COO George Mathew says the company’s mission is to be a one-stop shop for statistical analysis. It wants to be the one place where analysts and data scientists can blend their data, model it on it and then visualize it. Often, he noted, that same process might require two or three separate products.

Another feature that Alteryx hopes will set it apart is its collection of prebuilt models in what the company calls an analytics gallery. Users can share their own work or find models others have built for tackling similar issues. Alteryx also offers up its own pre-formatted datasets for analysis, often public data such as the U.S. census.

“The canvas for creating an analytics application should never be blank for the analyst when they’re getting started,” Mathew explained. They often need to understand external data as well as their internal data, so Alteryx’s software gives them easy access to it.

Because it’s based on the R statistical-programming language, heavy R user Walmart has been able to transition some workloads to Alteryx when employees need an easier user experience. McDonald’s uses it to analyze data about franchisees and about its growth strategy in China, and Bloomin’ Brands (parent of company of Outback Steakhouse and other restaurants) is using it to help build menus that take into account what diners in various parts of the country prefer to eat. Nine of the 10 leading top wireless providers providers are also users, Mathew said, trying to blend actual call data with traditional sources such as customer service data.

Mathew compares Alteryx’s current growth as analogous to software-as-a-service applications like Salesforce.com in the CRM space, or even Tableau in the traditional business-intelligence space. In a business world increasingly driven by at least the idea of big data, one might expect any vendor pushing a more consumer-like purchase and consumption experience to get interest from companies tired of dealing with legacy software or never wanting to experience it in the first place.

“The disruption that’s happening is creating a new space for ourselves,” Mathew said, “without having to go head to head, frankly, with the a status quo out there.”

Feature image courtesy of Shutterstock user ramcreations.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Sector RoadMap: Social customer service in 2013
• The Internet of things: creating tomorrow’s health care
• Dissecting the data: 5 issues for our digital future

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• How data warehousing is now a cost-effective solution for businesses
• How to use big data to make better business decisions
• How to manage big data without breaking the bank

“We’re thrilled,” Tableau co-founder and CEO Christian Chabot told me during a call after the market closed. One should hope so.

Chabot and his fellow co-founders stand to make a lot of money if today’s closing price holds up, as does its sole investor NEA. The firm put $15 million into Tableau since it launched in 2003, and has rode that sum to profitability and more than $127 million in annual revenue.

Here’s a quick chart (made using Tableau Public) showing who owns how many share and what they’re potentially worth.

The company didn’t really need more capital to operate, Chabot said, but one of the primary drivers was to raise awareness of the company. It has about 12,000 customers, he said, but there are millions more possible users. As part of attracting them, the company is going to expand globally and is working to improve its reach across mobile devices, the cloud and the Mac operating system.

“I don’t believe in the this whole ‘or’ philosophy with computers,” Chabot said. “It’s ‘and’” — meaning people will use desktops and tablets and smartphones.

More prominence and more users singing its praises might also dispel the notion that Tableau is just about visualization. It has some fairly advanced features under the covers (as a commenter to my earlier post about the company’s influence pointed out), even if they’re hidden by the relatively simple user experience.

“Tableau is not a visualization company, per se, it’s really an analytics company,” Chabot said.

However, if the company really wants to expand its reach to everyone one who wants to gain knowledge from data — something Chabot calls a “timeless human need” — it might actually need to get simpler. More marketing can let potential business users know about new features like forecasting and data-extraction, but it won’t make a dentist is Des Moines better at formatting his data.

After raising $254 million in its IPO, though, Tableau is in a good place to do whatever it has to.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• NewNet Q4: Platform mania and social commerce shakeout
• NewNet Q4: Platform mania and social commerce shakeout
• How data warehousing is now a cost-effective solution for businesses

The company’s product, Spire, which provided full SQL support on top of the HBase NoSQL database, was one of the first products to try to blend Hadoop’s scalability with the robustness and familiarity of SQL. That’s now an increasingly crowded space (and has grown since that linked graphic was created). In March, Drawn to Scale expanded its support to MongoDB, as well.

I wasn’t shocked when Stephens told me the news — questions about the four-year-old company’s financial health had been swirling for a while — but to hear of its financial woes was a bit surprising. His account in the post pretty much echoes what I had heard from others:

“It seemed we had everything going for us — paid customers such as American Express, Orange Telecom, Flurry, and 4 others. Our technology worked brilliantly, we had a big hiring pipeline, and we had great media presence against our competitors who raised 10-100x more cash.”

He added:

“Yet five days before we signed term sheets for a big A round or sold the company, we started getting hit by a series of black swans — and we just didn’t have what we needed to recover. I’ll leave the public detail at that level, but I will say that paying employees’ health insurance out of your meager savings is a powerful incentive to change course.”

Up to this point, the company had raised $925,000 from RTP Ventures, IA Ventures and SK Ventures. There’s no word yet on what will come of the company’s intellectual property.

As Stephens — who’s now doing an entrepreneur-in-residence gig at Ping Identity and helping out other startups (including popular wardrobe app Cloth) — succinctly put it during a phone discussion, “We just don’t have the horsepower to keep running the company.”

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• How data warehousing is now a cost-effective solution for businesses
• Sector RoadMap: SQL-on-Hadoop platforms in 2013
• How to use big data to make better business decisions

That’s an apt ticker symbol for a company that is in some ways a bellwether for the current fascination with all things data. Tableau isn’t a big data company, per se, but its visualization software breathes life into many big data calculations. Its focus on making software that’s easy to use and that creates visually captivating charts has turned people from numerous professions into amateur data analysts. (I’ve even used it in the past, including for the first time in 2011.)

Christian Chabot

As Chabot told me during a conversation in 2011, “In any field of human endeavor … there are a hundred to a thousand more people who understand the data of that field more than they understand reporting and analytics.”

Anytime you read about a hot new visualization or analytics startup promising the moon, you’re also seeing the results of what Tableau has sown in terms of the user experience. Many of those same companies will be quick to tell you how limited Tableau’s capabilities are. It’s memory-bound, it doesn’t have a database, it’s not available in the cloud (or on the Mac operating system), it can’t do predictive analytics. All true.

Of course, if it raises the kind of capital it expects to by going public, it can build and buy a lot of those capabilities. If pricing stays flat all day Friday, Tableau stands to make $155 million from its 5 million shares. Previous estimates had Tableau’s market cap at around $1.7 billion at a price of $29 per share (the company’s S-1 filing is available here).

If investors have really bought into the company and the concept of a data-driven world, then who knows. Machine-data expert Splunk wnet public in 2012, flying the big data banner, and saw shares peak at 91 percent above its original asking price of $17.

I’m not suggesting Tableau is the biggest name in data, or even that it will some day become it. This next-generation analytics field is very young, with startups and larger vendors alike sometimes competing against themselves to win wholly new accounts than trying to displace legacy vendors within large enterprises. And every month, it seems, I come across some new startup that was built with the same principles in mind as Tableau, but with the advantage of having today’s best practices baked into its software.

But Tableau definitely commands a lot of the mindshare. How it fares as a public company could be a strong indicator of just how powerful the data movement is, and how well it capitalizes on a new influx of cash will determine how long it stays on the top of customers’ minds.

This post was updated at 7:01 p.m. to include previous estimates of the company’s market capitalization and a link to its S-1 filing.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• How data warehousing is now a cost-effective solution for businesses
• The importance of putting the U and I in visualization
• 4 iPad apps to help wrangle data