Storage, databases, query processing and algorithms are all extremely important — heck, visualization is next to nothing without them — but in a data-driven world where is obsessed with insights, they’re just the foundational layers. They are to big data what server and network configurations are to mobile-app development on platforms like Parse. If you’re going to find out new things from massive and highly complex data sets, or going to give new types of people the ability to analyze even simple data, the presentation of that data and the ability to create consumable presentations are critical.

With that in mind, here are six startups I’ve seen trying to fundamentally change the way that data is visualized. Some are highly complex under the covers, some are not and none are perfect, but they’re all doing their part to make us rethink what it means to look at data and make spreadsheets and static charts look like relics. (And this list is by no means exhaustive, so feel free to add your favorite visualization tools in the comments.) We’ll be highlighting data visualization at our design-focused RoadMap conference in San Francisco in November (sign up here to get first access to tickets this Summer).

Ayasdi

The idea of network graphs isn’t new, but Ayasdi’s approach to it is. Under the covers, there’s an HBase data store, a technique called topographical topological data analysis and hundreds of machine learning algorithms to churn through complex data sets and determine the similarity among the data points. To the end user, though, there’s a map of the data set that looks a lot like a network graph (only it’s probably not network data) highlighting clusters of related data points that analysts might want to investigate further.

BeyondCORE

BeyondCore actually operates under the same basic premise as Ayasdi — show users the significant correlations so they don’t have to think of the queries that will uncover them — but it uses some different techniques to get there. It uses a different visualization method, too: BeyondCore sticks to standard charts, but actually offers the option of having an avatar talk users through the correlations the software has discovered.

ClearStory

ClearStory has a pretty unique product in the works — even if it’s keeping many details and all of its screenshots under lock and key until its formally launches. Essentially, though, it’s trying to tell stories via visualizations that display mashups of numerous data sources, update automatically when the source data changes, and invoke collaboration and social concepts. Here’s Co-founder and CEO Sharmila Mulligan explaining the idea behind ClearStory at Structure: Data in March.

Datahero

Unlike so many data startups, Datahero isn’t trying to woo people fed up with business-intelligence software or the difficulties of getting insights from Hadoop data. Rather, it’s trying to let people with simple business or personal data make simple charts without ever having to enter an Excel function or worry too much about how their spreadsheets are formatted. Early on, Datahero’s visualizations are still pretty commonplace (bars, pies, plots, etc.), but it’s the ease of creating them that’s so unique.

Platfora

Platfora has undertaken the ambitious task of trying to make analyzing mountains of data stored in Hadoop clusters as easy as analyzing their own Stripe data might be for developers using Datahero. It’s based on a foundation of Hadoop and massively parallel query processing, but is presented like an HTML5 version of current visualization golden boy Tableau that’s all about dragging, dropping, and visually slicing and dicing through data. The latter capability is actually critical in a big data world where there are likely more data points than you can ever digest at once.

Zoomdata

Zoomdata is far from the only analytics company to support mobile devices, but it’s one of the few I know of (Roambi also comes to mind) designed primarily for them. Zoomdata connects to standard business data sources, but takes advantage of touch screens and the D3.js visualization project to offer up some visually interesting charts that are designed to be manipulated like an artist’s palette.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• The importance of putting the U and I in visualization
• How data warehousing is now a cost-effective solution for businesses
• Cloud computing and trickle-down analytics

Once he finds a suitable topic, Feilding Cage, a New York-based developer and journalist for The Guardian, can easily spend three weeks generating the source information and designing a visualization for what’s become known as data journalism. The results bring understanding and reader engagement to topics that are otherwise discussed with a lot of words or static numbers. Readers can and do play around with the information, share it widely and discuss it for long periods after it appears online.

The Guardian’s interactive guide to gay rights in the United States

Along with The Guardian, a few other news organizations have been putting an emphasis on data-driven reporting and visualizations, apps and even games in the past few years, such as the Chicago Tribune, the Los Angeles Times and ProPublica (Check out the Data Journalism Handbook for more information on this sort of work.)

Data journalism and visualization stand out for the verification and occasional gray-area explanations that journalists provide. Cage, for example, accompanied his interactive visualization of gay rights in the United States with a blog post explaining his methodology and disclosing his assumptions.

Screenshot from the Zoomdata’s big data analytics iPad app

It would be neat to find a happy medium for enterprises that want original insights that every employee can see and use and act on but doesn’t take three weeks to generate. That’s especially true because the return on investment for work like Cage’s is hard to identify, although it’s possible the content could indirectly generate revenue by driving users to content they have to pay for.

Bridging the gap might be a matter of finding the perfect data scientist for the company. Or it might be a matter of time before the kind of work Cage does is automated. A computer already can write an earnings story, although it might be a few years before computers put wordsmiths out of business.

Maybe it just doesn’t make sense to cross data journalism visualizations with big data analytics apps. But I, for one, would like to play with such a tool.

Entrepreneurs from companies that work with and make visualizations from big data, such as Quid, will speak at the GigaOM Structure:Data conference on March 20-21 in New York.

Disclosure: The Guardian is an investor in Giga Omni Media, which publishes GigaOM.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Big data 2013: key trends and companies to watch
• Sector RoadMap: Social customer service in 2013
• How to use big data to make better business decisions

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Infrastructure Q1: Cloud and big data woo enterprises
• The fourth quarter of 2012 in cloud
• Putting Big Data to Work: Opportunities for Enterprises

Zoomdata Founder and CEO Justin Langseth started Zoomdata on the premise of building a company that envisioned BI free from decades of legacy baggage. In 2012, that means abandoning the desktop and designing for tablets, and taking advantage of the nearly unlimited computing power available in the cloud and even on our mobile devices. It also means designing a user experience so intuitive that users know how it works without ever really having to learn.

Just like someone can open up Google Earth and know they’re seeing the planet Earth, Langseth said Zoomdata users should open the app and say, “Oh, that’s my business.” And then they should be able to easily zoom in right where they want to go, using only their fingers. In a few swipes and pinches, Langseth said, users are soon uttering the business equivalent of “that’ my house, that’s my car, that’s my tree.”

It’s a pretty heady concept for a guy like Langseth who has been entrenched in the space for years, first at MicroStrategy in the 1990s and most recently doing a text-analysis startup, but he appears to have pulled it off thanks to the array of powerful components now floating around the web for free. Zoomdata is able to take data from anywhere — web apps, enterprise systems, Hadoop, email, you name it — and process it as it hits the system using an open source stream-processing engine called Storm. Once it’s processed, Zoomdata applies intelligence to figure out the best way to display that data visually and puts the result on the screen.

Data is like paint on a palette

That, Langseth said, is where the magic really comes in. “We’ve been thinking of data as kind of like paint,” he explained. The app takes many of the concepts from the D3.js project for creating HTML documents using data, but then makes them interactive and “lights them up with real-time data.”

The human interaction becomes a combination of watching a movie and finger painting. Combining data sources and sets by swiping your fingers is akin to blending colors from a palette. The interface comes with set of buttons for pausing, rewinding and fast-forwarding the visualization, too — because it’s a real-time engine, the data keeps coming and the visualization keeps changing until someone temporarily stops the flow.

Under the hood, of course, Zoomdata is a lot more complex than meets the eye. It’s all about scale, speed and huge amounts of data, Langseth explained.  The backend does all the work and only streams the data required at any given time, so as to save the processing load on the user’s device. If a user presses pause or rewinds, the system keeps processing new data while also letting the user interact with the older data unaffected. Zoomdata also supports historical data sitting inside databases and other data stores so that users can compare their real-time information against the past.

The future: Bigger screens, smarter visualization and stiff competition

As if all this doesn’t sound futuristic enough, Langseth’s plans to take the technology further. “There’s a whole bunch more intelligence we can add to the system,” he explained, referencing his plans to incorporate machine-learning algorithms that will make the system even better at choosing how to visualize the raw, often schemaless data it’s receiving.

He also likes the idea of big touch screens, like CNN Big Board big. Sometimes when he has his iPad display showing on this 50-inch office television to play music, people come in and just assume they can start interacting with it like a touch screen. “Not just being able to see it, but to touch it, really excites people,” Langseth said.

Of course, as with all companies trying to carve out their space in the lucrative BI market — including the newly cash-rich Platfora — Zoomdata will have to prove itself a worthwhile alternative to big, expensive legacy technologies. Langseth thinks the real-time, mobile nature of his company’s product will at least make it a nice complement to existing desktop-based BI tools for historical data. And like pretty much everything powered by the cloud and rendered on a mobile device, its simplicity might appeal to a lot of users who don’t need the price or complexity that comes along with much legacy software.

“Some people need 10,000 features,” he said, “but most people need 5 features.”

We’ll see if Langseth is right soon enough. Zoomdata is currently in private beta after development began in March, and the company hopes to keep refining the user experience and open it up for broader consumption next year.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• How data warehousing is now a cost-effective solution for businesses
• Sector RoadMap: Social customer service in 2013
• How to manage big data without breaking the bank