GigaOM » 1010data

1010data builds a big data bartering platform in the cloud

Derrick Harris — Tue, 18 Sep 2012 18:29:11 +0000

If you’re in the market to buy and sell data sets, or maybe to track down some particularly useful analytics applications, 1010data thinks it has just the thing for you. The company, best known for its cloud-based analytics service that lets users quickly analyze huge amounts of relational data using a spreadsheet-like interface, has launched a new platform on which users can build their own applications and buy, sell and mashup their data sets with the rest of the community.

Called the Analytical Dataspace, 1010data’s new offering is like the Walmart of data marketplaces in that it brings together everything you need for valuable analytics — data, applications and even some customer service. Or maybe it’s like the Goodwill of data, in that the data sets are used, but not used up. However you choose to characterize the Analytical Dataspace, though, it’s probably a good idea and probably a sign of things to come.

When people talk about the three Vs of big data — volume, velocity and variety — the latter can be more of a benefit than a challenge to overcome, although finding the right blend of data sources isn’t always possible. We’ve seen data marketplaces pop up and succeed to varying degrees, but their products tend to target application developers wanting API access to data, and the data sets themselves can be of questionable utility for many enterprises. In theory, 1010data’s new platform should help solve these problems because its customers are typically larger businesses with data that could actually be valuable to other organizations. Already, the company hosts a number of data sets covering everything from home prices to financial markets to weather.

It also doesn’t hurt that these data sets reside on a platform that also includes applications to analyze them. There’s 1010data’s general-purpose service, but also forthcoming industry- or application-specific services from software partners that will sit atop that. After all, all the data in the world is only useful if you know what to do with it, and 1010data thinks a collection of analytics applications will take away any guesswork.

The Analytical Dataspace is just a step in the evolution of data-centric communities, however. While it advances and puts an enterprise spin on the work of companies such as Factual and Datafiniti (and even Google’s Fusion Tables, to a degree), it’s not quite the nirvana that Cisco envisions with data infomediaries serving as the hubs for all the big data needs of entire industries. It’s not quite clear where, or whether, these types of communities or brokerages will catch on, but one has to expect there’ll be some traction if there’s a profit to be made.

Feature image courtesy of Shutterstock user JeremyRichards.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

Cloud computing and trickle-down analytics

Thu, 05 Jul 2012 06:55:01 +0000

Some people predict 2013 will be the year Hadoop becomes mainstream. Such an occurrence will only be possible if the technology trickles down to a broader base of users and lowers many of the barriers to adoption it carries today. A major limitation of big data, after all, is that the technologies used to analyze it are not easy to learn. It doesn’t have to be that way, and this research note looks in detail at how components of technologies like Hadoop are finding their way into tools that target less-sophisticated users — from business users to receptionists to high school students. Thanks to cloud-based services, data visualization tools and more, analytics can be made easier, and maybe even fun.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

Cloud computing infrastructure: 2012 and beyond

Wed, 20 Jun 2012 06:55:39 +0000

Cloud computing continues to change and shape the technology industry, and these days discussions are about more than simply reorganizing the IT department. New developments in chip and hardware architectures, finding greener data centers, regulatory concerns and simplifying data analytics are all discussions currently circling through the industry. For this report, GigaOM Pro has gathered six of its analysts to discuss these topics and others in current cloud market. Here we present several areas to consider when thinking about your business in the cloud.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

The importance of putting the U and I in visualization

Derrick Harris — Fri, 04 May 2012 06:55:34 +0000

Ask a venture capitalist about big data and she will probably tell you about visualization. Only it won’t be visualization in the usual sense. Instead, it will be about visualization of the user interface. We’re talking about strikingly intuitive UIs that let users visually work with data using charts and tools instead of with algorithms and code. It’s hard work to do right — especially when you’re talking about massive data sets and complex computations — but the payoff could be huge for businesses.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

Google opens up its BigQuery data analytics service to all

Stacey Higginbotham — Tue, 01 May 2012 17:00:56 +0000

Google sits on masses of traffic and advertising data, and has decided that it should take advantage of its expertise in building the infrastructure to handle those petabytes of information to offer a data analytics service in its cloud. Google’s BigQuery product, which debuted at our Structure Data conference in March, lets companies upload their information to Google and then perform business analytics in the cloud.

On Tuesday, the service launches to the general public providing up to 2 terabytes for storage at 12 cents for gigabyte and processing on up to 1000 queries per day or 20 terabytes of data at 3.5 cents per gigabyte. This is a boon for companies that might have large analytics needs, but doesn’t have a desire to invest in a data center or specialized equipment to process those queries.

The Google site and blog post detailing the service has more information, but the gist is, companies upload their data to Google (or maybe it’s already there) and then they have the ability to run MySQL-based queries against many terabytes of data. Customers can also combine data across different databases using JOIN queries. Companies can use it in-house for their own analytics, or they can build services on top of the product as French company, We Are Cloud, has done.

The trend of hosted data in a cloud environment is growing in popularity with companies like 1010data as has the lure of making ad hoc, unplanned queries across a huge set of information. Check out Facebook’s tool to perform such queries on its own infrastructure. For more, check out my chat with Ju-Kay Kwek, product manager at Google discussing BigQuery and how to keep data analytics products in the cloud from becoming a way to lock businesses into a single provider:

Watch live streaming video from gigaombigdata at livestream.com

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

Like your data big? How about 5 trillion records?

Derrick Harris — Wed, 04 Jan 2012 18:01:34 +0000

1010data, the analytics-as-a-service pioneer, says it now hosts more than 5 trillion — yes, trillion with a “t” — records for its customers. If 1010data’s growth over the past year is a microcosm of the greater market, it’s no wonder we’re seeing so much excitement around technologies such as Hadoop, NoSQL databases and massively parallel analytic databases. Organizations are gathering mountains of data and need some way to store and analyze it — if not with a service provider like 1010data, then on premise.

According to 1010data’s annual assessment, data volumes just keep climbing. The total volume of data it houses grew by 33 percent, while the number of records grew by 45 percent. The records are spread across thousands of tables, the largest of which tops out at around 500 billion rows. And although these numbers might seem small in comparison with the astronomical data growth predicted by reports such as IDC’s Digital Universe Index, it’s important to remember 1010data deals only with business data, not with the entirety of data produced in any digital form.

Perhaps the most telling aspects of 1010data’s growth have to do with people, though. The company claims increases of about 50 percent in both the number of customers and the number of people consuming data produced using the service. The number of business analysts using 1010data grew by 25 percent. Data volumes have been growing forever, but the growing number of people concerned with that data means we should only expect the pace of analytics innovation to pick up.

It’s noteworthy, however, that 1010data’s growth is primarily in structured data, the type generally stored in relational databases and data warehouses. For unstructured data, such as that stored in Hadoop, anecdotal evidence suggests growth is much higher among organizations. In July, Cloudera noted average cluster size among its customers had more than tripled since October 2010, and 22 Cloudera customers had clusters storing more than a petabyte of data apiece.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

6 companies doing big data in the cloud

Derrick Harris — Tue, 06 Sep 2011 20:02:06 +0000

Cloud computing and big data analytics are a match made in heaven. I’ve explained why before, but essentially it’s because the cloud model lets users leverage a service provider’s infrastructure investment and subject-matter expertise without having to build them in-house. Done right, big data in the cloud is like a marriage of managed services and Software-as-a-Service, only using very powerful software.

Thankfully, big data and the cloud have already found each other. Although it’s still very early in the evolution of this combination — experts predict major investment in this area going forward — several companies have already melded the two into a variety of unique services.

Quantivo. Quantivo just announced its foray into the space Tuesday with a cloud-based version of its analytics platform. The platform combines business data from multiple sources, transforms and enriches it, then lets customers work with it via Quantivo’s specialized interface. Because knowing the right questions to ask is often one of the more difficult aspects of big data, Quantivo says its technology also takes some of the guesswork out of the process by “intelligently auto-compiling lists of patterns” in customers’ datasets.

1010data. 1010data actually has been doing big data as a service for more than a decade, before anyone was talking about the cloud. It provides a variety of services for specific big data use cases, including data warehousing, business intelligence advanced analytics. ~~Although 1010data utilizes Hadoop and other traditional big data tools to power its service, its~~ Customers interact with the service using familiar tools such as spreadsheets that make it easier to find the connections and trends they’re looking for.

Opera Solutions. Opera Solutions is an interesting company, because although it’s doing $100 million in revenue a year, relatively few people have heard of it. But its service is pretty compelling: Customers upload their data to Opera’s platform, which then analyzes it and delivers results based on the relevant “signals” in a customer’s data set. Not content with providing generic analysis to customers, Opera focuses on each customer’s specific needs and employs experts in a variety of industries to help it cater unique analytics programs for each customer.

IBM. IBM has seemingly limitless options in terms of providing big data analytics as a cloud-based service, but its current strategy appears centered around Hadoop. When IBM launched its SmartCloud cloud computing platform in April, it promised that Hadoop workloads will be part of it. A likely candidate to provide that capability is InfoSphere BigInsights, IBM’s Hadoop-based software for analyzing and visualizing large quantities of unstructured data. BigInsights previously was available as a service on IBM’s test-and-development cloud that SmartCloud replaced.

Amazon Web Services. AWS isn’t providing actual analytics as a service, just the parallel processing framework and computing power necessary to do them at scale. Its Elastic MapReduce platform is a cloud-based Hadoop implementation onto which users port their Hadoop applications, then upload their data and run the workload. Like all things AWS, customers only pay for the resources used while the job’s running, as well as for storing the data in AWS’s S3 storage service.

HPCC Systems. LexisNexis spinoff and Hadoop alternative HPCC Systems plans to give customers cloud-based access to a system running the company’s HPCC data-processing software. During an interview during Structure 2011, CTO Armando Escalante noted the company might even offer up its own massive data sets — which span the financial, legal and intelligence sectors, among others — to be processed by customers’ applications.

Watch this video for free on GigaOM

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

Infrastructure Q2: Big data and PaaS gain more momentum

Tue, 19 Jul 2011 07:01:55 +0000

Big data and Platform-as-a-Service offerings highlighted the second quarter, suggesting that we can expect to see a shift in enterprise IT practices around application development and analytics very soon. On the PaaS front, we saw new projects like DotCloud and Cloud Foundry gain incredible momentum in just a few short months. The big-data activity ranged from major new Hadoop vendors to heavy investment in flash storage that will speed the serving of data to processing engines. In other areas, we saw an uptick in cloud-computing plans from large vendors, OpenStack continued to mature and pick up both contributors and users, and Facebook caught our eye by launching an open-source project around the designs for its specialized servers and data centers. Additional companies mentioned in this report include VMware, Salesforce.com, IBM, Heroku and Calxeda. For a full list of companies, and to read the full report, sign up for a free trial.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.