2 Comments

Summary:

A startup called BitDeli is trying to democratize analytics by letting developers create custom metrics and dashboards using Python scripts instead of having to try Hadoop. What frameworks like Rails were to web development, Co-founder and CEO Ville Tuulos says, BitDeli is to analytics.

San Francisco startup BitDeli launched on Thursday with a simple mission: To let as many people as possible measure as many things as they want to. If it’s correct, the secret to the company’s success will be just a few lines of Python code.

BitDeli is targeting small to medium-sized businesses that need something between the easy-but-limited Google Analytics on one end of the spectrum, and a full-fledged Hadoop cluster on the other. According to Co-founder and CEO Ville Tuulos, many companies start small with some simple web analytics, but eventually realize they need something more. Especially if that something more involves tracking custom metrics unique to their business (e.g., how many Mac users are buying T-shirts or how many people are buying products within social games), he said, the natural instinct is to do what they see everyone else doing and try deploying Hadoop or maybe a MongoDB database.

The problems with that approach are pretty easy to spot: Deploying and managing those types of systems can be difficult, and they’re not particularly helpful if you’re trying to track your data in real time. BitDeli fills the gap, Tuulos explained, by letting developers easily set up custom analytic functions using Python scripts. They can be as simple or complex as necesary — even incorporating techniques such as machine learning — and they’re consumable by the entire company via a simple collection of dashboards called “cards.”

The service, which is hosted on the Amazon Web Services cloud, also keeps a database for each user so they can analyze historical data as well as the streaming data that BitDeli really focuses on.

Tuulos compares what BitDeli is trying to do to how frameworks such as Rails democratized web development earlier this century. Development used to be a heavy and expensive process, but open, lightweight frameworks made it so “10 times more developers can do things,” he said. He analogizes Hadoop and MongoDB to those first-generation development methods, while BitDeli is the lightweight new tool for the masses.

Sure, Tuulos acknowledges, there are workloads for which Hadoop is definitely still the right solution, but if you were to start thinking today about what most companies need to fulfill their analytic requirements, “you don’t come up with Hadoop,” he said. Tuulos ought to know — his previous gig was working on Nokia’s data team alongside its petabyte-scale Hadoop cluster. But, he said, when the data team are the only guys who know how to interact with the system in order to run jobs, that team becomes a bottleneck.

With BiDeli, he explained, scripts can be written and analyzing data in production in minutes, and developers can easily iterate on them when they need to be tweaked.

Whatever tools people choose to use, though, Tuulos’s analogy to web development is hard to argue against. Other companies, including Structure 2012 Launchpad winner Keen.io, are doing very similar things for mobile analytics, and even developer-focused Hadoop frameworks such as Continuuity and Mortar Data are trying to make it fast and easy to analyze big data. Analytics has to become more of a continuous development situation than a waterfall situation, Tuulos said, and “you have to increase the number of people capable of doing these things themselves.”

Feature image courtesy of Flickr user ramcreations.

  1. WIth the shortage in talent in Big Data, I think companies like BitDeli will be successful in the future.. There are just so many potential use cases for Big Data that haven’t been explored yet.

    Share
  2. Henley, you got that right. There’s just not enough experts in the field.

    BitDeli might be able to fill that gap for small companies, but for scalability, Hadoop in the cloud is still the way to go. Qubole is another startup that Derrick Harris has written about – also committed to tackling the tech issues associated with Hadoop so that corporations can leverage their knowledge for BI, not to re-invent the wheel in Big Data.

    Share

Comments have been disabled for this post