GigaOM » data analysis

Private PaaS: the next generation platform for enterprises

Tue, 18 Jun 2013 06:55:59 +0000

How do you make PaaS work for your enterprise? The answer lies in understanding new models of delivery, such as private PaaS. An enterprise needs a good understanding of its requirements, and this paper aims to help business managers find the best path to understanding and deploying this technology.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

The quantified pup: Whistle wants to create the world’s largest database on dog behavior

Kevin Fitchard — Wed, 05 Jun 2013 15:19:21 +0000

Whistle is the latest San Francisco startup trying to crack the wearable device market, but it has one key differentiator over the Fitbits (see disclosure), FuelBands and Amiigos out there. You would look awfully silly wearing Whistle’s activity tracking device, since its intended to clip onto your dog’s collar.

Whistle on Wednesday announced it has raised a $6 million Series A round led by DCM Ventures and it debuted its eponymous device. The small metallic puck that clips contains a three-axis access accelerometer that tracks a wide range of canine motion – or lack thereof. The device has both Wi-Fi and Bluetooth Low Energy radios, which it not only uses to send its data payloads to the internet and your smartphone, but also for proximal location. For instance, Whistle knows your dog is at home if it’s sniffing your house’s Wi-Fi network, and if it’s detecting your smartphone’s Bluetooth signal it knows the you and dog are together.

Your dog probably doesn’t care about counting calories and steps, but in the interest of keeping their pups healthy, dog owners certainly do. And that’s why the cornerstone of Whistle’s service doesn’t live in the collar sensor or phone app, but in the cloud, CEO and co-founder Ben Jacobs told me.

Whistle has been working with leading veterinary schools to create a hefty database of dog health information. It tries to determine what a dog’s optimal exercise and sleep levels should be based on breed, weight and age, and then compares them to a dog’s actual activity patterns. Dog owners can access all of that data through a web interface, and track activity on a smartphone app. They can also use the software to send or print out reports for their vets.

“We translate that accelerometer data into info you can use and give you a snapshot of what your dog’s day looked like – walks, playtime and the time spent resting,” Jacobs said.

Whistle, however, isn’t just graphing data and comparing it against a few charts. According to Jacobs, its analyzing millions of individual data points per animal, looking for patterns or “vectors” such as restless sleep or interrupted play that would indicate that something isn’t right in Fido’s world.

“Dogs tend to hide their pain from you,” Jacobs said. “They will act how their owners want them to act. Dogs will run a marathon with their owners until their paws bleed. That’s not an indication of health. That’s an indication of loyalty.”

Whistle’s job is to ferret those patterns out from perceived behavior, he said. What’s more, Whistle plans to aggregate all of its canine data and offer it up to its research partners. Despite the high level of dog ownership in this country, there’s a lot vets don’t know about individual breeds, Jacobs said. Whistle wants to supply them with the country’s largest live case study – giving them real data on how real dogs behave in domestic environment, not just the lab.

Though the major value of Whistle comes from its cloud service, the company is taking an Apple approach to the market, making its money off of hardware. It’s selling each Whistle device for $99.95, but will never charge for the data collection and analytics service, Jacobs said. He added Whistle might pursue additional revenue streams through a version of its analytics portal optimized for vets so they can keep track of their patients.

The quantified pet is definitely going to be an interesting space, and it targets a market segment that goes nuts over their pets and has money to spend on them (I can speak from personal experience). According to Whistle, pet owners spend $50 billion annually on food, treats, equipment, medicine, veterinary bills and even massages for their pets. The idea of a Fitbit for your pooch might sound silly to many people, but to dog owners used to fussing over and pampering their pets, Whistle’s wearable doggie tech is by no means a stretch.

We’re starting to see the emergence of all kinds of internet of things devices designed to let our pets “talk” to us and interact with their environments. Whistle, however, is particularly compelling because it doesn’t appear to be building a one-trick application, such as GPS tracking collar or an NFC emergency tag.

According to Jacobs, Whistle is trying to build a platform from which it and partners can launch new hardware and services. Future software updates or versions of its collar could use proximal location to create geo-fences or even allow for remote doggie door entry. We humans can get into our homes with Bluetooth keys. Why not our pets?

Disclosure: Fitbit is backed by True Ventures, a venture capital firm that is an investor in the parent company of this blog, Giga Omni Media. Om Malik, founder of Giga Omni Media, is also a venture partner at True.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

How data warehousing is now a cost-effective solution for businesses

nraden — Mon, 13 May 2013 06:55:34 +0000

The new economics of data warehousing provide attractive alternatives in both costs and benefits. While big data gets most of the attention, evolved data warehousing will play an important role for the foreseeable future. In order to be relevant, data-warehouse design and operation need to be simplified, taking advantage of greatly improved hardware, software, and methods.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

Visualization startup Datahero opens its doors and delivers data analysis for the masses

Derrick Harris — Tue, 23 Apr 2013 14:00:46 +0000

When I first met Datahero Co-founder Chris Neumann a year ago, I was pretty excited about what he claimed his new company was going to do. Essentially, he told me, it was going to offer a simple, cloud-based data analysis and visualization service that anyone could use. About a month later, in late May, I got a demo of a very-early-stage Datahero and was impressed with the vision. On Tuesday, the company is officially opening its service to a public beta, and the more-finished product still strikes the right chord.

Before evaluating Datahero, though, it’s important to know what it’s not. Namely: it’s not enterprise software, it’s not even business intelligence software and it’s not designed for people who hope to run complex analyses. Neumann nicely summed up what Datahero is during a recent call: “We’re gonna make it usable by the masses,” he said, which means there are going to be some things advanced users want that the service doesn’t do.

By and large, what I wrote in May about the company’s vision and product holds true today (although the product is obviously more polished in terms of aesthetics and functionality). The service’s “data decoder” — which Neumann said received a majority of the development resources over the past year — does a good job characterizing data types and determining whether they’re quantities, dates, email addresses, currencies or whatever else they might be. As I explained in my “data for dummies” post in January, formatting data can be a pain, sometimes even after it’s uploaded to a service, so easing that burden is critical.

Now’s your chance to correct the data decoder’s choices.

Probably the biggest difference since last May is the addition of an import feature from data sources that lots of Datahero’s core audience of individual developers, startup employees, and other resource- and analytics-skill-strapped professionals are likely to use. These include Salesforce.com, Google Drive, Stripe, Github, MailChimp, Box and Dropbox. Really, Neumann explained about the decision-making process, the Datahero team just asked “what were the ones [its users] were clicking ‘export to Excel’ on the most.”

Salesforce.com easily has the most difficult APIs to work with when it came to building an easy import process, Neumann said, but it was necessary to connect with “if only because so many people use Salesforce and so many people hate it.” It’s pretty much held together with duct tape, he joked, and the recommended process for building a report is essentially to dump everything into Excel and go from there.

Once you’re actually visualizing, though, it’s quite literally as simple as drag, drop and maybe a few mouse clicks to filter stuff. Using a process Neumann and co-founder Jeff Zabel call “chart magic,” Datahero suggests certain charts based on the data set or let users make their own. Then, the app automatically creates what it thinks is the best chart for visualizng those particular variables (although switching to other types is really easy). Neumann said this capability is the result of lots of research about best practices for visualization, as well as the company’s own user testing and a little common sense.

Charting the frequency of terrorism incidents in the U.S. by year and target.

“Every time I would think something is simple and intuitive, [Zabel] would say, ‘You’re crazy, you don’t know what you’re talking about,’” Neumann joked. And, overall, Datahero’s Odd Couple approach to building an analytics service (Neumann is a data engineer who cut his teeth at Aster Data, while Zabel is a UI specialist formerly at BMW) seems to be working out.

Among planned features for later iterations is the ability to personalize chart selections based on a user’s past decisions and transitions from chart to chart.

But nothing’s perfect. After experimenting with it, other things that would be nice to have include the ability annotate points on a graph and perhaps to embed finished charts rather than just export as an image file (although perhaps that shortcoming is just in the pre-release version I have been using). And although Datahero isn’t trying to compete with Platfora in big data visualization or ClearStory in business analytics, it could take a page from their playbooks and render charts in HTML5 to make them easier to slice, dice, zoom in on and generally interact with.

Still, Datahero by and large does what it claims to do, and that’s important as we transition into a society consumed by data. But even with more and more data available, it will be more a dictatorship than a democracy if only a few people control the means to analyze it. I think Datahero, along with statistics-focused startup Statwing, fills a necessary place on the spectrum between Infogram and Tableau Public in terms of offering a simple product for doing analytics that does more than just make charts.

It’s not yet the easiest place to make a living, but that could change as more people get interested in digging into the piles of data they’re generating on their own terms.

You can check out the gallery below for a step-by-step process of the Datahero service and some example visualizations. I used two data sets from the Guardian’s DataBlog to experiment with — one on the 100 most-followed musicians on Twitter, and another on terrorism in the United States between 1970 and 2011.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

Sector RoadMap: Social customer service in 2013

Tue, 23 Apr 2013 06:55:38 +0000

“Social customer service” refers to those services that provide customer support via social media channels. Providing such services is no longer merely a niche or specialty sideline. Challengers, or disruptors who were early with the new technology, are working to expand and integrate their offerings into enterprise systems and processes.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

How fourth-quarter 2012 will affect IT spending in 2013

Mon, 11 Mar 2013 19:06:48 +0000

Worldwide IT spending finished out 2012 with a growth rate of 3.8 percent over 2011, the lowest growth rate since 2009. Fourth-quarter 2012 earnings reports and guidance were notable in their lack of any decisively positive news to raise 2013 spending expectations much.

Those optimistic about 2013 point to the second half of the year (now only four months away) as when we’ll begin to see some stronger growth. In their view, Europe will be better, the U.S. will be stronger, China will begin to reaccelerate in earnest, currency will be stable, and (perhaps most importantly) the year-over-year (YoY) comparisons between 2012 and 2013 will be easier because mid-2012 was so weak. Moreover, as EMC CEO Joe Tucci stated during the company’s recent earnings call, “You can’t starve IT for too long,” which suggests that better times must get rolling again. On the other hand, CEO John Chambers’ statement in the Cisco earnings call, that perhaps 2012 represents the new normal, might be a better indicator of what we can expect in 2013. We’ll see.

That said, here’s what we’re expecting for 2013.

Table 1. 2012 and 2013 worldwide IT spending (in $ billions)

Market segment	2011	2012	2012 growth rate	2013	2013 growth rate	Note
Smartphones and tablets	$213	$265	24.4%	$319	20.4%	Smartphones and tablets
Hardware	$429	$419	-2.3%	$423	1.0%	PCs, servers, storage, peripherals, network
Software	$256	$270	5.5%	$291	7.8%	Systems, middleware, solutions
Service	$797	$805	1.0%	$826	2.6%	Consulting, IT outsourcing, systems integration, BPO, education and training, maintenance
All spending	$1,695	$1,759	3.8%	$1,860	5.7%

Overall, smartphones and tablets will lead the way, with a 5.6 percent worldwide IT growth rate in 2013. While Apple’s tepid forward guidance in its recent earnings call and its implications for smartphones and tablets gives us pause, we expect the category to continue to lead, with growth in the 20.4 percent range in 2013. In hardware, 2013 will look like 2012, with smartphones and tablets buoying the otherwise-weak spending in the PC, peripheral, and server segments. (This will be offset somewhat by stronger storage and network hardware spending.) Software continues to flourish — especially solutions-related software like SaaS-enabled customer-relationship management (CRM), supply-chain management (SCM), and industry vertical systems like health care. This sector will continue to grow in 2013. Finally, services has experienced a punishing 2012, and this area will only look modestly better in 2013.

A methodological note: We’ve reevaluated our model and determined that we have underweighted business-process outsourcing (BPO) as a service category. As such, we are restating 2012 and 2013 growth rates to accommodate a higher weighting of BPO in our services spending. The net is that growth in services and all spending are a bit more robust, since BPO is growing faster than the aggregate of traditional IT services. The full methodology can be found at the end of this report.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

How to use big data to make better business decisions

Paul Miller — Thu, 07 Mar 2013 07:55:06 +0000

Companies are rushing to embrace the promise of big data to understand both their businesses and the ways in which customers interact with them. But effective data-based decisions are not made in response to simplistic data reporting; they are made in response to considered and ongoing data analysis.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

How Food Genius built the ultimate test kitchen out of menu data

Kevin Fitchard — Fri, 21 Dec 2012 22:27:33 +0000

Chicago startup Food Genius knows what America is eating when it dines or orders out. It knows, for instance, whether more pizzerias are starting to put basil on their cheese and tomato pies or whether restaurants are still piling their burgers high with bacon or have switched over to avocado.

What’s more, Food Genius knows exactly how much more restaurants are charging for adding that sprinkling of basil to turn a standard pie into a pizza Margherita. Food Genius can tell you what hot combinations of ingredients, flavors and culinary buzzwords can boost the appeal – and thus the price – of a dish. In short, Food Genius has built the country’s biggest virtual test kitchen with menu data.

Food Genius has compiled a mammoth database of 100,000 unique menus from independent and chain restaurants around the country, giving it a massive data set to play with, said Justin Massa, Food Genius co-founder and CEO. The startup gets its data from some key menu aggregation partners, the biggest being fellow Chicago food outfit GrubHub. It also takes in readily available menu data from the national and regional restaurant chains, sometimes manually scraping data off of restaurants websites.

Any company can aggregate menus, Massa admits, but Food Genius is only using those raw lists of ingredients and dish descriptions as a starting point. It’s built parsing and categorization algorithms that break down those menu items into 14,000 ingredients, techniques and concepts that make up its internally developed culinary taxonomy. From there, it ferrets out the relationships between the items, like what categories they fall into and how different ingredients and techniques are commonly paired.

By crunching that data, Food Genius had generated more than 1 billion different food concepts, each of which represents a sort of meta-dish. Some of those concepts have been produced millions of times over in restaurants all over the country, like cheeseburgers, but many more are specific to individual restaurants. And others are just concepts inferred from Food Genius’s data.

Teaching a Kraft to cook like a Wolfgang Puck

While it might be tempting to think that Food Genius wants to use its data to automate the creative process of cooking, Massa quickly dispelled that notion. While Food Genius can suggest ingredient combinations based on patterns its database detects, pure math could never replace the chef, Massa said.

Instead, Food Genius is doing the opposite. It’s tracking trends in independent restaurants and relaying that information to big food companies – restaurant chains, food distributors, consultants and most importantly food manufacturers.

“The big food companies take it on faith that innovation starts in independent kitchens,” Massa said. “The problem is the lifecycle of product development at a company like Kraft is two years. Meanwhile, in an independent restaurant, a new dish could be conceived and executed in 30 minutes.”

Food trends are becoming ever more fleeting. Bacon in desserts or gourmet sliders may be hot concepts today, but they could become passé quickly. By using Food Genius’s database, food companies can identify food trends with true mass appeal, and they can latch onto those trends early. The last thing a major food manufacturer wants to do is invest large sums of money and time bringing to market a new line of frozen southwestern chicken entrees, only to discover that the consumers are now into Thai cuisine with pork.

“We want to give them the confidence that they can catch a trend within nine months, rather than within two years,” Massa said.

Taking food analysis to the next level

The company plans to take its service live in January, and already has half a dozen food industry consultants and food product manufacturers signed up. It’s selling its data through a licensing model, with monthly fees starting at $2,000 per user.

Food Genius is one of the growing number of companies to emerge from Excelerate Labs’ accelerator program in Chicago. In September, it raised a $1.2 million funding round led by Hyde Park Venture Partners and Hyde Park Angels with participation by New World Ventures, IDEO, Amicus Capital and I2A Fund.

Massa said right now its customers are using Food Genius data to develop their own food concepts, but as the company scales it hopes to offer more custom analysis. Instead of trying to identify food trends themselves, Food Genius will tell them what trends they should be paying attention to, Massa said.

The startup is also looking for ways to refine its data by tracking what diners are actually ordering, not just what restaurants are offering. Knowing what dishes are in the menu doesn’t tell you which ones are popular. Food Genius would like to delve deeper into sentiment analysis, but Massa will that such data-mining techniques aren’t fully baked, especially when it comes to people’s mercurial tastes in food.

Food Genius could start pairing specific restaurant dishes with reviews in Yelp or dish sightings on apps like Foodspotting, but the information it would gather would be of questionable value.

“Say there’s this one guy who has written 10 Yelp reviews about 10 different brisket dishes he’s had, and he hated every single one of them,” Massa said. “Does he really hate brisket? Or does he really love it? Clearly he’s going to order brisket again at the next place he goes to.”

Saute pan photo courtesy of Shutterstock user Fedor Kondratenko; Dining photo courtesy of Shutterstock user Everett Collection

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

GigaOM has a new data channel — here’s what you’ll find if you stop by

Ernie Sander — Mon, 13 Aug 2012 13:25:41 +0000

It’s hard to overstate the importance of data these days. Just about everything we do online — from buying a pair of shoes to sharing a photo to searching for our soulmate — creates new data that may hold the secret to solving a problem.

Indeed, our ability to analyze this fast-growing mountain of data will increasingly shape the world that we live in. And not just in smaller ways, like whether we can find that perfect pair of jeans without ever setting foot in a store, or how quickly we discover that great new indie band we had never heard of. It will also enable use to tackle some of the biggest problems of our time, like making our healthcare system more efficient, making cities and governments more effective, or even reducing world hunger and the spread of disease.

We write about the power of big data on GigaOM every day. Just in the last few months, our data maven Derrick Harris has done stories on topics as wide ranging as bullying, intelligent transportation and genomics, showing how in each case big data is helping to power new breakthroughs. Starting today, in recognition of the growing importance of data in spurring new insights and even new technologies, we are rolling out a dedicated data channel. Now you’ll be able to find the best stories about the data revolution — from the most interesting use cases to the most important technologies to the smartest data scientists — all in one place

The launch of this new channel coincides with something of a perfect storm with big data over the last few years. Because so much of what we now do on the web — and even in our physical lives — generates new data, we’re swimming in the stuff. We have far more data at our disposal than we’ve ever had before, and we continue to collect it at a furious rate.

At the same time, it’s never been cheaper or easier to analyze large amounts of data. What used to take hours or days now takes minutes or even seconds – in some cases, the data synthesis is taking place in real time. Meanwhile, someone with absolutely no computer science training can create a chart to visualize a data set in just a few clicks using a variety of free web services.

All of this means that the business opportunity with data is palpable. The biggest internet brands of today — companies like Google, LinkedIn, Twitter, Facebook, Amazon and Netflix — are where they are in large part because of their ability to harness big data. And it’s not just the heavyweights of the web that are taking full advantage of this moment; almost every company of any note is thinking about how to use big data to take their business to a new level.

Data is king, and our new data channel will be the place to watch how it’s changing not only the internet but the world around us, too.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

The importance of putting the U and I in visualization

Derrick Harris — Fri, 04 May 2012 06:55:34 +0000

Ask a venture capitalist about big data and she will probably tell you about visualization. Only it won’t be visualization in the usual sense. Instead, it will be about visualization of the user interface. We’re talking about strikingly intuitive UIs that let users visually work with data using charts and tools instead of with algorithms and code. It’s hard work to do right — especially when you’re talking about massive data sets and complex computations — but the payoff could be huge for businesses.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.