Facebook knows something about big data — it collects more data and has built more tools than almost anybody else. Here, Facebook’s Jay Parikh and Accel Partners’ Ping Li talk about what lessons big data startups can take from Facebook to build businesses that can succeed. Read more »
Rather than rely on Hadoop or any other popular data-management tools to build a platform for democratizing data science, Precog decided to build its own system from scratch. That makes Precog stand out from the crowd, but it also means there’s little room for error. Read more »
It’s not for everyone, but if you’re storing petabytes of data Hadoop, Quantcast thinks it has the cure to your woes. Its newly open sourced Quantcast File System promises smaller clusters and better performance, and it has proven itself over exabytes of data inside Quantcast. Read more »
Sumall, a New York data analytics startup targeting small and medium sized online stores, mined its data to determine how time on site translates into revenue, which it says can be a helpful indicator in e-commerce. Read more »
Carter S. won his first-ever Kaggle competition — our own GigaOM WordPress Challenge — using a brute force method of data science he calls overkill analytics. Rather than spend untold hours perfecting complex models, Carter used simple algorithms and let powerful microprocessors do the rest. Read more »
Targeting consumers with relevant ads on their mobile devices is still an inexact science, but new data sources should make the process much more accurate. Knowing precisely where users shop and whether ads influenced their purchases could help everyone get what they expect from mobile ads. Read more »
Space-Time Insight has raised $14 million for its technology that visualizes geospatial data on a map so users can actually see what they’re dealing with and start to see correlations in entirely new ways. Although the energy sector drives business, the company is catching on elsewhere. Read more »
1010data has launched a new service called the Analytical Dataspace, which is essentially a marketplace for buying and selling data and accessing analytics applications. 1010data has been providing a cloud-based analytics service for years, but this is a chance to add value by adding data. Read more »
Listening platforms — or social media monitoring, analysis, and engagement tools — are bringing massive amounts of new information to businesses’ marketing departments. These tools are also leading the way in the growth of social media information and access throughout organizations so that customers’ wants and needs can be better understood and, presumably, satisfied. This report analyzes how the adoption of listening platforms is transforming marketing departments, their relationships with central IT, and the core systems that run the enterprise. Read more at GigaOM Pro »
I spent two days last week watching experts on big data and data science discuss how their companies are building businesses around data, or at least rethinking how they do business. Although most came from the web, these five ideas should matter across industries. Read more »
The big data world is full of small, scrappy startups using their ingenuity to build complex systems out of open source software, but the Walt Disney Company is not one of them. Here’s what goes into building a big data platform in a Fortune 100 company. Read more »
Are you a startup working on big data/high performance computing; gaming; consumer or business applications? Do you want $50,000? If so, Amazon has a challenge for you. Read more »
Photojournalist Rick Smolan has a new book coming out called The Human Face of Big Data. Far more than a book, it’s part of a project to show how much data each one of us generates, how it’s all connected and how it’s changing the world. Read more »
Data science competition platform Kaggle is opening up the leaderboards for its invitation-only private competitions, meaning anyone can watch and see how the world’s best data scientists are faring in these special challenges. Can data science actually become a spectator sport in the analytics community? Read more »
Cloud analytics provider GoodData has a new suite of offerings that let users connect their various data sources into a single platform that handles the analysis and visualization for them. It’s made possible by a confluence of technology trends that enable analytics as a service. Read more »
Cisco wants to rethink the big data business model by putting infomediaries at the center of vast data-sharing networks. It’s a vision of easy access to data and resources, where analyses can be conducted and acted upon in real time, but it’s a long way off. Read more »
Thanks to the popularity of everything from social media sites such as Twitter to email to mobile phones, it’s easier than ever to get data about who’s connected to whom. With the right tools, we can apply it solve certain problems faster and easier than ever. Read more »
A Boston startup called Predilytics is trying to meld machine learning and administrative data from health care companies to help the run more efficiently. It’s the latest case of big data techniques being applied to health care, where insights can both lives and lots of money. Read more »
Gojee has been using ingredient data to recommend recipes and track pantry staples on the web for a year. Now it’s final ready to make its service portable. Today it launched mobile apps for Android, the iPhone and the iPad. Read more »
Some of the largest companies in the world use Tenable Network Security’s vulnerability-management software, and now the company has a $50 million investment from Accel Partners. Between its voluminous vulnerability database and its ability to track mobile devices, Tenable thinks it’s poised to grow even more. Read more »
South Bend, Ind., is using a sensor network and IBM software to prevent its sewer system from dumping sewage into rivers or backing up into citizens’ home, but it’s just a microcosm of global trend toward solving urban maladies using big data. Read more »
Google designed BigQuery as a cloud service for running fast queries against massive datasets, but with lofty ambitions there’s always room to take a step back. Now, users that don’t require super speed can run batch queries, and can connect to the service using Microsoft Excel. Read more »
Placed is broadening its horizons by letting mobile web developers take advantage of its unique approach to location analytics. Previously available for mobile apps, Placed cross-references location data against a massive database to paint a picture of where apps or sites are being used most. Read more »
Splunk is taking its machine data show to the cloud with a new SaaS offering called Storm. It’s essentially Splunk’s flagship software tuned for cloud-generated data, but it plays into a big opportunity for fusing cloud computing with big data. Read more »
It’s slow going for the HP turnaround Meg Whitman is trying to engineer. There’s softness in the PC business, in printers, in servers — just more bad news atop the $8 billion write-down of enterprise services that HP pre-announced. Not a pretty picture. Read more »
Y Combinator ed tech startup Clever, which helps education-focused developers more easily integrate with school data, has reached 1,000 schools barely three months after launching. The company is part of the startup accelerator’s most recent class of companies. Read more »
Sqrrl plans to parlay technology from the Apache Accumulo project, which enables the lock-down of segments of a record, to make big data applications more secure and compliant with government regulations, said CEO Oren Falkowitz, an NSA veteran. Read more »
One big problem with big data is that most analytical queries are slow and non-interactive. That’s why MapR and the Apache Foundation are backing Drill, an open source version of Google’s Dremel, as a tool to address that problem Read more »
Since launching less than a year ago, LearnSprout, an API platform for education, has helped 150 schools and 15 developers bring new software into the classroom. With a suite of new products, it plans to do even more to open and clean up student data. Read more »
Etsy, Airbnb and the Climate Corporation are all using a combination of Cascading and Amazon Elastic MapReduce to make creating Hadoop jobs as simple as possible. But they’re not the only options for doing so — simplifying Hadoop usage is big business in the IT world. Read more »
Hopper’s multiyear mission to use big data tools to bring structure to the web’s travel-related information continues with a fresh infusion of cash. CEO Frederic Lalonde talked about how his company is taking advantage of acquisition-happy Google’s interest in travel information. Read more »
Although it’s still a work in progress, 0xdata thinks it has the answer to the problem of doing advanced statistical analysis at scale: Build on HDFS for scale, use the widely known R programming language and hide it all under a simple interface. Read more »
In yet another move to simplify the process of patent search, Google is now tackling the particularly tough problem of prior with a new feature. But all this innovation begs the question of whether Google, not the USPTO, should become the de facto patent search engine. Read more »
German startup ParStream raised a $5.6 million Series A round for its analytic database that goes head to head with larger vendors such as HP Vertica, EMC Greenplum and ParAccel. It’s a highly competitive database market right now, so we’ll see if ParStream has legs. Read more »
SpaceCurve has raised another $3.5 million for its technology that should allow users to analyze mountains of geospatial data. Although the company has yet to deliver product code, it already has Fortune 50 customers waiting in the wings to test it on some difficult problems. Read more »
In the era of cloud computing and big data, chief marketing officers can either sink or swim depending on their ability to recognize the importance of the consumer information available to them and are able to capture and put it to use. Read more »
A team of Swiss researchers thinks it has created an algorithm capable of tracking almost anything — from computer viruses to terrorist attacks to epidemics — back to the source using a minimal amount of data. The trick is focusing on time to figure out who “infected” whom. Read more »
There’s been something of a perfect storm in the data world over the past several years. Companies are sitting on more data than ever before, yet it’s never been easier or cheaper to analyze all that information to solve problems and create new business opportunities. Read more »
Satyamev Jayate, one of India’s highest-rated television shows, is using data as a means to effect meaningful change. The show’s producers are aggregating and analyzing the millions of messages they receive on controversial issues to do everything from planning future episodes to pushing for political change. Read more »
A group of researchers have created a computing system capable of identifying the unique features that help create a city’s look and feel, beyond just its famous landmarks. Their project is just the beginning, however, as the researchers note numerous artistic and commerical applications. Read more »
