IBM and Cisco have both launched specialized hardware designed to securely and efficiently handle big data, but is there a large market for specialized big data gear? If there is such a market, are these the boxes that will fill it? Read more »
Campaigns have been profiling potential voters for decades, but the glut of data available online changed the game in terms of how much they collect and how it’s used. Now, thanks to complex models and real-time ad platforms, poltiical advertising is becoming a personal affair. Read more »
In photojournalist Rick Smolan’s latest book, The Human Face of Big Data, he shares more than 100 stories revealing the concrete, mind-blowingly powerful ways big data is changing how we consume energy, receive healthcare, monitor the environment and more. Take a look at five examples. Read more »
A new startup called Trifacta, founded by UC-Berkeley professor Joe Hellerstein and Stanford professor Jeffrey Heer, wants to eliminate much of the hassle of making messy data usable. The company combines machine learning and human-computer interaction, and has raised $4.3 million from Accel Partners. Read more »
Big data company RainStor has raised $12 million is Series C funding for its database that’s designed to shrink data footprints by at least 95 percent. It also plays nice with Hadoop, meaning a system can handle ad hoc SQL queries as well as MapReduce jobs. Read more »
NASA and a couple other government agencies have kicked off a series of TopCoder challenges designed to find innovative solutions to the government’s big data problems. The first contest is all about making disparate, incompatible data sets usable and actually valuable across agencies. Read more »
A startup called Entelo is trying to make résumés a thing of the past by aggregating profiles of tech workers from their public data, and then feeding those results to recruiters. The secret sauce is an algorithm for spotting when someone might be looking for work. Read more »
Although Twitter is pushing itself as a platform to gauge public opinion around popular events — including the upcoming presidential election — not everyone is buying the hype. Stats about sentiment and tweet velocity are certainly interesting, but man cannot live on tweet data alone. Read more »
Interest graph specialist Gravity has raised $10.6 million to expand its business of personalizing the web for consumers. Thanks to a semantic engine that associates the content site visitors read with related topics, Gravity says it can show readers just what they want to see. Read more »
In-Q-Tel, a somewhat mysterious company, takes an undisclosed stake in Cloudant, the developer of a NoSQL-based data layer as a service. IQT’s blessing can help companies get their technologies deployed in government agencies, including the CIA and NSA. Read more »
Thanks to the rise of online business, companies must now get their products and services to market as fast as they can, and releasing software now means small releases that occur very frequently. Enter devops, which is disrupting traditional assumptions about the roles of development and operations. Read more at GigaOM Pro »
Building a robotic bee that acts like a real bee is a lot more complicated than programming a robot to fly around from flower to flower. A project called Green Brain aims to build an artificial intelligence system that can actually mimic a bee’s brain. Read more »
Facebook knows something about big data — it collects more data and has built more tools than almost anybody else. Here, Facebook’s Jay Parikh and Accel Partners’ Ping Li talk about what lessons big data startups can take from Facebook to build businesses that can succeed. Read more »
Rather than rely on Hadoop or any other popular data-management tools to build a platform for democratizing data science, Precog decided to build its own system from scratch. That makes Precog stand out from the crowd, but it also means there’s little room for error. Read more »
It’s not for everyone, but if you’re storing petabytes of data Hadoop, Quantcast thinks it has the cure to your woes. Its newly open sourced Quantcast File System promises smaller clusters and better performance, and it has proven itself over exabytes of data inside Quantcast. Read more »
Sumall, a New York data analytics startup targeting small and medium sized online stores, mined its data to determine how time on site translates into revenue, which it says can be a helpful indicator in e-commerce. Read more »
Carter S. won his first-ever Kaggle competition — our own GigaOM WordPress Challenge — using a brute force method of data science he calls overkill analytics. Rather than spend untold hours perfecting complex models, Carter used simple algorithms and let powerful microprocessors do the rest. Read more »
Targeting consumers with relevant ads on their mobile devices is still an inexact science, but new data sources should make the process much more accurate. Knowing precisely where users shop and whether ads influenced their purchases could help everyone get what they expect from mobile ads. Read more »
Space-Time Insight has raised $14 million for its technology that visualizes geospatial data on a map so users can actually see what they’re dealing with and start to see correlations in entirely new ways. Although the energy sector drives business, the company is catching on elsewhere. Read more »
1010data has launched a new service called the Analytical Dataspace, which is essentially a marketplace for buying and selling data and accessing analytics applications. 1010data has been providing a cloud-based analytics service for years, but this is a chance to add value by adding data. Read more »
Listening platforms — or social media monitoring, analysis, and engagement tools — are bringing massive amounts of new information to businesses’ marketing departments. These tools are also leading the way in the growth of social media information and access throughout organizations so that customers’ wants and needs can be better understood and, presumably, satisfied. This report analyzes how the adoption of listening platforms is transforming marketing departments, their relationships with central IT, and the core systems that run the enterprise. Read more at GigaOM Pro »
I spent two days last week watching experts on big data and data science discuss how their companies are building businesses around data, or at least rethinking how they do business. Although most came from the web, these five ideas should matter across industries. Read more »
The big data world is full of small, scrappy startups using their ingenuity to build complex systems out of open source software, but the Walt Disney Company is not one of them. Here’s what goes into building a big data platform in a Fortune 100 company. Read more »
Are you a startup working on big data/high performance computing; gaming; consumer or business applications? Do you want $50,000? If so, Amazon has a challenge for you. Read more »
Photojournalist Rick Smolan has a new book coming out called The Human Face of Big Data. Far more than a book, it’s part of a project to show how much data each one of us generates, how it’s all connected and how it’s changing the world. Read more »
Data science competition platform Kaggle is opening up the leaderboards for its invitation-only private competitions, meaning anyone can watch and see how the world’s best data scientists are faring in these special challenges. Can data science actually become a spectator sport in the analytics community? Read more »
Cloud analytics provider GoodData has a new suite of offerings that let users connect their various data sources into a single platform that handles the analysis and visualization for them. It’s made possible by a confluence of technology trends that enable analytics as a service. Read more »
Cisco wants to rethink the big data business model by putting infomediaries at the center of vast data-sharing networks. It’s a vision of easy access to data and resources, where analyses can be conducted and acted upon in real time, but it’s a long way off. Read more »
Thanks to the popularity of everything from social media sites such as Twitter to email to mobile phones, it’s easier than ever to get data about who’s connected to whom. With the right tools, we can apply it solve certain problems faster and easier than ever. Read more »
A Boston startup called Predilytics is trying to meld machine learning and administrative data from health care companies to help the run more efficiently. It’s the latest case of big data techniques being applied to health care, where insights can both lives and lots of money. Read more »
Gojee has been using ingredient data to recommend recipes and track pantry staples on the web for a year. Now it’s final ready to make its service portable. Today it launched mobile apps for Android, the iPhone and the iPad. Read more »
Some of the largest companies in the world use Tenable Network Security’s vulnerability-management software, and now the company has a $50 million investment from Accel Partners. Between its voluminous vulnerability database and its ability to track mobile devices, Tenable thinks it’s poised to grow even more. Read more »
South Bend, Ind., is using a sensor network and IBM software to prevent its sewer system from dumping sewage into rivers or backing up into citizens’ home, but it’s just a microcosm of global trend toward solving urban maladies using big data. Read more »
Google designed BigQuery as a cloud service for running fast queries against massive datasets, but with lofty ambitions there’s always room to take a step back. Now, users that don’t require super speed can run batch queries, and can connect to the service using Microsoft Excel. Read more »
Placed is broadening its horizons by letting mobile web developers take advantage of its unique approach to location analytics. Previously available for mobile apps, Placed cross-references location data against a massive database to paint a picture of where apps or sites are being used most. Read more »
Splunk is taking its machine data show to the cloud with a new SaaS offering called Storm. It’s essentially Splunk’s flagship software tuned for cloud-generated data, but it plays into a big opportunity for fusing cloud computing with big data. Read more »
It’s slow going for the HP turnaround Meg Whitman is trying to engineer. There’s softness in the PC business, in printers, in servers — just more bad news atop the $8 billion write-down of enterprise services that HP pre-announced. Not a pretty picture. Read more »
Y Combinator ed tech startup Clever, which helps education-focused developers more easily integrate with school data, has reached 1,000 schools barely three months after launching. The company is part of the startup accelerator’s most recent class of companies. Read more »
Sqrrl plans to parlay technology from the Apache Accumulo project, which enables the lock-down of segments of a record, to make big data applications more secure and compliant with government regulations, said CEO Oren Falkowitz, an NSA veteran. Read more »
One big problem with big data is that most analytical queries are slow and non-interactive. That’s why MapR and the Apache Foundation are backing Drill, an open source version of Google’s Dremel, as a tool to address that problem Read more »
