Big data is often talked about as a phenomenon that lets organizations create narratives from their volumes of data. That is an apt characterization when we are talking about connecting the dots among disparate and possibly disconnected data sets. However, when we are talking about anything ... Read more at GigaOM Pro »
1010data says it now hosts more than 5 trillion records for its customers. If 1010data’s growth is a microcosm of the greater market, it’s no wonder there’s so much excitement around scalable data stores such as Hadoop, NoSQL databases and massively parallel analytic databases. Read more »
In the world of science, cloud computing provides an ideal platform for crowdsourcing scientific problems across the whole world of researchers, giving them access to data sets and the computing resources to analyze them. If big data is any indicator, scientific crowdsourcing should catch on. Read more »
Digital recipes and cookbooks need to emulate the world of digital music. By creating a standard recipe format similar to the MP3, we could overcome the artificial barriers between cooking Websites, apps and our bookshelves. Only then could we be build truly comprehensive digital cooking libraries. Read more »
Mu Sigma has closed a $108 million investment round to expand its analytics-outsourcing business. Mu Sigma takes customers’ data and it turns it into business insights, meaning customers don’t have to built their own in-house big data expertise. It’s an already-profitable business that’s only getting bigger. Read more »
How does the cloud market for the next year look? There will be no silver linings or lemonade. This research note, part of GigaOM Pro’s outlook for 2012, examines startups’ chances to challenge traditional enterprise IT practices, cloud pricing models, the true state of federated clouds, and how IT jobs will change in the coming months. Expect plenty of defensive M&A, cloud lock-in, phantom data discovery, outages, availability headaches, pricing confusion and job displacement over the next 12 months. Read more at GigaOM Pro »
Google announced that it’s ending its Academic Cloud Computing Initiative, a joint program with IBM and the National Science Foundation that gave researchers access to a massive Hadoop cluster on which to run their data-intensive projects. The company says access to such resources is now common. Read more »
Money has turned the Hadoop community, once united under the Apache banner and the cuddly stuffed-toy-elephant logo, into something resembling a frat house: Everyone’s under the same roof, but there’s plenty of machismo to go around. If it’s not good business; it is good theater. Read more »
The discovery of a new type of chemical bond by the National Center for Supercomputing Applications at University of Illinois at Urbana-Champaign shows exactly how supercomputers and big data are combining to become the microscope of the future. Read more »
For those in a perpetual snit about personal data privacy, here’s bold proposal from Michael Driscoll: Donate your own medical data. And do it now — don’t wait till you’re dead. What better way to make big data truly relevant — and helpful — to real live people? Read more »
Beyond Hadoop, there’s a lot more to think about when it comes to big data, ranging from where companies will actually find workers to how they’ll deal with an impending privacy-policy onslaught. The answers won’t be easy to come by, but they could be critical. Read more »
It is becoming abundantly apparent that, while there is still focus on install rates of physical smart meters, what will matter in the future is how well the data from these smart meters can be secured by utilities and how well utilities can use that data ... Read more at GigaOM Pro »
Microsoft’s Windows Azure platform as a service (PaaS) now supports Node.js, the popular server-side JavaScript development framework. That could give Azure more traction beyond the Microsoft .NET faithful. Also new: a limited trial of Hadoop-based distribution for Azure. Read more »
Northwestern University is doing its bit to feed the need for analytics skills in this era of big data. NU’s McCormick School of Engineering launched a new Master of Science degree in analytics and is accepting applications for the inaugural class. Read more »
IBM plans to acquire retail data analytics company DemandTec in an all-cash deal valued at $440 million, continuing Big Blue’s buys in big data businesses. DemandTec has two especially compelling elements for IBM: It adds to its big data expertise, and it’s vertically focused. Read more »
LexisNexis is pressing MarkLogic’s technology into service for its just-launched Lexis Advance legal service. MarkLogic’s document storage, search and analytics technology replaces legacy home-built code as part of a platform modernization and big data push. Read more »
Greenplum has announced its Unified Analytics Platform, a packaging of the Greenplum Database and Hadoop distribution along with its long-awaited Chorus software. Chorus is really what ties everything together, providing a platform to explore both types of data and to share interesting data sets and findings. Read more »
Digital Reasoning, a somewhat shadowy specialist in big data analytics for the U.S. intelligence community, announced B Series funding and named industry vet John Brennan to its board. Funding came from CIA-backed In-Q-Tel, as well as some Silver Lake Sumeru partners and other unnamed investors. Read more »
However it’s defined, the concept of big data ultimately boils down to money. But I think there’s a line we shouldn’t cross when it comes to using analytics to squeeze every last dollar out of an operation. It’s between things we want and things we need. Read more »
What exactly is Carrier IQ doing with your smartphone data? It claims that only its selling network performance metrics to operators. But relationships it has with media analytics firms and handset makers imply otherwise. With 150 million smartphones tracked Carrier IQ has big data goldmine. Read more »
Analyst powerhouses IDC and Gartner both rolled out their latest cloud computing and big data predictions and statistics Thursday morning, and while some are bold, others should have you saying “Duh.” Here’s what they have to say, and how that squares with what I’m seeing. Read more »
Silk, the browser for Amazon’s new Kindle Fire, utilizes Amazon’s cloud. But don’t think AWS is the Kindle Fire’s only cloud connection. In a post on Tuesday, Pulse’s Greg Bayer explained how his company’s news-reading app actually runs atop Google’s App Engine Platform-as-a-Service offering. Read more »
By 2020 it is estimated that 20–50 billion devices will be connected to the Internet. Many of these devices will be collecting health data or will be connected to health and medical devices in the home, the hospital or the wider environment. The Internet of things (IoT), meanwhile, refers to the growth of sensors and things that connect to the Internet via RFID, Bluetooth, ZigBee and satellite. In health care, its growth is likely to open new disruptive business opportunities for services that add value to the data collected. This paper provides a preliminary overview of the landscape of opportunities and drivers in the current health and health care environments and highlights some of the challenges that remain. Companies mentioned in this report include IBM, Arrayent Health, Kaiser Permanente and Ford. For a full list of companies, and to read the full report, sign up for a free trial. Read more at GigaOM Pro »
HPCC Systems, the division of LexisNexis that’s pushing a big-data processing-and-delivery platform to compete with Hadoop, has tuned its software to run on Amazon’s cloud computing platform. Interested developers can now experiment with the open source software without having to wrangle physical servers. Read more »
Retroficiency, a startup out of MIT that uses software and big data to create energy retrofits for buildings, has raised a round of $3.2 million from Point Judith Capital and has acquired the energy efficiency division of Nexamp, called Clean Energy Solutions. Read more »
So the big data backlash begins. The Hadoop framework does a lot, but some experts — including those who push non-Hadoop options — say it’s not enough for many specialized apps where a build-your-own Hadoop implementation costs too much to be a real contender. Read more »
With its new Idol 10 software, Hewlett-Packard brings together key Vertica and Autonomy technologies acquired over the last year for an assault on knotty big data problems. HP says the software can collect and analyze unstructured data including video, audio and social net feeds. Read more »
Today, I read a press release from Sourcefire touting its “big data” approach to security and the fact that its Immunet anti-malware-for-PCs product is now monitoring 2 million endpoints. I reached out to find out what’s under Immunet’s covers and, no surprise, found Hadoop. Read more »
Hadoop gets plenty of attention from investors and the IT press, but it’s very possible we haven’t seen anything yet. All the action of the last year has just been setting the stage for what should be a big year. Read more »
Retailers already parse a ton of data — from ERP systems, from registers, from Twitter and Facebook. That pool of addressable data will only get bigger as more retailers revisit using RFID to prevent losses, says Steve Stone, former CIO of Lowe’s Home Improvement stores. Read more »
It appears as if Apple users’ willingness to shell out a little more cash for a premium experience doesn’t stop at computers. Orbitz’s data-crunching has found that Mac users also spend about $20 more a night on hotels than do Windows users. Read more »
DataSift, the British company that built its business filtering and sorting through reams of Twitter data in real time, has brought its act to the U.S., opening a San Francisco office. Businesses use DataSift to glean information about user impressions of their products and services. Read more »
When it comes to the promise of data as the currency of the web, the current state of affairs has privacy advocates and many consumers up in arms. But it doesn’t have to be the one-sided affair it is today, in which companies have all the data and all the rights, and we shouldn’t have to be afraid of who’s doing what with our information. With laws, products, practices and education, data can become a far more valuable currency than cash ever was. Keeping that in mind, this research note examines five issues that must be addressed by policy makers and entrepreneurs so that they can deliver on our data-driven digital future. Companies mentioned in this report include Twitter, Facebook and Foursquare. For a full list of companies, and to read the full report, sign up for a free trial. Read more at GigaOM Pro »
In the past decade supercomputers were dressed-up versions of Intel’s x86 machines, but increasingly supercomputers are borrowing innovations (and silicon in the form of ARM-based chips or DSPs) from the mobile and big data realms to add speed without guzzling too much power. Read more »
What happens when you place the equivalent of 1024 neurons in parallel on a chip? Well, you get a new form of computing for cloud computing and sensor networks as well as toys that can recognize cue cards, better artificial intelligence and pattern recognition. Read more »
A new startup called CrowdControl is launching today, and it aims to bring order to the world of crowdsourcing by using artificial intelligence to judge workers’ accuracy. Think Amazon Mechanical Turk, only with a quality control mechanism in place to help ensure jobs get done right. Read more »
Connectivity changes everything. That’s the credo driving just about every corner of our day-to-day lives. As human beings, we are now connected to one another through not just our social networks but also our cars, the books we read, the albums we download and even our own health and wellness habits (to name just a few areas). With that in mind, GigaOM Pro has singled out certain areas in the technology industry where we see this shift to constant connectivity taking place most drastically. Read more at GigaOM Pro »
The Supreme Court of the United States heard oral arguments in a case that could decide how connected the concept of big data is to constitutional expectations of privacy. How much data is too much before allowable surveillance crosses the line into an invasion of privacy? Read more »
Johns Hopkins is building a 100 gigabit per second network to shuttle data from the campus to other large computing centers. The network would be capable of transferring the amount of data equivalent to 80 million file cabinets filled with text each day. Read more »
Luxtera has developed an optical chip for the data center market that can achieve speeds of more than 100 gigabits per second. That’s the same speed delivered by long-haul networks under the sea, but now harnessed to move big data and deliver cloud computing. Read more »
