Hadoop startup Hadapt has made its unified Hadoop-and-SQL analytic architecture even easier by adding native advanced analytic functions and integrating tightly with Tableau’s powerful BI software. It’s a sign of things to come as Hadoop and traditional SQL-based BI become cozy across the board. Read more »
Traditionally databases have either focused on real-time transactions or longer-term analytics of data, but newer technologies like Hadoop and a related open-source system called Hbase can combine those two things, according to a panel at GigaOM’s Structure Europe conference in Amsterdam Read more »
According to Equinix CTO Lane Patterson, CIOs are still learning what can and what can’t be done in the cloud. Ultimately, it’s a matter of trust. Read more »
The usual suspects Amazon and VMware made significant announcements in cloud in the third quarter, while Hadoop remained the talk of the town in big data. Emerging trends in software-defined networking and flash storage stirred up lots of M&A and venture investment in the quarter. Read more at GigaOM Pro »
IBM and Cisco have both launched specialized hardware designed to securely and efficiently handle big data, but is there a large market for specialized big data gear? If there is such a market, are these the boxes that will fill it? Read more »
A new startup called Trifacta, founded by UC-Berkeley professor Joe Hellerstein and Stanford professor Jeffrey Heer, wants to eliminate much of the hassle of making messy data usable. The company combines machine learning and human-computer interaction, and has raised $4.3 million from Accel Partners. Read more »
Big data company RainStor has raised $12 million is Series C funding for its database that’s designed to shrink data footprints by at least 95 percent. It also plays nice with Hadoop, meaning a system can handle ad hoc SQL queries as well as MapReduce jobs. Read more »
NASA and a couple other government agencies have kicked off a series of TopCoder challenges designed to find innovative solutions to the government’s big data problems. The first contest is all about making disparate, incompatible data sets usable and actually valuable across agencies. Read more »
Facebook knows something about big data — it collects more data and has built more tools than almost anybody else. Here, Facebook’s Jay Parikh and Accel Partners’ Ping Li talk about what lessons big data startups can take from Facebook to build businesses that can succeed. Read more »
Rather than rely on Hadoop or any other popular data-management tools to build a platform for democratizing data science, Precog decided to build its own system from scratch. That makes Precog stand out from the crowd, but it also means there’s little room for error. Read more »
It’s not for everyone, but if you’re storing petabytes of data Hadoop, Quantcast thinks it has the cure to your woes. Its newly open sourced Quantcast File System promises smaller clusters and better performance, and it has proven itself over exabytes of data inside Quantcast. Read more »
Carter S. won his first-ever Kaggle competition — our own GigaOM WordPress Challenge — using a brute force method of data science he calls overkill analytics. Rather than spend untold hours perfecting complex models, Carter used simple algorithms and let powerful microprocessors do the rest. Read more »
I spent two days last week watching experts on big data and data science discuss how their companies are building businesses around data, or at least rethinking how they do business. Although most came from the web, these five ideas should matter across industries. Read more »
The big data world is full of small, scrappy startups using their ingenuity to build complex systems out of open source software, but the Walt Disney Company is not one of them. Here’s what goes into building a big data platform in a Fortune 100 company. Read more »
Observers of database technology should look closely at the non-relational database market to see where the most interesting growth lies in the world of applied information storage and retrieval. Read more at GigaOM Pro »
Etsy shared the details of its hardware architecture on Friday, showing the world a whole lot of Supermicro servers running everything from web servers to Hadoop. At this point, software is the name of the game at webscale, so hardware openness is just welcome community service. Read more »
How much of your data is Facebook collecting every day? Some new stats from the company reveal just how large its user base is, and what big data means to a company with 950 million users. Read more »
One big problem with big data is that most analytical queries are slow and non-interactive. That’s why MapR and the Apache Foundation are backing Drill, an open source version of Google’s Dremel, as a tool to address that problem Read more »
Etsy, Airbnb and the Climate Corporation are all using a combination of Cascading and Amazon Elastic MapReduce to make creating Hadoop jobs as simple as possible. But they’re not the only options for doing so — simplifying Hadoop usage is big business in the IT world. Read more »
Although it’s still a work in progress, 0xdata thinks it has the answer to the problem of doing advanced statistical analysis at scale: Build on HDFS for scale, use the widely known R programming language and hide it all under a simple interface. Read more »
While computing in the cloud can cost less than running servers in your enterprise data center, the question of how much less isn’t an easy one to answer. The cloud will get cheaper in the future, but not before these challenges are addressed and overcome. Read more at GigaOM Pro »
Nimbula and MapR say that combining the former company’s scalable private cloud infrastructure with the latter’s Hadoop distribution will enable companies to run and manage big data applications much more easily. The idea is that a cloud infrastructure will make Hadoop much more flexible and available. Read more »
Infochimps has released version 1.1. of its platform that the company has described as Heroku for Hadoop. The new version takes things a step further, though, turning the platform into an engine for easily creating streaming workflows that don’t require using Hadoop at all. Read more »
Cloud computing and open source software have freed IT practitioners from so much legacy vendor baggage over the past few years. Isn’t it time to free them from inane benchmark boasting, too? A crowdsourced platform where users share their real-world performance experiences could help. Read more »
Drawn to Scale’s Spire database is meant to be all things to all people — it combines Hadoop, HBase and SQL to provide a fast, scalable, robust experience — and now it has integrated with MapR’s Hadoop distribution. It’s no surprise the young company already claims big customers. Read more »
Organizations are coping with the challenge of processing unprecedented volumes of data. However, the processes involved with using a large cluster to run applications like Hadoop are error-prone. So IT managers are turning to cluster-management solutions to automate tasks associated with cluster creation, management and maintenance. Read more at GigaOM Pro »
Nodeable is now offering a cloud service for processing and analyzing streams of data in real time. Its new flagship service, called StreamReduce, is built atop Twitter’s open source Storm framework and acts as Hadoop’s faster, nimbler front-end partner that delivers users insights as they happen. Read more »
In cloud and big data, the second quarter of 2012 featured several high-profile deals and product launches that could reshape the marketplace for everyone. Google and Microsoft launched Infrastructure-as-a-Service offerings, software-defined networking took off, and all eyes stayed fixed on the continuing promise of data analytics. Read more at GigaOM Pro »
Slowly but surely, health care is becoming a killer app for big data. Whether it’s Hadoop, machine learning or natural-language processing, folks in the worlds of medicine and hospital administration understand that data is the key to helping them take their fields to the next level. Read more »
Big data has become the latest front for the patent troll epidemic as a shell company is suing firms for using a common software framework known as the Hadoop Distributed File System (HDFS). Read more »
Hadoop is on its way to becomig the de facto platform for the next-generation of data-based applications, but it’s not without some flaws. Ironically, one of Hadoop’s biggest shortcomings right now is also one of its biggest strengths going forward — the Hadoop Distributed File System. Read more »
The market for business analytics software grew 14 percent in 2011 and will hit $50.7 billion in revenue by 2016, according to IDC. The category will grow at a 9.8-percent-a-year rate until then, driven in part by the hype around big data. Read more »
Hey, mobile developers, have you ever wondered where users are when they interact with your apps — like down to the level of whether they’re in a Starbucks or the McDonald’s right across the street? A startup called Placed can tell you so you can act accordingly. Read more »
For better or worse, Hadoop has become synonymous with big data. In just a few years it has gone from a fringe technology to the de facto standard. But is the enterprise buying into a technology whose best day has already passed? Read more »
A major limitation of big data is that the technologies used to analyze it are not easy to learn. It doesn’t have to be that way, and technologies like data visualization and cloud-based tools target less-sophisticated users — from business users to receptionists to high school students. Read more at GigaOM Pro »
The Mount Sinai School of Medicine is about to get a lesson in big data thanks to Cloudera Chief Scientist Jeff Hammerbacher. He’s partnering with the school’s clinical and academic faculty to develop new methods and systems for analyzing data in a variety of important areas. Read more »
It was a rough weekend for the internet. While Friday’s problems with Amazon Web Services and other sites could be chalked up to some wicked thunderstorms, several sites went down Saturday for periods of time thanks to problems with the “leap second.” Read more »
To say there are a lot of companies involved in the Hadoop ecosystem would be an understatement. To say partnership strategies are broad would be one, too. The folks at Datameer created this infographic to show just how expansive and interconnected the Hadoop ecosystem is. Read more »
The results of a recently released survey from Hadoop-focused startup Karmasphere show that while Hadoop use is picking up among mainstream (read “non-web”) companies, it’s still far from the all-powerful and ubiquitous insight engine its supporters (myself included) believe it will become. Read more »
Discussions about the cloud now involve more than just the IT department. New developments in hardware architectures, more-energy-efficient data centers, regulatory concerns and simplifying analytics are all discussions currently circling through the industry. Here’s what to consider when thinking about your business in the cloud. Read more at GigaOM Pro »
