Social and mobile analytics startup Kontagent has expanded its business to include a data-mining service powered by Hive, the SQL-like interface for querying data stored within Hadoop. It’s a smart move by the company, and one that other cloud-based analytics providers would be wise to replicate. Read more »
The federal government talked a lot about grand scientific visions when it unveiled its big data agenda last week, but the government has consumers on its mind, too. Specifically, it doesn’t want to unduly hinder innovation, and it might even be willing to provide data. Read more »
If you have a lot of unstructured data, don’t have (or want) a Hadoop cluster and can write Python jobs, Mortar Data has got the service for you. The New York-based startup is jumping into the fray with possibly the most lightweight Hadoop service yet. Read more »
Managed hosting provider Sungard is getting into the big data space with a new Hadoop service that gives users on-demand access to the popular data-storage and processing platform. Called Unified Analytics Service, Sungard’s new offering joins the growing ranks of cloud-based Hadoop offerings. Read more »
By pumping hundreds of millions of dollars into big data research and development, the Obama administration thinks it can push the current state of the art well beyond what’s possible today, and into entirely new research areas. It’s a noble goal, but also a necessary one. Read more »
Key to understanding big data is to move beyond simply examining the technology for data storage and analytics engines. Organizations preparing for a data-centric economy should also examine the roles of data quality, data obesity and data markets in the future of modern enterprises. Read more at GigaOM Pro »
The problem for many companies is that user information is spread across hundreds or even thousands of different fields in various databases, and it’s difficult to compile it in real time. But doing that successfully is becoming increasingly important, says WiBiData at Structure:Data. Read more »
Two star hires and a well-reviewed phone-and-tablet operating system do not necessarily remake a company, but they do ease the perception — prevalent in recent years — that Microsoft is on its last legs. Could the once-dominant software giant be on the comeback trail? Read more »
One solution to the big data skills shortage has been consulting firms that specialize in deploying big data systems companies need to make sense of their information. These companies will continue to play a vital role in helping us make sense of the the data deluge. Read more at GigaOM Pro »
Big data now touches everything from enterprises to smart-meter startups, while Hadoop is fast becoming the leading tool to analyze that data, and debates around privacy abound. GigaOM Pro analysts offer insights on what to consider when it comes to big data decisions for your business. Read more at GigaOM Pro »
EMC’s acquisition of Pivotal Labs proves the company really understands the big data market. Namely, that big data won’t go anywhere without great applications, and EMC isn’t the company to help customers figure out how to build theirs. Read more »
With many utilities facing the task of storing petabytes of smart meter data for as long as seven years in order to satisfy regulatory requirements, the ability to house and leverage the massive load of data accumulating from the smart grid is a significant IT challenge. Read more »
Raghu Ramakrishnan, who was the top scientist for several of Yahoo’s key technology efforts, is now a technical fellow with Microsoft’s server and tools unit. This is the latest sign that Yahoo is struggling to retain key technologists. Read more »
Web companies like Google and Facebook invest incredible resources in making sure they know everything about their infrastructures and how server-level issues are affecting the applications that comprise their lifeblood. The rest of the business world is now catching on. Read more »
In September, I profiled six companies doing big data in the cloud, and here are nine more. One of the themes of Structure:Data is “putting big data to work,” and there’s no easier way to get started doing so than with a cloud service. Read more »
Everyone is eager to name the new big data stack — a LAMP stack equivalent for all your big data needs. But with literally dozens of different open-source projects all taking a shot at solving a piece of the stack, settling on a standard architecture is ... Read more at GigaOM Pro »
Amazon Web Services is indisputably the largest cloud service provider by far, and by all accounts. So what rivals could be No. 2 and maybe even give AWS a run for its money? Some of these names might surprise you. Read more »
Social media are a great source of information on consumer sentiment, but the next wave of analytics in social media will be influence. It’s easy to ridicule your friends and co-workers who are actually concerned with their Klout scores, but this stuff matters. Read more »
The emergence of the big data phenomenon is fundamentally changing everything from the way companies operate to the way people interact to how the world deals with outbreaks of infectious diseases. Here we highlight 10 case studies illustrating how big data is changing the world. Read more at GigaOM Pro »
The massive amount of data that is emerging from connected, digital systems, is fundamentally changing everything, from Internet search to entertainment, to disease management, to energy consumption. Here’s 10 case studies that highlight the power of big data. Read more »
The files J. Edgar Hoover kept are nothing compared to the data collected by the Republican and Democratic campaigns in the coming 2012 elections. Thanks to tools such as Hadoop and Hive, campaigns can now predict how to target their campaigns. What’s next? Read more »
Banks and their customers are under constant threat from cybercriminals trying to get at their money, but at least one bank is using Hadoop to help ensure those attacks aren’t successful. And it has changed the face of its analytics efforts. Read more »
Drawn to Scale, a two-year-old startup focused on making SQL ready for the world of big data by combining it with Hadoop, has raised an initial funding round of $925,000. Its product, Spire, utilizes Hadoop to increase scalability and reduce latency across large data sets. Read more »
Enterprises realize their big data is a strategic asset, and they have budget to spend against it. But without the usual yardsticks and metrics for measuring the success, these projects will fall flat. ... Read more at GigaOM Pro »
Twitter has been on a tear lately when it comes to open sourcing big-data tools. The latest two are Cassie, a client for managing Cassandra clusters, and Scalding, a MapReduce framework for simplifying the creation of Hadoop jobs. Big data won’t be black magic forever. Read more »
Look under the covers of almost any data-focused web application — including Klout — and you’ll find Hadoop. It helps Klout accurately measure and score its users’ social media influence. But Klout also has another important, and very not-open-source, weapon in its arsenal — Microsoft SQL Server. Read more »
Zynga is in a tricky position technologically thanks to its new frenemy status with Facebook. Zynga claims 240 million active users, and it wants a lot more, but scaling to those heights might require one heck of a computing infrastructure. Read more »
Matt Howard of Norwest Venture Partners predicts that 2012 and 2013 will be Hadoop’s breakout years. Howard gives us insight into the five factors that will accelerate Hadoop’s mainstream adoption over the next 18 months. Read more »
RSA says the way to handle security threats is by harnessing big data, but the tools aren’t out there yet for mainstream business ... Read more at GigaOM Pro »
Cisco and EMC have come up with a reference architecture featuring Cisco UCS server gear that’s designed to run the EMC Greenplum MR software, the company’s “enterprise-class” Hadoop distribution that features technology it OEMs from Hadoop startup MapR. Read more »
When Microsoft and VMware latch onto a technology, you know it’s for real. VMware is now pushing a project called Spring Hadoop that lets developers use the popular Spring Java framework to write big data applications atop Apache Hadoop. Read more »
Microsoft, which looked as if it might be the odd man out on Hadoop, might actually play a big role in taking the platform into the mainstream. On Tuesday, it exposed its plans to make Hadoop data analyzable via both a JavaScript framework and Microsoft Excel. Read more »
According to business intelligence provider Jaspersoft, MongoDB and Hadoop dominate the big data landscape among its users. The company today announced a new service, called the Big Data Index, which is a monthly measure of downloads of Jaspersoft’s “big data” connectors to its flagship BI product. Read more »
Rob Bearden, CEO of Hortonworks, the Hadoop startup that spun out of Yahoo in June 2011, knows a thing or two about making open source software profitable. And he thinks Hadoop has an opportunity to be bigger than the markets for JBoss, SpringSource and MySQL combined. Read more »
At some inderminate time, very possibly this year, business intelligence favorite Tableau Software will file for its initial public offering. When it does, it will be in good company, along with others that were smart enough to ride the twin waves of consumerization and big data. Read more »
Big data startup Skytree emerged from stealth mode on Thursday with a product that is designed to democratize the science of machine learning while improving significantly on the speed and scale of existing options. Skytree has raised $1.5 million from Javelin Venture Partners. Read more »
Infochimps, a startup best known for its data marketplace, is now offering a cloud-based service that takes the pain out of managing Hadoop and scale-out database environments. The company hopes its new cloud-based service can do for big data applications what PaaS did for web applications. Read more »
BloomReach emerged from stealth mode a message about how it will help ensure companies get their web pages heard above the noise online. Using a potent brew of big data techniques, BloomReach says it can significantly improve traffic by making pages more relevant to consumers. Read more »
Data-warehouse veteran Teradata has tightened its embrace of the Hadoop big data platform via a partnership with Hortonworks. The goal is to give customers big data environments that integrate everything from the Teradata Database for advanced SQL analytics and the Hortonworks Data Platform Hadoop distribution. Read more »
If you are a data scientist, or have other “big data” chops, you can write your own ticket, headhunters say. A search of popular job posting sites shows huge demand for anyone who can demonstrate these skills even in this tough economy. Read more »
