Whether they’re building big data applications or just trying to gather some insights from their mobile apps, developers have more need than ever for analytics tools. It’s a good thing so many companies are building tools designed with developers’ needs and skills in mind. Read more »
Former Yahoo cloud VP Todd Papaioannou and Facebook engineer Jonathan Gray are trying to big data for programmers with a new platform service called Continuuity. It’s a development environment and runtime layer that sits atop a company’s Hadoop infrastructure and abstracts the complexity of writing apps. Read more »
A number of new data markets have emerged in the past few years that provide access to data for decision making. It’s early days, however, with plenty of kinks to work out on both the technology side and with business models. Read more at GigaOM Pro »
Infochimps has released version 1.1. of its platform that the company has described as Heroku for Hadoop. The new version takes things a step further, though, turning the platform into an engine for easily creating streaming workflows that don’t require using Hadoop at all. Read more »
Ask a VC about big data and she will probably tell you about visualization of the user interface. We’re talking about intuitive UIs that let users visually work with data using charts and tools, not algorithms. It’s hard to do right, but the payoff could be huge. Read more at GigaOM Pro »
This quarter saw Amazon Web Services finally relaxing its public-cloud-only stance and launching services to support hybrid-cloud deployments. Meanwhile, Hadoop players moved to make their platforms more accessible to mainstream BI analysts and database administrators. A new quarterly report analyzes these trends and provides a near-term outlook. Read more at GigaOM Pro »
Key to understanding big data is to move beyond simply examining the technology for data storage and analytics engines. Organizations preparing for a data-centric economy should also examine the roles of data quality, data obesity and data markets in the future of modern enterprises. Read more at GigaOM Pro »
One solution to the big data skills shortage has been consulting firms that specialize in deploying big data systems companies need to make sense of their information. These companies will continue to play a vital role in helping us make sense of the the data deluge. Read more at GigaOM Pro »
Big data now touches everything from enterprises to smart-meter startups, while Hadoop is fast becoming the leading tool to analyze that data, and debates around privacy abound. GigaOM Pro analysts offer insights on what to consider when it comes to big data decisions for your business. Read more at GigaOM Pro »
In September, I profiled six companies doing big data in the cloud, and here are nine more. One of the themes of Structure:Data is “putting big data to work,” and there’s no easier way to get started doing so than with a cloud service. Read more »
Infochimps, a startup best known for its data marketplace, is now offering a cloud-based service that takes the pain out of managing Hadoop and scale-out database environments. The company hopes its new cloud-based service can do for big data applications what PaaS did for web applications. Read more »
Application developers can now use one API to access data from Factual’s marketplace that contains data on more than 55 million places worldwide. It’s not data nirvana just yet, but Factual points to what our data-driven future might look like. Read more »
Infochimps, the two-year-old company that runs a “data marketplace” where owners of datasets can charge for their usage, is swapping CEOs. Co-founder Nick Ducoff has stepped down from the CEO role, and the company is looking for a “seasoned executive” to take his place. Read more »
Hadoop has been used by large web companies for applications such as search engines, but the reality is that the project is so much more. This report takes a closer look, examining what Hadoop is (and isn’t), who’s doing what to productize it and why we can expect to see the market pick up serious steam in 2011. We profile the growing number of companies — from startups like MapR to Cloudera, the arguable leader in the space — using Hadoop, the challenges still hindering widespread adoption and where potential users can expect the market to go as we move through 2011 and beyond. Companies mentioned in this report include Yahoo, Facebook, EMC, Teradata and Appistry. For a full list of companies, and to read the full report, sign up for a free trial. Read more at GigaOM Pro »
A few providers are not just creating huge data dumps — collecting and structuring data from multiple sources — but creating marketplaces for others to find and use that data. They include Microsoft’s Windows Azure DataMarket and InfoChimp, who hope to connect enterprises with the data they want Read more »
Infochimps is attempting to build a data market, and in doing so, the company is wading into some of the messiest and most unstructured data around, attempting to clean it up and put it up for sale. I talk to co-founder Flip Kromer about the challenges. Read more »
With enterprise data volumes growing, business and IT leaders face significant opportunities and challenges from big data. Using cloud-computing technologies, organizations are experimenting with distributed data stores, cloud compute capacity for data analytics, hosted data integration and even operational databases in the cloud. Hadoop/MapReduce, meanwhile, has moved past test and development stage to become a viable extension or alternative to traditional relational databases. Though the space is not without its obstacles, including plenty of privacy concerns, there are numerous sales-growth opportunities and new business models finally surfacing in 2011. Companies mentioned in this report include Google, IBM, Apple, Oracle, Salesforce and VMware. For a full list of companies, and to read the full report, sign up for a free trial. Read more at GigaOM Pro »
Some might call this past quarter in the infrastructure space transformative. The rise of ARM-based processing suggests the days of x86 dominance might be coming to an end, while the Amazon Web Services-WikiLeaks controversy cast new light on the legal aspects of cloud computing. Big data got bigger, meanwhile, as the Hadoop ecosystem expanded, and amid all these cutting-edge technologies, two archaic topics — Novell and Java — proved they aren’t going anywhere soon. Companies mentioned in this report include Intel, AMD, Amazon Web Services, IBM, Yahoo, Appistry, VMware, Joyent and Microsoft. For a full list of companies, and to read the full report, sign up for a free trial. Read more at GigaOM Pro »
Bundle uses the billions of Citi customer transactions to draw correlations between spending habits and what other people might enjoy or buy. CTO Phil Kim explains that wrangling even highly structured data takes a lot of organizing, a lot of computing and a lot of time. Read more »
Although we’ve seen online office suites added to the portfolio of web worker productivity tools, database tools have been curiously absent from the mix. Even suites like Google Apps lack a dedicated application for managing and sharing specialized data, leaving users creating crude spreadsheet-based approximations. Read more »
Although we’ve seen online office suites added to the portfolio of web worker productivity tools, database tools have been curiously absent from the mix. Even suites like Google Apps lack a dedicated application for managing and sharing specialized data, leaving users creating crude spreadsheet-based approximations. Read more »
A decade ago, scientists would collect data over a period of years, upload that data to a supercomputer, then wait for the opportunity to run it during a scheduled time. The process took months — or even years. Now, thanks to cheap processing power and the ... Read more at GigaOM Pro »
A pair of slices from a massive scrape of Twitter’s API could be of great use to programmers and researchers alike — as long as users don’t mind. The company behind the mining effort, Infochimps, is trying to demonstrate and promote its data aggregation service while […] Read more »
