Companies are grappling with how to make use of all their data, facing the challenge of teasing out insights quickly and with flexibility. Moving to the cloud opens up security and privacy questions. But the effort can be worth it, says Google’s Ju-kay Kwek at Structure:Data. Read more »
Cloud technologies still lag behind on-premise tools in some key ways, but are unbeatable in enabling “one single guy” to complete large BI projects, We Are Cloud’s Rachel Delacour said at Structure:Data today. Read more »
Businesses understand now that big data can help them wring revenue out of once-unproductive assets. But that just fuels an exploding demand for bigger, faster, and more precise big data applications, experts speaking at Structure:Data say. Read more »
EMC’s acquisition of Pivotal Labs proves the company really understands the big data market. Namely, that big data won’t go anywhere without great applications, and EMC isn’t the company to help customers figure out how to build theirs. Read more »
With many utilities facing the task of storing petabytes of smart meter data for as long as seven years in order to satisfy regulatory requirements, the ability to house and leverage the massive load of data accumulating from the smart grid is a significant IT challenge. Read more »
Raghu Ramakrishnan, who was the top scientist for several of Yahoo’s key technology efforts, is now a technical fellow with Microsoft’s server and tools unit. This is the latest sign that Yahoo is struggling to retain key technologists. Read more »
Holding onto millions of pieces of archived content it still wanted to monetize, the Associated Press turned to MarkLogic’s NoSQL non-relational database designed for XML files. As publishers try to leverage their years worth of archived, often not tagged content, they’ll need new tools. Read more »
Web companies like Google and Facebook invest incredible resources in making sure they know everything about their infrastructures and how server-level issues are affecting the applications that comprise their lifeblood. The rest of the business world is now catching on. Read more »
Businesses can now leverage big data for the benefit of driving marketing insights. According to Ajay Agarwal of Bain Capital Ventures, this fundamental shift will create several multi-billion dollar winners, and new technology companies will emerge as the marketing equivalents of Salesforce and SAP. Read more »
EMC Corp., the Hopkinton, Mass.-based storage and cloud hardware company has bought Pivotal Labs, a San Francisco-based consulting firms well known for its tool Pivotal Tracker and also for its pioneering work on agile development methodology. I had first reported earlier this week. Read more »
In September, I profiled six companies doing big data in the cloud, and here are nine more. One of the themes of Structure:Data is “putting big data to work,” and there’s no easier way to get started doing so than with a cloud service. Read more »
You know how our social graphs are creeping into every aspect of our web lives, from search results to coupons? Well, get ready for something a lot more personal, a lot more targeted and, perhaps, a lot more creepy. Read more »
Everyone is eager to name the new big data stack — a LAMP stack equivalent for all your big data needs. But with literally dozens of different open-source projects all taking a shot at solving a piece of the stack, settling on a standard architecture is ... Read more at GigaOM Pro »
Amazon Web Services is indisputably the largest cloud service provider by far, and by all accounts. So what rivals could be No. 2 and maybe even give AWS a run for its money? Some of these names might surprise you. Read more »
Earlier this week, I weighed in on the NCAA tournament with my data-influenced picks, and now the experts have crunched the numbers and are weighing in with their purely objective picks. If you haven’t filled out your bracket, you might want to read this. Read more »
For most people, the only data they’ll ever really see about themselves are the pretty charts generated by applications like Mint or Jawbone. Some people take their personal data a little more seriously, tracking their lives for years, then showing the world what that looks like. Read more »
Social media are a great source of information on consumer sentiment, but the next wave of analytics in social media will be influence. It’s easy to ridicule your friends and co-workers who are actually concerned with their Klout scores, but this stuff matters. Read more »
We live in a big data world, full of complex algorithms among any type of information one can imagine. Gaining the skills to work with it requires a lot work, however — and the first step in changing that might be realizing that data can be fun. Read more »
March Madness is officially upon us, and this year I have a plan to dominate my betting pool by using big data. I hope. I’m going to balance my gut feelings and my heart’s desires with statistics and . . . I’m going to hope everyone else doesn’t do the same. Read more »
The massive amount of data that is emerging from connected, digital systems, is fundamentally changing everything, from Internet search to entertainment, to disease management, to energy consumption. Here’s 10 case studies that highlight the power of big data. Read more »
Legal scholars are always searching for ways to improve the patent system, sometimes via sweeping changes, but big data — especially techniques such as machine learning and natural-language processing — could help provide a technological fix to a big part of the problem. Read more »
The files J. Edgar Hoover kept are nothing compared to the data collected by the Republican and Democratic campaigns in the coming 2012 elections. Thanks to tools such as Hadoop and Hive, campaigns can now predict how to target their campaigns. What’s next? Read more »
Seemingly overnight, big data became the behemoth to conquer. But the truth is, tried and true technologies have been tackling the problem for years. Versant’s Robert Greene gives respect to three unsung heroes of big data. Read more »
There’s a reason Larry Ellison called cloud computing “nonsense” and why he still won’t permit Amazon-style metered pricing for Oracle’s mainstream database and middleware. George Gilbert lets us in on the company’s expensive secret. Read more »
To improve medicine, we need a big heaping dose of data. That’s the takeaway from a conversation with Aneesh Chopra, the former U.S. CTO, at SXSW in Austin on Friday. He discussed where startups interested in this space should focus on as well as privacy. Read more »
It all seems so easy: You log into Facebook, update your status, tell everyone where you are and — voila! — your Timeline is geospatial. Only, while it’s just one extra step for you to add location, building that capability was a tad more complicated for Facebook. Read more »
Banks and their customers are under constant threat from cybercriminals trying to get at their money, but at least one bank is using Hadoop to help ensure those attacks aren’t successful. And it has changed the face of its analytics efforts. Read more »
Drawn to Scale, a two-year-old startup focused on making SQL ready for the world of big data by combining it with Hadoop, has raised an initial funding round of $925,000. Its product, Spire, utilizes Hadoop to increase scalability and reduce latency across large data sets. Read more »
Enterprises realize their big data is a strategic asset, and they have budget to spend against it. But without the usual yardsticks and metrics for measuring the success, these projects will fall flat. ... Read more at GigaOM Pro »
Twitter has been on a tear lately when it comes to open sourcing big-data tools. The latest two are Cassie, a client for managing Cassandra clusters, and Scalding, a MapReduce framework for simplifying the creation of Hadoop jobs. Big data won’t be black magic forever. Read more »
IBM’s Jeopardy-playing supercomputer Watson is now getting a gig in the retail banking sector as part of an IBM partnership with Citi. The many careers of Watson illustrate a very big technological and business opportunity — the rendering of big data into human scale. Read more »
Look under the covers of almost any data-focused web application — including Klout — and you’ll find Hadoop. It helps Klout accurately measure and score its users’ social media influence. But Klout also has another important, and very not-open-source, weapon in its arsenal — Microsoft SQL Server. Read more »
On Super Tuesday, when 10 states hold their presidential primaries and 419 delegates are up for grabs, the election trackers at Yahoo and Fizziology have Mitt Romney as the favorite to win the day. The day might not go as well for Rick Santorum. Read more »
In 2012, the next big thing will be what we do with iPads in the office. Here are a few interesting iPad apps that give us new ways to gather, absorb and manipulate data necessary to get our jobs done. Read more at GigaOM Pro »
Zynga is in a tricky position technologically thanks to its new frenemy status with Facebook. Zynga claims 240 million active users, and it wants a lot more, but scaling to those heights might require one heck of a computing infrastructure. Read more »
Matt Howard of Norwest Venture Partners predicts that 2012 and 2013 will be Hadoop’s breakout years. Howard gives us insight into the five factors that will accelerate Hadoop’s mainstream adoption over the next 18 months. Read more »
The metro Boston area has good database DNA dating back to Digital’s Rdb. Those good genes are resurfacing in a fresh crop of database startups clustered in the area. Here are five hot database startups to watch in the Boston-Cambridge-Waltham nexus.. Read more »
There has been a lot of outrage — bordering on hysteria — about Google’s new privacy policy, with some critics advising users to delete their search histories. But is there that much to fear from Google’s tracking? Not really. In fact, in many ways it is beneficial. Read more »
RSA says the way to handle security threats is by harnessing big data, but the tools aren’t out there yet for mainstream business ... Read more at GigaOM Pro »
Mark Phillip wants to save cable television, kind of. He really wants to make life easier for sports fanatics like himself, but in doing so has created something he thinks could save cable and satellite from the cord-cutting craze. The secret to his possible success: data. Read more »
