Blind trust in black box, or click-and-run, software is a growing problem in science, and the concern extends to big data and high performance computing. Read more »
When it comes to using big data technology effectively, there’s a lot to like about SaaS. When companies like BloomReach create and analyze massive web-wide data sets, they automate insights that almost no individual company could discover on its own. Read more »
Graph databases and graph-processing applications have been popping up all over the place lately, and now they’re starting to go commercial. On Tuesday, popular open source project GraphLab joined the ranks of graph startups. Read more »
Hadoop startup Mortar Data is offering to build recommendation systems for 10 companies, with help from Hilary Mason, Drew Conway and Max Shron. It’s part of a bigger plan to democratize the science behind online recommendations. Read more »
The growing pains of big data were apparent at the Data 2.0 Summit on Tuesday in San Francisco. Here is a selection of visualization tools that came up at the meeting. Read more »
Machine learning startup Skytree has raised $18 million for its software that makes short work of pattern recognition across massive datasets. Read more »
The advent of big data is affecting Ford Motor Co. in some significant ways, from how it analyzes its supply chain to the features it puts into its cars. Read more »
Data scientist might be the sexiest job of the 21st century, but it’s hardly an easy gig to land. Here is some advice from practitioners at Netflix, Orbitz and Hortonworks on how get hired and even do the hiring. Read more »
More than a decade ago, Gracenote received some cryptic advice from Apple to buy more servers. What followed — the launch of iTunes and iPod — blew up Gracenote’s database to epic proportions and laid the groundwork for a metadata empire. Read more »
A data democracy built to last needs tools that empower everyone to work with data rather than relying on apps and data scientists. Tableau helped ignite the data revolution, and its IPO could help it keep going. Read more »
MapR is releasing open source code and partnering with Canonical on Ubuntu, while Netflix is releasing some data for for developers to play with. Sounds like a good day for openness. Read more »
Medical researchers are using a mathematical process similar to Google PageRank in order to identify organs most likely to spread lung cancer throughout the human body. Read more »
Call it whatever you want — big data, data science, data intelligence — but be prepared to have your mind blown. Imagination and technology are on a collision course that will change the world in profound ways. Read more »
When it comes to using big data, there are still bottlenecks. Many of these are around the tools that people use to try to make sense of massive amounts of information. Read more »
In his talk at Structure: Data, Quid’s Sean Gourley talked about the meaningful differences between “data science” and “data intelligence.” While one is concerned with correlations, the other is concerned about solving problems. Read more »
Even though a perception persists that machines can increasingly solve complex problems and process large amounts of data on their own, machine learning experts say humans still play a key role. Read more »
Many of us yell at the TV while watching our favorite sports teams. Many of us also want to get better at working with data. Statwing thinks it can help with both. Read more »
Unbound Concepts, a startup launched in Washington, DC, is applying machine learning and natural language processing to help teachers identify the most appropriate books for their students. Read more »
We talk a lot about big data, but only analyze 1 percent of what’s available. In order to take advantage of the other 99 percent, we need to reconsider how we do big data. Read more »
At the SXSWedu ed tech conference this week, I got an earful on plenty of new technologies aiming to remake education. But here are a few themes I still hope to hear more about. Read more »
Five years ago, LinkedIn was a shell of the technology company it is today. Here’s an inside look at where it came from, what it’s become and where it’s going. Read more »
Just when you thought spam was under control, a new breed of spammers is taking up new methods to infiltrate our inboxes, search results and social media feeds. Data science could make them very effective. Read more »
Google Flu Trends significantly overestimated the number of Americans afflicted with flu-like symptoms during the season’s peak a couple months ago, but assuring accuracy is a big part of the puzzle any time we’re talking about web data. Read more »
Two Indiana University researchers have developed a computer model they say can identify significantly better and less-expensive treatments than can doctors acting alone. It’s just the latest evidence that big data will have a profound impact on our health care system. Read more »
Not all data analysis is created equal, and understanding the difference is critical as our society places a greater value on listening to the data. Using big data to cure disease is one thing, using statistics to ruin my sports-watching is quite another. Read more »
For small business lender Capital Access Network, finding worthy borrowers is about a lot more than their credit scores. It’s taking real-world data into account in order to distinguish financial fools from savvy entrepreneurs. Read more »
In order to recommend new events for its members, online event-management company Eventbrite must build what it calls “implicit social graphs.” It’s just one of many approaches to figuring out what content users want to see. Read more »
While some are hoping for better software to reduce the need for data scientists, WibiData’s Omer Trajman thinks we need more of them. Better software, he argues, is actually just a tool to make it easier for data scientists to do world-changing work. Read more »
A lot happened in the world of data analysis this year. Here’s a list of the most-popular and generally most-interesting things I’ve had the fortune to cover in 2012 — from Hadoop to the Supreme Court to Bollywood stars. Read more »
According to the New York Times, National Football League teams are often years behind their peers in other professional sports when it comes to data analysis. Machine learning could help teams make more sense of the myriad variables that currently keep them relying on human intuition. Read more »
Online dating startup Zoosk uses big data techniques and technologies for a lot more than just computing which members might make the best mates. Data from its young audience well versed in social media lets the company design a user experience that matches their expectations. Read more »
Eight years after forming, a startup called BeyondCore is finally launching publicly with a product it claims can revolutionize analytics. Rather than making analysts search for the needle in the haystack, BeyondCore says it remove the human element and deliver that needle on a silver platter. Read more »
Even if you don’t agree with ZestFinance’s business, it’s hard to argue with its grasp of big data and applied mathematics. Now, the company has added a human touch to machine learning in order to make its models for assessing repayment risk even more accurate. Read more »
Mortar Data has raised $1.8 million for its cloud-based service that wraps Hadoop in a custom — and supposedly developer-friendly — blend of Pig and Python, meaning even novice Hadoop programmers can be writing jobs in about an hour. Read more »
Guess what, accurately predicting the outcomes of elections really isn’t a partisan affair. What Nate Silver and several others accomplished in perfectly predicting the election isn’t about finding data to support their desired outcomes. It’s about processing reams of imperfect data and figuring out what matters. Read more »
There’s shift happening in the world of online data collection, meaning consumers might expect to get paid for access to their data rather than always playing the role of uncompensated mark. Done right, it’s a system where both sides of the equation stand to win. Read more »
It’s one day before the presidential election, and the results from computer models and other data analyses are in, with most experts giving President Obama a higher probability of winning than challenger Mitt Romney. That’s no lock, however: while data doesn’t lie, models sometimes do. Read more »
In just a few years, big data has turned from a buzzword and concept best left for large web companies into a force that drives much of our digital lives. Here are five technological trends that will change how data is processed and consumed going forward. Read more »
An MIT researcher says he has created an algorithm that can identify Twitter trends hours before the service can itself. If the algorithm works as he says, it could help Twitter — and many more companies — make a lot of money. Read more »
A new research paper from Google highlights the importance of big data in creating consumer-friendly services such as voice search on smartphones. More data helps train smarter models, which can then better predict what someone say next — letting you keep your eyes on the road. Read more »
