Developers love the latest and greatest tooling. Whether it’s Sawzall, a Google language that bridges declarative and procedural worlds. Or Kafka, a real-time framework for managing data streams. Here are four or five tools that deserve a look. Read more »
As we move towards a quantified society, one shaped by data, we start to dismiss things that are unquantified. Empathy, emotion and storytelling — these are as much a part of business as they are of life. Here is why. Read more »
Google Drive and LinkedIn suffered snafus this week; and Amazon will reportedly build a not-so-secret cloud for the Central Intelligence Agency. Read more »
Call it whatever you want — big data, data science, data intelligence — but be prepared to have your mind blown. Imagination and technology are on a collision course that will change the world in profound ways. Read more »
For companies in Silicon Valley who want to prioritize design, the real question is how to do it while also shipping engineering products to market. At Pinterest, the designers and engineers are vocal that you can, in fact, do both. Read more »
A few trends emerged in more than 30 talks at this year’s GigaOM Structure:Data conference in New York on March 20-21. The big one: people play a crucial part in the big data equation. Read more »
In-memory, SQL, NoSQL and graph databases were on display in a fiesty discussion about databases that don’t involve Hadoop. The distinctions stand out amid growing interest in specialized databases in a big-data age. Read more »
Financial institutions have a lot of data — as in multiple petabytes– so storing that data for use in new products and for regulatory compliance will move to the public cloud. Read more »
For a company like Eventbrite, which manages ticket sales and audiences for events across the world, keeping up with the constant flow of data and information is a challenge. Read more »
We’ve pretty much got the data scale problem solved, now we have to focus on speed. When that big data utopia of scale and performance is achieved, the applications could be awesome. Read more »
Rather than store lots of data and then analyze to draw insights, Guavus puts rapid analysis first, yielding considerable return on investment for telecommunications companies, said CEO Anukool Lakhina. Read more »
There’s no doubt that Hadoop is the data tool of the present and future, but more can be done to make it really shine for business intelligence. Read more »
Entrepreneurs who build applications on top of Hadoop see lots of use cases, but the ecosystem needs to evolve further in order to support wider and more cost-effective implementations. Read more »
LucidWorks CTO Grant Ingersoll made his case at Structure:Data on Thursday for why companies tackling big problems related to large sets of data should give search another look. Read more »
Asking how something is better than Hadoop is not the right question. For strategic thinking around big data companies need to figure out what they want to achieve, not what tool to use. Read more »
Want to see big data in action? When it comes to planning out data center capacity, data can influence everything from the power usage to planning for disasters. Read more »
Former Yahoo Chief Cloud Architect Todd Papaioannou said his difficulties building a consistent, stable Hadoop platform at Yahoo directly led him to found his startup Continuuity. Read more »
All companies have data, all companies have people: the secret to big data analytics is incorporating people into the overall process, according to speakers at GigaOM’s Structure Data. Read more »
Natural language user interfaces show great promise for a variety of situations, but Nuance Communications CTO Vlad Sejnoha sees a need for standard formats if new entrants aren’t to be blocked as data sources. Read more »
Wrapping up the first day of GigaOM’s 2013 Structure:Data conference, entrepreneurs from six startups talked about big ideas that show ideals for how to derive valuable insights from large sets of data. Read more »
Just like internet cookies went from major privacy concern to an accepted part of web browsing, the sharing of personal information like location and DNA will become more acceptable once people understand the value they may get out of offering it. Read more »
Companies often need to decide between innovation and open standards when they put their data into the cloud. So how can we improve data portability? Read more »
While using machine learning over large data sets to serve up ads inside social networks isn’t new, there’s an era emerging where social network data can be used to help people solve important problems. Read more »
Big data and the horsepower needed to generate, store and manage it is all great. Now we need to make sure our data is reproducible, says AWS principle data scientist. Read more »
When it comes to using big data, there are still bottlenecks. Many of these are around the tools that people use to try to make sense of massive amounts of information. Read more »
Representatives at IBM and the New York Stock Exchange laid out a schematic for doing big data analytics and showed how it can work in practice. Read more »
Harvard recently threw a tough genomics problem to TopCoder’s crowdsourced community and discovered the contest not only revealed a much broader field of investigation but provided a high level of motivation to get the problem solved. Read more »
Just like any company, the Central Intelligence Agency is trying to filter the massive amounts of data that are being produced by both people and machines, and find the signal in a growing volume of digital noise. Read more »
Speaking at GigaOM’s Structure:Data conference on Wednesday, Aetna’s head of innovation, Michael Palmer, talked about the company’s efforts to use patient data to provide better care. Read more »
What does it take to move companies toward a data-driven future? EMC chief strategist and Pivotal Initiative leader Paul Maritz spoke at Strucuture:Data in New York on how to move toward the future through human leadership and strategy. Read more »
There’s already a ton of data flying around, but the amount just scratches the surface of the data deluge that will come with the Internet of Things. Better get ready, says Snaplogic CEO Gaurav Dhillon. Read more »
In his talk at Structure: Data, Quid’s Sean Gourley talked about the meaningful differences between “data science” and “data intelligence.” While one is concerned with correlations, the other is concerned about solving problems. Read more »
Even though a perception persists that machines can increasingly solve complex problems and process large amounts of data on their own, machine learning experts say humans still play a key role. Read more »
Speaking at Structure:Data in New York, Juniper Networks product management lead Jennifer Lin claimed application developers working with big data could find great value in the software-defined network. Read more »
Are algorithms actually making society dumber? Yes, says at least one big data expert. We can’t throw computers at our problems until we better define those problems though human input. Read more »
Lending firms like Zest and Kabbage are doing a better job than jobs at deciding if a person or small business should receive credit. Their advantage is thousands of data signals that banks don’t even consider. Read more »
When building successful apps, both designers and engineers have to remember that they are on the same team, said Kleiner Perkins’ Michael Abbott at Structure: Data 2013 Wednesday. Read more »
Riak CS distributed cloud storage technology has always been sort of open-sourcey but not really open sourced. That’s changing now with Basho putting it under the Apache 2 license. Read more »
You can find all of our coverage of Structure:Data 2013 here, along with links to more info on the conference and a livestream of the action. Read more »
Who better to show the CIA how to build a cloud than Amazon Web Services? No one’s confirming anything but an AWS-CIA contract would make sense for both parties. Read more »
