Computers became way more productive when operating systems allowed computers to run different programs without having to be re-jiggered. Palantir thinks the same thing is about to happen to data.

Ari Gesher, Engineering Ambassador, Palantir Technologies Structure Data 2014

The modern computing era didn’t really start to take off until we figured out how to make operating systems that could remove a lot of the manual labor required to program early computers. According to Ari Gesher of Palantir Technologies, it’s time for a similar breakthrough in data to become more widely used.

Speaking at Structure Data Thursday, Gesher took attendees on a bit of a computer history lesson, recalling how computers once required their users to manually reconfigure the machine each time they wanted to run a new program. This took a fair amount of time and effort: “if you wanted to use a computer to solve a problem, most of the effort went into organizing the pieces of hardware instead of doing what you wanted to do.”

Operating systems brought “abstraction,” or a way to separate the busy work from the higher-level duties assigned to the computer. This is the foundation of modern computing, but it’s not widely used in the practice of data science.

In other words, the current state of data science is like “yak shaving,” a techie meme for a situation in which a bunch of tedious tasks that appear pointless actually solve a greater problem. “We need operating system abstractions for data problems,” Gesher said.

Palantir, a notoriously secretive data analysis firm that counts the federal government among the customers of its software, has built such abstractions into its system, Gesher said. This can allow organizations to “take teams of teams and have them all looking at the same data to tease out things you couldn’t do by looking in isolation.”

Check out a video embed of the session below:

Photo courtesy of Jakub Mosur.

Structure Data Ticker

You’re subscribed! If you like, you can update your settings

  1. Chris Saunders Friday, March 21, 2014

    Reblogged this on Capacitive Flux and commented:
    I enjoyed watching the video here – great context of modern compute, and overview of Palantir’s approach to next gen big data.

    1. @Chris, can you help me with the video? I can’t find the link.

Comments have been disabled for this post