Regards,
Marc

Use a semantic data model ( RDF ) that encodes contextual data relationships enabling machines and knowledge workers to discover meaning for sharing and reuse across applications and platforms without data relationships having to be predefined like for SQL, NoSQL, Hadoop ( ergo need for data scientists ).

So if you want to know the semtech startups that have their act together than ping me ;)

From the article:

Ford’s data chief joined many other top executives who are bemoaning the lack of simple tools to solve big data problems — namely the fact that running Hadoop clusters or performing analytics is still a job that requires a specialist.

Assumption #1: it’s possible to “dumb down” the analytics tools to the point that a non-specialist can effectively replace a specialist.

My response: the basic tools of carpentry (hammer, nail gun, etc.) are easy enough to use, but nobody would want me to build a house for them. Such tasks are better left in the hands of a specialist, one who is skilled in understanding the problem space, formulating a solution, and applying the tools to get the job done.

To use another analogy, easy-to-use software like TurboTax hasn’t put all CPAs out of work.

I mean, honestly, is the author really suggesting that we roll-back the historical drive towards division of labor, or is she just bemoaning the fact that it’s hard to find & hire good people who are capable of doing the advanced analytical work he needs? General-purpose computer programming isn’t something that a non-specialist can perform, but good programmers aren’t as hard to find as good data scientists (a relatively new title that serves as an umbrella term for a very broad set of specialties).

If we want to [sic] big data revolution to scale, then we need to make it as easy as Netscape made the web surfing experience.

Assumption #2: performing advanced analytics on big data is somehow comparable in any way to browsing the web for photos of LOLcats.

My response: not every problem lends itself to being solved by an “easy button.” Easy-to-use tools are great, but you still need to be a skilled practitioner (i.e., a specialist) in order to use them effectively.

=====

It seems that author of this article somewhat misinterpreted what Ford’s data chief actually said:

I’d love us to move toward [...] where data specialists — but not computer scientists — could go in and interrogate it and look for correlations that might not have been able to look at before. That’s a beautiful future state, but we’re not there yet.

To me, what he’s is saying is that he wants to be able to have data specialists (who are not computer scientists by training) to be able to do the work currently being performed by data scientists, who are largely computer scientists with an aptitude for analyzing big data.

This already exists in some ways within the business intelligence (BI) community. Really good BI tools exist to enable analysts (who are not computer scientists) to analyze business operations and add value to organizations.

The question is whether it is possible to build general-purpose data analytics tools capable of analyzing any kind of data as well as BI tools analyze business-related data. I suppose it’s possible, but a more likely scenario involves lots of different specialized tools focused on different problem domains (e.g., Orion for IC applications, RedOwl for corporate communications), since each domain has different kinds of data problems to solve.

The tools outlined in the article could be useful, especially for a non-computer scientist “data specialist” to perform some initial exploration and analysis of data (cf. TurboTax), but there’ll still be a need for someone (i.e., a traditional data scientist) to build the custom tools necessary for more advanced analytics.

a) a lot of this data is confidential and it is not going to leave the LAN. The more valuable it is the more this applies. Conversely if a company doesn’t care about uploading the data to an unknown third party – how valuable can this data be? Bottom line this “outsourced data science” is a solution for the long tail. But the money is at the other end of the power law curve ie the enterprise which is not uploading its data “family jewels” anytime soon.

b) It takes a lot of bandwidth to upload any meaningful amount of data – AWS allows you to send in hard drives in the mail which they will then upload locally. This should tell you that the bottleneck is not lack of data scientists, its the thin pipe between the ground and the cloud.

c) We are at a structural inflection point – we need to integrate data driven thinking into everything we do and we can’t delegate this. There is no silver bullet here.
There is a chasm and only companies that are in it fo rthe marathon rather than the sprint will survive.
Good luck to the sprinters.

Did I already say there was no short cut?