Blog Post

Three reasons why the Semantic Web has failed

The semantic web is the vision of a web of interconnected data and meaning. This global web of knowledge would be something computers could understand and therefore provide us with a new frontier of information retrieval and intelligent agents.

After two decades of failed attempts, semantic web has become a dirty word with investors and consumers. So what exactly went wrong? Why are we still so far away from the web of data? Here’s my take on it.

The web of Obsoledge

Most attempts at creating a knowledge repository have involved converting “expert knowledge” into a web of data. The result is an inherently boring web of data. Google’s Knowledge Graph promotional video is a great example of how boring this web can be. “Let’s say you’re searching for Renaissance Painters”…. Really? Who searches for that?

More accessible technology is causing an explosion of information. This has the effect of making the shelf-life of knowledge shorter and shorter. Alvin Toffler has – in his seminal book Revolutionary Wealth – coined the term Obsoledge to refer to this increase of obsolete knowledge.

If we want to create a web of data we need to expand our definition of knowledge to go beyond obsolete knowledge and geeky factoids. I really don’t care what Leonardo DaVinci’s height was or which Nobel prize winners were born before 1945. I care about how other people feel about last night’s Breaking Bad series finale. How did they find the ending? What other series or movies might I enjoy based on those experiences?

We are living in the Now. The Now is eating ever greater quantities of our attention. It’s drowning out the obsolete past. Human attention, sentiment and emotion are key elements to today’s information age. They cannot be ignored. They need to be at the very core of any web of data.

Documents are dead

Deriving structured information from Wikipedia documents – a common practice – is fundamentally flawed. Not only does this create a web of boring facts, it assumes that documents are the source of knowledge somehow. They’re not. They are only a small sliver of the stuff that matters. And it’s the underlying conversation and activity that matters.

There is a sea change happening in the web and how we use it. It?s an evolution to a second phase of the web – the real-time web, or what I call the “Stream.” In the Stream, the focus is on messages not web pages. These vast amounts of messages are generated by social interaction, by conversation, by attention, by ideas, by little chunks of thought unleashed into a gigantic stream of data.

This also changes the way machines communicate with each other. Machines are still programmed by humans, and humans – especially programmers – are going to be lazy. They will use the easiest most pragmatic way to get machines to communicate. They aren’t going to spend days learning complicated RDF or OWL specs. They will use simple communication using JSON. And all the cool kids have abandoned XML.

Information should be pushed, not pulled

One less obvious problem is one of information retrieval. For the past two decades we’ve gotten so used to keyword search that google became an actual verb. Unfortunately, keyword search is now fundamentally broken. The more information is out there, the worse keyword search performs.

Advanced query systems like Facebook’s Graph Search or Wolfram Alpha are only marginally better than keyword search. Even conversation engines like Siri have a fundamental problem. No one knows what questions to ask.

We need a web in which information (both questions and answers) finds you based on how your attention, emotions and thinking interconnects with the rest of the world.

Meet the synaptic web

Keyword search is broken and we’re drowning in an unstoppable stream of information. The need for a next generation of information retrieval is now higher than ever. Is the semantic web going to be that next paradigm? I don’t think so. Not unless we radically revisit what a ‘web of data’ means.

It’s time to ditch the old paradigms of documents, knowledge and keyword search. We live in a world of big data, real-time streams and human emotions. It’s time for a revolution in information retrieval. We need a web that’s dynamic and centered around humans. A web in which data flows in a smarter way. A web that understands you and makes the proper data find you. This web doesn’t look like a database or a graph. It’s a web that’s intelligent, dynamic and sometimes chaotic. It’s the digital equivalent of the human brain. I call it the Synaptic Web.

Dominiek ter Heide is the CTO and co-founder of Bottlenose, which combines big data technologies with specialized data mining to make sense out of streams.

47 Responses to “Three reasons why the Semantic Web has failed”

  1. I have seen what you do, dear Sir, and I must say I am very tempted to sell organs from my body just to fly over there and kick you in the shins.

    While I admit the coding must be an impressive feat, and agree that contextualization is indeed key, your mindset qualifies you as the poster-child for everything that’s wrong with science and technology in this day and age.

    You are disregarding everything that made any sense (and was ever cool) about how people use the web, flooding us with drivel in the name of centering things on human emotions.

    I’ll have you know (speaking for people) that we have more important things to put the Internet to work on! Like how not to overcook the food and kill the plants, learning to code and DIY, or assessing emerging sustainable technology in order to die along with the rest of our failed civilization in the smoldering rubble of runaway climate change with a semblance of dignity. Or generally not fail at life.

    I have a dream… that one day the web will enable all men to become as enlightened cyborgs, and humanity be merged into a hivemind of trascendent wisdom and goodness.

    And there you are sitting there on the monstrous pile of dirty cash I’ll never make, rotting your gifts away on perpetuating this insane celebrity culture of dumb people, without harboring even a shadow of an ethical thought.

    And the Mission Control, being all teacher’s pet for corporations who want to up their tech game and sell more expensive nothing… Makes me want to puke. Who give a flying f*** what morons tweet about the NBA?

    Semantic Web failed because of reliance on human editors and bad luck at arriving at good standards.

  2. Sebastien LORBER

    “We are living in the Now. The Now is eating ever greater quantities of our attention. It’s drowning out the obsolete past. Human attention, sentiment and emotion are key elements to today’s information age. They cannot be ignored. They need to be at the very core of any web of data.”

    This can perfectly work with the semantic web.
    Read about the W3C WebID protocol which provides you an online semantic identity.
    It is not yet a mature spec but you can still play with it here: https://my-profile.eu/
    There’s also other stuff you may be interested in: ReadWriteWeb, WebAccessControl, LinkedDataPlatform…

    The semantic web is not static, not necessarily about wikipedia’like static interconnected data and can also be real time, event driven.

    It is not dead, and it hasn’t really even started.

  3. David Byrden

    An interesting article, Mr. te Heide. You are mining expanses of ephemera; massive amounts of transient and ultimately worthless data, from which you propose sell nuggets during the 15 minutes when anybody still cares about them.
    So be it. That’s your product. And I’m sure you will need different tools and formats to those used in semantic webs.
    But please don’t dismiss the historical record, or the semantic tools used to build it, as “boring”. They are not your competitors. They don’t have to be discarded to make way for your rivers of froth.

  4. This article is proof that more philosophers need to learn code and get involved in the creation of the connected frontier. Otherwise we’re all leaving uninspired entertainment junky morons like this guy in charge of creating the framework from which future ideas wil manifest.

    Seriously, this article sounds like it was written a couple of decades before the time period in which the movie “Idiocracy” takes place.

    That someone with this lack of intellectual curiosity is the Chief Officer of anything anywhere is indicative of a far greater problem than the one addressed in the article.

  5. Kingsley Uyi Idehen

    Below is a simple breakdown, hopefully putting to rest some common confusions which have had devastating effect on comprehension of what the Semantic Web is about.

    A *relationship* (or *property*) is an association between 2 or more entities (things).

    A *sentence* is how relationships are expressed in natural language using a string of words.

    A sentence is constrained by rules of *syntax* (or *grammar*) that cover word roles in sentence construction.

    A sentence is comprised of specific word roles including *subject,* *predicate,* and *object*.

    A sentence predicate (or *relation*) provides the kind of relationship expressed by that sentence (i.e., its meaning, or semantics).

    A sentence can be constructed using a variety of *notations*, without losing or changing its meaning.

    We all know the World Wide Web has been demonstrably useful, across a plethora of fronts. By simply extending what HTTP URIs “refer to” (or denote), we end up with a global Web of documents, sentences, and words, that are structurally decipherable and semantically comprehensible, by both humans and machines. That’s it!

    Links:

    [1] Revisiting Linked Data principles using natural language — http://bit.ly/1fdTTPE
    [2] Webby Documents, Sentences, Words, Syntax, and Semantics — http://t.co/m2wuPd47OL

  6. eddyvanderlinden

    When we search the web for failures, we find of course hundreds of thousands. Be it the failure of BPM, of SOA, of IT or enterprise architecture, even the failure of the cloud,… There is a remarkable constant however: the declaration of failure originates from people with commercial interests in the failure. It is by itself sad that people need to destroy other’s work so that theirs sound better. How long do you think it will last before the next self-declared guru will shout big data is a failure. In the case of the just born semantic technologies, we commit a baby murder. This is striking because I discover every day new fantastic applications developed with very cheap tools by competent and enthusiastic people. The technologies reached a maturity stage in 2007. As strategic tools for the EU and USA, the expectation of adoption was 10-15 years. What I see is a significantly higher speed. Which is unprecedented for earlier technologies. Declaring the failure of ST is a slam in the face of all those performing exceptionally well with that little means. I hope you got enough sentiment as requested in the article. The others will find enough evidence of the increasing success of semantic technologies.

  7. dark_gabriel

    Wikipedia is the biggest source of data ever made by humanity in known history, and you dismiss it as fundamentally flawed ?!?!?

    you must really hate knowledge..

    and you dismiss…….. knowledge……. for.. what ?
    for “how other people feel about last night’s Breaking Bad series finale.” ?!?!?

    are you serious ?!?

    if people like you exist….. humanity must be nearing extinction…
    ..spontaneous ignorance-caused extinction…/

  8. dark_gabriel

    ” Really? Who searches for that? ”
    – please speak for yourself.

    “Human attention, sentiment and emotion are key elements to today’s information age. They cannot be ignored. They need to be at the very core of any web of data.”
    – taping into this information could be a very powerful and dangerous thing.
    – sentiment and emotion means quantum computing, you will discover this in a few decades.

    – giant-social-networks -> data-mining serving corporations with raw data for perfecting neuro-lingvistic-programming persuasion techniques.

    – handing out the very elements that makes us human, like sentiment,attention,emotion to the social web would be the last mistake humanity makes. it’s as simple as the biggest corporations will get the data.

    – such an upgrade to social web sets corporations on the road to acquire quantum computing power in a few decades, using masses as farms of calculus drones.
    – because corporations simply care only about profits, even (especially) in the detriment of the consumers, as they will become unimaginably powerful, mankind will choke and perish by their hand.

    hey but what do I know.. just ignore me.

  9. It’s an awful article, really. I wonder how the writer has come to the conclusion that Semantic Web has failed. It may not have gotten to where the originators of Semantic web wanted it to get; but it is certainly the direction. Siri and Google Now with their knowledge graphs and intricately connected pieces of information are found fascinatingly useful by many. Not everyone out there is (as Linda has earlier commented) a social media crack addict. There are a lot of people out there who look for knowledge and information. These are the same people which makes Wikipedia one of the most visited website year after year. And the evolution of semantic web is towards what the author is referring to as Synaptic web. The author wants to know how people liked last night\s episode of Breaking Bad, but without the Semantic Web, the Synaptic Web wouldn’t know where to get that information from; or it would always get the information from the same place which is far from ideal; or it would give you wrong information half the time (the chaos the writer has referred to); and I don’t see how that would be “better”.

  10. Step outside your own life for one moment. Who searches for Renaissance painters? I don’t know, art history majors maybe? Current artists looking to draw on themes of the great artists of the last 500 years? People who care about more than TV shows and the continued expansion of their library of available, enjoyable entertainment?

    No matter what fact you are referring to, someone, somewhere, at some point, will be looking for it for some reason. That person may be researching ideas that will change the fabric of society, help determine causes for specific diseases or even just hep us learn to understand one another. While tech has changed all of our lives, it is philosophy, sociology, the humanities that help us understand who we are and were. There’s no reason we shouldn’t create systems that help these people do the important work of synthesizing new, lasting ideas . Make no mistake, It is important, way more important than Walter White.

    If the vacant headed, in their lust for more and more of the disposable can get pointed to more fodder for the vacuous space they feel an undying need to fill to the brim with anything other than original thoughts and ideas, then those of us who think, study, write, critique, research and make actual differences to the actual world should be able to as well. Perhaps your assertion that semantic web isn’t marketable enough, because of people’s lack of involvement in anything outside their netflix cue coupled with human laziness is true. If so, that’s just sad.

    “I don’t care about or see the relevance of certain historical facts to the ‘Now’ so the entire internet and humanity itself should just forget about making them easy to find.”
    Have fun repeating mistakes of the past.

    If everything is always catered to your likes and tastes where is the discovery in life? Is the solution really to spoon feed everyone things we know they will like? I agree it’s great having the option to sit back and let google show you all kinds of cool stuff, but is that really the only web we need? Is that really the only way we should learn more about our world? What about the search for truth, personal and absolute?