11 Comments

Summary:

Last night I was schooled at playing Jeopardy by Watson in an exhibition match and discovered that despite our fear of the robot overlords, humans are much smarter than we think. Case in point: Watson could never use Apple’s personal assistant Siri.

Me, Watson and Robert Walker of Sierra Ventures onstage.

Me, Watson and Robert Walker of Sierra Ventures onstage.

Updated: Tuesday night, I was schooled at playing Jeopardy by Watson in an exhibition match at the Computer History Museum, and discovered that despite our fear of the robot overlords, humans are much smarter than we think. Case in point: Watson could never use Apple’s personal assistant Siri.

While both services seemingly understand what we’re saying to them and can respond with amazingly functional or accurate answers, the truth is they are both still programmed for specific tasks and could never actually converse with one another or a human outside of a narrow context. So Watson can’t take dictation, and Siri can’t play Jeopardy. Understanding why shows how far we have to go when it comes to true artificial intelligence and those fears of the robots taking over.

Dr. David Ferrucci of IBM

As David Ferrucci, the guy at IBM behind Watson’s creation, explained during a conversation before the match, that as intuitive as the interactions with Siri or Watson appear to us, they are fundamentally task oriented. So the questions Watson gets are in effect “translated” not just into the zeros and ones of digital signals but also to a series of words that are then broken down into related concepts.

After that point, Watson tries to ascribe “meaning” to those contexts based on searches of unstructured data to derive an answer. It then determines which answer is most likely to be correct, and how confident it is in that top answer, because in Jeopardy, if someone guesses the wrong answer, he or she (or it) is penalized. Thus Watson’s tasks are figuring out the context associated with a question, figuring out which answer is the likeliest based on that context and if it’s confident enough in its probabilities to bother to answer.

Watson is really confident that Chile is the right answer.

Siri, on the other hand, does two important things: It recognizes speech (Watson actually doesn’t understand speech, but is fed a text version of the question), and it can figure out what steps to take in a limited number of applications once it understands the words in a natural language process  related to the process by which Watson does. The sense from IBMers (unsurprisingly) is that Siri doesn’t have the natural language depth that Watson does. Siri certainly doesn’t have the computing horsepower behind it (2,880 processor cores and 15 terabytes of RAM), or the 100 GB of text data that Watson uses to figure out how different words relate to each other.

Watson started strong and dominated the board all night.

The net result of their differences? Not only could Siri and Watson not communicate because each relies on different input methods, but even if they could, their tasks are fundamentally far apart. Both have an ability to do natural language processing, but one uses that skill to find related information and figure out which information is most correct, while the other uses it to open applications and perform a set number of tasks.

So while Alan Turing proposed the best test of artificial intelligence is that a human can’t tell if it’s a human or computer he or she is interacting with, it may be more accurate to say the best test will be creating a machine that can not only understand natural language like Siri and Watson can, but has Watson’s ability to then determine the best course of action and Siri’s ability to take that action.

Update: IBM took issue with me saying that Watson can’t talk to Siri because Watson could indeed tell Siri, “What is Chile?” and Siri could go look it up. However, I meant talking as a synonym for conversing. Watson would have to be fed a text-based version of a question for it to come up with “What is Chile?” and Siri would have to overhear the question and could then provide the answer. That is technically talking, but it’s not a conversation.

All images, except for the Watson probabilities shot, courtesy of the Computer History Museum. The Watson probabilities shot is courtesy of IBM.

  1. Watson and Siri cannot understand one another, so we’re safe from the robot apocalypse … for now. http://t.co/dC3z3MQ6

    Share
  2. Watson and Siri cannot understand one another, so we’re safe from the robot apocalypse … for now. http://t.co/dC3z3MQ6

    Share
  3. Watson and Siri cannot understand one another, so we’re safe from the robot apocalypse … for now. http://t.co/dC3z3MQ6

    Share
  4. Watson and Siri cannot understand one another, so we’re safe from the robot apocalypse … for now. http://t.co/B7bME8hN

    Share
  5. RT @nytimesbits: Watson and Siri cannot understand one another, so we’re safe from the robot apocalypse … for now. http://t.co/B7bME8hN

    Share
  6. RT @nytimesbits: Watson and Siri cannot understand one another, so we’re safe from the robot apocalypse … for now. http://t.co/B7bME8hN

    Share
  7. RT @nytimesbits: Watson and Siri cannot understand one another, so we’re safe from the robot apocalypse … for now. http://t.co/9rvjdrjS

    Share
  8. RT @nytimesbits: Watson and Siri cannot understand one another, so we’re safe from the robot apocalypse … for now. http://t.co/9rvjdrjS

    Share
  9. RT @nytimesbits: Watson and Siri cannot understand one another, so we’re safe from the robot apocalypse … for now. http://t.co/9rvjdrjS

    Share
  10. RT @nytimesbits: Watson and Siri cannot understand one another, so we’re safe from the robot apocalypse … for now. http://t.co/B7bME8hN

    Share
  11. RT @nytimesbits: Watson and Siri cannot understand one another, so we’re safe from the robot apocalypse … for now. http://t.co/B7bME8hN

    Share
  12. RT @joshuabrustein: Watson and Siri cannot understand one another, so we’re safe from the robot apocalypse … for now. http://t.co/1wO2B92y

    Share
  13. RT @joshuabrustein: Watson and Siri cannot understand one another, so we’re safe from the robot apocalypse … for now. http://t.co/1wO2B92y

    Share
  14. RT @nytimesbits: Watson and Siri cannot understand one another, so we’re safe from the robot apocalypse … for now. http://t.co/B7bME8hN

    Share
  15. RT @nytimesbits: Watson and Siri cannot understand one another, so we’re safe from the robot apocalypse … for now. http://t.co/B7bME8hN

    Share
  16. Misconceptions in AI: Or why Watson can’t talk to Siri http://t.co/UMgACboy #Apple #iPodtouch

    Share
  17. #apple Misconceptions in AI: Or why Watson can’t talk to Siri http://t.co/7byU3iPP

    Share
  18. RT @nytimesbits: Watson and Siri cannot understand one another, so we’re safe from the robot apocalypse … for now. http://t.co/mtKmcwYp

    Share
  19. RT @nytimesbits: Watson and Siri cannot understand one another, so we’re safe from the robot apocalypse … for now. http://t.co/mtKmcwYp

    Share
  20. RT @nytimesbits: Watson and Siri cannot understand one another, so we’re safe from the robot apocalypse … for now. http://t.co/mtKmcwYp

    Share
  21. “@nytimesbits: Watson and Siri cannot understand one another, so we’re safe from the robot apocalypse … for now. http://t.co/bjP0MR1n

    Share
  22. Misconceptions in AI: Or why Watson can’t talk to Siri — Tech News and Analysis http://t.co/vBP2DfhN #nothistype

    Share
  23. Misconceptions in AI: Or why Watson can’t talk to Siri — Tech News and Analysis http://t.co/vBP2DfhN #nothistype

    Share
  24. Misconceptions in AI: Or why Watson can’t talk to Siri — Tech News and Analysis http://t.co/vBP2DfhN #nothistype

    Share
  25. Rt @joshuabrustein: Watson and Siri cannot understand one another, so we’re safe from the robot apocalypse… for now. http://t.co/xxTnUBz7

    Share
  26. RT @nytimesbits: Watson and Siri cannot understand one another, so we’re safe from the robot apocalypse … for now. http://t.co/B7bME8hN

    Share
  27. RT @nytimesbits: Watson and Siri cannot understand one another, so we’re safe from the robot apocalypse … for now. http://t.co/B7bME8hN

    Share
  28. RT @nytimesbits: Watson and Siri cannot understand one another, so we’re safe from the robot apocalypse … for now. http://t.co/B7bME8hN

    Share
  29. RT @nytimesbits: Watson and Siri cannot understand one another, so we’re safe from the robot apocalypse … for now. http://t.co/B7bME8hN

    Share
  30. Misconceptions in AI: Or why Watson can’t talk to Siri http://t.co/OISPAjXs

    Share
  31. Misconceptions in AI: Or why Watson can’t talk to Siri: Me, Watson and Robert Walker of… http://t.co/vybqxqM6

    Share
  32. Misconceptions in AI: Or why Watson can’t talk to Siri: Me, Watson and Robert Walker of… http://t.co/vybqxqM6

    Share
  33. Misconceptions in AI: Or why Watson can’t talk to Siri: Me, Watson and Robert Walker of… http://t.co/vybqxqM6

    Share
  34. GrumpyMokeyPants Wednesday, November 16, 2011

    You obviously have not seen Colossus: The Forbin Project
    http://www.imdb.com/title/tt0064177/

    Share
    1. Computers will only “take over” if we allow them to. They are our creation.

      Share
  35. Misconceptions in AI: Or why Watson can’t talk to Siri – GigaOm http://t.co/Isxg2viE #ai

    Share
  36. Misconceptions in AI: Or why Watson can’t talk to Siri – GigaOm http://t.co/Isxg2viE #ai

    Share
  37. Misconceptions in AI: Or why Watson can’t talk to Siri http://t.co/zEgRSqKE

    Share
  38. Misconceptions in AI: Or why Watson can’t talk to Siri http://t.co/EdVU5Qxh

    Share
  39. @katiefehren with some nice writing on Siri and Watson: http://t.co/PR93AEjW

    Share
  40. Misconceptions in AI: Or why Watson can’t talk to Siri http://t.co/ddmskN31

    Share
  41. I think Wolfram Alpha which is used by Siri is a very powerful knowledge search engine so I am not sure that the statement that Siri doesn’t have the computing horsepower that Watson has could be way off base…

    Share
  42. recognition && categorization != understanding

    Also both systems ignore modern neurology, none takes timing into account to create context or sub-conscious responses. They are so far away from what we know today about neuronal systems it’s rather ….
    ————-
    In this study, Yuichi Yamashita and Jun Tani demonstrate that even
    without explicit spatial hierarchical structure a, functional hierarchy can self-organize through multiple timescales in neural activity. Their model was proven viable when tested with the physical body of a humanoid robot.

    Results suggest that it is not only the spatial connections between neurons, but also the timescales of neural activity, that act as important mechanisms in neural systems.[2]
    ———-
    Neurons discriminate among signals based on the signals’ “shape,” (how a signal changes over time), and Forger and coauthors found that, contrary to prior belief, a neuron’s preference depends on context. Neurons are often compared to transistors on a computer, which search for and respond to one specific pattern, but it turns out that neurons are more complex than that. They can search for more than one signal at the same time, and their choice of signal depends on what else is competing for their attention.[1]

    “Second, we found that the optimal stimulus is context-dependent,” he said, “so the best signal will differ, depending on the part of the brain where the implant is placed.” [1]
    ————

    1.http://www.sciencedaily.com/releases/2011/07/110708124538.htm
    2.http://www.sciencedaily.com/releases/2008/11/081107071820.htm

    Share
  43. Misconceptions in AI: Or why Watson can’t talk to Siri: Last night I was schooled at playing Jeopardy by Watson … http://t.co/WqY7HMdf

    Share
  44. The Apple Blog – Misconceptions in AI: Or why Watson can’t talk to Siri http://t.co/dAlMzf7B

    Share
  45. The Apple Blog – Misconceptions in AI: Or why Watson can’t talk to Siri http://t.co/dAlMzf7B

    Share
  46. “@CloudTopics: Misconceptions in AI: Or why Watson can’t talk to Siri: http://t.co/tcQ2NQb1 CloudComputingTopics” – interesting article #yam

    Share
  47. Misconceptions in AI: Or why Watson can’t talk to Siri | http://t.co/Z90O7Usg

    Share
  48. RT @matylda: I find this fascinating. Misconceptions in AI: Or why Watson can’t talk to Siri: http://t.co/NE3pJfNX

    Share
  49. RT @matylda: I find this fascinating. Misconceptions in AI: Or why Watson can’t talk to Siri: http://t.co/NE3pJfNX

    Share
  50. Misconceptions in AI: Or why Watson can’t talk to Siri http://t.co/auVCE9ss

    Share
  51. Misconceptions in AI: Or why Watson can’t talk to Siri http://t.co/auVCE9ss

    Share
  52. RT @nytimesbits: Watson and Siri cannot understand one another, so we’re safe from the robot apocalypse … for now. http://t.co/B7bME8hN

    Share
  53. RT @gigaom: Misconceptions in AI: Or why Watson can’t talk to Siri http://t.co/m0tKf0dg

    Share
  54. Move over Siri, Apparently humans are much smarter than we think http://t.co/BWuxTaLr

    Share
  55. Move over Siri, Apparently humans are much smarter than we think http://t.co/BWuxTaLr

    Share
  56. Move over Siri, Apparently humans are much smarter than we think http://t.co/BWuxTaLr

    Share
  57. RT @AppleMacGeek: The Apple Blog – Misconceptions in AI: Or why Watson can’t talk to Siri http://t.co/dAlMzf7B

    Share
  58. Misconceptions in AI: Or why Watson can’t talk to Siri: Me, Watson and Robert Walker of… http://t.co/K3S38yg8

    Share
  59. Misconceptions in AI: Or why Watson can’t talk to Siri: Me, Watson and Robert Walker of… http://t.co/K3S38yg8

    Share
  60. When building the semantic web you quickly learn how smart humans really are. More from GigaOm. http://t.co/KXpLa8dN

    Share
  61. When building the semantic web you quickly learn how smart humans really are. More from GigaOm. http://t.co/KXpLa8dN

    Share
  62. Misconceptions in AI: Or why Watson can’t talk to Siri http://t.co/62meoH57

    Share
  63. Misconceptions in AI: Or why Watson can’t talk to Siri http://t.co/62meoH57

    Share
  64. Misconceptions in AI: Or why Watson can’t talk to Siri: Last night I was schooled at playing Jeopardy by Watson … http://t.co/7fHS72ar

    Share
  65. You can simply go back to Heidegger’s Being & Time from over 70 years ago to know why AI is a fool’s errand

    Share
  66. Good post and furthermore if you took http://www.ai-class.com and followed @aiclass you would know exactly why ;)

    Share
  67. Marcus L Endicott Thursday, November 17, 2011

    Its not strictly true that Watson can’t, or rather couldn’t, talk to Siri. Of course, Watson can’t talk at all, without assistance. If IBM were to release a Watson API, then that could be integrated with Siri (once Apple releases a Siri SDK).

    Share
  68. “Watson actually doesn’t understand speech, but is fed a text version of the question” – this is why Watson isn’t something that everyone is still talking about. Watson is a news story but Siri was built for the masses and brings very advanced technology into everyone’s home. Plus, Apple invested in fun by making sure it would respond to people’s wacky questions. That’s why Siri has inspired tons of spin-offs from music videos to beeri. What has Watson inspired? Oh, that’s right. Nothing. Learn more from Siri! 3 Lessons Siri Can Teach You About Marketing
    http://blog.sfcopywriter.com/2011/10/25/3-lessons-siri-can-teach-you-about-marketing/

    Share
  69. Misconceptions in AI: Or why Watson can’t talk to Siri  http://t.co/3DJGkF5N

    Share
  70. RT @xivicinc: Misconceptions in AI: Or why Watson can’t talk to Siri  http://t.co/3DJGkF5N

    Share
  71. Misconceptions in AI: Or why Watson can’t talk to Siri http://t.co/TnOyY3mQ

    Share
  72. Mosaic Technology Friday, November 18, 2011

    I guess humans are still smarter than computers…. For now.

    Share
  73. RT @nytimesbits: Watson and Siri cannot understand one another, so we’re safe from the robot apocalypse … for now. http://t.co/B7bME8hN

    Share
  74. Misconceptions in AI: Or why Watson can’t talk to Siri http://t.co/D4TFxT9N via @zite

    Share
  75. Misconceptions in AI: Watson can’t talk to Siri http://t.co/5lJSOWI4 -both constrained (programmed for specific tasks). middleware agent?

    Share
  76. JReuben1 (@JReuben1) Friday, November 18, 2011

    So build a middle-ware agent to coordinate these disparate agents.

    Share
  77. Misconceptions in AI: Or why Watson can’t talk to Siri – by Stacey Higginbotham http://t.co/YOMZenVn

    Share
  78. Misconceptions in AI: Or why Watson can’t talk to Siri http://t.co/o6TsMvVn via @zite

    Share
  79. Misconceptions in AI: Or why Watson can’t talk to Siri http://t.co/P8HxG2On via @zite

    Share
  80. Misconceptions in AI: Or why Watson can’t talk to Siri — Tech News and Analysis http://t.co/KTi7OvuY

    Share
  81. Misconceptions in AI: Or why Watson can’t talk to Siri http://t.co/8vWBv6p9 via @zite

    Share
  82. Misconceptions in AI: Or why Watson can’t talk to Siri — Tech News and Analysis http://t.co/psYq4j8X

    Share
  83. Misconceptions in AI: Or why Watson can’t talk to Siri http://t.co/xEHJYc6M via @zite

    Share
  84. Misconceptions in AI: Or why Watson can’t talk to Siri http://t.co/zg36vh4m via @zite

    Share
  85. Watson vs. Siri http://t.co/KtJQi8GV #tech #technology

    Share
  86. Tech; 1) Misconceptions in AI: Or why Watson can’t talk to Siri – http://t.co/BWyNO5SI

    Share
  87. Misconceptions in AI: Or why Watson can’t talk to Siri – GigaOm http://t.co/J5NaSIfu

    Share
  88. Why Watson can’t talk to Siri: http://t.co/K31gn0Z0

    Share
  89. RT @vardi: Why Watson can’t talk to Siri: http://t.co/K31gn0Z0

    Share
  90. RT @vardi: Why Watson can’t talk to Siri: http://t.co/K31gn0Z0

    Share
  91. I don’t think Siri is more than a marketing hype. There’s plenty of bots (http://www.chatterboxchallenge.com/) that work much like Siri. Siri added speech recognition as bot’s input, which the masses use for fun and pleasure. It’s like playing with google and bing’s voice search for fun.

    I believe, Siri can be created and replicated in large number but not Watson. Might be that’s the key point of Siri.

    Share
  92. Understanding @IBMResearch technologies related to human-computer interaction/voice. @IBMWatson anyone? http://t.co/uLjPQ6Fe #IBMSocialBiz

    Share

Comments have been disabled for this post