11 Comments

Summary:

Last night I was schooled at playing Jeopardy by Watson in an exhibition match and discovered that despite our fear of the robot overlords, humans are much smarter than we think. Case in point: Watson could never use Apple’s personal assistant Siri.

Me, Watson and Robert Walker of Sierra Ventures onstage.

Me, Watson and Robert Walker of Sierra Ventures onstage.

Updated: Tuesday night, I was schooled at playing Jeopardy by Watson in an exhibition match at the Computer History Museum, and discovered that despite our fear of the robot overlords, humans are much smarter than we think. Case in point: Watson could never use Apple’s personal assistant Siri.

While both services seemingly understand what we’re saying to them and can respond with amazingly functional or accurate answers, the truth is they are both still programmed for specific tasks and could never actually converse with one another or a human outside of a narrow context. So Watson can’t take dictation, and Siri can’t play Jeopardy. Understanding why shows how far we have to go when it comes to true artificial intelligence and those fears of the robots taking over.

Dr. David Ferrucci of IBM

As David Ferrucci, the guy at IBM behind Watson’s creation, explained during a conversation before the match, that as intuitive as the interactions with Siri or Watson appear to us, they are fundamentally task oriented. So the questions Watson gets are in effect “translated” not just into the zeros and ones of digital signals but also to a series of words that are then broken down into related concepts.

After that point, Watson tries to ascribe “meaning” to those contexts based on searches of unstructured data to derive an answer. It then determines which answer is most likely to be correct, and how confident it is in that top answer, because in Jeopardy, if someone guesses the wrong answer, he or she (or it) is penalized. Thus Watson’s tasks are figuring out the context associated with a question, figuring out which answer is the likeliest based on that context and if it’s confident enough in its probabilities to bother to answer.

Watson is really confident that Chile is the right answer.

Siri, on the other hand, does two important things: It recognizes speech (Watson actually doesn’t understand speech, but is fed a text version of the question), and it can figure out what steps to take in a limited number of applications once it understands the words in a natural language process  related to the process by which Watson does. The sense from IBMers (unsurprisingly) is that Siri doesn’t have the natural language depth that Watson does. Siri certainly doesn’t have the computing horsepower behind it (2,880 processor cores and 15 terabytes of RAM), or the 100 GB of text data that Watson uses to figure out how different words relate to each other.

Watson started strong and dominated the board all night.

The net result of their differences? Not only could Siri and Watson not communicate because each relies on different input methods, but even if they could, their tasks are fundamentally far apart. Both have an ability to do natural language processing, but one uses that skill to find related information and figure out which information is most correct, while the other uses it to open applications and perform a set number of tasks.

So while Alan Turing proposed the best test of artificial intelligence is that a human can’t tell if it’s a human or computer he or she is interacting with, it may be more accurate to say the best test will be creating a machine that can not only understand natural language like Siri and Watson can, but has Watson’s ability to then determine the best course of action and Siri’s ability to take that action.

Update: IBM took issue with me saying that Watson can’t talk to Siri because Watson could indeed tell Siri, “What is Chile?” and Siri could go look it up. However, I meant talking as a synonym for conversing. Watson would have to be fed a text-based version of a question for it to come up with “What is Chile?” and Siri would have to overhear the question and could then provide the answer. That is technically talking, but it’s not a conversation.

All images, except for the Watson probabilities shot, courtesy of the Computer History Museum. The Watson probabilities shot is courtesy of IBM.

You’re subscribed! If you like, you can update your settings

  1. Watson and Siri cannot understand one another, so we’re safe from the robot apocalypse … for now. http://t.co/dC3z3MQ6

  2. Watson and Siri cannot understand one another, so we’re safe from the robot apocalypse … for now. http://t.co/dC3z3MQ6

  3. Watson and Siri cannot understand one another, so we’re safe from the robot apocalypse … for now. http://t.co/dC3z3MQ6

  4. NYTimes Bits Blog Wednesday, November 16, 2011

    Watson and Siri cannot understand one another, so we’re safe from the robot apocalypse … for now. http://t.co/B7bME8hN

  5. RT @nytimesbits: Watson and Siri cannot understand one another, so we’re safe from the robot apocalypse … for now. http://t.co/B7bME8hN

  6. RT @nytimesbits: Watson and Siri cannot understand one another, so we’re safe from the robot apocalypse … for now. http://t.co/B7bME8hN

  7. RT @nytimesbits: Watson and Siri cannot understand one another, so we’re safe from the robot apocalypse … for now. http://t.co/9rvjdrjS

  8. RT @nytimesbits: Watson and Siri cannot understand one another, so we’re safe from the robot apocalypse … for now. http://t.co/9rvjdrjS

  9. RT @nytimesbits: Watson and Siri cannot understand one another, so we’re safe from the robot apocalypse … for now. http://t.co/9rvjdrjS

  10. Tom Ternquist Sr Wednesday, November 16, 2011

    RT @nytimesbits: Watson and Siri cannot understand one another, so we’re safe from the robot apocalypse … for now. http://t.co/B7bME8hN

  11. Tom Ternquist Sr Wednesday, November 16, 2011

    RT @nytimesbits: Watson and Siri cannot understand one another, so we’re safe from the robot apocalypse … for now. http://t.co/B7bME8hN

  12. RT @joshuabrustein: Watson and Siri cannot understand one another, so we’re safe from the robot apocalypse … for now. http://t.co/1wO2B92y

  13. RT @joshuabrustein: Watson and Siri cannot understand one another, so we’re safe from the robot apocalypse … for now. http://t.co/1wO2B92y

  14. RT @nytimesbits: Watson and Siri cannot understand one another, so we’re safe from the robot apocalypse … for now. http://t.co/B7bME8hN

  15. RT @nytimesbits: Watson and Siri cannot understand one another, so we’re safe from the robot apocalypse … for now. http://t.co/B7bME8hN

  16. Misconceptions in AI: Or why Watson can’t talk to Siri http://t.co/UMgACboy #Apple #iPodtouch

  17. #apple Misconceptions in AI: Or why Watson can’t talk to Siri http://t.co/7byU3iPP

  18. Andryan Viryadi T Wednesday, November 16, 2011

    RT @nytimesbits: Watson and Siri cannot understand one another, so we’re safe from the robot apocalypse … for now. http://t.co/mtKmcwYp

  19. Andryan Viryadi T Wednesday, November 16, 2011

    RT @nytimesbits: Watson and Siri cannot understand one another, so we’re safe from the robot apocalypse … for now. http://t.co/mtKmcwYp

  20. Andryan Viryadi T Wednesday, November 16, 2011

    RT @nytimesbits: Watson and Siri cannot understand one another, so we’re safe from the robot apocalypse … for now. http://t.co/mtKmcwYp

  21. “@nytimesbits: Watson and Siri cannot understand one another, so we’re safe from the robot apocalypse … for now. http://t.co/bjP0MR1n

  22. Misconceptions in AI: Or why Watson can’t talk to Siri — Tech News and Analysis http://t.co/vBP2DfhN #nothistype

  23. Misconceptions in AI: Or why Watson can’t talk to Siri — Tech News and Analysis http://t.co/vBP2DfhN #nothistype

  24. Misconceptions in AI: Or why Watson can’t talk to Siri — Tech News and Analysis http://t.co/vBP2DfhN #nothistype

  25. Rt @joshuabrustein: Watson and Siri cannot understand one another, so we’re safe from the robot apocalypse… for now. http://t.co/xxTnUBz7

  26. RT @nytimesbits: Watson and Siri cannot understand one another, so we’re safe from the robot apocalypse … for now. http://t.co/B7bME8hN

  27. RT @nytimesbits: Watson and Siri cannot understand one another, so we’re safe from the robot apocalypse … for now. http://t.co/B7bME8hN

  28. RT @nytimesbits: Watson and Siri cannot understand one another, so we’re safe from the robot apocalypse … for now. http://t.co/B7bME8hN

  29. Vasilis Vassalos Wednesday, November 16, 2011

    RT @nytimesbits: Watson and Siri cannot understand one another, so we’re safe from the robot apocalypse … for now. http://t.co/B7bME8hN

  30. Katie Fehrenbacher Wednesday, November 16, 2011

    Misconceptions in AI: Or why Watson can’t talk to Siri http://t.co/OISPAjXs

  31. Misconceptions in AI: Or why Watson can’t talk to Siri: Me, Watson and Robert Walker of… http://t.co/vybqxqM6

  32. Misconceptions in AI: Or why Watson can’t talk to Siri: Me, Watson and Robert Walker of… http://t.co/vybqxqM6

  33. Misconceptions in AI: Or why Watson can’t talk to Siri: Me, Watson and Robert Walker of… http://t.co/vybqxqM6

  34. GrumpyMokeyPants Wednesday, November 16, 2011

    You obviously have not seen Colossus: The Forbin Project
    http://www.imdb.com/title/tt0064177/

    1. Computers will only “take over” if we allow them to. They are our creation.

  35. Nova-box AI News Wednesday, November 16, 2011

    Misconceptions in AI: Or why Watson can’t talk to Siri – GigaOm http://t.co/Isxg2viE #ai

  36. Nova-box AI News Wednesday, November 16, 2011

    Misconceptions in AI: Or why Watson can’t talk to Siri – GigaOm http://t.co/Isxg2viE #ai

  37. Misconceptions in AI: Or why Watson can’t talk to Siri http://t.co/zEgRSqKE

  38. Misconceptions in AI: Or why Watson can’t talk to Siri http://t.co/EdVU5Qxh

  39. @katiefehren with some nice writing on Siri and Watson: http://t.co/PR93AEjW

  40. Misconceptions in AI: Or why Watson can’t talk to Siri http://t.co/ddmskN31

  41. I think Wolfram Alpha which is used by Siri is a very powerful knowledge search engine so I am not sure that the statement that Siri doesn’t have the computing horsepower that Watson has could be way off base…

  42. recognition && categorization != understanding

    Also both systems ignore modern neurology, none takes timing into account to create context or sub-conscious responses. They are so far away from what we know today about neuronal systems it’s rather ….
    ————-
    In this study, Yuichi Yamashita and Jun Tani demonstrate that even
    without explicit spatial hierarchical structure a, functional hierarchy can self-organize through multiple timescales in neural activity. Their model was proven viable when tested with the physical body of a humanoid robot.

    Results suggest that it is not only the spatial connections between neurons, but also the timescales of neural activity, that act as important mechanisms in neural systems.[2]
    ———-
    Neurons discriminate among signals based on the signals’ “shape,” (how a signal changes over time), and Forger and coauthors found that, contrary to prior belief, a neuron’s preference depends on context. Neurons are often compared to transistors on a computer, which search for and respond to one specific pattern, but it turns out that neurons are more complex than that. They can search for more than one signal at the same time, and their choice of signal depends on what else is competing for their attention.[1]

    “Second, we found that the optimal stimulus is context-dependent,” he said, “so the best signal will differ, depending on the part of the brain where the implant is placed.” [1]
    ————

    1.http://www.sciencedaily.com/releases/2011/07/110708124538.htm
    2.http://www.sciencedaily.com/releases/2008/11/081107071820.htm

  43. Misconceptions in AI: Or why Watson can’t talk to Siri: Last night I was schooled at playing Jeopardy by Watson … http://t.co/WqY7HMdf

  44. The Apple Blog – Misconceptions in AI: Or why Watson can’t talk to Siri http://t.co/dAlMzf7B

  45. The Apple Blog – Misconceptions in AI: Or why Watson can’t talk to Siri http://t.co/dAlMzf7B

  46. “@CloudTopics: Misconceptions in AI: Or why Watson can’t talk to Siri: http://t.co/tcQ2NQb1 CloudComputingTopics” – interesting article #yam

  47. Misconceptions in AI: Or why Watson can’t talk to Siri | http://t.co/Z90O7Usg

  48. RT @matylda: I find this fascinating. Misconceptions in AI: Or why Watson can’t talk to Siri: http://t.co/NE3pJfNX

  49. RT @matylda: I find this fascinating. Misconceptions in AI: Or why Watson can’t talk to Siri: http://t.co/NE3pJfNX

  50. Misconceptions in AI: Or why Watson can’t talk to Siri http://t.co/auVCE9ss

  51. Misconceptions in AI: Or why Watson can’t talk to Siri http://t.co/auVCE9ss

  52. RT @nytimesbits: Watson and Siri cannot understand one another, so we’re safe from the robot apocalypse … for now. http://t.co/B7bME8hN

  53. RT @gigaom: Misconceptions in AI: Or why Watson can’t talk to Siri http://t.co/m0tKf0dg

  54. Move over Siri, Apparently humans are much smarter than we think http://t.co/BWuxTaLr

  55. Move over Siri, Apparently humans are much smarter than we think http://t.co/BWuxTaLr

  56. Move over Siri, Apparently humans are much smarter than we think http://t.co/BWuxTaLr

  57. Joe iPhoneography  Wednesday, November 16, 2011

    RT @AppleMacGeek: The Apple Blog – Misconceptions in AI: Or why Watson can’t talk to Siri http://t.co/dAlMzf7B

  58. Misconceptions in AI: Or why Watson can’t talk to Siri: Me, Watson and Robert Walker of… http://t.co/K3S38yg8

  59. Misconceptions in AI: Or why Watson can’t talk to Siri: Me, Watson and Robert Walker of… http://t.co/K3S38yg8

  60. When building the semantic web you quickly learn how smart humans really are. More from GigaOm. http://t.co/KXpLa8dN

  61. When building the semantic web you quickly learn how smart humans really are. More from GigaOm. http://t.co/KXpLa8dN

  62. Misconceptions in AI: Or why Watson can’t talk to Siri http://t.co/62meoH57

  63. Misconceptions in AI: Or why Watson can’t talk to Siri http://t.co/62meoH57

  64. Misconceptions in AI: Or why Watson can’t talk to Siri: Last night I was schooled at playing Jeopardy by Watson … http://t.co/7fHS72ar

  65. You can simply go back to Heidegger’s Being & Time from over 70 years ago to know why AI is a fool’s errand

  66. Good post and furthermore if you took http://www.ai-class.com and followed @aiclass you would know exactly why ;)

  67. Marcus L Endicott Thursday, November 17, 2011

    Its not strictly true that Watson can’t, or rather couldn’t, talk to Siri. Of course, Watson can’t talk at all, without assistance. If IBM were to release a Watson API, then that could be integrated with Siri (once Apple releases a Siri SDK).

  68. “Watson actually doesn’t understand speech, but is fed a text version of the question” – this is why Watson isn’t something that everyone is still talking about. Watson is a news story but Siri was built for the masses and brings very advanced technology into everyone’s home. Plus, Apple invested in fun by making sure it would respond to people’s wacky questions. That’s why Siri has inspired tons of spin-offs from music videos to beeri. What has Watson inspired? Oh, that’s right. Nothing. Learn more from Siri! 3 Lessons Siri Can Teach You About Marketing
    http://blog.sfcopywriter.com/2011/10/25/3-lessons-siri-can-teach-you-about-marketing/

  69. Misconceptions in AI: Or why Watson can’t talk to Siri  http://t.co/3DJGkF5N

  70. RT @xivicinc: Misconceptions in AI: Or why Watson can’t talk to Siri  http://t.co/3DJGkF5N

  71. Misconceptions in AI: Or why Watson can’t talk to Siri http://t.co/TnOyY3mQ

  72. Mosaic Technology Friday, November 18, 2011

    I guess humans are still smarter than computers…. For now.

  73. RT @nytimesbits: Watson and Siri cannot understand one another, so we’re safe from the robot apocalypse … for now. http://t.co/B7bME8hN

  74. Misconceptions in AI: Or why Watson can’t talk to Siri http://t.co/D4TFxT9N via @zite

  75. Misconceptions in AI: Watson can’t talk to Siri http://t.co/5lJSOWI4 -both constrained (programmed for specific tasks). middleware agent?

  76. JReuben1 (@JReuben1) Friday, November 18, 2011

    So build a middle-ware agent to coordinate these disparate agents.

  77. Misconceptions in AI: Or why Watson can’t talk to Siri – by Stacey Higginbotham http://t.co/YOMZenVn

  78. Misconceptions in AI: Or why Watson can’t talk to Siri http://t.co/o6TsMvVn via @zite

  79. Misconceptions in AI: Or why Watson can’t talk to Siri http://t.co/P8HxG2On via @zite

  80. Misconceptions in AI: Or why Watson can’t talk to Siri — Tech News and Analysis http://t.co/KTi7OvuY

  81. Misconceptions in AI: Or why Watson can’t talk to Siri http://t.co/8vWBv6p9 via @zite

  82. Misconceptions in AI: Or why Watson can’t talk to Siri — Tech News and Analysis http://t.co/psYq4j8X

  83. Misconceptions in AI: Or why Watson can’t talk to Siri http://t.co/xEHJYc6M via @zite

  84. Misconceptions in AI: Or why Watson can’t talk to Siri http://t.co/zg36vh4m via @zite

  85. Courtney Wallin Tuesday, November 22, 2011

    Watson vs. Siri http://t.co/KtJQi8GV #tech #technology

  86. Tech; 1) Misconceptions in AI: Or why Watson can’t talk to Siri – http://t.co/BWyNO5SI

  87. Misconceptions in AI: Or why Watson can’t talk to Siri – GigaOm http://t.co/J5NaSIfu

  88. Why Watson can’t talk to Siri: http://t.co/K31gn0Z0

  89. RT @vardi: Why Watson can’t talk to Siri: http://t.co/K31gn0Z0

  90. Maria Tsiakmaki Sunday, November 27, 2011

    RT @vardi: Why Watson can’t talk to Siri: http://t.co/K31gn0Z0

  91. I don’t think Siri is more than a marketing hype. There’s plenty of bots (http://www.chatterboxchallenge.com/) that work much like Siri. Siri added speech recognition as bot’s input, which the masses use for fun and pleasure. It’s like playing with google and bing’s voice search for fun.

    I believe, Siri can be created and replicated in large number but not Watson. Might be that’s the key point of Siri.

  92. Christina Howell Friday, December 2, 2011

    Understanding @IBMResearch technologies related to human-computer interaction/voice. @IBMWatson anyone? http://t.co/uLjPQ6Fe #IBMSocialBiz

Comments have been disabled for this post