3 Comments

Summary:

Truth Teller is a prototype launched by the Washington Post — with funding from the Knight Foundation — that is designed to fact-check political speeches in real time. But can it do this? And will anyone care?

Funded in part by the Knight Foundation for Journalism, the Washington Post has just launched a new project called “Truth Teller,” which is designed to fact-check political speeches and comments in real time. It’s an ambitious effort that is based on an idea the Post‘s political editor had in 2011, and makes use of speech-to-text technology and a number of other tools. But the project raises a number of thorny questions, including: Is it even possible to fact-check the most egregious types of political disinformation? And if Truth Teller does manage to do this, will anyone care?

The funding for the venture comes from the Knight Prototype Fund, a relatively new effort from the foundation that is designed to help jump-start new ideas and startups — it invests via grants of up to $50,000 and says that the program is intended to help entrepreneurs “build fast [and] fail fast” while they try to reinvent the way that journalism works online.

The Washington Post‘s political editor, Steven Ginsberg, got the idea for a real-time fact-checking service while he was watching a speech given by former Republican presidential candidate Michele Bachmann in 2011. During her talk at a sports bar in Iowa, the candidate repeatedly misled her audience and Ginsberg said he envisioned a mobile app that would have allowed anyone to check whether she was telling the truth as she was speaking, and then challenge her statements.

Truth Teller checks facts against a Post database

The beta prototype that the Post launched on Tuesday isn’t quite there yet — it is only capable of checking facts related to taxation law and proposed changes to those laws, information that has been hand-fed to the system. But eventually, the newspaper says that the application should be able to do this about a wide range of topics by checking the Post‘s database in real time.

Truth Teller screenshot scaled

The Post‘s executive producer for digital news, Cory Haik, explains that the application turns speech from a video into text and then checks statements from that text version with an algorithm (a modified Rabin-Karp using Levenshtein distance). While watching a video of House Speaker John Boehner, for example, when he says that raising the top tax rate would “kill 700,000 jobs,” the application pops up a large warning in red, saying: “False.”

Below the video, there is a transcript of the speech, and users can click on any sentence and go to that section of the video — and they can also “view source” and see the basis for the newspaper’s claim that something is false. In the case of the Speaker of the House’s claim, it’s a piece from Glenn Kesler, who writes a column called The Fact Checker, and wrote a post specifically addressing Boehner’s statement — and ultimately awarded it three Pinocchios, which are the newspaper’s version of a thumbs down on the facts.

Political truth can be a slippery thing

The topic of fact-checking political statements became the focus of much debate during the run-up to the election, especially after the former public editor for the New York Times wrote a post asking whether the newspaper’s reporters should be “truth vigilantes” — in other words, whether they should challenge politicians on their statements during the campaign, or just report what they said and fact-check them later. The reaction from readers was astonishment that the newspaper wouldn’t want to do the former, but Brisbane argued that fact-checking is often harder than it seems.

This is one of the potential flaws with an approach like the Post is taking with Truth Teller: such automated systems likely work best when they have specific facts to drawn on, such as Boehner’s claim about 700,000 jobs being lost. But in many cases, the mis-statements that politicians make are much more vague and subject to interpretation than that, and so would be difficult to check against a database of facts.

The other problem is that even Ginsberg’s example of the Bachmann speech assumes that any of those listening would have wanted to fact-check her comments in real time. In the same way, the Truth Teller system — and other similar efforts such as the “Truth Goggles” project, or even more established fact-checking services like Politifact — assume that anyone will take the time to actually go there and check the facts. But will they?

Post and thumbnail images courtesy of Shutterstock / Donskarpo

You’re subscribed! If you like, you can update your settings

  1. One of the biggest flaws in any fact-checking system is built-in: namely that they only check asserted facts (“errors of commission”). They aren’t designed to handle the harder-to-detect “errors of omission”.

    A quick example:

    Fact: drones kill terrorists.

    The above statement would get a pass in any fact-checking system but a knowledgeable reader would immediately discern that plenty of other facts about drone warfare are omitted. Facts which may have a bearing on drones and terrorists.

    Fact-checking systems don’t take into account how humans communicate. At best they will be 100% correct but largely irrelevant, at worst they will be hopelessly easy to game.

    1. Unless the software is sophisticated enough to provide a comprehensive analysis, taking all of your objections into account and giving a summary opinion, pro and con. The sky is the limit on the potential of this kind of thing; I would NOT write it off at all!

  2. What’s fact checking without logical fallacy checking, too?

Comments have been disabled for this post