@ SxSW: Bryan Caplan: Questioning The Wisdom Of Crowds


We see tons of startups predicated on the so-called “wisdom of crowds”, an idea that’s been so thoroughly internalized by people, the actual premise, that crowds are wise, are rarely questioned. But maybe when the individuals within the crowd don’t know anything, the crowd doesn’t either. Bryan Caplan, an econ prof at George Mason University, said he was invited to speak at SxSW Interactive to provide a counterpoint to James Suriowiecki’s popular book The Wisdom of Crowds. Caplan’s own book, The Myth of The Rational Voter argues that typical voting schemes often result in bad policies. He makes a somewhat elitist argument, which he’ll acknowledge, but coming from an economist, there’s a lot more depth to it than, say, similar arguments from Andrew Keen, whose viewpoint can pretty much be boiled down to “experts rule and the plebes drool.” While Caplan’s presence here may seem a bit far afield form everything else, it’s not hard to make the connection.

The Mirage of Aggregation: The idea that voting works is based on the so-called “Miracle of aggregation”, which Caplan likens to a form of alchemy. You take some uninformed opinions, mesh them together, and shake it all up and voila, it’s informed.” While Surowiecki’s book cited a number of examples of crowd wisdom (groups of people collaboratively guessing the number of jellybeans in a jar or guessing the weight of a cow), Caplan criticized what he saw was a “(weak) hunt for counter-examples.” His one-off counter-example: In polls asking voters what percentage of the budget is spent on foreign aid, averages come around 10-20 percent, when in fact it’s actually just 1.2 percent. Ultimately, in Caplan’s view: “The miracle of aggregation fails and it fails very directly.” As that relates to democracy: “What’s interesting about this miracle of aggregations is it gives people a way to believe democracy works despite the public’s deficiencies.”

Predictions markets and the web: I asked during the Q&A what the implication of his work on stuff like prediction markets and other wisdom of crowds-based startups. As he argued, there’s a huge difference between public opinion polling and betting markets. For one thing, there’s a financial penalty for being wrong. So ignorance gets blinded down, so to speak. The problem is when there’s no policy for being wrong, and no way to keep out the ignorant: “On almost any question, if the betting market says something I believe it.” He didn’t address it directly, but the question for a site like, say, Digg, is to determine whether there’s an adequate penalty for being wrong. Typically, for a non-financial site, the currency is reputation.

Bottom line: Caplan’s presentation and work should be though-provoking from a web perspective, but it’s not ultimately damning. Nor is he in as much contradiction with Surowiecki as he suggests. He argues persuasively that you can’t bake wisdom by cooking with ignorance, but it doesn’t mean that things like prediction markets, or other collaborative filtering services (Last.fm, Netflix (NSDQ: NFLX), etc.) can’t work, when people voluntarily.



I don't think Bryan Caplan's foreign aide example correctly compares with James Surowiecki's jelly beans and pigs examples. He may still be right in his distrust of the "miracle of aggregation," but I think he's comparing apples to oranges in this sense. All the information necessary to correctly guess the number of jelly beans in a jar is directly in front of the guesser, the whole problem is in plain view, as is the pig's weight, there is no problem of asymmetric information at the time the guess was placed.

The problem of the foreign aide example is that the concept is a bit more abstract, only the people who are familiar with federal budgets would even get close. Given a sense of how large the budget is or how much is spent on aide would have surely gotten the group closer without giving them the answer to the question. Your average person knows very little about foreign aide budgets.

Anton Mannering

Talk about missing the point. User voting sites are not about being "accurate" their about relevance. No one is better qualified to define what's relevant to an interest area than those interested in it. If anyone thinks they are then they are either being disingenuous to protect the staus quo or they are just not very bright. That's what opinion polls and consumer surveys that experts reference are for!
Comparing this to peoples knowledge of foreign aid expenditure or anything else is facetious at best. It is just totally irrelevant. The reason people don't know is because they don't care, they aren't really intrerested.
Get an aggregate of answers from, for instance, foreign aid workers or people who work in charities or have an interest int he field and your answer will be MUCH more accurate.
Please can someone stop trotting out "experts" whose only interest is self preservation. This is deliberately misleading to make political capital. journalists regularly try this one as well.


I don't think Caplan was arguing about the viability of democracies. Rather he was pointing out that democracies don't produce the best or optimal solutions. In that respect, he's right but only because democracies were never designed to generate ideal policy. A democracy reflects the the desires of its members regardless of what's sound from a long-term policy or economic standpoint. Post-9/11, our nation's policies reflected a large portion of the population that was ready to bomb someone to oblivion. Right or wrong, the resulting policies reflected part of the democracy.

Frankly, I think Suriowiecki's arguments are weak at best. 'Wisdom of Crowds' just pulls together a number of central principles involving large numbers: surveying, power laws, law of averages, mean/median/SD, collective work output. For subjective concepts, sample enough individuals and you get a fairly accurate representation of preferences. Woohoo – that's called a survey! For a work product like Google or Wikipedia, it's just harnessing tiny amounts of work from millions of people. There's no 'wisdom' at play – all we're seeing is the result of tools that are effective at organizing opinions or work units.

Sue Sparks

The example of people's misperceptions on foreign aid is interesting – I wonder if he asks why that is. Surely it has to do with the (still remaining) power of the mainstream media in shaping perceptions. The same would apply to fear of crime (e.g. elderly white women being most afraid, when young black men are most at risk) and attitudes to immigration and asylum seekers (people in the UK think it takes in the most refugees of any country, whereas Pakistan does and the UK is not even the highest in Europe). Look to the front pages of popular newspapers for the reasons behind this 'ignorance'. So the question becomes how to ensure citizens are better informed, rather than concluding that democracy can't work because of ignorance.

Comments are closed.