What a relief to talk about something other than my distinguished colleague Prof. Sachs…. over to you, Dambisa Moyo!
Now back to real work: the reader survey generated a great response – thank you readers! It confirmed a well known psychology experiment, but also contained surprises I did not expect.
The question was which was more probable, (A) or (B)?
(A) a country succeeds at economic development, or
(B) a country succeeds at economic development with a wise and capable leadership.
60 percent of you readers voted A, and 40 percent chose B. (Interestingly, a small sample on my Facebook page voted in the same percentages on the same question.)
On one level, A is the right answer, because B is a subset of A. A contains all successes, both (1) those achieved with wise leadership, and (2) those achieved with any other means. B only contains (1), and so is less likely than A. Well known psychology experiments find the same thing — that many people have what is called the “conjunction fallacy” (again from my continuing Mlodinow and Kahneman obsession) that would cause them to choose (B). A set of outcomes that fits a plausible story is thought to be larger than one unrestricted by ANY story, even though ANY restriction on the set of possible outcomes always makes that set less likely than an unrestricted set. An explanation usually trumps no explanation, even if it gets the probabilities wrong!
But on another level, the reaction of many readers made me aware of how I had phrased the alternatives too sloppily, which taught me something about how the language we commonly use is often fuzzy on exactly what probabilities we are talking about. I think many of those who voted B were interpreting the question differently: when is development success more likely? With good leadership (B)? Or when the quality of the leadership is unspecified, and so could be either good or bad (A)? Obviously (B). Neither our brain wiring nor our education is good enough to give us linguistic precision about probability and randomness. So my sloppy language created a coalition in favor of (B) between an incorrect answer and a correct answer! How many such coalitions exist on development issues?
There is another related bias that is, called the “halo effect,” often discussed in recruiting job candidates (and also the subject of a great business book). An interviewer who quantifies one positive trait in a candidate excessively assumes that the candidate also performs well on other traits. Later quantitative evaluation finds the traits are not as correlated as the interviewer (or any of the rest of us) assume. So, for example, a beauty queen is not as likely to be a nice person as you think (could this be an excuse to mention the hilarious Miss California parody by Lisa Nova?).
What does this have to do with development? Well, a country that performs well on GDP per capita is also assumed to perform well on having wise and capable leadership. The latter is hard to quantify, so in many cases, our halo effect bias never gets corrected.
So sloppy language about probability, the conjunction fallacy, and the halo effect all make us assume that if the country has a good economic outcome, there is also a good political outcome (wise and capable leadership), even if we have no independent evidence that the leadership is wise. We do this in all countries (and assume bad leaders in unsuccessful countries), and then we notice a strong association between quality of leaders and development success! Therefore (adding the correlation=causation fallacy for good measure, which has its own cartoon) good leaders cause success! This amounts to the most elementary fallacy of them all – circular reasoning – which is still amazingly common in development debates.
Now would anybody like to go back and reread the “Asian success mythology” discussion, and think from yet another angle about whether East Asian successes were due to wise and capable leaders? And maybe if the leaders were not so perfectly wise in East Asia, we don’t necessarily want to imitate everything they did?
I guess the lesson in the end is to be precise about probabilities, and demand independent evidence on the whole “wise and capable leadership” thing, rather than just assuming it when things turn out well.
9 Comments
I think a number of readers wondered at the time about the wording of the choice set. I personally took (A) to mean the unconditional P(successful economic development) and (B) to mean the conditional P(successful economic development | wise and capable leadership), NOT the intersection P(successful economic development ∩ wise and capable leadership). This makes the choice a question of whether leadership is positively correlated with economic outcomes, which I turn to Jones and Olken (2005) to suggest that it is.
I think one of the key problems in your question, and indeed this is probably a lesson for the debate itself, was that questions for clarification seemed to be disallowed by the “with no elaboration” comment. I think the failure to elaborate or clarify the terms of the debate lead to a lot more errors than any conjunction fallacy bias at work.
“when is development success more likely? With good leadership (B)? Or when the quality of the leadership is unspecified, and so could be either good or bad (A)? Obviously (B).”
I would like to hear more from you on how good leaders ‘obviously’ make developmental success more likely. Could it be that not all ‘Planners’ are hellbent on driving their countries towards disaster? Surely not!
I didn’t vote, but I would have voted for the “fallacy”, B (by the way, how nice of you to call those with another opinion proponents of a fallacy… I didn’t realize we were dealing with mathematics here, rather than diverging opinions).
The reason I chose B (and, I believe, many others chose B) is that A is basically a non-sensical statement. It has no meaning. It doesn’t explain anything. It looks like it was torn out of context, especially when compared to B. When people read statements, they look for meaning. Hence, they may read “between the lines”, and alter somewhat the phrasing into something like “when is development…” as you indicate.
I think this is a clear case of poor survey design. Obviously the countries succeeding at economic development are a subset of the countries succeeding at economic development AND enjoying a wise and capable leadership, but both (A) and (B) would be equally probable if all countries succeeding at economic development happened to enjoy wise and capable leaders. I guess you would qualify as “wise” and “capable” the leaders of a country that fails at economic development if they show their wisdom and capacity in other areas of governance, but then we are discussing semantics, not probablilty. You want to define what is a good political outcome in a way that is completely independent from what is a good economic outcome. This sounds as nonsense to me. Even in developed countries leaders are usually judged (by voters) on the basis of economic performance. So, please, think of a better example (and keep with these Mlodinow/Kahneman posts).
Defining wise and capable leadership: The leadership in any country which experiences economic development is wise and capable by default. Ergo, “(B)” is more probable.
My own “(A) ⊇ B” – answer was considered a spoiled ballot and hence deleted though.
I saw that one was a subset of the other and voted accordingly.
@Filip the the word fallacy is being used in its technical sense. As in, which is more likely:
A. That I am writing this comment on a laptop
B. That I am writing this comment on a black laptop
A is more likely because it includes B within it. Picking B is a fallacy, a logical error.
Filip, You raise a good point about whether I am equating an opinion with a fallacy. If I was doing this, I would be committing a fallacy myself. At best, the experiment may have showed: IF you fall for such and such fallacy, THEN you hold such and such opinion. But it does NOT follow that IF you hold such and such opinion, THEN you fall for the fallacy. Beware of the fallacy, but of course you can make the case for an opinion on other, totally logical grounds. Best regards, Bill Easterly
Ok Bill, good to have that out of the way. What I think this interesting experiment shows is that surveys, however useful (and I’m a statistician myself), violate a basic rule of the search for truth (or better opinions if you want), namely dialogue, or the “public use of reason” in Kant’s phrase. Just taking a survey doesn’t give one the opportunity to understand how the survey population understands the survey, which leads to false conclusions about their understanding. I’m not sure I’m getting my point across here and I see that this is already taking too much space in this comment section, so I’ll just redirect you here (I promise I won’t make a habit of linking to myself!)
What I think is interesting is the fact that 60% of respondents answered that A is more probable. I wonder why, especially when it seems that thinking in many ODA circles is better characterized by B.
Perhaps your readers know this experiment, and so chose A, or perhaps theyve read your books and others about how economic development is an extremely decentralized process, and that it can happen with good or bad leadership (do we really think that we’ve had only good leadership during periods of growth in the West?).
So, maybe there is a sampling bias in the fact that you asked the people who read your blog to respond, when these people are already familiar with your arguments, and so do not really represent the more mainstream ODA community.