As major world leaders jet from the UN General Assembly yesterday to the Pittsburgh G-20 today, the UN and World Bank have bombarded them with messages and statistics about the effect of the crisis on the global poor:
(1) We need to know how many are suffering where, so that help can be targeted to those in most need,
(2) Here are our precise numbers of how many additional poor have been created by the crisis,
(3) Since we based the numbers in (2) on thin evidence or no evidence whatsoever, you should also give us more money to expand our abuse of statistics.
Here are some examples to illustrate these three points:
(1) Secretary General Ban Ki-Moon announced yesterday, “We need to know who is being hurt, and where, so we can best respond.” He handed out a new UN report, “Voices of the Vulnerable” with lots of these numbers.
The need to know precise numbers is not so obvious, since the international aid system lacks any central authority that has the skill or power to redeploy aid resources from areas of less poverty to areas of increased poverty. Even in normal times, the relationship between level of poverty by country and aid given to that country (even correcting for quality of government) is not that strong.
(2) “Voices of the Vulnerable” says “in 2009 about 100 million more people will be trapped in extreme poverty … than was anticipated before the …crisis.” This figure is based on a World Bank paper prepared for the G20, which actually said “the crisis will leave an additional 89 million people in extreme poverty … at the end of 2010.” This number is similar to the number in today’s FT oped by WB President Zoellick, except that he said that 90 million had already been pushed into poverty by “food, fuel and now financial crises” (i.e. 2007-2009)
The Bank’s 89 million claim, in turn, is based on a paper by Chen and Ravallion, which actually predicted “the crisis will add 53 million people to the 2009 count of the number of people living below $1.25 a day.” “Voices of the Vulnerable” report also cites figures from the ILO that “as many as 222 million additional workers worldwide run the risk of joining the ranks of the extreme working poor over the period 2007–2009.”
So precise estimates guide us to redeploy resources to the 100 million, or 89 million, or 53 million, or 222 million that were driven into poverty either in 2009, or 2009-2010, or 2007-2009, or 2008-2009.
There is an obscure piece of theoretical statistics called “garbage in, garbage out.” Calculating “additional poor in poverty due to crisis” requires (a) knowing what growth would have been in absence of crisis in every country, (b) knowing what growth will actually turn out to be in 2009 or 2010 in every country, not to mention in 2008, since the World Bank’s World Development Indicators do not yet have estimates for that year, (c) having good data on the current level of world poverty, (d) knowing the effect of growth on poverty, (e) projecting the effect of food and fuel prices on poverty, not to mention projecting food and fuel prices.
The reality: (a) is impossible, (b) is almost impossible, (c) Voices of the Vulnerable says the last real global poverty numbers were in 2005, which themselves reflected an upward revision of 40 percent ,(d) is unreliable and volatile, and (e) is impossible.
Economists can do useful projections sometimes, but the castles in the air implied by (a) through (e) should have caused a responsible analyst to NOT invent such a number.
Unfortunately, the made-up poverty numbers look positively respectable compared to other claims in the UN Voices of the Vulnerable that are based on no known statistics whatsoever:
“Women and children are likely to bear the brunt of the crisis…depression and drug and alcohol abuse could be on the rise….{including} consumption of strong local brews….{There are} rises in domestic violence…{There are} increased social tensions within communities.”
(3) In the same report, the Secretary-General admits: “More than a year in, what we do not know overshadows what we know.” But he offers to produce: “a networked 21st Century global system for real-time monitoring of the impacts of this and future global crises on the most vulnerable and poor: a Global Impact and Vulnerability Alert System (GIVAS). This system will require … resources.”
So … give more “resources” to serial inventers of numbers to invent more numbers.
Why does all this matter? Because serious numbers are useful in analyzing how best to help alleviate poverty. The onslaught of imaginary numbers weakens that cause while accomplishing nothing for the poor.
19 Comments
It’s time economists and people who generate/use/disseminate numbers included margins of error, as any decent physicist or statistician would.
And we should expect to be given these margins along with every single number — just to make sure they did their homework.
I agree with phme above. Imagine what range a 90% confidence interval would be for some of these statistics? Presumably some of them have not done their homework, or at least not to the degree that they could share it without losing credibility.
It was high time Economists truly have numbers that will will predict the true nature of how the crisis has affected the poor.
Great post.. This is the kind of thing I go to Aid Watch for.
The truth is that is pays to inflate numbers. People in this business with less skepticism or a reasonable fear of being accused of insensitivity to suffering will not question papers that trot out big numbers with no back up. Then the numbers take on a life of their own and grow as they are cited again and again. See Elizabeth Pisani’s great book Wisdom of Whores for an account of how “punching up the numbers” helped UNAIDS increase its budget.
Brent and phme, You are correct of course that the problem would be alleviated if everybody understood standard errors of the estimate and confidence intervals. Under some circumstances, confidence intervals blow up to infinite. That is the confidence interval is between minus infinity and plus infinity. The strongest form of our critique would be that is the case with some or all of these numbers. A weaker form of our argument is that the confidence interval is finite but so large as to make the number meaningless. That is if the confidence interval for the poverty number was (-100 million to +300 million), that is pretty useless as there is even a possibility that poverty fell because of the crisis (seems implausible, but if the estimate is so imprecise you can’t statistically exclude that possibility). Since the policy makers and newspaper readers usually do NOT understand standard errors and confidence intervals, that is all the more reason to refuse to make any projection if the confidence interval is too large. all the best, Bill
Great post. Right after reading this post, I read an article in the Guardian that cites the ILO’s made-up 222 million more people sliding into poverty. Good thing i read this article first. This is precisely why i like reading this blog.
Hats off …
Devil’s advocate then has to ask;
- why would it be impossible that UN, WB, ILO etc ever published a report without these ‘exact’ numbers?
- are these ‘made-up’ numbers better or worse than more refined and ‘robust’ results that are based on not so obvious proxies, coded linear scores … ?
Interesting post, but I think it’s a bit unfair to lump together Ravallion and Chen with all the other guys you cite. Ravallion and Chen are internationally respected poverty experts who don’t make up numbers. Their study on the effects of the recession, to which you link, is of high quality. Citing it among other, more “holding your thumb in the wind” estimates makes it sound like it’s just another one of those. You should know that people are all too quick to attribute “guilt by association”. Just thought this needed to be said.
These numbers remind of the “beer game” and the problems that arise from poor sharing of data along a fairly simple supply chain when there is a basic shock in demand.
Keep up the good work – we need more of your questioning the ‘facts’ and an evidence-based approach to aid.
I agree with most of the description, but what is the prescription? I guess there are three possibilities:
1) give up entirely on collecting reliable and valid data on poverty and making projections, due to the inherent difficulty.
2) continue with the current system.
3) fund something live GIVAS or existing statistics authorities to do a better job.
I would opt for # 3 in hopes of improving the current system. But then I guesss Easterly would say the solution is in the incentive system for statisticians and international bureaucrats. But do we fire them if their predictions/estimates aren’t correct?
Raphael:
There is a fourth option that Mr. Easterly talked about. Limit statistical predictions to those which there is good data for. In other words, show a little restraint.
Interesting/good post.
Also re comment by Mukhtar Amin about the Guardian (UK)..there was also this today that, given the interest in poverty in the ‘developing world’ I thought was interesting. (title of piece: A new report claims the media is to blame for the UK public’s lack of concern over poverty in their own country” http://www.guardian.co.uk/media/2009/sep/28/dispatches-media-uk-charities-poverty
(one part says this – “Searching for an answer to why there isn’t much popular concern over UK poverty, the Joseph Rowntree Foundation (JRF), in a report published last week, places much of the blame on the media, saying there is little appetite to address themes of poverty. In newspapers, the subject is “worthy, not newsworthy”, and journalists found it was often “difficult to give poverty a focus, since it is ongoing and amorphous rather than a specific ‘event’”.) Which is interesting as clearly poverty is not what interests people but the idea of it affecting non-western people (the whole the ‘other’/edward said factor (I suppose not much a surprise but thought it was mildly interesting!)
Also, I agree with much of the post and can see where you’re going with this -
“Unfortunately, the made-up poverty numbers look positively respectable compared to other claims in the UN Voices of the Vulnerable that are based on no known statistics whatsoever:
“Women and children are likely to bear the brunt of the crisis…depression and drug and alcohol abuse could be on the rise….{including} consumption of strong local brews….{There are} rises in domestic violence…{There are} increased social tensions within communities.”
But, surely, using what we do know about vulberable members of society (esp factoring in poverty/conflict) this is not that unlikely to occur? In Canada the rise of alcoholism amongst native canadians is absolutely a factor of poverty and India (where my experience lies) I would say, without question women and children are usually the worst affected when times are hard. I don’t have first-hand working expereince in Africa, but my brother who has been in Sudan (UN) for some years now (and fyi, after 10 years working in various countries with the UN would agree with 99% of your views about the UN!) also says woman and children are usually the wordt affected – and, of course, clearly lack of resources are a major source of conflict.
Perhaps I’m missing something and took your post too literally? (I have been up with two under-3’s most of the night so can’t claim to be that sharp right now!) But using my scant knowledge of economics (Occam’s razor) – surely we don’t need precise numbers…indeed isn’t that your point in some respects… to come to some conclusions about who might be most affected during times of poverty/conflict?
Interesting blog thank you.
Anything in life is like a mosaic, the empty spaces are as important as the fill in spaces- in aid and development as well- we (IO, BO, rich countries and rich people) are obsessed with poverty may be because we are so rich so we think we have to count the poor, and when the numbers go up, we become anxious – why don’t we count successes in Africa and elsewhere? Why don’t we count positive indicators instead and see if they go up or down- it could stimulates a positive energy instead of a depressing one and will show Africa in a different light. Africa is way more than Aids, poverty, and corruption; it is a place where you can find entrepreneurs and strong women, etc. They are heroes of modern myths and their stories could nourish our collective unconsciousness. It is by blowing gently on a spark that we make a fire, it is by adding yeast and letting it doing its work that we raised the bread, etc; in the same way by providing (also) data of positive indicators (among other positive actions) could provide”the dynamic poor” the psychological little blow they need to keep their spark alive.
Thank you.
cf. last paragraph, how do you get serious numbers if you do not strengthen the statistical offices of international organizations?
What is needed is these organizations to have more power (more resources). They have to be supported by the constituent states (not undermined), and allowed them to be supra-national agencies.
IMF and World Bank Take On Istanbul: A Links Round-up
- Zoellick speech on the eve of Istanbul: Current upheaval = French revolution, Africa’s growth potential = Europe’s with Marshall Plan. Earth-shaking changes: “Bretton Woods is being overhauled before our eyes.” – Impartial observers like Nancy Birds…
Pull your head out of the sand Bill Easterly!
Faced with all these perceived “impossibilities,” Easterly and Freschi would apparently prefer to wait and see rather than take action when it is needed, based on the information available at the time. Forecasting is impossible in their eyes. What then is possible? The crisis will probably be over before we will no longer need to make forecasts or estimates to fill in for missing data. Counterfactual analysis of the impact of a crisis is also deemed to be “impossible,” even though the pre-crisis expectations for growth in developing countries are a matter of public record—hardly impossible to know! My Economix article last week defended forecasting against this type of analytic paralysis in the face of uncertainty.
Easterly and Freschi also suggest that the numbers coming from the international agencies are a muddle. Granted there are differences, but Easterly and Freschi have manufactured a good deal of the perceived muddle by mixing forecasts of different things made at different times (and hence with different information). As they could have readily verified, the 89 million figure quoted in the World Bank’s G20 paper is the estimated impact of the crisis on the number of people living below $1.25 a day by the end of 2010 based on our latest growth forecasts, as of mid 2009. “Impact” is assessed relative to the pre-crisis trajectories, as expected at the beginning of 2008.
The uncertainty about these numbers is, of course, acknowledged. But they appear to be the best estimates we can currently make given the information available.
I don’t understand how some bureaucrat can be called an “internationally respected poverty expert”?
The only poverty expert I respect, is that person who actually starts a business in a poor country, trades, makes a profit, expands, employs more and more people in a productive way and actually makes the pie in the country bigger.
But a civil servant, reading newspapers and spreadsheets, designing plans and enjoying his rich life as a parasite of poverty, I can not respect as an expert of the poor.
geckonomist,
You seem not to believe in science nor in public policy. 500 years of economic development undermines that belief!
Jose,
No I don’t.
i am not the only one who thinks like that:
http://www.denverpost.com/harsanyi/ci_13501105
He is more eloquent than me, but he says exactly the same, but one day later.
At cafe hayek Prof. Boudreaux wrote: harsanyi hit a home run!