Despite Climategate, even a superficial reading seems to indicate that there is enough evidence for effects of man-made activity on the climate.
Surprisingly, there is a lot less evidence for effects of man-made activity on something that actually is completely man-made: the rate of economic growth in each country.
I had this frustrating thought as I was reading an important new paper, “Determinants of Economic Growth: Will Data Tell?” [1]
The paper gives a conclusive and resounding answer to the question in the title: no.
It has taken economists a lot of hard work to attain this level of sublime ignorance. There were three steps in the the great History of Evolving Cluelessness:
- Economists spent the past two decades trying every possible growth determinant in sight. They found evidence for 145 different variables (according to an article published in 2005). That was a bit too many in a sample of only about one hundred countries. What was happening is there would be evidence for Determinants A, B, C, and D when tried one at a time to explain growth. But the evidence for A disappeared when you also controlled for some combination of B, C, and D, and/or vice versa. (Interestingly enough, foreign aid never even merited inclusion in the list of 145 variables.)
- The Columbia economist Xavier Sala-i-Martin and co-authors ran millions of regressions on all possible combinations of 7 variables out of the many possible determinants of growth. Skipping a lot of technical detail, they essentially averaged out the millions of regressions to see which determinants had evidence for them in most regressions. There was hope: some were robust! For example, the idea that malaria prevalence hinders growth found consistent support.
- This new paper by Ciccone and Jarocinski found that every time the growth data are revised, or if the sample is changed to another equally plausible one, the results vanish on the “robust” variables and new “robust” variables appear. Goodbye, malaria, hello, democracy. Except the new “robust” determinants are no longer believable if minor differences between equally plausible samples changes what is robust. So nothing is robust.
There are two possible ways to describe what had happened over the past two decades:
- The growth research was at least partially fraudulent, in that we researchers were searching among many different econometric exercises till we got the “determinants of growth” we wanted all along.
- There was a good faith effort by us researchers to test different theories of growth, which led to some results. We didn’t realize until later that these results were not robust.
Description (1) would be a “GrowthGate,” but since so many people would be guilty (of “data mining”), and since we really can’t tell for any individual study or researcher whether it was (1) or (2), “GrowthGate” never became a story.
The only guilty ones might be those who continue to run growth econometrics today without acknowledging that our Three-Act Tragi-Comedy is so OVER. Like for example, I wonder a little why pay attention to some hot new study that claims to have finally found that big POSITIVE effect of aid on growth, or POSITIVE anything on growth.
Of course, the policy world abhors the great Vacuum of Ignorance, which opens the door to empty pontificators like a certain bestselling writer of books about Flat Worlds, in which You Cannot Have Growth Unless You Do Precisely What I Tell You.
Thank goodness, many economists did good economics before the Dark Age of Growth Econometrics, and many economists have still managed to do it during and after. Economics is so much more, even if the cross-country econometric data refuse to tell us The Exact Determinants of Growth.
—–
[1] By Antonio Ciccone and Marek Jarocinski, ICREA-Universitat Pompeu Fabra; and European Central Bank.
[2] Durlauf, Steven N., Paul A. Johnson, and Jonathan R. W. Temple, 2005, “Growth Econometrics,” in Philippe Aghion and Steven N. Durlauf, eds., Handbook of Economic Growth, North-Holland.
[3] Sala-i-Martin, Xavier, Gernot Doppelhofer, and Ronald I. Miller, 2004, “Determinants of Long-Term Growth: A Bayesian Averaging of Classical Estimates (BACE) Approach,” The American Economic Review, 94(4):813–835.
22 Comments
Bill:
I said on Dec 6 in a post at Marginal Revolution that, fear not, there will soon be a developmentgate — but didn’t think it would be revealed quite so promptly!
I’m sure most of the effort was, as you say, in good faith. But what you describe as the “evolving cluelessness” of growth econometrics is, if true, a terrible indictment of the brazen confidence with which findings were disseminated to a generation of students, politicians, activists. Perhaps the interim damage done to developing countries is even more incalculable than the data itself?
But let’s look on the bright side. At one level all that this tells us is that some economists have not yet achieved the degree of scientific sophistication that would allow them to replicate the century-old results of social scientific logic in the sub-discipline of social economics or economic sociology pioneered by Max Weber and Joseph Schumpeter.
Let me try briefly to explain what I mean. In the early twentieth century Weber applied new kinds of sociological logic to the analysis of then-available historical evidence. He came up with the idea that the determinants of economic development (being synonymous with growth and market expansion) are to be found in the organized institutional architecture of a society. Specifically, he singled out the formal procedural impersonality that characterizes interactive subsystems of market regulation, the legal system, public administration, and political representation in modern capitalism. And he discussed how each of these preconditions are achieved in policy terms. He implicitly suggested a sequence of institutional constructions in pursuit of growth.
In my recent book ‘Capitalism, Institutions, and Economic Development’ I attempt to provide the first comprehensive explanation of the implicit Weberian-Schumpeterian development strategy (read ‘growth strategy’ if you will). I outline a sequence of feasible reforms for the depersonalization of institutions that can, in theory, generate durable peace/prosperity. These reforms are culture-neutral, i.e. they are universally applicable across countries.
My point is this – (1) We have ‘an outcome’ of sustainable growth before our very eyes in the shape of most of the OECD countries. (2) Weber argued that ultimately the only reliable guide to provable causality is “the actual outcome”. (3) The probable causality of institutional architecture as a condition of modern growth was implicitly predicted by Weber in the early twentieth century. (4) Based on this evidence (comparing the original institutional hypothesis with actual outcomes) I think you have to agree that more was achieved by rational social scientific logic in the early twentieth century than by development economics in the late twentieth. The theory is shown to be correct by the outcome.
A pointer to the healthy future of a more sociological (and of course post-post-Marxian) development studies? Throw in Schumpeterian economics and I do believe the circle is convincingly closed. Contrary to what some self-reproducing groups of puzzle-posing, mystery-solving institutional theorists like to tell us, the recipe of OECD-style peace & prosperity was and is not a secret-unknown. That’s not to say that econometricians shouldn’t continue to look for hard data to support a patently productive Weberian theory. And I will read the papers you’ve listed. Thanks for your disclosure of ‘growthgate’!
I sat through undergrad lectures at Cambridge nearly a decade ago by one of the chief protagonists of this intellectual escapade. It wasn’t particularly inspiring then, and 4 years in the field have only reinforced quite how misplaced the quest was.
There is clearly a lot to be said for the pursuit of empirical regularity, but it was obvious from quite early on that there were serious issues (Levine and Renelt 1991/2 anyone?)
There’s no way it was any kind of conspiracy or climate-gate type episode, not from anything I have seen. But it is a rather sobering example of the limits of contemporary neoclassical economics when so very far abstracted from reality.
As you say though, William, there has been, and continues to be, a lot of very good, country-specific, microeconomic and political economy work on growth. Long may that continue.
Does this mean you are renouncing your own work, such as Easterly and Rebelo (1993) “Fiscal Policy and Economic Growth”, Easterly and Levine (1997) “Africa’s Growth Tragedy”, and recently Easterly (2007) “Inequality does cause underdevelopment”? I assume you would place yourself in the well-meaning second category of researchers if so. If this critique does not apply to your own work, could you please help me understand what the difference is? I could see, for instance, a caveat that papers I mention largely discuss NEGATIVE results (this harms growth), and you very specifically discuss only POSITIVE results (this causes growth) in this post. Are there other differences?
The real GrowthGate is Acemoglu, Johnson, and Robinson (2001). There results turn out to be not just in error, but overwhelmingly and inconceivably convenient, as pointed out by Albouy:
http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1152672
Last I’d heard, Albouy’s paper was revise and resubmit at the AER (one of the most prestigious economics journals), but was being forced through endless revisions by “anonymous reviewers.” I’m pretty sure there was something in the ClimateGate emails about redefining what peer review means, so I think the parallels are clear.
A, J and R I’ll agree has some very serious issues indeed, my favourite being the assumption that entire races of people being wiped out (in the US, Australia etc) can in some way be reduced to “creating a stable, non-extractive system”.
So we haven’t been able to use regression analysis to identify causes of growth. You seem to imply that regressions cannot identify the causes of growth. If this is your argument, then why is it necessarily the case? Is the growth data too noisy? Is it the IVs? Is it both? Is it selection effects and endogeneity? If it’s any of these, can we solve it? If it can’t be solved, then how do we analyze economic growth? Since growth is a macro-level phenomenon, I’m skeptical about the value of experiments here; although I expect experiments would be more valuable in terms of micro-level economic development outcomes (health, education, household income changes, etc).
D. Watson, quick response, more later. Easterly and Rebelo was about a NON-result, the failure of tax rates to predict growth, so that has held up. “Africa’s Growth Tragedy” and “Inequality” were results in levels of per capita income, which I will argue in a future post is much more reliable. Having said that, I confess the “Africa’s Growth Tragedy” did have a correlation with growth that did not pass the the robustness checks in the paper cited in this post. It survived replication with new data in Alesina et al. 2003, and is robust as a simple correlation which might be a super reduced form, but otherwise suffered the same fate as other growth regressions.
Chris, I will have a future post on this, how one of central problems is the large transitory component of growth (“random” in some non-mechanical sense), and how doing regressions in levels of per capita income is one way to average away the huge transitory component.
Against the CoC/Christianist conventional “wisdom,” we should get more actual (ie, per capita) growth out of slow-growing than fast growing populations because the latter soak up effort and resources accomodating and preparing for the extra people. The argument about taking care of old people is bunk: we have to front-load the care of the young, and of course each person cancels out as a net producer/consumer anyway. The argument that there’s more to do in the latter case is also fase; it’s a broken-window fallacy.
Related to the Ciccone and Jarocinski PWT revision result is this recent work — http://www.nber.org/papers/w15455 & http://www.voxeu.org/index.php?q=node/4339
The annual data move a lot with PWT revisions — growth rates over longer periods (10-20 years) are more consistent. That may fit with Bill’s story about the “large transitory component of growth”.
I am not an Austrlian, but I am reminded of what Hayek said about statistical aggregates.
Obviously, I stuttered between ‘Austrian’ (which I am not) and ‘Australian’ (which I am). Oops
Growth is a non-starter anyway. What is “growth”? It’s an increase in an arbitrary index, GDP, which, of course, only measure monetized economic transactions and excludes all the rest.
3. Knowing about malaria made us do something about the problem, so it used to slow down growth but this is no longer the case. Meanwhile, we neglected democracy for some reason, and this developed to become the constraint on growth…
Lorenzo – would that make you an Oztralian?
This may lead either to condemnation of econometrics OR to questioning of data quality.
Reliability of national accounts data from developing countries (especially, more historic data) remains doubtful. Arguably, GDP is one of the most difficult variables to survey. Cross-country growth regressions are affected by this problem.
With varying degrees, same quality problem might apply to the independent variables of GDP growth.
Bill, I would be interested to hear whether you think the growth regressions in your article “Freedom Versus Collectivism In Foreign Aid” also stand up to this kind of scrutiny. You seemed particularly sure at the time that this was conclusive evidence that economic ‘freedom’ (defined according to some highly complex index) was, if not a sure-fire recipe for income growth (as well as high income levels), something damn close to it.
‘Growthgate’ is the failure of the aid investment of hundreds of billions of dollars to reduce poverty. One could conclude that donors do not really wish to reduce poverty, but prefer to invest in areas seen as providing short term political benefits. Aid agencies should be as independent as central banks, with a mandate for economic growth, climate security and emergency humanitarian response. Poverty reduction is an effect of growth, not a cause.
This is a very nice post. But I wonder if we should be at least a little wary of the implicit message in it.
What is clear is that we have a hard time discerning what causes what with observational data. Yes, we know much less about development and growth than climate scientists know about the implications of CO2 (even though many economists I know would be loath to admit that). But your implicit message here (correct me if I’m wrong) seems to be that absence of evidence implies evidence of absence, and that’s obviously not true.
I’m no development economist and believe you are correct in being skeptical about who truly benefits from aid. But the fact is that many of the insignificant correlates could actually be causal, perhaps even strongly so, but their effects are nonetheless confounded by other factors or by reverse causation (e.g., extreme poverty causes greater aid?).
We simply don’t know.
A couple of issues with cross country regressions:
1) Firstly the small sample sizes used due to data limitations means that assymptotic properties are unlikely to hold. Thus a lot of these studies would struggle to get consistent estimates.
2) Panel data techniques that are used to increase sample sizes often assume fixed effects which it is unlikely with so many possible explantory variables
3) I have big problems with the assumption that parameters are linear across countries at different levels of development. It seems clear that certain countries rely on different industries, therefore it is likely that different variables will be of importance. However controlling for this means reducing sample sizes even further.
“even a superficial reading seems to indicate that there is enough evidence for effects of man-made activity on the climate”
A superficial reading of what?
More than anything else, “GrowthGate” seems to be a problem of our lenses.
It reminds of what Bill Quiqley wrote to a law student about social justice:
“Our laws, by and large, are what those with
power think should apply to those without power… If you are interested in real social justice, you must seek out the voices of the people whose voices are not heard in
the halls of Congress or in the marbled courtrooms.”
The full letter is found at http://www.law.berkeley.edu/files/QuigleyLetterToLawStudent.pdf
Economists hate to admin their data is so incomplete it is meaningless. This world is dominated by the “informal sector.” Most people have no financial records. I predict in 5 years the world will suddenly APPEAR to have grown, when in fact we just got better at counting people because they are using mobile phones.
And “best” of all, some economist will run out and attribute all this sudden growth to X,Y,Z again and not to a change in the lens we use to measure it – phones.
7 Trackbacks
[...] to Bill Easterly for the [...]
[...] Why there’s no “GrowthGate:” Frustration vs. Chicanery in Explaining Growth [...]
[...] Why there’s no “GrowthGate:” Frustration vs. Chicanery in Explaining Growth Via KB [...]
[...] Easterly, “Why there’s no “GrowthGate:” Frustration vs. Chicanery in Explaining Growth”, Aid Watch, 10 December [...]
[...] something that actually is completely man-made: the rate of economic growth in each country.” William Easterly discusses whether or not we actually know what causes economic growth in the first place. Possibly [...]
[...] Interesting post by developmental economist Bill Easterly on “GrowthGate.” Contains mocking of Thomas Friedman, something every blog post could use a little more of. [...]
[...] on something that actually is completely man-made: the rate of economic growth in each country.” William Easterly discusses whether or not we actually know what causes economic growth in the first place. [...]