I think you would agree that theroetically, there are variables that explain labour participation but not wages. Empirical identification of those variable is the tricky part. The problem is therefore one of identification, not selection bias. If we have a situation of perfect identification, selection bias is not an issue, and there’s no need for experiments. Obviously this is wildly unrealistic, so there is a need for experiments.

I wouln’t say experiments are necessarily the ‘gold standard’, although I concede they are certainly most fashionable right now. Maybe the gold standard for a very particular type of problem, but not generally. A perfectly identified selection biased corrected regresseion model can be equally or more convinving. Experiments are easier but obviously have the scalability issue that prof. Easterly is so worried about; respectfully though, I think too much so.

Most of these papers, the good ones anyway, don’t pretend to be something they’re not, and are still contributing to our understanding of how important mechanisms work. They are generally read with full awareness of the limitations of experiments. We don’t have perfect methods; all we can hope for is an improved understanding (as opposed to a perfect one), which is something god experiments are doind a pretty good job of providing so far.

It’s true that there is at least one established method for correcting for selection bias but the problem remains that you need to find the right variable that would allow you to make the necessary correction. For example, It’s a common procedure to correct for selection when estimating returns to education since wage earners are unlikely to be representative. But it is not easy finding a variable that affects labor market participation and not wage rate.

e.g. Olken shows corruption follows standard rules of IO. How do you identify the mechansims of corruption without an experiment? You don’t.

e.g. Kling, Liebman and Katz show neighbourhood effects cause reduction in obesity and anxiety. This one does feature selction bias prominently. But, its not really overcoming the selection bias that’s important. If we could predict neighborhood decisions, in a correctly specified model, we could correct for selection bias in an equation explaining anxiety or obesity using old-fashioned Heckman lamda stuff, or new-fangled nonparametric methods. But these aren’t things that are easily identified using traditional data. That’s the reason it gets in econometrica, not just because it overcomes selection bias.

The post includes a short review of the Easterly et. al. paper “Was the Wealth of Nations
Determined in 1000 B.C.?*” It was definitely an interesting-looking paper which everyone interested in development should read.

This is one of the most refreshing, succinct explanations of evaluation methods I’ve seen. Truth often lies in the middle two extremes; so while I’m certainly a fan of RCT’s, it’s foolish to consider them the only helpful evaluation tool.

This post was mentioned on Twitter by bill_easterly: Aid Watch: A voice of sanity addresses “The Civil War in Development Economics” http://bit.ly/5Jx3bj...

Perhaps every randomized trial should be accompanied by a study of the organisational practices and the implementation chain through which the program is delivered. Also, this would involve very little additional costs, since high quality RCTs usually have dedicated field researchers, who are potentially store-houses of critical organisational information.