Skip to content

Why does aid hate critics, while medicine appreciates them?

Two stories ran today in the New York Times that showed the important role of critics in medicine.

In the first, medical researchers found that the usual methods screening for prostate and breast cancer was not as effective as previously advertised. Screening successfully identifies small tumors and the rate of operating to remove such tumors has skyrocketed. But the screening regimen has failed to make much of a dent in the prevalence of large prostate and breast tumors, so their preventative value is not as great as previously thought. Many other researchers had already pointed out that there is no evidence that the relatively new PSA prostate screening test has reduced prostate cancer deaths (a message that failed to make it to my own doctor, who tells me I am definitely OK once the PSA comes back normal). To make things even worse, some of the operations on small tumors were unnecessary and even harmful: “They are finding cancers that do not need to be found because they would never spread and kill or even be noticed if left alone.” The American Cancer Society concluded that too much emphasis on screening “can come with a real risk of overtreating many small cancers while missing cancers that are deadly.”

In the second story, earlier reports of positive results of an AIDS vaccine trial are coming under more and more doubt. The issue is one very familiar to any statistical researcher – did the apparently positive results from the vaccine trial come from random fluctuations in noisy data, or were the positive outcomes definitely more than could have happened by chance? We have the arcane concept of “statistical significance” to answer this. The NYT ran a story a month ago on the same vaccine trial that suggested definite positive outcomes (“statistically significant”), while today’s story features critics of the original trial results who fear the results were just due to random noise (“not statistically significant.”)

Suppose these critics were operating in the aid world. Aid defenders would accuse the critics of not being constructive – these studies were 100 percent negative (so what’s your plan for eliminating prostate cancer deaths, you fancy-pants researcher, if you don’t like ours?) They would accuse them of hurting the cause of financing cancer and AIDS treatment. The attacks on the critics might even get personal.

If this were the aid world, the mainstreamers would dismiss the arguments over statistical significance as some obscure academic quarrel that needn’t concern them. How do I know this? I have criticized Paul Collier on numerous occasions for failing to establish statistical significance for many of his aid & military intervention results. I have argued that he is doing “data mining,” which is pretty much the equivalent of producing lots of results on the AIDS vaccine and reporting only the positive results. But I have yet to find anyone who cares about these critiques – on the contrary the whole American and British armies seem to base their strategies on Collier’s statistical results. In contrast, it’s almost comical to see the heroic lengths to which the writer Donald McNeil Jr. goes in the latest NYT AIDS vaccine story to explain statistical significance to NYT readers. He is saying, hey you really have to get this if you want to know: Did the vaccine in the trial Work — or — Not.

The other feature of both stories is that both throw doubt on excessive confidence in simple panaceas – screening and vaccines. They suggest reality is more complex and that we need to think of new ways of attacking difficult problems like cancer and AIDS. If you are familiar with the aid world, you will know the analogy is exact to how we discuss solving difficult problems like poverty.

So why does medicine welcome critics and aid hates them? Perhaps us aid critics are just not as good as the medical critics. Or perhaps it is because we care so much more whether medicine really works than whether aid or military intervention really works?

This entry was posted in Data and statistics, Global health. Bookmark the permalink. Follow any comments here with the RSS feed for this post. Both comments and trackbacks are currently closed.

25 Comments

  1. D. Watson wrote:

    Possibility 1: There aren’t remarkably few groups opposing all medicine, hence critics are seen as critics of the METHOD only. But there are groups opposed to all aid, so there’s someone to confuse you with.

    Poss 2: There is a much greater acceptance of the notion that aid recipients are Voiceless, rather than that (e.g.) breast cancer patients are. Thus people feel more free to speak on someone else’s behalf, and humans generally argue loudest and least properly when they imagine they are arguing for someone else. To the extent that breast cancer patients do have more voice than aid recipients, false impressions can be corrected. Few Sahelian villages rise up to smack down bad donors.

    Poss 3: Detachment is regarded as a virtue in medicine, but as cold-bloodedness in development where empathetic passion is king.

    Posted October 21, 2009 at 12:18 pm | Permalink
  2. Elise wrote:

    It’s not that people don’t care whether aid works. Everyone has their own personal mission to contribute something good to the world, and when their approach is criticized they feel like people are saying they have been wasting their time, and that’s not a fun thing to discover. This criticism forces them to re-evaluate what they have devoted themselves to for the past 2, 5, 10 years and either try to solve some difficult problems or look into a new approach, neither of which are easy choices. I think that’s why aid professionals can be particularly defensive when it comes to criticism.

    Posted October 21, 2009 at 2:19 pm | Permalink
  3. jose wrote:

    It is clear to me that “aid” is a more political and ideological topic than medical cures, which is more technical. This generates more polarization in aids topics.

    And I think that option 1 of D. Watson is important: the fact that we have an important group of people that sees the whole enterprise (of reducing poverty) worthless or nearly. When you criticize aid programs without giving alternatives it is difficult not to feel that that group has been empowered, that if reducing worldwide poverty is very expensive we should drop it from our priority list. Perhaps the best way to make your criticisms better heard is to distinguished yourself clearly from that group.

    Posted October 21, 2009 at 2:29 pm | Permalink
  4. Jeff Toohig wrote:

    I’m struck by the misleading comparisons and conclusions this post makes. In it, Easterly suggests that the Aid Industry response to criticism of research findings is structurally different than the medicine industry’s response to criticism of research findings. Yet the examples cited indicate quite the opposite. Apparently, cancer screening is going on in exactly the same way as before, despite evidence that it isn’t as effective as believed. Indeed, according to Easterly’s own experience prostate screening, in practice, is indeed used as a panacea. So it seems that Donald McNeil Jr. is playing a similar role as Aid Watch; that is preaching outside the gates. Generally, critics of Aid are met with some interest, some disinterest and no general change of behavior. This seems to be precisely the same in the medicine world. Ironically, the post seems to be data mining (despite Aid Watch’s general excoriation of the sins of such behavior when practiced by researchers) to prove a predetermined point. Using these examples as representative of a larger whole is simply misleading. Exactly one click away from the McNeil article I found this example of a horrendous example of the medicine industry’s approach: http://www.nytimes.com/2009/10/21/opinion/21johnson.html So, instead of a “look how bad we are compared to them” approach, this same post would have benefited from a “look how bad we are, and so are they” perspective.

    Posted October 21, 2009 at 2:40 pm | Permalink
  5. Garrett Schmitt wrote:

    If you take a closer look at medicine and focus on the ones with public policy implications, I think you will find a good deal more hatred for critics bearing contrary evidence in good-faith against a consensus. Consider the controversies behind policy toward medical marijuana and the legal drinking age. Those arguing for tighter restrictions or prohibition take very dim views of their critics supporting more moderate policies based on scientific evidence, such as the impact of legal drinking ages on binge behavior or the potential medical uses of marijuana.

    And US policy essentially takes military intervention is a useful tool for public health in the prosectution of the Drug War in places like Afghanistan and the Andes, despite the admitted lack of results on consumption in the US.

    I don’t think Aid hates critics more than Medicine. I think advocates for public policy hate critics more than scientists.

    Posted October 21, 2009 at 3:29 pm | Permalink
  6. Irene Guijt wrote:

    I am inclined to align with Tohig’s and Schmitt’s observation that medicine (I’m assuming we’re talking about mainstream medicine?) is not as open to criticism as is perhaps suggested. Ineffective or dangerous drugs keep being prescribed, the advocates of modern ‘real’ medicine continue to take serious swipes at alternatives, corruption is rife in drug patenting and marketing, etc.

    An additional issue is that medicine cannot be and will not be abolished with a policy change. On the other hand, aid is under serious scrutiny with considerable probability of reduced % being allocated through policy decisions, at least in the Netherlands where I’m based. So it can be wiped out – and some of it should undoubtedly be.

    However, the decision about what gos and what stays is not based on a solid discussion and ‘evidence’ (assuming we’d all be adhering to similar standards of rigour). Instead, we have extremely generalised public rants, particularly from the (far) right here. Some of these politicians take absolutely any critique to argue for 100% dismantling of aid in any form. So there is a caution here about criticism in part due to the deeply superficial and xenophobic treatment of the aid debate.

    Posted October 21, 2009 at 4:03 pm | Permalink
  7. Mozza wrote:

    I think the tone of your critiques, often mocking, make them even harder to accept for people who are… mocked. They make for an entertaining read, but they don’t elicit the kind of enthusiastic answer that you are expecting.

    This being said, I think you’re taking things too personal and both overestimate resistance and underestimate support.

    Posted October 21, 2009 at 5:06 pm | Permalink
  8. Sceptical Secondo wrote:

    Perhaps if you applied the same rigour to the works of those you endorse to those you critique…

    Posted October 21, 2009 at 5:46 pm | Permalink
  9. As someone who juggles an aid profession with a medical one, I understand the polarity that Mr. Easterly describes. Most medical practitioners love new research, especially critical research. It doesn’t mean they will stop their panaceas, or standard prescriptions, which they KNOW are not very effective – these time-honoured traditions are often still the BEST FIT. Furthermore, medicine is often reactionary – if Solution A doesn’t work, no harm in admitting it and trying B, or a combination. There are so many variables!

    I don’t however, see most aid practitioners to be as accepting of criticism. When critics identify negative side effects to prescribed aid solutions, agencies shouldn’t:
    1. Plug their ears and pretend they are practicing “holistic transformational development”, the be-all and end-all of aid.
    2. Give up their project and jump on a new band wagon.

    Research and criticism are designed to identify problems and promote solutions. Good “mainstream” medical practitioners are not afraid of failed vaccine trials, alternative medicine “miracles” or new “wonderdrugs”. All of these are tools in a continued learning process.

    Oh, that we could have a better “lessons learned” peer-reviewed process of critical review in aid, where practitioners described the shortcomings of their work, and recommended changes for the future!

    Posted October 21, 2009 at 7:25 pm | Permalink
  10. April wrote:

    As several of other commenters have noted, I think one critical distinction between interventions to identify early (or prevent) cancer in the US vs interventions to (say) reduce maternal mortality in developing countries is that – funding for cancer research isn’t likely to go away. So people working in the field don’t have to present a facade to the outside (funding) world, that they have solutions that work to the key problems in order to attract resources – it is enough merely to remind people that they or a loved one could easily be a cancer victim…and the money will keep flowing.
    Jeremy Shiffman wrote a nice paper in the WHO Bulletin that is relevant: http://www.who.int/bulletin/volumes/87/8/08-060749.pdf

    He suggests that aid issues jockey for resources – and that they attract support by (among other things) getting their issue (say TB) seen as tractable, and the programs as predictably generating sizable benefits.
    To the degree that this is true, aid critics (reminding the public of some unfortunate findings about your program impact) can directly reduce funding available (and attention).
    I’m sure there is some of this also in the world of medical research – but the total envelope for research in the field is less susceptible to precipitous drops or shifts. Certainly global health (within development) suffers from such swings, and it is perhaps no wonder that folks are less welcoming than they ought to be when problems with their programs are pointed out.

    Posted October 21, 2009 at 8:18 pm | Permalink
  11. Michel S. wrote:

    That’s a curious example of the term “data mining” — in computer science, data mining is the extraction of non-obvious, but real, patterns from a haystack of data. As it is used to, e.g., help supermarkets organize their shelves, you actually want to be as objective as possible.

    Surely you mean to say cherry-picking, a term that’s perfectly good for this situation :)

    Posted October 21, 2009 at 9:32 pm | Permalink
  12. Justin Kraus wrote:

    Toohig and Schmitt do a good job of setting up a straw man and knocking it down. Of course Easterly’s point is NOT that modern medicine is perfect, rather that it “appreciates” criticism and that Aid by-and-large does not. As Easterly acknowledges with the PSA, medicine moves slowly like every other human enterprise. However admitting this is NOT the same thing as saying that Medicine and Aid are equally (in)effective as Toohig does (“look how bad we are, and so are they”). If we compare modern Medicine’s track record over the past 50 years with that of Aid’s it is clear that Medicine has been much more effective at curing disease and at prolonging, and making more enjoyable, human life than Aid has been at helping developing countries develop.
    And of course Easterly’s point is that Medicine’s appreciation of criticism may be one of the keys to its successes while Aid’s disdain for criticism may be one of the reasons for its lack of success..

    The fact that many of people have responded to this very non-radical idea (i.e. that Aid should be more open to criticism because through it Aid’s effectiveness may improve) by stating that criticism hurts peoples feelings (Elise and Mozza) and by arguing that Aid isn’t really any worse than any other human enterprise only demonstrates more clearly the extreme dysfunction within the Aid community. To those thin-skinned Aid workers, one can only say “Get over yourselves. This is NOT about you, its about improving desperately poor people’s lives.” While to those who are quick to justify Aid’s ineffectiveness by pointing out the ineffectiveness of other human enterprises, I can only ask why are your standards so low?

    Posted October 21, 2009 at 9:51 pm | Permalink
  13. geckonomist wrote:

    Why do they hate critics?

    if a new medical procedure doesn’t work, bad luck, let’s hope it did not do any harm,
    but the doctor will keep his job since his work is still needed.

    If aid doesn’t work and too many people realize that through critics, 500.000 people will lose their job.
    Why are you surprised they hate critics?

    Posted October 22, 2009 at 3:50 am | Permalink
  14. Sceptical Secondo wrote:

    I wanted to steer clear of this medicine vs aid, but: How succesful has medicine actually been in ‘eradicating’ sicknesses, vira etc.

    One may claim that ‘aid’ too is rather succesful in succesful ‘treatment’ of poor people. Check out Scandinavia for example.

    However, both ‘treatments’ are rather expensive, which might be why neither works close to perfectly in the global south.

    Posted October 22, 2009 at 6:07 am | Permalink
  15. Jim wrote:

    It’s not my impression that those working in aid are particularly hostile to criticism. Here in the UK, the Overseas Development Institute is constantly debating aid effectiveness, and DFiD seems fairly open to constructive criticism too. Also, isn’t the story about the AIDS vaccine an example of what would be a major aid-funded intervention being carefully and rigorously evaluated? I find it somewhat bizarre that you’re using it as an example of how aid interventions don’t get properly analysed.

    Also, it seems to me that the common factor in the heated debates about aid you refer to in this post is that they all involve you. Are you really surprised that people react defensively when you use arguments that are almost always derisive or insulting, usually ideologically slanted, frequently cherry-picking or misleading and sometimes just plain wrong?

    Compare the reaction your contributions to the debate get to those of, say, Owen Barder, who is also trying to get aid to be more accountable but who goes about it in a constructive and carefully reasoned way, perhaps because he cares more about making things better than about making other people look stupid.

    I seem to recall Amartya Sen made a similar comment in a review of your book, and you subsequently accused him of being a closet Maoist. If that’s how you ‘discuss’ things, no wonder it gets people’s backs up.

    Posted October 22, 2009 at 9:04 am | Permalink
  16. Anonymous wrote:

    Dude.

    If you think medicine appreciates critics, I should introduce you to some of the surgeons around the med school here.

    Posted October 22, 2009 at 9:20 am | Permalink
  17. Ennis wrote:

    I don’t know why you think doctors welcome evidence based medicine. They welcome new treatments that emerge, yes, but they hate news that says that remedies don’t work and most importantly they had being constrained by evidence.

    Doctors also hate evidence that takes the enjoyment out of things. Let me give you an example. I have a friend who is a doctor and a HUGE football fan. She has asked me not to share articles about football causing premature dementia with her. She’s asking me not to expose her to evidence because she wants to watch football without feeling guilty. My guess is that other doctors feel the same way.

    Posted October 22, 2009 at 9:47 am | Permalink
  18. Jim wrote:

    As usual, Daniel Davies puts it much better:

    “If you start a fight, you can hardly be surprised that you’re in a fight. It’s the definition of passive-aggression and really quite unseemly, to set out to provoke people, and then when they react passionately and defensively, to criticise them for not holding to your standards of a calm and rational debate.”

    http://crookedtimber.org/2009/10/22/rules-for-contrarians-1-dont-whine-that-is-all/

    Posted October 22, 2009 at 12:25 pm | Permalink
  19. Justin Kraus wrote:

    So according to Daniel Davies (and Jim) calm and rationale debate is now an impossibility? Must every disagreement be characterized as a “fight”?
    Again those defending the Aid community justify their emotional responses and hurt feelings with poor arguments and low standards.

    Posted October 22, 2009 at 7:15 pm | Permalink
  20. Jim wrote:

    No Justin, that’s pretty much the opposite of what I said. Try again!

    Posted October 23, 2009 at 7:03 am | Permalink
  21. Justin Kraus wrote:

    Enlightenment me then Jim on how I should interpret your quotation of Mr. Davies.

    Posted October 23, 2009 at 9:01 am | Permalink
  22. Jim wrote:

    Justin,

    It is perfectly possible to have a calm, rational debate about aid. In fact, I spent my lunchtime today watching one at the Overseas Development Institute. But Prof Easterly doesn’t seem to want that kind of debate. Instead he uses a derisive (sorry, ‘satirical’) tone and cherry-picks facts and arguments to make his target look as bad as possible. In other words, he goes around picking fights with people, then acts shocked when they get annoyed in response.

    Posted October 23, 2009 at 1:53 pm | Permalink
  23. Justin Kraus wrote:

    “Prof Easterly doesn’t seem to want to have [calm and rational debate about aid].”

    I very much doubt he would agree with that. And again your on focus on Easterly’s perceived tone, and on his imputed motivations (“make…people look as bad as possible”) instead of engaging with his argument means that you are actually the one who has chosen to elevate the discussion to the level of a “fight.”

    Enough with the perceived insults and hurt feelings. Engage with the arguments.

    Posted October 23, 2009 at 7:48 pm | Permalink
  24. Jim wrote:

    Justin,

    If you want, look up my comments on this site – I’ve done plenty of engaging with the arguments. Prof Easterly was asking why so many of the debates he gets into get so emotional, so I suggested an answer. I would much rather focus on the facts, as by the sounds of it would you. I look forward to more calm, rational debates!

    Jim

    Posted October 24, 2009 at 5:02 am | Permalink
  25. Robert Tulip wrote:

    “because we care so much more whether medicine really works than whether aid or military intervention really works”

    This hits the nub of who is the real constituency for aid. It is not the poor, but the rich taxpayers whose conscience is salved by giving charity. Unlike in medicine, the aid constituency does not benefit from aid, except emotionally. Emotional benefit for aid donors is maximised by direct engagement with the poor, even though indirect approaches (eg private sector development) are more effective in delivering growth and poverty reduction. The real objective of aid is not to reduce poverty, but to respond to the assumptions and prejudices of its funders. Achievement of results is an incidental byproduct.

    Posted October 29, 2009 at 12:20 am | Permalink

4 Trackbacks

  1. By uberVU - social comments on October 22, 2009 at 12:00 am

    Social comments and analytics for this post…

    This post was mentioned on Twitter by bill_easterly: Aid hates critics while medicine appreciates them. Do we care more whether medicine works than whether aid works? http://bit.ly/1YuOr1...

  2. [...] William Easterley reflects on why critics of AIDS vaccine research are applauded for discovering a fraud, but critics of international aid are like unwelcome guests at the party. Easterley argues that those who offer ways can end poverty through “simple panaceas” are not unlike the researchers who faked their vaccine research. Their work may not be supported by the facts. So why does medicine welcome critics and aid hates them? Perhaps us aid critics are just not as good as the medical critics. Or perhaps it is because we care so much more whether medicine really works than whether aid or military intervention really works? [...]

  3. [...] Bill Easterly wonders why critical research is welcomed in medicine, and abused/ignored in aid. [...]

  4. By El blog de Albert Esplugas on October 29, 2009 at 6:10 pm

    Easterly: la importancia de criticar tratamientos ineficaces contra el sida… y contra la pobreza…

    William Easterly en Aid Watch se refiere a dos noticias publicadas en el NYT en las que varios investigadores cuestionan la efectividad de determinados tratamientos médicos, y contrasta la naturalidad con la que se aceptan estas críticas con la hostili…