The history of foreign aid for global health has seen a cycling back and forth between two alternative approaches. The “vertical” approach focuses on fighting one disease at a time, and in Africa has been very effective in targeting smallpox, Guinea worm, measles, and river blindness, to name a few examples. After large initial successes though, diminishing returns to vertical programs set in. The “horizontal” approach instead invests sector-wide to make health systems work to administer prevention and treatment for all diseases. (For more on the history and pros and cons of these approaches, see Can the West Save Africa, pp 57-60).
Since the late 1990s, the Bank and other donors have shifted resources to back the idea that “it’s the health system, stupid.” (According to the Institute for Health Metrics and Evaluation, health sector support shot up from $2 million in 1998 to $937 million in 2007, and surpassed specific funding for TB and malaria for the first time in 2006.)
Strangely enough, whether this resource shift has actually improved health has never really been tested.
A new report funded by the Bill and Melinda Gates Foundation found that sector-wide approaches (aka SWAps—the development industry never misses the chance to make a silly acronym) “are not yet being implemented in a way that has led to improvements in health outcomes in effective, efficient, measurable, or sustainable ways.” In other words… SWAps don’t work.
Written by Richard Skolnik, Paul Jensen and Robert Johnson of ACTION (Advocacy to Control TB Internationally), the report looks especially at whether the Bank’s sector-wide programs are associated with success in TB detection and treatment, and concludes with a number of alarming or surprising findings. (We don’t know if the authors have a predisposition towards the vertical approach given their affiliation with advocacy on one disease, but they do seem to ask the right questions.)
First, the authors find little evidence of the impact of SWAps on health outcomes, and what little there is, is mixed at best. The World Bank’s own evaluation picks up on a “general lack of attention to results,” “insufficient attention to ensuring that SWAps are technically sound,” “a general failure to monitor country expenditures,” and “very weak monitoring and evaluation of the health programs that SWAps are supporting.” In the history of SWAps, there has been only one rigorous, independent evaluation, in Tanzania.
Second, only three of the 15 Bank SWAp projects in sub-Saharan Africa from 2001-2008 even included indicators for detection of TB cases and successful treatment of TB. And in only one country (Tanzania), a SWAp “might” be linked to an actual health outcome: higher rates of TB treatment success.
Third, the aid workers and health experts interviewed for the evaluation said that SWAps focus on the process of coordinating aid delivery, which has become an end in itself, obscuring the need to actually increase successful treatment and decrease deaths. NONE of them questioned the need to work through SWAps BUT they almost all agreed there is “little evidence” that SWAps are associated with improved health outcomes.
This suggests to us that it’s not only about correctly choosing the right mix of horizontal and vertical but whether ANY approach will work unless it has feedback and accountability. Is this why SWAps were a good idea in theory but a disaster in practice?
What to do? The authors have some suggestions, which are a little hard to believe aren’t already being done as a matter of course: Create incentives to focus on results not the process, drastically increase transparency of project information and evaluation, and do independent program evaluation.
Come to think of it, the donors’ behavior reminds us of Aid Watch’s analogy from Monday. Here, the Bank sends truckloads of money down the same SWAps road, ignoring increasingly obvious and urgent signs that the Bank should change course. But still it hurtles along, unfazed by even its own evaluators shouting from the side of the road that what it’s doing isn’t working.
11 Comments
Richard Skolnik, Paul Jensen and Robert Johnson of ACTION really know what they are talking about in this report. The fact remains that the World Bank has been practicing a strategy which only succeeds on paper and fails in execution. Skolnik, Jensen and Johnson are experts who clearly have a grasp on the growing problems facing the global health community today. When will the bank wake up and listen to the professionals here?
The report looks at two complex issues, and does justice to neither.
issue 1) why is TB control going so poorly?
issue 2) on balance, how useful or not are SWAps as a form of development assistance in health compared to investment projects and compared to vertical programs.
Among the many global and national actors, the WB is a pretty minor actor in TB control. So it weakens the report that they spend so much time trying to link what the Bank does in TB to weak results in TB control. You could much more compellingly ask: Why has the Stop TB Partnership failed?” or why has the Global Fund TB support failed or perhaps best of all: why have the governments in countries with a high TB burden failed? both to focus attention and then, to implement effective control efforts when they do take action.
The SWAps question is an important one, and the report does bring together useful analysis of the problems with SWAps – but it misinterprets and/or misrepresents the results of this analysis. It also focuses especially on the WB when it is really the SWAps approach the evidence is linked to. Why? Just not clear from the report. It is the approach that should be examined, and all the actors who support it should be questioned. It looks to me like the Bank and DFID/HLSP have done more critical analysis of SWAps than anyone else (note where the references cited in the report came from)…and they appear to be taking on-board the insights from the analysis – so again, why not point that out???
SWAps by design reduce donors’ ability (and it should be pointed out, the ability of international advocacy NGOs ) to target donor and government funds on their favorite issue (like TB, like family planning, like gender). That’s the point of SWAps. So to turn around and say, see SWAps are a bad form of development assistance because they enabled such shifts is plain silly and/or disingenuous.
And frankly, to say they have failed because they can’t be linked to desired changes in health outcomes is either ignorant or disingenuous.
SWAps are best judged precisely by process indicators. As are all health system reforms. If you look at the literature from health reforms in OECD countries they use these same indicators. The Obama reforms will be judged by various indicators of coverage for population subgroups in the medium term, some indicators for out-of-pocket spending (e.g. bankruptcies from catastrophic health expenditures) in the longer term, and by some indicators of cost control. These are the results of health systems reforms. No thoughtful commenter (5 years from now) would say, yep, those Obama reforms failed because spending on diabetes has gone down, or because incidence of diabetes has gone up.
This report is not a helpful contribution to an important discussion of the effectiveness of SWAps vis a vis other forms of health development assistance. It manifests real or feigned ignorance about health systems development and donor support thereof. Interested readers would be well-served to examine the evidence and analysis presented in the reports referenced (the ones from DfID/HLSP, the World Bank, CGD and CSIS) if they want to get a better understanding of the issues.
Actually, this thoughtful commentator at least will say that the Obama reforms have failed if health spending increases dramatically and outcomes don’t improve significantly.
April, wow I amost never disagree with you. It seems to me that there is a tradeoff here. Making the health system work is obviously critical, and you have convinced me on this more than anyone. Yet aid to accomplish this could be very hard to get useful feedback on, and hold anyone accountable on, and it seems like the Bank didn’t even try very hard to do whatever limited stuff might be possible.
It’s important to distinguish two things: (1) what is important, and (2) what aid can accomplish. Something can be very important and yet beyond the capability of aid to change. I am not sure if health systems reform is in this category, but I have to wonder.
best, Bill
PEPFAR, PMI and the Global Fund rigorously report on the impact they achieve. The World Bank doesn’t on with the billions it spends on Global Health. Why do we continue to let the WB throw US taxpayer’s dollars down a rat hole?
April hit the nail on the head. I can’t add anything to her thoughtful analysis of the report. But I can add that is amazing at how the Bank is singled out in the report when SWAps are multi-donor approaches that involve regional development banks, bilateral donor agencies, and the governments themselves, often contributing more than the World Bank’s share of funding. The only development actor really left out of the process is the much-maligned private sector, which you would expect to be crowing the loudest, but the report is from a targeted advocacy organization. I’m a strong believer in watchdog functions, and we should all be learning, listening development actors, but this is an unscientific report based on anecdotal conversations – not structured interviews with the Bank staff it attributes the internal information to. Disappointed.
Thank you April for your points which i fully endorse. Having halped design, implement or monitor health SWAps around the world I am painfully aware of (and often frustrated by) their shortcomings. But I do not develop SWAps from an ideological perspective -as the Aid Without Impact report does- but from a pragmatic standpoint that development assistance should be better integrated with and support priority national policies and programmes. While the report quotes our work (DFID/HLSP) it does not trully reflect or summarise the main findings emerging from it. One key finding is that SWAps are no better or different from those individuals who have tried to make them work on a day to day basis. Their efforts to develop consensus on national health priorities and to ensure that donor resources support those priorities deserve a lot of respect. What if any are the alternatives that the authors propose to progress in that direction? SWAps do not include or ecxclude TB (or any other health matter) from national priorities. It is those working in those thematic areas in countries who have developed health SWAps that often exclude themselves from designing or improving sector wide approaches. There are a number of examples of countries where TB and other communicable diseases are core elements of the national SWAp or even receive core funding from its pooled resources. The decisions to allocate more or less resources to TB rest with those implementing each national SWAp, not with the SWAp principles. It is more often the case that agencies who traditionally support interventions like TB -including those mentioned in April´s response- prefer to work in the margins of the SWAp rather than becoming active part of it. Working in a SWAp is time consuming and can be frustrating. It may expose policy inconsistencies on both the donor and the national side that are hard to resolve. But SWAps -where and when they work- have the advantage of bringing these issues to the surface and thence to deal with them. What alternative mechanism if at all do the authors of the `Aid without Impact`report propose to better deal with those matters? More donor led or donor driven projects? More cosy deals between agencies and individual officers in ministries of health to perpetuate clientelism? More funds to international NGOs to provide parallel services? What proposals have the authors got to offer that are drastically different from the essence of a health SWAp: a plan with expected results, a clear financing arrangement, and a workable coordination and monitoring arrangement?
I find it astonishing to see he authors of the Action report being regarded as the ultimate experts in SWAps and their impact on Tb. Where are the government and country voices in these superficial reviews of SWAps ? Where’s the acknowledgement of country ownership ? Or is this a case of paternalism on the part of RESULTS and Action ? I see lots of interviews with experts at the global level but almost nothing from government ministers and country experts whose overall health plans are financed by SWAp donor financing. In the case of the Tanzania health SWAp, the government is the major financier.
One other thing is that since 2007, the Bank, at the request of developing country governments and the UN health agencies and other key partners, has been committed to helping low-income countries strengthen their underlying national health systems. This means being able to provide prevention, care, and treatment services for people regardless of what illness or disease may be making them unwell. A very Bank recent example of how a stronger health system can work across all diseases is the 63 million dollar East Africa Public Health Laboratory Networking Project which will speed up testing for and treatment of many communicable diseases, including Tb and multi-drug resistant Tb strains, in Kenya, Tanzania, Rwanda, and Uganda.
Lastly, the Bank is currently involved in 50 health operations in Africa to improve health, nutrition, and population outcomes. Of these less than 10 percent of these support SWAps. For RESULTS and ACTION to then say that SWAps are the Bank’s preferred approach to working in health would appear to be erroneous. Furthermore, why is the Bank being singled out for critical attention al critical treatment when many partners contribute to the overall implementation of SWAps. Consider in the case of the Tanzanian SWAp, partners currently contributing to the pooled funds are Canada, Denmark, Germany (GTZ and KfW), Ireland, Netherlands, Norway, Switzerland, UNFPA, UNICEF and WB. Other major donors outside the pool include DfID, USAID and Global Fund.
Let’s have substantive discussions without the megaphone point-scoring which RESULTS and ACTION appear to be engaged in, without balancing their advocacy efforts with sufficient African government perspective.
“Experts who clearly have a grasp on the growing problems facing the global health community today”? As I believe I encountered Richard on his first visit to Africa a few years back (when he was representing Harvard as a PEPFAR contractor), I would assume he has had little or no opportunity to discuss SWAps with the African leaders who opt to use the approach. I also recall that the staff of ACTION who came to the Bank (ostensibly to better understand how the Bank supports TB –I naively believed, as did the other Bank HNP specialists they interviewed, that they had left with a better understanding thereof), had no experience in developing countries. Paul Jensen is a TB expert, and Richard obtained his international health knowledge working 25 years in the World Bank. What Richard and his team are, are advocates for TB. This is what they have received funding from the Gate Foundation to do, advocate for tuberculosis (not produce this report per se, as is being somewhat disingenuously implied). Disease advocates have an important responsibility in the international health community. However, let us appreciate that such advocates are not likely experts on health sector development more broadly.
A preference that donors would earmark development assistance for what they believe ought to be a country’s priorities, is a viewpoint that understandably will continue to be espoused by advocates. In this case, the TB advocacy group prefers stand alone TB projects and wishes that development partners would insist more funds be used exclusively for TB. Others will equally advocate that funds need to be targeted/earmarked for AIDS, malaria, maternal health, child health, reproductive health, mental health, road safety, etc. A stand alone TB project (or one which uses TB indicators to trigger the release of funds as this NGO also proposes) will have a better chance at achieving results in TB indicators than a program that aims to address multiple priorities. In some countries, this approach may make sense, but a choice for a stand alone TB project or linking funds to TB indicators is equally a choice not to prioritize maternal health, child health, malaria, pharmaceutical quality, the production of health workers, water and sanitation, etc., etc. If the World Bank were to insist that a larger share of its financing would be earmarked for inputs to support tuberculosis, it would be immediately criticized by all the other advocates (and recipient countries would be dismayed given the availability of TB earmarked resources through the Global Fund).
Much to the dismay of some, the Bank’s role is not to advocate for specific health issues, but to ensure countries are setting priorities and defining strategies in a manner that will have the greatest impact on their overall health, nutrition and population outcomes. Whether those priorities are supported through targeted projects or SWAps or other approaches depends upon each country’s individual situation, preferences and needs. Many countries and development partners have preferred to take a Sector-Wide Approach to development assistance because they recognize the fungibility of external financing, wish to reduce the transaction costs of too many individual donor approaches, and have come to the conclusion that development is not simply about realizing short-term (often unsustained) improvement certain disease-of-the-month indicators, but about building their capacity to identify and effectively respond to their own priorities.
There is no dispute that recipient countries, donors and the multilateral agencies need to improve how we measure and increase accountability for such results (this is something the global health community continually strives to improve and would welcome constructive criticism on), but suggesting that the only alternative is to target one disease at a time (reproducing PEPFAR, PMI and the Global Fund) is nonsensical, and reflects no appreciation for what has transpired in development assistance for health over the past four decades. And, a conclusion that “SWAps don’t work” because “only one rigorous, independent evaluation” has been recorded (one which concluded they do work…) does not raise the bar for monitoring development effectiveness.
There will be a discussion of this matter tomorrow night at the Global Health Conference. I may write in more detail after that, but do not want to bias that discussion.
However, I do want to make just a few points now in my personal capacity.
First, I want to remind those who have written that our report never argues for a “targeted approach” to TB. Never. We accept that the development partners will largely be working through broad based health system strengthening approaches.
Our report also says that if the development partners want to work through SWAps as they intend to do, then SWAps must be judged in low-income countries in Africa largely by the extent to which they reduce neo-natal, infant, child and maternal deaths; TB and malaria deaths; and contribtute to HIV cases and deaths averted. These are the most important burdens of disease for the poor.
If the development partners are not measuring the contribution of their work to these health outcomes, (and they are not) then something is wrong. If their approaches are found not contributing to these health outcomes sufficiently, then something is also wrong.
I am very sorry that some have decided to make this a personal matter, rather than answering the question: what is the contribution of these efforts to reducing the leading causes of illness and death among the poor in low-income countries in Africa? What can be done to make them better at enabling better health for the poor.
The World Bank’s Internal Evaluation Group, after the only rigorous study to date, has said that the contribution of these efforts to health outcomes is limited and must be greatly improved.
There is no debate that development assistance for health should be assessed by the extent to which it enables poor countries to “reduce neo-natal, infant, child and maternal deaths; TB and malaria deaths; and contribute to HIV cases and deaths averted”, etc. We would all agree that good questions would be “what is the contribution of these efforts to reducing the leading causes of illness and death among the poor in low-income countries in Africa?” and “What can be done to make them better at enabling better health for the poor”. However, this report does not address these questions and does not assess which modes of development assistance are more likely to contribute to such results. It does not conduct even a simple analysis to consider whether health outcomes (including TB) have improved more in countries employing a sector-wide approach than those employing an alternative. But, it does conclude that the Bank and partners do not sufficiently support health outcomes in Africa, because most Bank project documents include only one (and not two) indicators for measuring TB results.
This paper explicitly builds upon ACTION’s previous report which concludes that the Bank should “become a leader in financing TB control in Africa” and should “increase funding for TB and TB-HIV specific activities”. So it is difficult to accept that the authors are not arguing for a targeted approach to TB. When the press releases by a group whose mandate is explicitly TB advocacy (“increasing public resources directed to TB programs”) state ” As donor governments provide financing for the Bank, they must demand that support for Africa deliver better health for the poorest people. Our report shows that donors may get a bigger bang for their buck elsewhere”, one might understandably assume that the “elsewhere” envisioned would be to targeted TB programs.
I will again note that the report provides no evidence to support its sensationalist title and language. At the Global Health Council one participant noted that the burden of proof lies with the authors. The ACTION-invited discussant on the panel also concluded that they simply have not substantiated their conclusions.
3 Trackbacks
[...] This post was mentioned on Twitter by @mikegechter's RSS, Dr. Nancy Philips. Dr. Nancy Philips said: The World Bank's “horizontal” approach to health falls horizontal?: The history of foreign aid for global health h… http://bit.ly/9IqbUI [...]
The Critical Links: Thurs 10th…
Blog Author: Timothy Cox Our daily round-up of what other think-tanks and commentators are saying on the big issues: Don’t fear the trade deficit: Dan Ikenson on how imports raise real incomes. Over at…
[...] This post was mentioned on Twitter by . said: [...]