Editors’ Note: This will be the last Aid Watch post until Monday after the holiday weekend. Happy Thanksgiving!
Which attribute of an aid project makes it more likely to succeed:
- It will have rigorous evaluation based on some output indicators to make sure it’s working, OR
- It is staffed by people who really, really want it to succeed?

Sister Shewaye Alemu, Area Director for Addis Ababa, introduces the staff of Marie Stopes
This question came out of a tour of maternity and family planning clinics of Marie Stopes International in Addis Ababa. The dedicated staff of Marie Stopes courageously confronts a sensitive issue responsible for about a third of the deaths behind Ethiopia’s high maternal mortality rate – deaths of mothers during unsafe abortions. Marie Stopes workers offer safe abortions consistent with Ethiopian law. They also provide the whole package for reproductive choices AND safe childbirth for women: contraception alternatives, testing and counseling for HIV, prevention of mother to child transmission of HIV, prenatal care, and a clinic for difficult, life-threatening deliveries referred from Ethiopia’s official hospitals. (Although I’m NOT a fan of family planning fanatics who decide on behalf of the poor that they should have less children; I AM a fan of family planning people who respect their clients enough to just give them more choices.)

Asfaw Fantaye, laboratory technician at Marie Stopes in Addis Ababa
One afternoon’s visit is not enough to verify a great aid project, and my brief stop at the Marie Stopes project is pathetically inadequate. But since informal site check-ups are much cheaper and more universal than more rigorous methods like randomized controlled trials (RCTs), which will only EVER be available for a small sample of aid projects, it’s worth pointing out a few advantages of the humble site visit.
First, a site visit tells you something about clean, well-maintained, high quality facilities, whether medicines and equipment are available, not to mention whether the health workers are present and whether there are patients waiting because they value the services. A government hospital in a regional town failed many of these same tests during another brief visit during this trip.
Second, gut instincts tell you at least a little something about the attitudes of the PEOPLE involved, workers and patients.[1] At Marie Stopes, I was very impressed with the eloquence and dedication of our host, Sister Shewaye Alemu, an Ethiopian who is the Addis Ababa Area Manager. The Danish country director of Marie Stopes (the only non-Ethiopian employee), Grethe Petersen, told me her mission was to be the LAST non-Ethiopian country director of Marie Stopes.
RCTs, on the other hand, don’t have a good way of getting at the intangible human element in aid projects –is there good team spirit and morale? Are there good relationships between management and workers, and amongst coworkers, and between workers and patients? There are no scientific recipes on how to DO human relations, just tacit knowledge on managing people, and personal attributes like trust, humility, patience, and respect. Getting to know the people involved can give you a sense of how well this intangible stuff is going.

Sister Shewaye shows saintly patience while pestered by inquisitive farangi
RCTs could possibly identify the right actions, but if PEOPLE’S motivation to get good results is low, these actions will not be implemented in the right way, or not at all. This will usually be obvious in a site visit.
I am not saying that getting to know the aid workers and more rigorous methods like randomized controlled trials are mutually exclusive – both have value. But even one brief visit to Marie Stopes in Addis was enough to increase HOPE in the potential for determined PEOPLE to make a difference in aid, and was strangely more persuasive than randomized trials.
Think of the analogy to the private sector: venture capitalists don’t do randomized trials but they DO talk to the entrepreneur and inspect the operation in situ. We need venture capitalists and entrepreneurs as well as randomized experimenters in aid.
[1] A related observation: the best evaluations of actual project implementation I have ever read BY FAR are written by anthropologists, such as James Ferguson’s all-time classic on a World Bank project on Lesotho.




11 Comments
One wonders, if in the final analysis, while number 2 is ever all that really works, number 1 is pointless to even talk about. Clearly, number 2 is the main reason things succeed, but taking that into account you could argue that any time 1) shows that, through rigorous testing, a project is successful, it is only down to 2 anyway. The point of 1 (rigorous testing) might be redundant (have studies comparing the two above been done?)
I’ve been thinking of this in relation to a dept in local UK govt (that is not unlike the OS – if I have the correct name, that oversees the CIA (FBI?). But one could argue that if you need a higher evaluation unit to ensure that people/dept in, say CIA and FBI are working properly, you assume that those that work for (say) the CIA/FBI are corrupt (or likely to be) and that those working for the OS are morally more grounded – surely probaility would show this to be unlikely (there is just as much likelihood in all 3 orgs, that people on staff will have good or bad morals – which are, to an extent, subjective in certain contexts). So going by that, the only point of a higher evaluation body (like OS) would be that is they were more transparent and accountable than the depts they are evaluating, monitoring and overseeing, but this is unlikely…as any org/dept will have more security and less transparency the higher up it is…which would mean, for any ‘overseeing’ org/dept to truely work there would need to be an OS overseeing/monitoring them and on and on.
In the UK, in local govt there are depts that serve as the OS does – that oversee other depts (say planning, or strategy) to ensure they are “working properly” and transparent. But like with the OS/CIA example, you could argue that such evaluating/monitoring bodies are, in the final analysis, pointless, as any dept that works well, will do so due to the people that staff it (their level of committment, expertise etc) and not because a ‘higher’ (and in these cases more secretive, less transparent body, that also doesn’t have the day-to day knowledge of that dept nor the expertise of that staff for the projets being undertaken there) are giving their opinion on it. And like the Us example, in the UK, also one cannot assume that someone working for an overseeing/evaluating body will have more expertise or morals (one could arge that in fact they might have less, as they are not working in a sector that they have specific expertise, and one could assume, passion for) – so who is overseeing them?
Logically the point of evaluating and monitoring outcomes of orgs seems important, but clearly any project that wants to do well will – and vice versa. Some added value comes from input (constructive critism) but is that likely to come from an evaluating body not connected to a project and perhaps without the inside knowledge of what it is about (and therefore what is likely to work or not?)
The point of all this, like (e.g.) the endless reports and MDG goals, does an evaluation of a project have any weight, in reality, to the outcome os a project? Of course, I’m kind of just making a very simplistic point using extreme cases, and of course some independent evaluation of projects is necessary – but I wonder, once all the dust has settled – what is their contribution (if any) to the outcome of a given project, in real terms.
ps – can I just apologise, to anyone who read my post, for the atrocious spelling mistakes and weird paragraph formatting…!
Reading this blog entry made my morning. Thanks Bill.
I’ve been working in global health for 13 years, and I too have become a fan of Marie Stopes International. I have interacted with their staff in a number of countries, and am consistently impressed. I’ve often tried to figure out what makes them such a standout in among global health international NGOs.
I believe one important factor is that they are all about getting the job done; most of their staff are actively involved with services delivery and management. I think this helps to explain why they are so pragmatic, and not ideological. Many other global health NGOs are more active in the advocacy realm – without the grounding that comes from delivering services in 40+ countries every day. And it shows.
Also, their operations are always striving for sustainability, so they are not involved in quick fix approaches which are often appealing to donors/funders, but a lot less helpful to the people and communities they are aiming to serve.
Also, many of their patients I believe pay for their services – so they are not serving passive users who feel lucky to get something (anything). I should add that they also work hard to get subsidies flowing for poor users so that they can reduce cost barriers and serve as many low-income people as possible.
I also think they are unique in being service providers throughout many poor AND RICH countries. This means that, as an organization, they understand health systems better than most organizations which only operate in poor countries. And again, in my experience, this deeper understanding of health systems contributes to their better model for organizing and delivering health services; and it means when they do weigh in at the global/ advocacy level, I often find their positions and logic compelling.
I love what you said about their country director aspiring to be the last expatriate country director. That really reveals a lot of what makes them so effective: a constant focus on helping people help themselves.
Re: best evaluations done by anthropologists: couldn’t agree with you more.
Having worked in the evaluation department of a large international organization (which is staffed largely by economists) I found the dry and bland style of writing, coupled with an incessant bias towards quantitative methods (“we can’t do qualitative studies, they’re just subjective descriptions!!”) led to very poor evaluation papers which did an abysmal job of really capturing what had gone on in a project. Apparently if you can’t reduce it to a number it has no meaning.
This is strange because just today I shared the same sentiment with my coworkers here in West Africa. I work for a US-based nonprofit that does randomized evaluations, and I just got back from a field visit checking out a community bank that has seen success offering credit to farmers via warehousing. Even though inventory credit is an innovative way to offer finance to poor farmers, the bank officers said it works simply because the people involved are honest, diligent and they want it to work. I couldn’t agree more – loans were processed in minutes, not weeks, and they were not hamstrung by nonsensical storage requirements.
In this case, it was clear that these factors are important to helping poor people, perhaps more so than the innovative spin on their loan product..
“PEOPLE to make a difference in aid’ — the argument that I made in my paper (Bill, you’ve seen it). I wonder if the journal I submitted it to will agree? http://ssrn.com/abstract=964058
Sir,
I completely agree with today’s post. Thank you.
I’m, however, quite puzzled that you -at Aidwatchers.com of all places- would reference Ferguson in such a positive light.
His methodology (I’m not thinking of specific method here) is so much in contrast to most of your own research.
If “The anti-politics machine” is the best evaluation you’ve ever read, wouldn’t that be quite a dismissal of mainstream economics’ methodology and then obviously methods?
Do you see a way out?
Totally agree. That was the intent in a previous article -to demonstrate the importance of the individual in the aid delivery process: http://www.aidprocurement.com/2009/03/ethics-and-aid-procurement-and-supply.html
People doing what?
Now I will be the first to tell you that I am certainly not fan of option1. I have said here several times that I believe randomized control trials violate human dignity (poor people have mouths…just ask).
At the same time, I think it matter what people want to use aid to “succeed” in. It could be (and too often in fact it is) harmful to people you re trying to develop.
And this points to the larger problem I have with “aid” as a tool of development a lot of the time (don’t worry, its not Moyo sanctioned); it assumes the best interests of the poor. I think that matters a lot.
Even better than having people who “really, really want “something” to succeed”, your project must be representative of the needs of the poor who “really really want, themselves, to succeed.”
Your emphasis of the “people-factor” in development, reminded me of a book I bought, and had been intending to read, in which the author emphasizes the same point: “Helping People Help Themselves: From the World Bank to an Alternative Philosophy of Development Assistance”, by David Ellerman.
http://www.amazon.com/Helping-People-Help-Themselves-Alternative/dp/0472114654
I’m part way through it now, and I think your readers may find it worth reading. Ellerman talks about the ways in which development agencies either override or undercut beneficiaries intrinsic motives to help themselves, and makes some suggestions for how development assistance might be made more effective by paying more attention to the starting point of where people are and involving them more in efforts to make themselves better-off.
His approach to analyzing the problems is interdisciplinary, underscoring your point about needing to use broader tools of analysis than just economics/ econometrics.
Are venture capitalists really the answer?
Couldn’t begin to know, though my gut cringes at the thought. Venture capitalism seems at odd with the wise view that people are always our most valuable human resource.
Hope the model for venture capitalism is something like the evolving “conscious capitalism,” which must always walk hand in hand with “conscious socialism.” It’s a checks and balances system, seems to me.
Never the either — or. But a dialogue between people bringing different insights to the table.
That written, I think your point that “Second, gut instincts tell you at least a little something about the attitudes of the PEOPLE involved . . .” is a good axiom, if that clause isn’t an oxymoron.
Effective entry. Though I’d love to quibble about some things, I feel less certain in a true economic discussion than playing the social critic. Pretty sure I’d just be further flaunting my lack of understanding.
Best.
One Trackback
[...] more here: The secret to aid is people :approaches, bookmark-the-permalink, development, feed-for, is–bookmark, permalink-, policies, [...]