PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of bmjThis ArticleThe BMJ
 
BMJ. 2007 June 2; 334(7604): 1123–1124.
PMCID: PMC1885342

Evaluation of HIV programmes

Independent national evaluations would mitigate global donors' desire to claim sole success

The HIV implementers' meeting in Kigali, Rwanda, 16-19 June 2007, will bring together programme implementers, researchers, representatives of donors who are funding HIV programmes, and international agencies tasked with controlling the global HIV epidemic. The meeting will focus on three initiatives that account for about 64% of international financing towards fighting HIV1—the Global Fund to Fight AIDS, TB, and Malaria; the President's Emergency Plan for AIDS Relief; and the World Bank's Multi-country AIDS Programme. The meeting aims to facilitate an open dialogue about the future direction of HIV programmes, and it will focus on identifying crucial barriers to and best practices in expanding efforts to control the epidemic.2

In this week's BMJ, Reithinger and colleagues3 discuss the difficulties in evaluating the effectiveness of programmes to prevent transmission of HIV from mother to child. Difficulties include the inadequacy of current indicators used to monitor and evaluate operational programmes; weak health information systems, especially in the poorest countries of Africa which, for example, make it difficult to tell whether pregnant women who tested positive subsequently received treatment; and poor quality of interventions in terms of care offered and adherence to treatment. All of these challenges prevent reliable judgments being made about the impact of programmes.

The challenges are political as much as programmatic. Global initiatives such as the President's Emergency Plan and the Global Fund have features, inherent in stand alone project approaches (rather than pooling of aid from different donors), which make them attractive to donors.4 5 They offer opportunities to channel funds into politically high profile areas, with short time frames that meet politicians' needs. By focusing on disease specific interventions rather than on the health services needed to deliver them, donors can claim that improvements are the result of their contributions. Donors can demand reports in formats that meet their specific needs, which can mean that recipient countries have to prepare several reports in different formats for donors who are jointly funding programme activities. Also, constraints such as shortages of health workers and managers can often be overcome through financial incentives that attract scarce staff to well funded donor projects.

These features are generally associated with the President's Emergency Plan. However, the Global Fund has shifted its approach in the past year and is attempting, with some success, to escape the shackles of the project approach. This is evident in its February 2007 results report,1 in which it judges its own performance according to five globally agreed commitments6 and indicators of progress7 on making aid more effective—the need for country ownership; alignment, which means that donors will target countries' priorities and use countries' management systems; harmonisation of donor procedures and sharing of analyses; focusing on results at the country level; and mutual accountability.

The committee for the evaluation of the President's Emergency Plan has also recently signed up to the need for improved harmonisation and coordination.8 In some respects Global Fund, the President's Emergency Plan, and World Bank's AIDS programme have out performed both stand alone project approaches and the pooling of aid from different donors, which several European donor agencies have championed. They have disbursed money quickly, with high proportions—30% of Global Fund support1—reaching non-governmental organisations that are central to the fight against HIV. The need to show accountability and performance is a requirement that these initiatives have grasped with enthusiasm, as in the Global Fund's early mantra, “raise it, spend it, prove it.”

Plans for Global Fund's five year evaluation (2007-8) are well advanced9; the World Bank will publish an evaluation of its AIDS programme at the Kigali meeting; and the President's Emergency Plan is planning a large scale evaluation. Global Fund and the President's Emergency Plan both intend to provide evidence that the expansion in HIV control has reduced the incidence of HIV, as well as morbidity and mortality from the disease. There has been an important shift in the language used by these initiatives, one that avoids ludicrous claims of being solely responsible for providing antiretroviral treatment to hundreds of thousands of patients. Both now use terms such as Global Fund (or the President's Emergency Plan) supported programmes,1 8 reflecting the reality that multiple sources of funds, notably from recipient country governments and their populations, collectively support the expansion of treatment.

The meeting in Rwanda next month will show whether or not the tensions that are inherent in donors' need to claim successes, so as to generate sustainable long term funding, continue to undermine global efforts at coordination. The desire to avoid mutually destructive competition has led the Global Fund to declare that its five year evaluation (2007-8) will measure “contribution” rather than “attribution.”9 However, this should not detract from another pressing question—not just whether the expansion is working—but how the collective efforts of all the global and national players, whether focused on diseases or on strengthening country systems, are working to support expansion and impact.10 Answering this question in the context of different countries means entering sensitive terrain. One solution would be for managers of national programmes and national AIDS councils to work with independently funded researchers, who will be less vulnerable to the pressures of attribution.

Notes

Competing interests: RB was a principal investigator in a study of the Global Fund (2003-4), commissioned by four European donor agencies: Irish Aid, Danish International Development Agency (Danida), UK Department for International Development, Directorate-General for International Cooperation (DGIS, Netherlands). He is currently a principal investigator on a network, funded by Irish Aid and Danida (2006-9), which is supporting 15 country studies of HIV global initiatives (www.ghin.lshtm.ac.uk). RB has intermittently been contracted by Irish Aid to undertake reviews, including participation in the 2005 high level forum on global health partnerships; he has also been contracted as a consultant by the EuroHealth consultancy group, through funding from the Global Fund, on a study of Global Fund proposal preparation processes (2005-6) and one on local fund agent processes (2007); and was contracted by the Global Fund to support a review, in 2006, of how the fund allocates its resources.

Provenance and peer review: Non-commissioned; not externally peer reviewed.

References

1. Global Fund. Partners in impact, results report. Geneva: Global Fund to Fight AIDS, Tuberculosis and Malaria. 2007. www.theglobalfund.org/en/files/about/replenishment/oslo/Progress%20Report.pdf [PubMed]
2. 2007 HIV/AIDS implementers' meeting, June 16-19, Kigali. www.hivimplementers.org/pdf/pr_call_for_abstracts.pdf
3. Reithinger R, Megazzini K, Durako SJ, Harris DR, Vermund SH. Monitoring and evaluation of programmes to prevent mother to child transmission of HIV in Africa. BMJ 2007. doi: 10.1136/bmj.39211.527488.94 [PMC free article] [PubMed]
4. Oliveira-Cruz V, Kurowski C, Mills A. Delivery of health interventions: searching for synergies within the vertical versus horizontal debate. J Int Dev 2003;15:67-86.
5. Cairncross S, Peries H, Cutts F. Vertical health programmes. Lancet 1997;349(suppl III):20-2. [PubMed]
6. Organisation for Economic Cooperation and Development. Paris declaration on aid effectiveness March 2005. www.aidharmonization.org/ah-overview/secondary-pages/editable?key=205
7. Organisation for Economic Cooperation and Development. The Paris declaration on aid effectiveness. Indicators of progress. 2005. www.oecd.org/dataoecd/57/60/36080258.pdf
8. Committee for the Evaluation of the President's Emergency Plan for AIDS Relief (PEPFAR) Implementation, Sepulveda J, Carpenter C, Curran J, Holzemer W, Smits H, Scott K, et al, eds. PEPFAR implementation: progress and promise Washington: National Academies Press, 2007
9. Schwartlander B. Technical Evaluation Reference Group (TERG) of the Global Fund. Partners in impact forum. The Global Fund five year evaluation 2007. www.theglobalfund.org/en/files/about/terg/announcements/impact_forum/PIF_IntroductionFive-YearEvaluation.pdf
10. Bennett S, Boerma T, Brugha R. Scaling up HIV/AIDS evaluation. Lancet 2006;367:79-82. [PubMed]

Articles from The BMJ are provided here courtesy of BMJ Publishing Group