PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of bmjBMJ helping doctors make better decisionsSearch bmj.comLatest content
 
BMJ. Jun 2, 2007; 334(7604): 1143–1146.
PMCID: PMC1885318
Monitoring and evaluation of programmes to prevent mother to child transmission of HIV in Africa
Richard Reithinger, epidemiologist,1 Karen Megazzini, project director,1 Stephen J Durako, vice president,1 D Robert Harris, senior epidemiologist,1 and Sten H Vermund, director2
1Clinical Trials Area, Westat, Rockville, Maryland, USA
2Institute for Global Health, Vanderbilt University School of Medicine, Nashville, Tennessee, USA
Correspondence to: R Reithinger ; rreithinger/at/yahoo.co.uk
Accepted May 6, 2007.
 
Many countries are expanding the coverage of programmes to prevent mother to child transmission of HIV. Although the need is unquestionable, Richard Reithinger and colleagues are concerned that without true measures of effectiveness we may not be making the best use of resources
In 2006, an estimated 2.3 million children under 15 years were living with HIV and about half a million babies became infected with HIV before birth, during delivery, or through breast feeding.1 Prevention of mother to child transmission of HIV is therefore a priority for agencies fighting the global HIV epidemic, but many questions remain about the effectiveness of the current programmes. We use the President's Emergency Plan for AIDS Relief2 as an example to examine how programmes to prevent mother to child transmission are monitored and evaluated and to highlight the problems.
Estimates of the efficacy of antiretroviral prophylaxis3 suggest that at least half of the world's children who are at risk of HIV infection might be protected if a mother receives antenatal care, is offered HIV counselling and testing, and, if infected, she and her baby receive prophylaxis. Prophylaxis is the mainstay of the strategy to prevent mother to child transmission.3 Several antiretroviral regimens are recommended in resource constrained settings, although nevirapine (either alone or with other drugs) is usually favoured because it is cheap, easy to administer, rapidly absorbed, and has a long half life.3 Depending on the regimen and the mother's choice of infant feeding, the risk of HIV transmission can be reduced to <2%.3 Whenever feasible, programmes should strive to provide highly active antiretroviral therapy to pregnant women.
Although formula feeding can reduce HIV infection rates among infants,4 it is often not acceptable, feasible, affordable, sustainable, or safe in resource limited settings.5 6 Programmes should provide counselling and support on various feeding options (including exclusive breastfeeding), highlighting the potential benefits and risks of each.7 Other activities include promotion of optimal obstetrical practices; improvement of antenatal, postnatal, and child health services; and treatment of maternal diseases. Community based activities are also often implemented to improve community knowledge of HIV and AIDS and counter negative attitudes to people with HIV.8
Because most countries do not have the money or laboratory resources to determine HIV infection in infants using polymerase chain reaction tests, health facility and patient indicators are generally used to assess the effectiveness of programmes (table 11).). The effectiveness of the programme is typically measured in terms of the estimated percentage (change) of HIV infected infants who are born to HIV infected mothers.9 10 However, table 11 shows that effective prevention relies on a cascade of steps. A pregnant woman must receive antenatal care at a centre offering HIV testing. If she consents to testing she must receive her HIV test result and, if infected, receive appropriate prophylaxis. She must then take the prophylaxis during labour and her infant must receive prophylaxis after delivery. Finally, the mother and infant must receive follow-up care to ensure that they have any necessary treatment and that the infant is tested for HIV, typically in the second year of life when HIV antibody testing is reliable.
Table 1
Table 1
 Indicators for monitoring and evaluation of programme to prevent mother to child transmission of HIV used by the President's Emergency Plan for AIDS Relief
The few studies that have attempted to evaluate such programmes show the logistical, managerial, and technical challenges in delivering effective preventive services (table 2)2).5 11 12 13 14 15 16 17 18 Coverage falls progressively the further women are along the prevention cascade. Thus, 12 months after delivery, only a fraction (19% in one study in Malawi5) of HIV positive mothers who received antiretroviral drugs will attend health services to have their infant tested for HIV. Clearly, this may have lethal consequences for those children who become HIV positive.
Table 2
Table 2
 Studies monitoring and evaluating the effectiveness of programmes to prevent mother to child transmission of HIV
The effectiveness of the programmes is compromised if there is a low coverage at any stage of the cascade. Monitoring and evaluation of each stage are thus critical to identify any weakness in the implementation and to develop strategies for improving coverage where necessary.
Relying exclusively on the indicators listed in table 11 has several limitations. Firstly, indicator data are commonly collected from several patient registers in the form of unlinked monthly summary reports.16 Consequently, for example, data on the number of pregnant women who tested HIV positive, received prophylaxis, and then delivered children who also received prophylaxis are not readily available. To obtain linked data, patient registers would need to be thoroughly cross-checked, which would be time consuming and cumbersome without computerisation and planning. The multitude of registers also adds a potential source of human error, with health workers sometimes forgetting to complete them. As a result, different registers may report different denominators.16
Secondly, the reported HIV prevalence rate in pregnant women is typically derived from those accepting voluntary counselling and testing. These women have a lower prevalence than those who refuse testing, probably because women at risk of HIV are less likely to volunteer for testing.11 16
Thirdly, in countries where same day HIV testing is unavailable, 15-67% of women who are tested do not obtain their test results and, hence, may not receive prophylaxis.17 18 19 Even in those countries where rapid testing is available, integration of testing services into antenatal care is essential to ensure that women receive prophylaxis.
Fourthly, and perhaps most importantly, none of the indicators listed in table 11 objectively measure adherence to the prophylactic drugs, and there can be substantial discordance between the proportion of women who agree to prophylaxis and the proportion of women who actually take the drugs. In an extensive surveillance study carried out in Zambia in 2003, only 68% of HIV infected women given a nevirapine tablet during antenatal care to take at the onset of labour had the drug in their umbilical cord blood at delivery.11 Moreover, both the woman and her infant received nevirapine in only 30% of cases despite preventive services being universally available in public antenatal clinics at the time of the evaluation.11 The unexpectedly low level of population coverage highlights the importance of using objective and definitive surrogates to measure the effectiveness of programmes. Once alerted, the Zambian team was able to make improvements at every level of the prevention cascade, increasing coverage as a consequence.
Fifthly, most indicators of programme effectiveness are quantitative rather than qualitative. The quality of services offered at health facilities affects supply and demand for voluntary testing, uptake of antiretroviral drugs, and other activities such as family planning.7 11 16 17 18 Also, current indicators do not capture possible adverse effects of the activities, whether related to the antiretroviral drugs or to the revelation of maternal or child infection status (such as, stigmatisation and domestic violence).
Finally, current indicators fail to evaluate effectiveness in terms of coverage and linkage to care for mothers, infants, and family members. As shown above, rates of postnatal follow-up can be very low.5 11 12 13 14 15 16 17 18 None of the indicators in table 11 capture postnatal follow-up rates so that, for example, the effect of breast feeding can be accurately assessed. Instead, estimates of infant coverage and infection prevented are modelled on nevirapine uptake data or from a limited number of cohort (research) studies.
Ideally programmes should carry out sequential prevalence studies or cohort studies in infants. These studies would reliably estimate infant HIV infections under operational conditions rather than extrapolate figures from clinical trial or “model programme” data. Although these studies could certainly be included in larger programmes, they are labour intensive and costly. For example, infants born to HIV infected mothers would have to be followed up for 15-18 months to rule out transmission. The care benefits of proper follow-up that accrue to mother and infant are substantial, so evaluation costs will be linked to service expenditures. Depending on venue and source of funding, ethical mandates to improve the standard of care for study participants would also reduce the generalisability of evaluations of enhanced prevention programmes and follow-up care. Thus, this type of evaluation would represent a best case scenario. Moreover, cohort studies will not yield data about the programme's coverage unless explicitly designed to do so.
One approach to measuring effectiveness is through anonymous screening of umbilical cord blood samples for HIV antibodies. Positive samples would then be tested for nevirapine (an indicator of maternal nevirapine coverage) using high performance liquid chromatography11 or the cheaper but less sensitive thin layer chromatography followed by high performance retesting of samples with negative results.20 Investigators in Zambia have used cord blood analysis combined with chart documentation of infants' receipt of nevirapine to measure true population coverage.11 When combined with patient level indicator data extracted from a woman's antenatal record at delivery, this approach provides an objective measure of effectiveness both for women who participated in the programme and in the broader population of women delivering in public labour wards.
Collection of patient data must be simplified and linked. In the absence of fully computerised, real time collection of patient health data,21 such linkage could be obtained by incorporating a small coded stamp into the medical record of women attending antenatal care (figure(figure).). A first generation of such a stamp was used when evaluating nevirapine coverage in pregnant women in Zambia22 and is being completed by health staff at each Zambian programme site during routine antenatal care and labour ward visits. The stamp provides a place to record important data for monitoring and evaluating prevention programmes. After delivery, the medical records are left in the health facilities' maternal and child health department, where the data can be readily entered on to computerised databases for analytical purposes or to aid clinic staff in postnatal outreach of infants born to HIV infected women.
figure reir438077.f1
Example of data collection stamp for use on antenatal care and labour records. Spaces shown represent where the dates of the given activity are recorded; multiple lines are provided for documentation of counselling or testing during follow-up visits. (more ...)
Use of a data stamp, combined with the umbilical cord blood analysis, permits collection of anonymous linked maternal and infant data—for example, by combining data on maternal programme indicators, maternal population drug coverage, and data on infant population drug coverage. This is crucial when attempting to identify shortcomings in the implementation of programmes and to estimate the proportion of children who will become infected with HIV. Moreover, inclusion of a modified version of the stamp in an infant's immunisation card may remind health staff of the need to test the infant for HIV and thereby improve postpartum follow-up rates (at least among those who return for immunisations).
The data stamps must be coded to protect the confidentiality of the mother and infant. Confidentiality can be violated only if a healthcare provider reveals the meaning of the stamp to a third party. This is analogous to a medical provider revealing the contents of a patient's medical record. In fact, a coded stamp would be more secure than standard medical record notes because it could not be interpreted without an explanation, whereas a third party might read records successfully without help. Nevertheless, before stamps are widely implemented we need pilot research to assess their feasibility and acceptability in a given clinical and cultural context.
We recognise that the above approach may be unsuitable for areas where a small proportion of women deliver in public labour wards (such as Haiti), where there is insufficient laboratory capacity to process and store cord blood specimens, and where programmes lack funds for high performance liquid chromatography analysis. The cost of analysing each specimen is around $50 (£25; €37),20 although high volumes should reduce costs. Nevertheless, a coded stamp could still enhance monitoring and evaluation efforts by allowing for simplified, periodic extraction of data. These data could verify the data presented in monthly monitoring reports and identify any shortfalls in the programme.
For programmes that use more complex antiretroviral regimens (such as short course zidovudine plus single dose nevirapine or nevirapine-containing highly active antiretroviral therapy), the absence of nevirapine in the cord blood would not necessarily imply a woman was not at least partly covered, assuming she adhered to the zidovudine portion of the regimen. Although zidovudine can be detected in cord blood,23 it is a less perfect surrogate for infant infection than nevirapine because it has a shorter half life and detection in the cord blood does not prove full compliance with the longer zidovudine regimen.
Other methods of measuring the effectiveness of these programmes have been reported. In South Africa, for example, facility based case finding approaches have been used to identify infants born to women accessing programme services,24 and mathematical modelling has also been used.25 However, these approaches do not separate each element of the prevention cascade, making it hard to pinpoint a programme's failures and successes.
Currently, less than 10% of HIV infected pregnant women receive antiretroviral prophylaxis.26 Scaling-up of programmes to prevent mother to child transmission of HIV remains a huge challenge. Programmes supported by organisations with a high media profile, such as the President's Emergency Plan for AIDS Relief, are often subject to substantial political and public pressure to report successes rather than failures. Given the amount of public and private sector investments into these programmes and the programmes' potential effect on a country's societal and economic development, we believe that rigorous monitoring and evaluation of operational programmes is a moral and ethical imperative. Evaluation is essential to identify shortcomings in the programme and to conceptualise approaches to improve services. This, in turn, will improve a programme's cost effectiveness and long term sustainability and save the most infant lives.
Summary points
  • Programmes to prevent mother to child transmission of HIV are an important part of global AIDS initiatives
  • Data on the effectiveness of these programmes are scarce
  • Current indicators to monitor and evaluate the programmes are inadequate
  • Comprehensive indicators that monitor all stages of the prevention cascade are urgently needed
Notes
We thank James Korelitz and the peer reviewers for insightful comments on the manuscript. We also thank Elizabeth Stringer, Moses Sinkala, and Jeffrey Stringer for allowing us to include a stamp similar to the one used in Zambia.
Contributors and sources: RR, KM, SJD, DRH, and SHV have all worked extensively in clinical and operational infectious disease research. The article arose following extensive discussions and planning for activities to monitor and evaluate programmes to prevent mother to child transmission of HIV. Part of this process included a subject review based on comprehensive literature search of medical databases as well as non-medical search engines. RR, KM, and SHV cowrote the manuscript; DRH and SJD contributed conceptually. All authors reviewed the manuscript. RR and SHV are the guarantors.
Competing interests: None declared.
Provenance and peer review: Not commissioned, externally peer reviewed.
1. UNAIDS. AIDS epidemic update, December 2006 http://data.unaids.org/pub/EpiReport/2006/2006_EpiUpdate_en.pdf.
2. United States Department of State. The president's emergency plan for AIDS relief. A US five year global HIV/AIDS strategy Washington, DC: US Department of State, 2006. www.state.gov/documents/organization/29831.pdf.
3. World Health Organization. Antiretroviral drugs for treating pregnant women and preventing HIV infection in infants in resource-limited settings Geneva: WHO, 2006. www.who.int/hiv/pub/mtct/guidelines/en/
4. Nduati R, John G, Mbori-Ngacha D, Richardson B, Overbaugh J, Mwatha A, et al. Effect of breastfeeding and formula feeding on transmission of HIV-1: a randomized clinical trial. JAMA 2000;283:1167-74. [PubMed]
5. Manzi M, Zachariah R, Teck R, Buhendwa L, Kazima J, Bakali E, et al. High acceptability of voluntary counselling and HIV-testing but unacceptable loss to follow-up in a prevention of mother-to-child HIV transmission programme in rural Malawi: scaling up requires a different way of acting. Trop Med Int Health 2005;12:1242-50.
6. WHO Collaborative Study Team on the Role of Breastfeeding on the Prevention of Infant Mortality. Effect of breastfeeding on infant and child mortality due to infectious diseases in less developed countries: a pooled analysis. Lancet 2000;355:451-5. [PubMed]
7. Chopra M, Doherty T, Jackson D, Ashworth A. Preventing HIV transmission to children: quality of counselling of mothers in South Africa. Acta Pediatr 2005;94:357-63.
8. Global HIV Prevention Working Group. New approaches to HIV prevention www.gatesfoundation.org/nr/downloads/globalhealth/aids/pwg_2006_report.pdf.
9. USAID. Expanded response guide to core indicators for monitoring and reporting on HIV/AIDS programmes Washington, DC: USAID, 2003. www.synergyaids.com/documents/Expanded_Response_IndicatorGuide03_5_03.pdf.
10. USAID. The President's Emergency Plan for AIDS Relief: indicators, reporting requirements, and guidelines Washington, DC: USAID, 2005. www.state.gov/documents/organization/58497.pdf.
11. Stringer JS, Sinkala M, Maclean CC, Levy J, Kankasa C, Degroot A, et al. Effectiveness of a city-wide programme to prevent mother-to-child HIV transmission in Lusaka, Zambia. AIDS 2005;19:1309-15. [PMC free article] [PubMed]
12. Ayouba A, Tene G, Cunin P, Foupouapouognigni Y, Menu E, Kfutwah A, et al. Low rate of mother-to-child transmission of HIV1 after nevirapine intervention in a pilot public health programme in Yaoundé, Cameroon. J Acquir Immune Def Syndr 2003;34:274-80.
13. Perez E, Peña R, Tavarez-Rojas M, Peña C, Quinonez S, Buttler M, et al. Preventing mother-to-child transmission in a developing country: the Dominican Republic experience. J Acquir Immune Defic Syndr 2003;34:506-11. [PubMed]
14. Temmerman M, Quaghebeur A, Mwanyumba F, Mandaliya K. Mother-to-child HIV transmission in resource poor settings: how to improve coverage? AIDS 2003;17:1239-42. [PubMed]
15. Van't Hoog A, Mbori-Ngacha DA, Marum LH, Otieno JA, Misore AO, Nganga LW, et al. Preventing mother-to-child transmission of HIV in Western Kenya. Operational issues. J Acquir Immune Defic Syndr 2005;40:344-9. [PubMed]
16. Sherman GG, Jones SA, Coovadia AH, Urban MF, Bolton KD. PMTCT from research to reality—results from a routine service. S Afr Med J 2004;94:289-92. [PubMed]
17. Doherty TM, McCoy D, Donohue S. Health system constraints to optimal coverage of the prevention of mother-to-child HIV transmission programme in South Africa: lessons from the implementation of the national pilot programme. Afr Health Sci 2005;5:213-18. [PubMed]
18. Perez F, Orne-Gliemann J, Mukotekwa T, Miller A, Glenshaw M, Mahomva A, et al. Prevention of mother-to-child transmission of HIV: evaluation of a pilot programme in a district hospital in rural Zimbabwe. BMJ 2004;329:1147-50. [PMC free article] [PubMed]
19. Unicef. Evaluation of United Nations-supported pilot projects for the prevention of mother-to-child transmission of HIV New York: Unicef, 2003:46.
20. Chi BH, Lee A, Acosta EP, Westerman LE, Sinkala M, Stringer JS. Field performance of a thin-layer chromatography assay for detection of nevirapine in umbilical cord blood. HIV Clin Trials 2006;7:263-9. [PubMed]
21. Siika AM, Rotich JK, Simiyu CJ, Kigotho EM, Smith FE, Sidle JE, et al. An electronic medical record system for ambulatory care of HIV-infected patients in Kenya. Int J Med Inform 2005;74:345-55. [PubMed]
22. Stringer EM, Sinkala M, Stringer JS, Mzyece E, Makuka I, Goldenberg RL, et al. Prevention of mother-to-child transmission of HIV in Africa: successes and challenges in scaling-up a nevirapine-based program in Lusaka, Zambia. AIDS 2003;17:1377-82. [PMC free article] [PubMed]
23. Bhadrakom C, Simonds RJ, Mei JV, Asavapiriyanont S, Sangtaweesin V, Vanprapar N, et al. Oral zidovudine during labour to prevent perinatal HIV transmission, Bangkok: tolerance and zidovudine concentration in cord blood. Bangkok Collaborative Perinatal HIV Transmission Study Group. AIDS 2000;14:509-16. [PubMed]
24. Coetzee D, Hilderbrand K, Boulle A, Draper B, Abdullah F, Goemaere E. Effectiveness of the first district-wide programme for the prevention of mother-to-child transmission of HIV in South Africa. Bull World Health Organ 2005;83:489-94. [PubMed]
25. Alioum A, Cortina-Borja M, Dabis F, Dequae-Merchadou L, Haverkamp G, Hughes J, et al. Estimating the efficacy of interventions to prevent mother-to-child transmission of human immunodeficiency virus in breastfeeding populations: comparing statistical methods. Am J Epidemiol 2003;158:596-605. [PubMed]
26. Secretary General of United Nations. Declaration of commitment on HIV/AIDS: five years later Geneva: UN, 2006.
Articles from BMJ : British Medical Journal are provided here courtesy of
BMJ Group