PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of intjepidLink to Publisher's site
 
Int J Epidemiol. 2010 April; 39(Suppl 1): i11–i20.
Published online 2010 March 23. doi:  10.1093/ije/dyq017
PMCID: PMC2845856

Comparing modelled predictions of neonatal mortality impacts using LiST with observed results of community-based intervention trials in South Asia

Abstract

Background There is an increasing body of evidence from trials suggesting that major reductions in neonatal mortality are possible through community-based interventions. Since these trials involve packages of varying content, determining how much of the observed mortality reduction is due to specific interventions is problematic. The Lives Saved Tool (LiST) is designed to facilitate programmatic prioritization by modelling mortality reductions related to increasing coverage of specific interventions which may be combined into packages.

Methods To assess the validity of LiST outputs, we compared predictions generated by LiST with observed neonatal mortality reductions in trials of packages which met inclusion criteria but were not used as evidence inputs for LiST.

Results Four trials, all from South Asia, met the inclusion criteria. The neonatal mortality rate (NMR) predicted by LiST matched the observed rate very closely in two effectiveness-type trials. LiST predicted NMR reduction was close (absolute difference <5/1000 live births) in a third study. The NMR at the end of the fourth study (Shivgarh, India) was overestimated by 39% or 16/1000 live births.

Conclusions These results suggest that LiST is a reasonably reliable tool for use by policymakers to prioritize interventions to reduce neonatal deaths, at least in South Asia and where empirical data are unavailable. Reasons for the underestimated reduction in one trial likely include the inability of LiST to model all effective interventions.

Keywords: child survival, neonatal mortality, modelling, Lives Saved Tool, Bangladesh, India, Pakistan

Introduction

Each year almost 4 million neonatal deaths (first 28 days of life) occur, accounting for an estimated 41% of all under-5 deaths.1,2 There is increasing policy and programme attention to reducing neonatal deaths, which have previously been relatively neglected in both maternal and child health programmes. Given limited funding and the short time to the Millennium Development Goal deadline of 2015, the imperative to invest in the most effective interventions is clear. However, a ‘one size fits all’ approach is unlikely to work since the effect of interventions will depend on the local cause-of-death profile and the health system platforms available for scale up.3 Decision-makers require information on the likely impacts of different interventions in their own setting, together with information on the incremental cost of adding interventions to an existing package either at facility or community level.

The Lives Saved Tool (LiST) has been designed for the purpose of facilitating programmatic decisions based on mortality effects. LiST allows users to model the estimated impact of scaling up a specific maternal, newborn and child health intervention by increasing the coverage for a defined population which may be one country, state or district. An important feature of LiST is that it models explicitly the effects of changes in individual-level interventions such as exclusive breastfeeding of a newborn or ORS treatment of a child with diarrhoea. The public health strategies by which coverage of these interventions is increased (e.g. peer counselling, community mobilization, etc.) are not modelled explicitly in LiST. It is up to the user to decide what levels of coverage can be achieved with the strategies and resources at their disposal. LiST is built into the widely accepted demographic software package ‘Spectrum’, designed to model population changes over time by age and sex4,5 and includes recent mortality rates by country with neonatal and under-5 cause-of-death estimates. The neonatal cause-of-death estimates are based on methods published previously,6 but have been updated for the year 2004 and cleared by WHO with countries.7,8 The neonatal mortality module in LiST is based upon evidence-based reviews for intervention impact originally undertaken for The Lancet Neonatal series.9 A standard methodology has been developed by the Child Health Epidemiology Reference Group (CHERG)10 for systematic review, rating of the evidence and estimation of mortality effect using an adapted GRADE methodology. Reviews of individual interventions are presented elsewhere in this supplement.11–15

Objective

To assess the validity of LiST for modelling neonatal mortality by comparing the reductions in neonatal mortality predicted by LiST with observed changes in mortality in field trials of packages of interventions.

Methods

Model

The development of LiST and the modelling assumptions that it makes are described in detail in several articles in this supplement. Briefly, LiST is a cohort model of neonatal and child survival, up to 5 years of age, embedded within the Spectrum Policy Modelling System (http://www.futuresinstitute.org). It provides estimates of the effects on cause-specific mortality (e.g. on deaths and mortality rates) of changing the coverage of different interventions. Several interventions in the model affect specific causes of neonatal death and thus neonatal mortality rates. The current effect sizes that LiST assumes for each intervention and each cause of neonatal death (except diarrhoea and congenital anomalies) are shown in Table 1.

Table 1
Cause-specific percent reduction in mortality estimated for interventions in LiST which affect neonatal mortality

Analysis

Validation data were selected based on the following inclusion criteria:

  1. Published studies with neonatal mortality data at baseline and endline, including cause-specific data or data from a similar population that could be used for the cause-of-death profile.
  2. Coverage data at baseline and endline for indicators to allow at least two interventions to be modelled.
  3. The study results had not been used as inputs to the effect estimates in the model.

Four studies met the inclusion criteria (Table 2).16–22 One was an effectiveness trial (Hala)16 whereas another simulated near program conditions (Sylhet).17,22 In the remaining two studies, the research team played an important role in assuring the delivery of the intervention package. In the intervention areas, packages of interventions designed to affect neonatal mortality were delivered. Only the primary intervention area was modelled for each study.

Table 2
Summary description of the four validation studies and the observed reductions in neonatal mortality, study time-lines, interventions and results

Neonatal researchers involved in these studies or other studies of neonatal interventions (ZB, GD and JL) met to map the coverage indicators of the interventions which were implemented and/or measured in each study onto the coverage indicators included in LiST. For each study, there was extensive discussion about which indicators were the most appropriate (Table 3). For components of facility delivery, such as comprehensive emergency obstetric care, basic emergency obstetric care, neonatal resuscitation, antenatal corticosteroids and antibiotics for preterm prolonged rupture of membranes, coverage data were not available. Thus, based on previous reviews by the Lancet Neonatal Series team, a standard proportion of women who deliver at a facility were assumed to receive each of the components.9 See the Supplementary appendix for the relevant fractions. The final mapping of the study indicators onto LiST indicators was performed prior to running the model and seeing the results. Study indicators which could not be mapped within LiST are shown in Table 4. For each study, a LiST model of the study population at baseline was created, including total population, fertility trends, health status (including nutritional status), neonatal mortality rates and cause-specific proportions of neonatal deaths (cause-of-death profile).

Table 3
Baseline and endline coverage estimates (%) used in the LiST modelling
Table 4
Interventions or indicators which could not be mapped to LiST, by study

Each study was then modelled in LiST to determine the predicted change in neonatal mortality given the changes in intervention coverage observed in the study. The changes in mortality predicted by the model were then compared with the changes actually observed in each study. The specific modelling details for each study, including non-standard-coding methods are described below. All modelled interventions are described in Table 5.

Table 5
Mapping of study coverage measures to LiST

Description of studies used for validation and modelling assumptions

Sylhet, Bangladesh17,22

The home-care arm of the study was modelled. The neonatal mortality rate measured in the trial population at baseline was used together with the study-specific cause-of-death profile. The Bangladesh national fertility trends in Spectrum were used without additional adjustment. For antenatal care (ANC), the coverage indicator used was one or more visits during pregnancy, although the ideal LiST indicator is four or more visits. Within LiST, ANC is only linked to the intervention of syphilis detection and treatment. As this was not measured in the trial, it was calculated based upon standard formulas built into LiST under the assumption that a given proportion women would have received this when visiting providers for routine ANC.9 For tetanus toxoid (TT) immunization, the coverage indicator used was two doses of TT during pregnancy. For multiple micronutrient supplementation, maternal report of taking an iron-containing compound during pregnancy was the coverage indicator used. The available indicator of facility birth was used and linked to selected facility-based interventions (antenatal corticosteroids for women with anticipated preterm labour, antibiotics for preterm prolonged rupture of membranes (pPRoM), basic emergency obstetric care, comprehensive emergency obstetric care and neonatal resuscitation) using standard formulas built into LiST. Clean delivery at home was defined as use of a clean delivery kit or a boiled cutting instrument at delivery and based on trial data. In the home-care arm of the study, case management of serious neonatal illness included successful referral to a facility, or if referral was not successful, provision of injectable antibiotics in the community. Of the children who were identified to have a serious neonatal illness, 31.9% were successfully referred (and modelled as full supportive care for serious neonatal illness) and an additional 41.5% received injectable antibiotics in the community (and modelled as ‘injectable antibiotics for case management of serious neonatal illness’). Data were unavailable for the type of treatment received at health facilities by the referred patients.

SEARCH, India19,20

The fertility trends were based on the Maharashtra population data in the National Family Health Survey23 and matched the crude birth rates observed in the intervention area. The cause-of-death data used were modified from Bang et al.19 by the investigator to fit the CHERG Neonatal group standard hierarchy, with some early infection deaths in very preterm infants being reclassified to the category of preterm as a direct cause of death. Village health workers visited homes during pregnancy to promote breastfeeding, and also visited the home within 24 h after delivery. The coverage of these home visits was used as the indicator for both preventive postnatal care and breastfeeding promotion. Coverage of neonatal resuscitation in the home was based on the percentage of births with a village health worker present at birth, since all study health workers were trained in resuscitation. Most sepsis cases were treated in the community using a combination of injectable and oral antibiotics. This was modelled as ‘injectable antibiotics for case management of serious neonatal illness’.

Hala, Pakistan16

Pakistani national data regarding cause-specific neonatal mortality were used,24 as were standard national fertility trends from Spectrum. Coverage of one or more antenatal visits was used for ANC and linked interventions. Data for coverage of two doses of TT immunization were collected in the study surveys, as was skilled birth attendance. For the indicator of clean home birth, the coverage with lady health workers at home for birth was used. Early initiation of breastfeeding was the indicator used for breastfeeding promotion.

Shivgarh, India21

The fertility trends used were based on the rural Uttar Pradesh population data in the National Family Health Survey.25 In this trial, mortality was measured both through household surveys and prospectively by a demographic surveillance system. As surveys may underestimate mortality, we used only surveillance data and used the observed mortality rate at endline in the comparison arm as the proxy for the baseline mortality rate in the intervention arm. The third arm of the study, (intervention package plus Thermospot), was excluded from the analysis as we chose only one arm to evaluate for each study and it would be difficult to model the additional impact of the ThermoSpot within LiST. However, intervention coverage figures and mortality impact in the two arms were not significantly different. Since neonatal cause-of-death data were not yet available from the study or from the state of Uttar Pradesh, the baseline cause of death profile used was the same as the SEARCH study, which is from a similar context in rural India. The indicator used in LiST for ANC and linked interventions was one or more ANC visits from the trial data as the ideal indicator of four or more routine ANC visits was not reported. Data were available for the coverage of two doses of TT vaccine. ‘Delivery into hands’ instead of on the ground was the indicator used for a clean home delivery. Although several thermal care indicators were reported, the most important one was considered to be skin-to-skin care within 24 h of birth. This was used as the indicator for appropriate preventive postnatal care. Similar to the Hala study, early initiation of breastfeeding was the indicator used for breastfeeding promotion.

Results

More than 40 interventions were included in the version of LiST used at the time of this analysis (version 4.0, which is not publically available at the time of writing). Of these, 19 have an effect on neonatal mortality (Table 1). For each study included in the validation exercise, all the coverage indicators that were available were mapped to interventions in the LiST model (Table 5). For each of the studies, several indicators or interventions were not able to be mapped to the model (Table 4).

Model outputs and observed mortality reductions are presented in Table 6. For three of the four studies, the modelled effect of the interventions closely matched the observed study results, with the predicted impact lying within the confidence bounds for the observed results. The one outlier was the Shivgarh trial. In this study, control area mortality (used as a proxy for baseline mortality in the modelling) was extremely high (84 per 1000 live births) and a very large reduction (58%) to 41 per 1000 live births was observed in the intervention area. In this study, LiST overestimated the observed NMR at the end of the study by 39%. Across the Hala, Sylhet, SEARCH and Shivgarh studies, the estimates derived using LiST differed from the observed rates of neonatal mortality by 0, 2, 5 or 16 per 1000 live births, respectively.

Table 6
Neonatal mortality rates for four validation studies, measured at baseline, measured at endline (or concurrent control), and modelled for endline using LiST

Discussion

This is the first attempt to validate the modelling of neonatal lives saved which has been developed over the past 5 years and is now incorporated within LiST. NMRs predicted by LiST were remarkably close to those observed in three out of the four studies used for validation. For the two studies implemented under routine (or close to routine) programme conditions, the model predictions were almost identical to the observed results (Hala, Pakistan16 and Sylhet, Bangladesh17). For the two studies which were closer to efficacy-type studies, the mortality reductions observed were greater than those predicted by LiST: i.e. LiST appears to produce conservative estimates of the mortality effect. This is a reassuring finding for a tool that aims to model programme effectiveness. These results suggest that LiST is a reasonably reliable tool for use by policymakers to prioritize interventions for maximal effect on neonatal deaths, at least in South Asia.

There are a number of possible explanations for the underestimation of the neonatal mortality effect by LiST in these trials. The first is that these were efficacy-type trials and set in populations with especially high mortality, suggesting that the model is conservative, as per the stated objective of including effectiveness estimates in the model. In addition, this model only includes interventions which have an evidence-based, cause-specific effect on mortality and are included in LiST. Thus, interventions which are likely to have an effect, but for which insufficient high-quality data have been collected to adequately generate a valid effect size were excluded from the modelling exercise, such as thermal care improvements or early initiation of breastfeeding. Another possible explanation for the conservative nature of the model is that some interventions may have a synergistic effect when delivered together that is difficult to quantify. Finally, in all environments, there are always changes which cannot be quantified, such as quality changes when intensive implementation has occurred, or on which data have not been collected. However, this should not be taken to mean that these changes have no effect. This also applies to non-health or distal factors that we are unable to model in LiST such as changes in education, women’s empowerment, food availability, and other factors which are known to affect overall mortality rates but do not have a clear cause-specific effect that can be modelled over and above their impact on coverage changes.

Only in one study did the observed mortality reduction differ by more than five neonatal deaths per 1000 live births. In this trial, in Shivgarh,21 a more comprehensive set of behaviour change interventions were implemented, including monthly community and folk song meetings with behaviour change messages, birth preparedness and education of a variety of newborn care stakeholders. More of the study activities and indicators could not be mapped in LiST than in the other trials. Perhaps more importantly, this study did not have measured neonatal mortality at baseline in the modelled arm of the trial; rather, the concurrent control area neonatal mortality rate was used as a proxy for the baseline rate. In addition, this area’s neonatal mortality rate was extremely high. Also, the use of a cause-of-death profile from another site introduced additional uncertainty. Thus, it was expected that the model would underestimate the neonatal mortality reduction.

Although the results of this validation exercise are encouraging, it should be noted that all four studies were performed in South Asia. A similar validation to compare LiST estimates of mortality reduction with study results from Africa, East Asia or South or Central America would be helpful, but unfortunately no published studies from these regions met our inclusion criteria. However, there are neonatal outcome studies now in progress in these regions so such comparative analysis may soon be possible. It would also be of interest to perform additional validations with nationally representative databases such as in the Demographic and Health (DHS) and Multiple Indicator Cluster (MIC) surveys, for example, similar to that presented by Hazel and Bhutta.27 in this journal issue, using the default CHERG cause-of-death profiles.

In summary, LiST appears to model well the effect of interventions on neonatal mortality. One limitation of LiST is in the number and variety of interventions included in the model for which high quality empirical coverage data are also available. Given the relatively recent attention to reducing neonatal mortality, the comparability and availability of coverage indicators is an important area to advance within large scale surveys, health facility assessments and in programmes and research. Although further validation of the model is desirable, the results obtained here suggest that LiST can be used as a reasonable guide for decision making when adequate data are unavailable or to extrapolate to future outcomes for lives saved with given changes in coverage. Important gaps remain in the evidence for neonatal mortality reduction, with ironically more rigorous evidence now available from community level than from facility level. In addition, important changes in community interactions and behaviour change remain challenging to model, but may be especially important in high mortality settings, such as Shivgarh, where almost 10% of babies die, many of whom are term babies with preventable and treatable problems. In addition, important changes in community interactions and behaviour change remain challenging to model, but may be especially important in high mortality settings, such as the study from Shivgarh, where currently almost 10% of babies die, many of whom are term babies dying from preventable and treatable conditions.

Supplementary data

Supplementary data are available at IJE online.

Funding

This analysis was funded by the US Fund for UNICEF through a grant from the Bill & Melinda Gates Foundation to ‘Promote evidence-based decision making in designing maternal, neonatal and child health interventions in low- and middle-income countries’ (No. 43386); and Save The Children-US through a grant from the Bill & Melinda Gates Foundation for ‘Saving Newborn Lives’ (No. 50124).

The individual studies were supported by the United States Agency for International Development (USAID), through cooperative agreements with the Johns Hopkins Bloomberg School of Public Health and the International Centre for Diarrhoeal Disease Research, Bangladesh (ICDDR,B), and by the Saving Newborn Lives program of Save the Children-US through a grant from the Bill & Melinda Gates Foundation (Sylhet); The John D. and Catherine T. MacArthur Foundation, The Ford Foundation, Saving Newborn Lives, Save the Children-US, The Bill & Melinda Gates Foundation, the International Women's Health Coalition, Oxfam and the Indian Council of Medical Research (SEARCH); WHO and the Saving Newborn Lives program of Save the Children-US, funded by the Bill & Melinda Gates Foundation (Hala); and the USAID, Delhi Mission and the Saving Newborn Lives program of Save the Children-US through a grant from the Bill & Melinda Gates Foundation (Shivgarh).

Acknowledgements

The authors are grateful to Becky Ferguson at the Bill and Melinda Gates Foundation for facilitating the meeting to undertake this analysis.

Conflict of interest: None declared.

KEY MESSAGES

  • The LiST modelled results matched well with the observed results of 4 published community trials of neonatal interventions in South Asia.
  • LiST can aid in program planning by modelling the potential impacts of community-based neonatal interventions, although more validation is needed, especially in African contexts.

References

1. Lawn JE, Cousens S, Zupan J. 4 million neonatal deaths: when? Where? Why? Lancet. 2005;365:891–900. [PubMed]
2. Lawn JE, Kerber K, Enweronu-Laryea C, Massee Bateman O. Newborn survival in low resource settings – are we delivering? Br J Obstetrics Gyneco. 2009;116(Suppl 1):49–59. [PubMed]
3. Knippenberg R, Lawn JE, Darmstadt GL, et al. Systematic scaling up of neonatal care in countries. Lancet. 2005;365:1087–98. [PubMed]
4. Stover J, Walker N, Grassly NC, Marston M. Projecting the demographic impact of AIDS and the number of people in need of treatment: updates to the Spectrum projection package. Sex Transm infect. 2006;82(Suppl 3):iii45–50. [PMC free article] [PubMed]
5. Stover J, Johnson P, Zaba B, Zwahlen M, Dabis F, Ekpini RE. The Spectrum projection package: improvements in estimating mortality, ART needs, PMTCT impact and uncertainty bounds. Sex Transm Infect. 2008;84(Suppl 1):i24–30. [PMC free article] [PubMed]
6. Lawn JE, Wilczynska-Ketende K, Cousens SN. Estimating the causes of 4 million neonatal deaths in the year 2000. Int J Epidemiol. 2006;35:706–18. [PubMed]
7. Bryce J, Daelmans B, Dwivedi A, et al. Countdown to 2015 for maternal, newborn, and child survival: the 2008 report on tracking coverage of interventions. Lancet. 2008;371:1247–58. [PubMed]
8. World Health Organization. The Global Burden of Disease. 2004 Update; 2008. Switzerland: WHO Press.
9. Darmstadt GL, Bhutta ZA, Cousens S, Adam T, Walker N, de Bernis L. Evidence-based, cost-effective interventions: how many newborn babies can we save? Lancet. 2005;365:977–88. [PubMed]
10. Walker N, Fischer-Walker C, Bryce J, Bahl R, Cousens S. writing for the CHERG review groups on Intervention Effects. Standards for CHERG Reviews of Intervention Effects on Child Survival. Int J Epidemol. 2010;39(Suppl 1):i21–31. [PMC free article] [PubMed]
11. Blencowe H, Lawn J, Vandelaer J, Roper M, Cousens S. Tetanus toxoid immunization to reduce mortality from neonatal tetanus. Int J Epidemol. 2010;39(Suppl 1):i102–109. [PMC free article] [PubMed]
12. Blencowe H, Cousens S, Modell B, Lawn J. Folic acid to reduce neonatal mortality from neural tube disorders. Int J Epidemol. 2010;39(Suppl 1):i110–21. [PMC free article] [PubMed]
13. Cousens S, Blencowe H, Gravett M, Lawn JE. Antibiotics for pre-term pre-labour rupture of membranes: prevention of neonatal deaths due to complications of pre-term birth and infection. Int J Epidemol. 2010;39(Suppl 1):i134–43. [PMC free article] [PubMed]
14. Mwansa-Kambafwile J, Cousens S, Hansen T, Lawn JE. Antenatal steroids in preterm labour for the prevention of neonatal deaths due to complications of preterm birth. Int J Epidemol. 2010;39(Suppl 1):i122–33. [PMC free article] [PubMed]
15. Lawn JE, Mwansa-Kambafwile J, Horta BL, Barros FC, Cousens S. ‘Kangaroo mother care’ to prevent neonatal deaths due to preterm birth complications. Int J Epidemol. 2010;39(Suppl 1):i44–54. [PMC free article] [PubMed]
16. Bhutta ZA, Memon ZA, Soofi S, Salat MS, Cousens S, Martines J. Implementing community-based perinatal care: results from a pilot study in rural Pakistan. Bull World Health Organ. 2008;86:452–59. [PubMed]
17. Baqui AH, Arifeen SE, Williams EK, et al. Effectiveness of home-based management of newborn infections by community health workers in rural Bangladesh. Pediatr Infect Dis J. 2009;28:304–10. [PMC free article] [PubMed]
18. Bang AT, Bang RA, Sontakke PG. Management of childhood pneumonia by traditional birth attendants. The SEARCH Team. Bull World Health Organ. 1994;72:897–905. [PubMed]
19. Bang AT, Bang RA, Baitule SB, Reddy MH, Deshmukh MD. Effect of home-based neonatal care and management of sepsis on neonatal mortality: field trial in rural India. Lancet. 1999;354:1955–61. [PubMed]
20. Bang AT, Reddy HM, Deshmukh MD, Baitule SB, Bang RA. Neonatal and infant mortality in the ten years (1993 to 2003) of the Gadchiroli field trial: effect of home-based neonatal care. J Perinatol. 2005;25(Suppl 1):S92–107. [PubMed]
21. Kumar V, Mohanty S, Kumar A, et al. Effect of community-based behaviour change management on neonatal mortality in Shivgarh, Uttar Pradesh, India: a cluster-randomised controlled trial. Lancet. 2008;372:1151–62. [PubMed]
22. Baqui AH, El Arifeen S, Darmstadt GL, et al. Effect of community-based newborn-care intervention package implemented through two service-delivery strategies in Sylhet district, Bangladesh: a cluster-randomised controlled trial. Lancet. 2008;371:1936–44. [PubMed]
23. International Institute for Population Sciences (IIPS) and Macro International. National Family Health Survey (NFHS-2), India, 1998–99: Maharashtra. 2001. Mumbai, India: IIPS.
24. Bhutta ZA, Ali S, Cousens S, et al. Alma-Ata: Rebirth and Revision 6 Interventions to address maternal, newborn, and child survival: what difference can integrated primary health care strategies make? Lancet. 2008;372:972–89. [PubMed]
25. International Institute for Population Sciences (IIPS) and Macro International. National Family Health Survey (NFHS-3), 2005–06: Uttar Pradesh. 2008. Mumbai, India: IIPS.
26. Bhutta ZA, Ahmed T, Black RE, et al. What works? Interventions for maternal and child undernutrition and survival. Lancet. 2008;371:417–40. [PubMed]
27. Haider BA, Bhutta ZA. Cochrane Database of Systematic Reviews. Multiple-micronutrient supplementation for women during pregnancy. 2006, Issue 4. Art. No.: CD004905. DOI: 10.1002/14651858.CD004905.pub2. [PubMed]

Articles from International Journal of Epidemiology are provided here courtesy of Oxford University Press