|Home | About | Journals | Submit | Contact Us | Français|
Background There is an increasing body of evidence from trials suggesting that major reductions in neonatal mortality are possible through community-based interventions. Since these trials involve packages of varying content, determining how much of the observed mortality reduction is due to specific interventions is problematic. The Lives Saved Tool (LiST) is designed to facilitate programmatic prioritization by modelling mortality reductions related to increasing coverage of specific interventions which may be combined into packages.
Methods To assess the validity of LiST outputs, we compared predictions generated by LiST with observed neonatal mortality reductions in trials of packages which met inclusion criteria but were not used as evidence inputs for LiST.
Results Four trials, all from South Asia, met the inclusion criteria. The neonatal mortality rate (NMR) predicted by LiST matched the observed rate very closely in two effectiveness-type trials. LiST predicted NMR reduction was close (absolute difference <5/1000 live births) in a third study. The NMR at the end of the fourth study (Shivgarh, India) was overestimated by 39% or 16/1000 live births.
Conclusions These results suggest that LiST is a reasonably reliable tool for use by policymakers to prioritize interventions to reduce neonatal deaths, at least in South Asia and where empirical data are unavailable. Reasons for the underestimated reduction in one trial likely include the inability of LiST to model all effective interventions.
Each year almost 4 million neonatal deaths (first 28 days of life) occur, accounting for an estimated 41% of all under-5 deaths.1,2 There is increasing policy and programme attention to reducing neonatal deaths, which have previously been relatively neglected in both maternal and child health programmes. Given limited funding and the short time to the Millennium Development Goal deadline of 2015, the imperative to invest in the most effective interventions is clear. However, a ‘one size fits all’ approach is unlikely to work since the effect of interventions will depend on the local cause-of-death profile and the health system platforms available for scale up.3 Decision-makers require information on the likely impacts of different interventions in their own setting, together with information on the incremental cost of adding interventions to an existing package either at facility or community level.
The Lives Saved Tool (LiST) has been designed for the purpose of facilitating programmatic decisions based on mortality effects. LiST allows users to model the estimated impact of scaling up a specific maternal, newborn and child health intervention by increasing the coverage for a defined population which may be one country, state or district. An important feature of LiST is that it models explicitly the effects of changes in individual-level interventions such as exclusive breastfeeding of a newborn or ORS treatment of a child with diarrhoea. The public health strategies by which coverage of these interventions is increased (e.g. peer counselling, community mobilization, etc.) are not modelled explicitly in LiST. It is up to the user to decide what levels of coverage can be achieved with the strategies and resources at their disposal. LiST is built into the widely accepted demographic software package ‘Spectrum’, designed to model population changes over time by age and sex4,5 and includes recent mortality rates by country with neonatal and under-5 cause-of-death estimates. The neonatal cause-of-death estimates are based on methods published previously,6 but have been updated for the year 2004 and cleared by WHO with countries.7,8 The neonatal mortality module in LiST is based upon evidence-based reviews for intervention impact originally undertaken for The Lancet Neonatal series.9 A standard methodology has been developed by the Child Health Epidemiology Reference Group (CHERG)10 for systematic review, rating of the evidence and estimation of mortality effect using an adapted GRADE methodology. Reviews of individual interventions are presented elsewhere in this supplement.11–15
To assess the validity of LiST for modelling neonatal mortality by comparing the reductions in neonatal mortality predicted by LiST with observed changes in mortality in field trials of packages of interventions.
The development of LiST and the modelling assumptions that it makes are described in detail in several articles in this supplement. Briefly, LiST is a cohort model of neonatal and child survival, up to 5 years of age, embedded within the Spectrum Policy Modelling System (http://www.futuresinstitute.org). It provides estimates of the effects on cause-specific mortality (e.g. on deaths and mortality rates) of changing the coverage of different interventions. Several interventions in the model affect specific causes of neonatal death and thus neonatal mortality rates. The current effect sizes that LiST assumes for each intervention and each cause of neonatal death (except diarrhoea and congenital anomalies) are shown in Table 1.
Validation data were selected based on the following inclusion criteria:
Four studies met the inclusion criteria (Table 2).16–22 One was an effectiveness trial (Hala)16 whereas another simulated near program conditions (Sylhet).17,22 In the remaining two studies, the research team played an important role in assuring the delivery of the intervention package. In the intervention areas, packages of interventions designed to affect neonatal mortality were delivered. Only the primary intervention area was modelled for each study.
Neonatal researchers involved in these studies or other studies of neonatal interventions (ZB, GD and JL) met to map the coverage indicators of the interventions which were implemented and/or measured in each study onto the coverage indicators included in LiST. For each study, there was extensive discussion about which indicators were the most appropriate (Table 3). For components of facility delivery, such as comprehensive emergency obstetric care, basic emergency obstetric care, neonatal resuscitation, antenatal corticosteroids and antibiotics for preterm prolonged rupture of membranes, coverage data were not available. Thus, based on previous reviews by the Lancet Neonatal Series team, a standard proportion of women who deliver at a facility were assumed to receive each of the components.9 See the Supplementary appendix for the relevant fractions. The final mapping of the study indicators onto LiST indicators was performed prior to running the model and seeing the results. Study indicators which could not be mapped within LiST are shown in Table 4. For each study, a LiST model of the study population at baseline was created, including total population, fertility trends, health status (including nutritional status), neonatal mortality rates and cause-specific proportions of neonatal deaths (cause-of-death profile).
Each study was then modelled in LiST to determine the predicted change in neonatal mortality given the changes in intervention coverage observed in the study. The changes in mortality predicted by the model were then compared with the changes actually observed in each study. The specific modelling details for each study, including non-standard-coding methods are described below. All modelled interventions are described in Table 5.
The home-care arm of the study was modelled. The neonatal mortality rate measured in the trial population at baseline was used together with the study-specific cause-of-death profile. The Bangladesh national fertility trends in Spectrum were used without additional adjustment. For antenatal care (ANC), the coverage indicator used was one or more visits during pregnancy, although the ideal LiST indicator is four or more visits. Within LiST, ANC is only linked to the intervention of syphilis detection and treatment. As this was not measured in the trial, it was calculated based upon standard formulas built into LiST under the assumption that a given proportion women would have received this when visiting providers for routine ANC.9 For tetanus toxoid (TT) immunization, the coverage indicator used was two doses of TT during pregnancy. For multiple micronutrient supplementation, maternal report of taking an iron-containing compound during pregnancy was the coverage indicator used. The available indicator of facility birth was used and linked to selected facility-based interventions (antenatal corticosteroids for women with anticipated preterm labour, antibiotics for preterm prolonged rupture of membranes (pPRoM), basic emergency obstetric care, comprehensive emergency obstetric care and neonatal resuscitation) using standard formulas built into LiST. Clean delivery at home was defined as use of a clean delivery kit or a boiled cutting instrument at delivery and based on trial data. In the home-care arm of the study, case management of serious neonatal illness included successful referral to a facility, or if referral was not successful, provision of injectable antibiotics in the community. Of the children who were identified to have a serious neonatal illness, 31.9% were successfully referred (and modelled as full supportive care for serious neonatal illness) and an additional 41.5% received injectable antibiotics in the community (and modelled as ‘injectable antibiotics for case management of serious neonatal illness’). Data were unavailable for the type of treatment received at health facilities by the referred patients.
The fertility trends were based on the Maharashtra population data in the National Family Health Survey23 and matched the crude birth rates observed in the intervention area. The cause-of-death data used were modified from Bang et al.19 by the investigator to fit the CHERG Neonatal group standard hierarchy, with some early infection deaths in very preterm infants being reclassified to the category of preterm as a direct cause of death. Village health workers visited homes during pregnancy to promote breastfeeding, and also visited the home within 24 h after delivery. The coverage of these home visits was used as the indicator for both preventive postnatal care and breastfeeding promotion. Coverage of neonatal resuscitation in the home was based on the percentage of births with a village health worker present at birth, since all study health workers were trained in resuscitation. Most sepsis cases were treated in the community using a combination of injectable and oral antibiotics. This was modelled as ‘injectable antibiotics for case management of serious neonatal illness’.
Pakistani national data regarding cause-specific neonatal mortality were used,24 as were standard national fertility trends from Spectrum. Coverage of one or more antenatal visits was used for ANC and linked interventions. Data for coverage of two doses of TT immunization were collected in the study surveys, as was skilled birth attendance. For the indicator of clean home birth, the coverage with lady health workers at home for birth was used. Early initiation of breastfeeding was the indicator used for breastfeeding promotion.
The fertility trends used were based on the rural Uttar Pradesh population data in the National Family Health Survey.25 In this trial, mortality was measured both through household surveys and prospectively by a demographic surveillance system. As surveys may underestimate mortality, we used only surveillance data and used the observed mortality rate at endline in the comparison arm as the proxy for the baseline mortality rate in the intervention arm. The third arm of the study, (intervention package plus Thermospot), was excluded from the analysis as we chose only one arm to evaluate for each study and it would be difficult to model the additional impact of the ThermoSpot within LiST. However, intervention coverage figures and mortality impact in the two arms were not significantly different. Since neonatal cause-of-death data were not yet available from the study or from the state of Uttar Pradesh, the baseline cause of death profile used was the same as the SEARCH study, which is from a similar context in rural India. The indicator used in LiST for ANC and linked interventions was one or more ANC visits from the trial data as the ideal indicator of four or more routine ANC visits was not reported. Data were available for the coverage of two doses of TT vaccine. ‘Delivery into hands’ instead of on the ground was the indicator used for a clean home delivery. Although several thermal care indicators were reported, the most important one was considered to be skin-to-skin care within 24 h of birth. This was used as the indicator for appropriate preventive postnatal care. Similar to the Hala study, early initiation of breastfeeding was the indicator used for breastfeeding promotion.
More than 40 interventions were included in the version of LiST used at the time of this analysis (version 4.0, which is not publically available at the time of writing). Of these, 19 have an effect on neonatal mortality (Table 1). For each study included in the validation exercise, all the coverage indicators that were available were mapped to interventions in the LiST model (Table 5). For each of the studies, several indicators or interventions were not able to be mapped to the model (Table 4).
Model outputs and observed mortality reductions are presented in Table 6. For three of the four studies, the modelled effect of the interventions closely matched the observed study results, with the predicted impact lying within the confidence bounds for the observed results. The one outlier was the Shivgarh trial. In this study, control area mortality (used as a proxy for baseline mortality in the modelling) was extremely high (84 per 1000 live births) and a very large reduction (58%) to 41 per 1000 live births was observed in the intervention area. In this study, LiST overestimated the observed NMR at the end of the study by 39%. Across the Hala, Sylhet, SEARCH and Shivgarh studies, the estimates derived using LiST differed from the observed rates of neonatal mortality by 0, 2, 5 or 16 per 1000 live births, respectively.
This is the first attempt to validate the modelling of neonatal lives saved which has been developed over the past 5 years and is now incorporated within LiST. NMRs predicted by LiST were remarkably close to those observed in three out of the four studies used for validation. For the two studies implemented under routine (or close to routine) programme conditions, the model predictions were almost identical to the observed results (Hala, Pakistan16 and Sylhet, Bangladesh17). For the two studies which were closer to efficacy-type studies, the mortality reductions observed were greater than those predicted by LiST: i.e. LiST appears to produce conservative estimates of the mortality effect. This is a reassuring finding for a tool that aims to model programme effectiveness. These results suggest that LiST is a reasonably reliable tool for use by policymakers to prioritize interventions for maximal effect on neonatal deaths, at least in South Asia.
There are a number of possible explanations for the underestimation of the neonatal mortality effect by LiST in these trials. The first is that these were efficacy-type trials and set in populations with especially high mortality, suggesting that the model is conservative, as per the stated objective of including effectiveness estimates in the model. In addition, this model only includes interventions which have an evidence-based, cause-specific effect on mortality and are included in LiST. Thus, interventions which are likely to have an effect, but for which insufficient high-quality data have been collected to adequately generate a valid effect size were excluded from the modelling exercise, such as thermal care improvements or early initiation of breastfeeding. Another possible explanation for the conservative nature of the model is that some interventions may have a synergistic effect when delivered together that is difficult to quantify. Finally, in all environments, there are always changes which cannot be quantified, such as quality changes when intensive implementation has occurred, or on which data have not been collected. However, this should not be taken to mean that these changes have no effect. This also applies to non-health or distal factors that we are unable to model in LiST such as changes in education, women’s empowerment, food availability, and other factors which are known to affect overall mortality rates but do not have a clear cause-specific effect that can be modelled over and above their impact on coverage changes.
Only in one study did the observed mortality reduction differ by more than five neonatal deaths per 1000 live births. In this trial, in Shivgarh,21 a more comprehensive set of behaviour change interventions were implemented, including monthly community and folk song meetings with behaviour change messages, birth preparedness and education of a variety of newborn care stakeholders. More of the study activities and indicators could not be mapped in LiST than in the other trials. Perhaps more importantly, this study did not have measured neonatal mortality at baseline in the modelled arm of the trial; rather, the concurrent control area neonatal mortality rate was used as a proxy for the baseline rate. In addition, this area’s neonatal mortality rate was extremely high. Also, the use of a cause-of-death profile from another site introduced additional uncertainty. Thus, it was expected that the model would underestimate the neonatal mortality reduction.
Although the results of this validation exercise are encouraging, it should be noted that all four studies were performed in South Asia. A similar validation to compare LiST estimates of mortality reduction with study results from Africa, East Asia or South or Central America would be helpful, but unfortunately no published studies from these regions met our inclusion criteria. However, there are neonatal outcome studies now in progress in these regions so such comparative analysis may soon be possible. It would also be of interest to perform additional validations with nationally representative databases such as in the Demographic and Health (DHS) and Multiple Indicator Cluster (MIC) surveys, for example, similar to that presented by Hazel and Bhutta.27 in this journal issue, using the default CHERG cause-of-death profiles.
In summary, LiST appears to model well the effect of interventions on neonatal mortality. One limitation of LiST is in the number and variety of interventions included in the model for which high quality empirical coverage data are also available. Given the relatively recent attention to reducing neonatal mortality, the comparability and availability of coverage indicators is an important area to advance within large scale surveys, health facility assessments and in programmes and research. Although further validation of the model is desirable, the results obtained here suggest that LiST can be used as a reasonable guide for decision making when adequate data are unavailable or to extrapolate to future outcomes for lives saved with given changes in coverage. Important gaps remain in the evidence for neonatal mortality reduction, with ironically more rigorous evidence now available from community level than from facility level. In addition, important changes in community interactions and behaviour change remain challenging to model, but may be especially important in high mortality settings, such as Shivgarh, where almost 10% of babies die, many of whom are term babies with preventable and treatable problems. In addition, important changes in community interactions and behaviour change remain challenging to model, but may be especially important in high mortality settings, such as the study from Shivgarh, where currently almost 10% of babies die, many of whom are term babies dying from preventable and treatable conditions.
Supplementary data are available at IJE online.
This analysis was funded by the US Fund for UNICEF through a grant from the Bill & Melinda Gates Foundation to ‘Promote evidence-based decision making in designing maternal, neonatal and child health interventions in low- and middle-income countries’ (No. 43386); and Save The Children-US through a grant from the Bill & Melinda Gates Foundation for ‘Saving Newborn Lives’ (No. 50124).
The individual studies were supported by the United States Agency for International Development (USAID), through cooperative agreements with the Johns Hopkins Bloomberg School of Public Health and the International Centre for Diarrhoeal Disease Research, Bangladesh (ICDDR,B), and by the Saving Newborn Lives program of Save the Children-US through a grant from the Bill & Melinda Gates Foundation (Sylhet); The John D. and Catherine T. MacArthur Foundation, The Ford Foundation, Saving Newborn Lives, Save the Children-US, The Bill & Melinda Gates Foundation, the International Women's Health Coalition, Oxfam and the Indian Council of Medical Research (SEARCH); WHO and the Saving Newborn Lives program of Save the Children-US, funded by the Bill & Melinda Gates Foundation (Hala); and the USAID, Delhi Mission and the Saving Newborn Lives program of Save the Children-US through a grant from the Bill & Melinda Gates Foundation (Shivgarh).
The authors are grateful to Becky Ferguson at the Bill and Melinda Gates Foundation for facilitating the meeting to undertake this analysis.
Conflict of interest: None declared.