|Home | About | Journals | Submit | Contact Us | Français|
To evaluate the effectiveness of the American Academy of Pediatrics Neonatal Resuscitation Program (NRP) in improving knowledge, skills, and self-efficacy of nurse midwives in low-risk delivery clinics in a developing country.
We used the content specifications of the NRP material applicable to college-educated nurse midwives working in low-risk clinics in Zambia to develop performance and self-efficacy evaluations focused on principles of resuscitation, initial steps, ventilation, and chest compressions. These evaluations were administered to 127 nurse midwives before and after NRP training and 6-months later.
After training, written scores (knowledge evaluation) improved from 57% ± 14% to 80% ± 12% (mean ± SD; P < .0001); performance scores (skills evaluation) improved the most from 43% ± 21% to 88% ± 9% (P < .0001); self-efficacy scores improved from 74% ± 14% to 90% ± 10% (P < .0001). Written and performance scores decreased significantly 6 months after training, but self-efficacy scores remained high.
As conducted, the NRP training improved educational outcomes in college-educated practicing nurse midwives. Pre-training knowledge and skills scores were relatively low despite the advanced formal education and experience of the participants, whereas the self-efficacy scores were high. NRP training has the potential to substantially improve knowledge and skills of neonatal resuscitation.
Birth asphyxia is one of the most frequent causes of early deaths, accounting for approximately 23% of 4 million neonatal deaths worldwide.1 Neonatal resuscitation is a simple, inexpensive, readily available, and cost-effective intervention that can prevent many of the deaths and reduce disabilities in survivors from birth asphyxia.2,3 However, the World Health Organization has concluded that resuscitation is often not initiated or the methods used are inadequate or wrong.4 This is especially true in the developing world, which accounts for 98% of the 4 million neonatal deaths and could benefit dramatically from effective neonatal resuscitation training of birth attendants.
The Neonatal Resuscitation Program (NRP) is the official educational program of the American Academy of Pediatrics and the American Heart Association for educating and training health care providers in the knowledge and skills of neonatal resuscitation. During the decade after the introduction of NRP in 1987, deaths caused by birth asphyxia in the United States decreased by 42%,5,6 although this improvement cannot be ascribed to NRP alone. Although controlled non-randomized studies in China, India, and Africa suggest that training in resuscitation with the NRP or other programs may reduce neonatal mortality caused by birth asphyxia,2,3,7–9 studies do not show a consistent decrease in mortality and a meta-analysis is inconclusive on all-cause neonatal mortality.10 The NRP has been widely adopted for use throughout many highly developed and lesser-developed countries.
The NRP course improved healthcare providers’ knowledge, skills, and attitudes in developed countries10; however, there has been no rigorous evaluation of its educational impact and retention in developing countries. Furthermore, the evaluation of the educational impact of other resuscitation training in developing countries has been limited.11,12 Evaluation of the educational effectiveness of NRP training in low-resource environments is important because the impact may differ on the basis of differences in education, practices, and resources. The purpose of this study was to determine whether NRP training improves knowledge, skills, and self-efficacy of primary healthcare providers in low-risk delivery centers.
Qualified NRP instructors administered the NRP course to 127 college-educated (4 years) nurse midwives who were employed in 18 primary-level delivery clinics in Lusaka and Ndola, Zambia in 2005. The midwives had been in practice for 16 ± 9 years (range, 3–30 years; median, 14 years). The midwives were trained previously in neonatal resuscitation, but had not been trained with the NRP course. The midwives were encouraged to read the NRP textbook before the training. The course was conducted in English, with group meetings held at a high-delivery service referral hospital (University Hospital of Zambia) in 2 phases with the trainer-of-trainers model. The first phase, led by an experienced NRP regional trainer (W.C.), consisted of a 4-day course for 15 nurse midwives who became trainers. Subsequently, these 15 trainers trained an additional 112 nurse midwives in small groups with the same format.
The standard NRP course content and format used cognitive and psychomotor activities to allow individuals to complete the course through group meetings following the materials and guidelines published in 2000.13–15 Participants not only completed self-directed and self-assessment exercises with textbooks, but also participated in extensive hands-on practice scenarios and actual deliveries with the guidance of trained instructors. Each nurse midwife completed the written, performance, and self-efficacy evaluations before and after the NRP training, and a subgroup repeated the evaluations 6-months after completion of training. The institutional review boards of the University of Alabama at Birmingham, the University of Zambia, and the Research Triangle Institute approved the study.
The NRP course has a written evaluation to test knowledge and a performance evaluation to test skills. To determine educational effectiveness of NRP training on knowledge in Zambia, the standard NRP written evaluation was used. The evaluation (scale) was split in subscales by grouping the related individual items. To evaluate performance, 2 physicians (W.C., L.W.), a nurse/educator (M. Collins), and an educator (B.M.) identified 29 content items from the standard NRP Megacode (NRP Textbook, 2000) pertinent to Lessons 1 to 4 that were determined to be essential and appropriate for the low-resource low-risk delivery centers in urban Zambia. Thus, the performance evaluations excluded intubation and medications. These items were compiled on a checklist that included specific and objective instructions for the participant and observer. An observer recorded the midwives’ skills as they performed procedures on a mannequin following the instructions. The same group of physicians and educator/nurse developed a 14-item self-efficacy evaluation and used a 5-point Likert scale (1-not very confident to 5-very confident). All 3 evaluation tools were pre-tested with local midwives and modified as needed by the group of physicians and educator/nurse. For the purposes of scoring the evaluations, each item within the written, performance, and self-efficacy evaluations were given equal value. The same evaluations were administered before and after NRP training and again 6 months after completion of training.
Each item on the written and performance evaluations in the lessons “Overview and Principles of Resuscitation,” “Initial Steps,” “Ventilation,” and “Chest Compressions” was scored as correct or incorrect. Although the written evaluation tested the concepts of intubation and medications, these contents were not included in the performance evaluation or in the analyses of the written evaluations. Scores for the scales and their subscales were computed as the percentage of items on the scale or subscale answered correctly. Items on the self-efficacy scale were scored from 1 to 5, and scale scores were calculated as the mean of the item responses.
We computed Cronbach’s alphas16 to assess the internal consistency of the 3 evaluations (scales) at each time point. In addition, we explored the relationship between the participants’ scores on the written and performance evaluations. We divided participants in 2 groups on the basis of their scores on the performance evaluation after the training: high performance (score ≥85), or low performance (score <85). t tests were used to compare mean written evaluation scores after the training for the 2 groups. When the written and performance evaluations were measuring similar constructs, we would expect participants in the high performance group to have significantly higher written evaluation scores.
To assess the impact of the training, we conducted 2 sets of analyses. First, we examined the change from before to after the training for all participants. The items on the scales were analyzed individually or in groups of subscales on the basis of content (eg, ventilation or chest compressions) to identify specific topics in which participants either performed well or had difficulty to target future training efforts or modify the evaluations. Paired t tests were used to test for changes in mean scores on the overall scale or subscale scores, and Mc-Nemar tests were used to compare responses with the individual items between pre-training and post-training.
The second set of analyses focused on retention of knowledge through the 6 months after the training. We used generalized linear models with generalized estimating equations estimation to test for changes in the evaluation (scale) scores from the pre-training to the 6-month follow-up. Generalized estimating equation models were selected because they account for the correlation between repeated measurements with time and permit the inclusion of participants with missing data. This approach allowed us to use data from all participants, even when they were not in the 6-month follow-up evaluation. In addition, we explored whether retention of knowledge and skills differed according to participants’ reported self-efficacy by testing a time by self-efficacy interaction in the models. A P value < .05 was considered to be statistically significant.
On the basis of Cronbach’s alpha, overall the 3 evaluation scales demonstrated good internal consistency (generally defined as alpha >0.70). The Cronbach’s alphas for the scales were: written evaluation (pre-training = 0.80, post-training = 0.84, 6 months = 0.87), performance evaluation (pre-training = 0.87, post-training = 0.60, 6 months = 0.87), and self-efficacy (pre-training = 0.90, post-training = 0.84, 6 months = 0.87). The alpha for the performance evaluation decreased from pre-training to post-training, suggesting that there may be less consistency in how participants performed across the items after the training.
The percentage of items on the written evaluation answered correctly increased from 57% ± 14% pre-training to 80% ± 12% post-training (P < .001; Table I). Similar levels of improvement were observed across the 4 subscales (Table I) and for almost all items on the subscales (Appendix 1; available at www.jpeds.com). Scores on the performance evaluation scale and subscales showed the largest improvements of all 3 scales. Overall, scores increased from 43% ± 21% before training to 88% ± 9% after training (P < .001; Table I). Scores improved in all 4 subscales and for almost all items on the subscales (Appendix 2; available at www.jpeds.com). There was a wide variation in performance. For example, although there was almost uniform provision of infant warming both before and after training (79% ± 41% to 90% ± 30%), techniques for depth of chest compressions improved from as low as 20% ± 40% to 92% ± 27% for correct depth of chest compressions.
The comparisons of mean written evaluation scores by birth attendants with high and low scores on the performance evaluation at post-training are shown in the on line material (Appendix 3; available at www.jpeds.com). As might be expected, birth attendants in the high performance group had significantly higher scores (P < .05) on the written evaluation overall and all the subscales, except the Ventilation subscale.
The pre-training score average for the self-efficacy test was the highest of any scale, but also increased from 3.7 ± 0.7 to 4.5 ± 0.5 on a scale of 1 to 5, or an average of 68% ± 17% to 87% ± 12% (both P < .001) of the total possible maximum score (Table I). Participants’ scores on the self-efficacy scale increased for each of the items (Appendix 4; available at www.jpeds.com) after the training. Self-efficacy scores for “Initial Steps” and “Ventilation” were highest before training and increased the least after training. The largest increases in self-efficacy were in the most advanced aspects of resuscitation (Intubation, Medications), although they were not the focus of the training (Figure 1).
The mean evaluation scores at each time point for the birth attendants who participated in the 6-month follow-up are shown on Table II. Scores on all 3 evaluations increased significantly from pre-training to post-training. Written and performance evaluation scores declined from post-training to 6-month follow-up, but remained significantly higher than pre-training scores.
With generalized linear models, we explored possible differences in changes in skills with time for birth attendants with high self-efficacy versus birth attendants with low self-efficacy (highest 50% versus lowest 50%). As suggested by non-significant time by self-efficacy interaction term (P = .6), the 2 groups had similar patterns of change in written evaluations with time (Figure 1).
A slightly different pattern emerges for the performance evaluation scores for which the interaction term is significant (P = .04; Figure 2). The high and low self-efficacy groups have similar levels of improvement in scores from pre- to post-training. However, the low self-efficacy group retained their skills (maintained the performance scores from 88% ± 10% to 86% ± 15%) from post-training to 6-month follow-up, whereas the high self-efficacy group experienced a decline in performance scores (from 88% ± 7% to 74% ± 21%) during the same period.
We report a comprehensive evaluation of the educational impact of NRP training in the developing world to address several domains of educational outcomes. All 3 evaluations had acceptable internal consistency and thus were valid evaluations of knowledge, skills, and self- efficacy of resuscitation. The improvements in scores between the pre-training period and post-training period demonstrated that the NRP training significantly increased scores on evaluations of knowledge, skills, and self-efficacy in midwives despite their earlier advanced formal education and many years of clinical experience. NRP training was more effective in increasing the scores on the performance evaluation than on the written evaluation, although scores of both evaluations were low pre-NRP training. In contrast to the low knowledge and skills scores pre-NRP training, self-efficacy scores were relatively high pre-NRP training; thus, the nurse midwives rated themselves more knowledgeable and skilled than reflected by their evaluations pre-NRP training.
Performance training was most effective; almost all (99%) the participants increased their performance scores by 20% or more. In contrast, only approximately 50% of the participants increased their knowledge or self-efficacy scores by at least 20%. The marked increase in performance scores may reflect the effectiveness of NRP training’s emphasis on psychomotor skills. The high self-efficacy scores pre-training despite the low knowledge and skills scores are concerning because clinicians may not believe they need the training. Furthermore, self-efficacy scores were not predictive of written or performance evaluation scores or their improvement with training. It is also concerning that birth attendants with high self-efficacy had a decline in performance scores 6 months post-training. However, in birth attendants with low self-efficacy, the performance scores remained high 6 months post-NRP training.
Although this study was well designed for a broad assessment of the effect of NRP training on a body of knowledge, skills, and self-efficacy attitudes, it has several important limitations. We used the standard NRP written evaluation materials although the midwives had limited opportunities for advanced resuscitation because they worked in low-risk delivery centers. The smaller increase in written versus performance scores may reflect a difficulty with written test skills although the midwives were college-educated. In evaluating performance, the unmasked observer who recorded the midwives’ skills was aware of whether the test was pre-training or post-training. However, because the performance evaluation had a checklist of skills and specific coding instructions, the potential for bias was limited. Although there was acceptable internal consistency of the evaluations, another limitation is that the performance and self-efficacy evaluations have not been thoroughly validated before implementation and, therefore, may need further testing to document their validity in assessing skills and self-efficacy. A potential confounder to the study was the wide range of the midwives’ years of practice; because of sample size limitations, subgroup analysis was not planned. However, the midwives shared the same foundation of a college education and some resuscitation training. Such variety in years of practice is difficult to avoid with any group of individuals and may suggest good generalizability because NRP training had such a consistent effect on all 3 areas tested.
Most studies of the effectiveness of NRP have been performed in developed countries.17–22 Typically, these studies were conducted in hospitals in which NRP was taught to residents and other healthcare providers with advanced levels of education. In this setting, NRP was effective in improving educational outcomes. A study to evaluate knowledge gained by pediatric residents during NRP showed comparable improvement by pediatric residents in both a developed and in a developing country.23 Two other studies have documented the educational effectiveness of resuscitation (but not NRP) training to primary healthcare providers in very low-resource countries.11,12 This study used carefully designed evaluation tools to determine whether NRP training would have the same beneficial impact in low-resource environments. Judging from the large effects on educational outcomes, it can be inferred that NRP training can effectively increase resuscitation knowledge, skills, and self-efficacy of midwives in developing countries.
Although NRP training may improve test scores acutely, several studies show that knowledge is lost as early as 3 to 5 months after training.11,17,19,24 Some studies advocate for boosters to improve knowledge and performance.19 Maintaining midwives’ knowledge and skills through retraining or reinforcement could be pivotal to maintaining the high levels of resuscitation skills in midwives required to improve infant mortality. Ultimately, the effect of NRP on patient’ outcomes depends on retention of the knowledge and skills learned and their correct application. Finally, it is necessary to evaluate the effect of resuscitation training on the quality of care, neonatal outcomes, and cost effectiveness.11,25,26 To date, there have been only limited studies to determine whether resuscitation training reduces neonatal mortality and other important adverse outcomes, and those results are inconclusive.10 Whether NRP training is sufficient to significantly reduce neonatal mortality in developing countries needs further evaluation.
In summary, this study shows low written and performance scores pre-NRP training despite earlier advanced formal education and long-term clinical practice, but relatively high self-efficacy scores in midwives in a developing country. NRP training increased scores in all 3 scales evaluated, but retention of knowledge and performance skills decreased significantly by 6 months post-training. Further research is needed to determine optimal timing for retraining in NRP in different clinical settings. NRP training has the potential to substantially improve knowledge and skills of neonatal resuscitation.
Supported by the NICHD Global Network for Women’s and Children’s Health Research (HD43475, HD404636) and the Bill and Melinda Gates Foundation.
The authors declare no conflicts of interest.