Related Articles

Background

It is increasingly appreciated that the interpretation of health systems research studies is greatly facilitated by detailed descriptions of study context and the process of intervention. We have undertaken an 18-month hospital-based intervention study in Kenya aiming to improve care for admitted children and newborn infants. Here we describe the baseline characteristics of the eight hospitals as environments receiving the intervention, as well as the general and local health system context and its evolution over the 18 months.

Methods

Hospital characteristics were assessed using previously developed tools assessing the broad structure, process, and outcome of health service provision for children and newborns. Major health system or policy developments over the period of the intervention at a national level were documented prospectively by monitoring government policy announcements, the media, and through informal contacts with policy makers. At the hospital level, a structured, open questionnaire was used in face-to-face meetings with senior hospital staff every six months to identify major local developments that might influence implementation. These data provide an essential background for those seeking to understand the generalisability of reports describing the intervention's effects, and whether the intervention plausibly resulted in these effects.

Results

Hospitals had only modest capacity, in terms of infrastructure, equipment, supplies, and human resources available to provide high-quality care at baseline. For example, hospitals were lacking between 30 to 56% of items considered necessary for the provision of care to the seriously ill child or newborn. An increase in spending on hospital renovations, attempts to introduce performance contracts for health workers, and post-election violence were recorded as examples of national level factors that might influence implementation success generally. Examples of factors that might influence success locally included frequent and sometimes numerous staff changes, movements of senior departmental or administrative staff, and the presence of local 'donor' partners with alternative priorities.

Conclusion

The effectiveness of interventions delivered at hospital level over periods realistically required to achieve change may be influenced by a wide variety of factors at national and local levels. We have demonstrated how dynamic such contexts are, and therefore the need to consider context when interpreting an intervention's effectiveness.

Philip Ayieko and colleagues report the outcomes of a cluster-randomized trial carried out in eight Kenyan district hospitals evaluating the effects of a complex intervention involving improved training and supervision for clinicians. They found a higher performance of hospitals assigned to the complex intervention on a variety of process of care measures, as compared to those receiving the control intervention.

Background

In developing countries referral of severely ill children from primary care to district hospitals is common, but hospital care is often of poor quality. However, strategies to change multiple paediatric care practices in rural hospitals have rarely been evaluated.

Methods and Findings

This cluster randomized trial was conducted in eight rural Kenyan district hospitals, four of which were randomly assigned to a full intervention aimed at improving quality of clinical care (evidence-based guidelines, training, job aides, local facilitation, supervision, and face-to-face feedback; n = 4) and the remaining four to control intervention (guidelines, didactic training, job aides, and written feedback; n = 4). Prespecified structure, process, and outcome indicators were measured at baseline and during three and five 6-monthly surveys in control and intervention hospitals, respectively. Primary outcomes were process of care measures, assessed at 18 months postbaseline.

In both groups performance improved from baseline. Completion of admission assessment tasks was higher in intervention sites at 18 months (mean = 0.94 versus 0.65, adjusted difference 0.54 [95% confidence interval 0.05–0.29]). Uptake of guideline recommended therapeutic practices was also higher within intervention hospitals: adoption of once daily gentamicin (89.2% versus 74.4%; 17.1% [8.04%–26.1%]); loading dose quinine (91.9% versus 66.7%, 26.3% [−3.66% to 56.3%]); and adequate prescriptions of intravenous fluids for severe dehydration (67.2% versus 40.6%; 29.9% [10.9%–48.9%]). The proportion of children receiving inappropriate doses of drugs in intervention hospitals was lower (quinine dose >40 mg/kg/day; 1.0% versus 7.5%; −6.5% [−12.9% to 0.20%]), and inadequate gentamicin dose (2.2% versus 9.0%; −6.8% [−11.9% to −1.6%]).

Conclusions

Specific efforts are needed to improve hospital care in developing countries. A full, multifaceted intervention was associated with greater changes in practice spanning multiple, high mortality conditions in rural Kenyan hospitals than a partial intervention, providing one model for bridging the evidence to practice gap and improving admission care in similar settings.

Trial registration

Current Controlled Trials ISRCTN42996612

Please see later in the article for the Editors' Summary

Editors' Summary

Background

In 2008, nearly 10 million children died in early childhood. Nearly all these deaths were in low- and middle-income countries—half were in Africa. In Kenya, for example, 74 out every 1,000 children born died before they reached their fifth birthday. About half of all childhood (pediatric) deaths in developing countries are caused by pneumonia, diarrhea, and malaria. Deaths from these common diseases could be prevented if all sick children had access to quality health care in the community (“primary” health care provided by health centers, pharmacists, family doctors, and traditional healers) and in district hospitals (“secondary” health care). Unfortunately, primary health care facilities in developing countries often lack essential diagnostic capabilities and drugs, and pediatric hospital care is frequently inadequate with many deaths occurring soon after admission. Consequently, in 1996, as part of global efforts to reduce childhood illnesses and deaths, the World Health Organization (WHO) and the United Nations Children's Fund (UNICEF) introduced the Integrated Management of Childhood Illnesses (IMCI) strategy. This approach to child health focuses on the well-being of the whole child and aims to improve the case management skills of health care staff at all levels, health systems, and family and community health practices.

Why Was This Study Done?

The implementation of IMCI has been evaluated at the primary health care level, but its implementation in district hospitals has not been evaluated. So, for example, interventions designed to encourage the routine use of WHO disease-specific guidelines in rural pediatric hospitals have not been tested. In this cluster randomized trial, the researchers develop and test a multifaceted intervention designed to improve the implementation of treatment guidelines and admission pediatric care in district hospitals in Kenya. In a cluster randomized trial, groups of patients rather than individual patients are randomly assigned to receive alternative interventions and the outcomes in different “clusters” of patients are compared. In this trial, each cluster is a district hospital.

What Did the Researchers Do and Find?

The researchers randomly assigned eight Kenyan district hospitals to the “full” or “control” intervention, interventions that differed in intensity but that both included more strategies to promote implementation of best practice than are usually applied in Kenyan rural hospitals. The full intervention included provision of clinical practice guidelines and training in their use, six-monthly survey-based hospital assessments followed by face-to-face feedback of survey findings, 5.5 days training for health care workers, provision of job aids such as structured pediatric admission records, external supervision, and the identification of a local facilitator to promote guideline use and to provide on-site problem solving. The control intervention included the provision of clinical practice guidelines (without training in their use) and job aids, six-monthly surveys with written feedback, and a 1.5-day lecture-based seminar to explain the guidelines. The researchers compared the implementation of various processes of care (activities of patients and doctors undertaken to ensure delivery of care) in the intervention and control hospitals at baseline and 18 months later. The performance of both groups of hospitals improved during the trial but more markedly in the intervention hospitals than in the control hospitals. At 18 months, the completion of admission assessment tasks and the uptake of guideline-recommended clinical practices were both higher in the intervention hospitals than in the control hospitals. Moreover, a lower proportion of children received inappropriate doses of drugs such as quinine for malaria in the intervention hospitals than in the control hospitals.

What Do These Findings Mean?

These findings show that specific efforts are needed to improve pediatric care in rural Kenya and suggest that interventions that include more approaches to changing clinical practice may be more effective than interventions that include fewer approaches. These findings are limited by certain aspects of the trial design, such as the small number of participating hospitals, and may not be generalizable to other hospitals in Kenya or to hospitals in other developing countries. Thus, although these findings seem to suggest that efforts to implement and scale up improved secondary pediatric health care will need to include more than the production and dissemination of printed materials, further research including trials or evaluation of test programs are necessary before widespread adoption of any multifaceted approach (which will need to be tailored to local conditions and available resources) can be contemplated.

Additional Information

Please access these Web sites via the online version of this summary at http://dx.doi.org/10.1371/journal.pmed.1001018.

WHO provides information on efforts to reduce global child mortality and on Integrated Management of Childhood Illness (IMCI); the WHO pocket book “Hospital care for children contains guidelines for the management of common illnesses with limited resources (available in several languages)

UNICEF also provides information on efforts to reduce child mortality and detailed statistics on child mortality

The iDOC Africa Web site, which is dedicated to improving the delivery of hospital care for children and newborns in Africa, provides links to the clinical guidelines and other resources used in this study

Background

We wanted to try to account for worker motivation as a key factor that might affect the success of an intervention to improve implementation of health worker practices in eight district hospitals in Kenya. In the absence of available tools, we therefore aimed to develop a tool that could enable a rapid measurement of motivation at baseline and at subsequent points during the 18-month intervention study.

Methods

After a literature review, a self-administered questionnaire was developed to assess the outcomes and determinants of motivation of Kenyan government hospital staff. The initial questionnaire included 23 questions (from seven underlying constructs) related to motivational outcomes that were then used to construct a simpler tool to measure motivation. Parallel qualitative work was undertaken to assess the relevance of the questions chosen and the face validity of the tool.

Results

Six hundred eighty-four health workers completed the questionnaires at baseline. Reliability analysis and factor analysis were used to produce the simplified motivational index, which consisted of 10 equally-weighted items from three underlying factors. Scores on the 10-item index were closely correlated with scores for the 23-item index, indicating that in future rapid assessments might be based on the 10 questions alone. The 10-item motivation index was also able to identify statistically significant differences in mean health worker motivation scores between the study hospitals (p < 0.001). The parallel qualitative work in general supported these conclusions and contributed to our understanding of the three identified components of motivation.

Conclusion

The 10-item score developed may be useful to monitor changes in motivation over time within our study or be used for more extensive rapid assessments of health worker motivation in Kenya.

Background

Delivering an intervention to a group of patients to improve health outcomes is increasingly popular in public health and primary care, yet "group" is an umbrella term which encompasses a complex range of aims, theories, implementation processes and evaluation methods. We propose a framework for the design and process evaluation of health improvement interventions occurring in a group setting, which will assist practitioners, researchers and policy makers.

Methods

We reviewed the wider literature on health improvement interventions delivered to patient groups and identified a gap in the literature for designing, evaluating and reporting these interventions. We drew on our experiences conducting systematic reviews, intervention, mixed method and ethnographic studies of groups for breastfeeding and weight management. A framework for health improvement group design and delivery evolved through an iterative process of primary research, reference to the literature and research team discussion.

Results

Although there is an extensive literature on group processes in education, work, politics and psychological therapies, far less is known about groups where the aim is health improvement. Theories of behaviour change which are validated for individual use are often assumed to be generalisable to group settings, without being rigorously tested. Health improvement or behaviour change interventions delivered in a group setting are complex adaptive social processes with interactions between the group leader, participants, and the wider community and environment. Ecological models of health improvement, which embrace the complex relationship between behaviour, systems and the environment may be more relevant than an individual approach to behaviour change.

Conclusion

The evidence for effectiveness and cost-effectiveness of group compared with one-to-one interventions for many areas of health improvement in public health and primary care is weak or unknown. Our proposed framework is the first step towards advocating a more systematic approach to designing, evaluating and reporting interventions in group settings, which is necessary to improve this currently weak evidence base. This framework will enable policy makers and practitioners to be better informed about what works, how it works and in which contexts when aiming to improve health in a group setting.

This paper presents the results of a qualitative study to investigate the perceptions and experiences of health workers involved in a a cluster-randomized controlled trial of a novel intervention to improve health worker malaria case-management in 107 government health facilities in Kenya. The intervention involved sending text-messages about paediatric outpatient malaria case-management accompanied by “motivating” quotes to health workers’ mobile phones. Ten malaria messages were developed reflecting recommendations from the Kenyan national guidelines. Two messages were delivered per day for 5 working days and the process was repeated for 26 weeks (May to October 2009). The accompanying quotes were unique to each message. The intervention was delivered to 119 health workers and there were significant improvements in correct artemether-lumefantrine (AL) management both immediately after the intervention (November 2009) and 6 months later (May 2010). In-depth interviews with 24 health workers were undertaken to investigate the possible drivers of this change. The results suggest high acceptance of all components of the intervention, with the active delivery of information in an on the job setting, the ready availability of new and stored text messages and the perception of being kept ‘up to date’ as important factors influencing practice. Applying the construct of stages of change we infer that in this intervention the SMS messages were operating primarily at the action and maintenance stages of behaviour change achieving their effect by creating an enabling environment and providing a prompt to action for the implementation of case management practices that had already been accepted as the clinical norm by the health workers. Future trials testing the effectiveness of SMS reminders in creating an enabling environment for the establishment of new norms in clinical practice as well as in providing a prompt to action for the implementation of the new case-management guidelines are justified.

Background

It is well-known that health care workers in today’s general hospitals have to deal with high levels of job demands, which could have negative effects on their health, well-being, and job performance. A way to reduce job-related stress reactions and to optimize positive work-related outcomes is to raise the level of specific job resources and opportunities to recover from work. However, the question remains how to translate the optimization of the balance between job demands, job resources, and recovery opportunities into effective workplace interventions. The aim of the DISCovery project is to develop and implement tailored work-oriented interventions to improve health, well-being, and performance of health care personnel.

Methods/Design

A quasi-experimental field study with a non-equivalent control group pretest-posttest design will be conducted in a top general hospital. Four existing organizational departments will provide both an intervention and a comparison group. Two types of research methods are used: (1) a longitudinal web-based survey study, and (2) a longitudinal daily diary study. After base-line measures of both methods, existing and yet to be developed interventions will be implemented within the experimental groups. Follow-up measurements will be taken one and two years after the base-line measures to analyze short-term and long-term effects of the interventions. Additionally, a process evaluation and a cost-effectiveness analysis will be carried out.

Discussion

The DISCovery project fulfills a strong need for theory-driven and scientifically well-performed research on job stress and performance interventions. It will provide insight into (1) how a balance between job demands, job resources, and recovery from work can be optimized, (2) the short-term and long-term effects of tailored work-oriented effects, and (3) indicators for successful or unsuccessful implementation of interventions.

Background

A cluster randomised controlled trial (RCT) of a national Kenyan mental health primary care training programme demonstrated a significant impact for health workers on the health, disability and quality of life of their clients, despite a severe shortage of medicines in the clinics. In order to better understand the potential reasons for the improved outcomes in the intervention group, the experiences of the participating health workers were explored through qualitative focus group discussions, as focus group methodology has been found to be a useful method of obtaining a detailed understanding of client and health worker perspectives within health systems.

Methods

Two ninety minute focus groups were conducted in Nyanza province, a poor agricultural region of Kenya, with 10 health workers from the intervention group clinics where staff had received the training programme, and 10 health workers from the control group where staff had not received the training during the earlier randomised controlled trial.

Results

These focus group discussions suggest that the health workers in the intervention group perceived an increase in their communication, diagnostic and counselling skills, and that the clients in the intervention group noticed and appreciated these enhanced skills, while health workers and clients in the control group were both aware of the lack of these skills.

Conclusion

Enhanced health worker skills conferred by the mental health training programme may be responsible for the significant improvement in outcome of patients in the intervention clinics found in the randomised controlled trial, despite the general shortage of medicines and other health system weaknesses. These findings suggest that strengthening mental health training for primary care staff is worthwhile even where health systems are not strong and where the medicine supply cannot be guaranteed.

Trial registration

ISRCTN 53515024

Background

The prevalence of both overweight and musculoskeletal disorders (MSD) in the construction industry is high. Many interventions in the occupational setting aim at the prevention and reduction of these health problems, but it is still unclear how these programmes should be designed. To determine the effectiveness of interventions on these health outcomes randomised controlled trials (RCTs) are needed. The aim of this study is to systematically develop a tailored intervention for prevention and reduction of overweight and MSD among construction workers and to describe the evaluation study regarding its (cost-)effectiveness.

Methods/Design

The Intervention Mapping (IM) protocol was applied to develop and implement a tailored programme aimed at the prevention and reduction of overweight and MSD. The (cost-) effectiveness of the intervention programme will be evaluated using an RCT. Furthermore, a process evaluation will be conducted. The research population will consist of blue collar workers of a large construction company in the Netherlands.

Intervention

The intervention programme will be aimed at improving (vigorous) physical activity levels and healthy dietary behaviour and will consist of tailored information, face-to-face and telephone counselling, training instruction (a fitness "card" to be used for exercises), and materials designed for the intervention (overview of the company health promoting facilities, waist circumference measuring tape, pedometer, BMI card, calorie guide, recipes, and knowledge test).

Main study parameters/endpoints

The intervention effect on body weight and waist circumference (primary outcome measures), as well as on lifestyle behaviour, MSD, fitness, CVD risk indicators, and work-related outcomes (i.e. productivity, sick leave) (secondary outcome measures) will be assessed.

Discussion

The development of the VIP in construction intervention led to a health programme tailored to the needs of construction workers. This programme, if proven effective, can be directly implemented.

Trial registration

Netherlands Trial Register (NTR): NTR2095

Background

We have reported the results of a cluster randomized trial of rural Kenyan hospitals evaluating the effects of an intervention to introduce care based on best-practice guidelines. In parallel work we described the context of the study, explored the process and perceptions of the intervention, and undertook a discrete study on health worker motivation because this was felt likely to be an important contributor to poor performance in Kenyan public sector hospitals. Here, we use data from these multiple studies and insights gained from being participants in and observers of the intervention process to provide our explanation of how intervention effects were achieved as part of an effort to better understand implementation in low-income hospital settings.

Methods

Initial hypotheses were generated to explain the variation in intervention effects across place, time, and effect measure (indicator) based on our understanding of theory and informed by our implementation experience and participant observations. All data sources available for hospitals considered as cases for study were then examined to determine if hypotheses were supported, rejected, or required modification. Data included transcriptions of interviews and group discussions, field notes and that from the detailed longitudinal quantitative investigation. Potentially useful explanatory themes were identified, discussed by the implementing and research team, revised, and merged as part of an iterative process aimed at building more generic explanatory theory. At the end of this process, findings were mapped against a recently reported comprehensive framework for implementation research.

Results

A normative re-educative intervention approach evolved that sought to reset norms and values concerning good practice and promote 'grass-roots' participation to improve delivery of correct care. Maximal effects were achieved when this strategy and external support supervision helped create a soft-contract with senior managers clarifying roles and expectations around desired performance. This, combined with the support of facilitators acting as an expert resource and 'shop-floor' change agent, led to improvements in leadership, accountability, and resource allocation that enhanced workers' commitment and capacity and improved clinical microsystems. Provision of correct care was then particularly likely if tasks were simple and a good fit to existing professional routines. Our findings were in broad agreement with those defined as part of recent work articulating a comprehensive framework for implementation research.

Conclusions

Using data from multiple studies can provide valuable insight into how an intervention is working and what factors may explain variability in effects. Findings clearly suggest that major intervention strategies aimed at improving child and newborn survival in low-income settings should go well beyond the fixed inputs (training, guidelines, and job aides) that are typical of many major programmes. Strategies required to deliver good care in low-income settings should recognize that this will need to be co-produced through engagement often over prolonged periods and as part of a directive but adaptive, participatory, information-rich, and reflective process.

Background

Although considerable efforts are directed at developing international guidelines to improve clinical management in low-income settings they appear to influence practice rarely. This study aimed to explore barriers to guideline implementation in the early phase of an intervention study in four district hospitals in Kenya.

Methods

We developed a simple interview guide based on a simple characterisation of the intervention informed by review of major theories on barriers to uptake of guidelines. In-depth interviews, non-participatory observation, and informal discussions were then used to explore perceived barriers to guideline introduction and general improvements in paediatric and newborn care. Data were collected four to five months after in-service training in the hospitals. Data were transcribed, themes explored, and revised in two rounds of coding and analysis using NVivo 7 software, subjected to a layered analysis, reviewed, and revised after discussion with four hospital staff who acted as within-hospital facilitators.

Results

A total of 29 health workers were interviewed. Ten major themes preventing guideline uptake were identified: incomplete training coverage; inadequacies in local standard setting and leadership; lack of recognition and appreciation of good work; poor communication and teamwork; organizational constraints and limited resources; counterproductive health worker norms; absence of perceived benefits linked to adoption of new practices; difficulties accepting change; lack of motivation; and conflicting attitudes and beliefs.

Conclusion

While the barriers identified are broadly similar in theme to those reported from high-income settings, their specific nature often differs. For example, at an institutional level there is an almost complete lack of systems to introduce or reinforce guidelines, poor teamwork across different cadres of health worker, and failure to confront poor practice. At an individual level, lack of interest in the evidence supporting guidelines, feelings that they erode professionalism, and expectations that people should be paid to change practice threaten successful implementation.

Background

Improving the health of school-aged children can yield substantial benefits for cognitive development and educational achievement. However, there is limited experimental evidence on the benefits of school-based malaria prevention or how health interventions interact with other efforts to improve education quality. This study aims to evaluate the impact of school-based malaria prevention and enhanced literacy instruction on the health and educational achievement of school children in Kenya.

Design

A factorial, cluster randomised trial is being implemented in 101 government primary schools on the coast of Kenya. The interventions are (i) intermittent screening and treatment of malaria in schools by public health workers and (ii) training workshops and support for teachers to promote explicit and systematic literacy instruction. Schools are randomised to one of four groups: receiving either (i) the malaria intervention alone; (ii) the literacy intervention alone; (iii) both interventions combined; or (iv) control group where neither intervention is implemented. Children from classes 1 and 5 are randomly selected and followed up for 24 months. The primary outcomes are educational achievement and anaemia, the hypothesised mediating variables through which education is affected. Secondary outcomes include malaria parasitaemia, school attendance and school performance. A nested process evaluation, using semi-structured interviews, focus group discussion and a stakeholder analysis will investigate the community acceptability, feasibility and cost-effectiveness of the interventions.

Discussion

Across Africa, governments are committed to improve health and education of school-aged children, but seek clear policy and technical guidance as to the optimal approach to address malaria and improved literacy. This evaluation will be one of the first to simultaneously evaluate the impact of health and education interventions in the improvement of educational achievement. Reflection is made on the practical issues encountered in conducting research in schools in Africa.

Trial Registration

National Institutes of Health NCT00878007

Background

The demand for high quality hospital care for children in low resource countries is not being met. This paper describes a number of strategies to improve emergency care at a children's hospital and evaluates the impact of these on inpatient mortality. In addition, the cost-effectiveness of improving emergency care is estimated.

Methods and Findings

A team of local and international staff developed a plan to improve emergency care for children arriving at The Ola During Children's Hospital, Freetown, Sierra Leone. Following focus group discussions, five priority areas were identified to improve emergency care; staff training, hospital layout, staff allocation, medical equipment, and medical record keeping. A team of international volunteers worked with local staff for six months to design and implement improvements in these five priority areas. The improvements were evaluated collectively rather than individually. Before the intervention, the inpatient mortality rate was 12.4%. After the intervention this improved to 5.9%. The relative risk of dying was 47% (95% CI 0.369–0.607) lower after the intervention. The estimated number of lives saved in the first two months after the intervention was 103. The total cost of the intervention was USD 29 714, the estimated cost per death averted was USD 148. There are two main limitation of the study. Firstly, the brevity of the study and secondly, the assumed homogeneity of the clinical cases that presented to the hospital before and after the intervention.

Conclusions

This study demonstarted a signficant reductuion in inpatient mortality rate after an intervention to improve emergency hospital care If the findings of this paper could be reproduced in a larger more rigorous study, improving the quality of care in hospitals would be a very cost effective strategy to save children's lives in low resource settings.

The Projahnmo-II Project in Mirzapur upazila (sub-district), Tangail district, Bangladesh, is promoting care-seeking for sick newborns through health education of families, identification and referral of sick newborns in the community by community health workers (CHWs), and strengthening of neonatal care in Kumudini Hospital, Mirzapur. Data were drawn from records maintained by the CHWs, referral hospital registers, a baseline household survey of recently-delivered women conducted from March to June 2003, and two interim household surveys in January and September 2005. Increases were observed in self-referral of sick newborns for care, compliance after referral by the CHWs, and care-seeking from qualified providers and from the Kumudini Hospital, and decreases were observed in care-seeking from unqualified providers in the intervention arm. An active surveillance for illness by the CHWs in the home, education of families by them on recognition of danger signs and counselling to seek immediate care for serious illness, and improved linkages between the community and the hospital can produce substantial increases in care-seeking for sick newborns.

A cost-effective analysis conducted by Edwine Barasa and colleagues estimates that a complex intervention aimed at improving quality of pediatric care would be affordable and cost-effective in Kenya.

Background

To improve care for children in district hospitals in Kenya, a multifaceted approach employing guidelines, training, supervision, feedback, and facilitation was developed, for brevity called the Emergency Triage and Treatment Plus (ETAT+) strategy. We assessed the cost effectiveness of the ETAT+ strategy, in Kenyan hospitals. Further, we estimate the costs of scaling up the intervention to Kenya nationally and potential cost effectiveness at scale.

Methods and Findings

Our cost-effectiveness analysis from the provider's perspective used data from a previously reported cluster randomized trial comparing the full ETAT+ strategy (n = 4 hospitals) with a partial intervention (n = 4 hospitals). Effectiveness was measured using 14 process measures that capture improvements in quality of care; their average was used as a summary measure of quality. Economic costs of the development and implementation of the intervention were determined (2009 US$). Incremental cost-effectiveness ratios were defined as the incremental cost per percentage improvement in (average) quality of care. Probabilistic sensitivity analysis was used to assess uncertainty. The cost per child admission was US$50.74 (95% CI 49.26–67.06) in intervention hospitals compared to US$31.1 (95% CI 30.67–47.18) in control hospitals. Each percentage improvement in average quality of care cost an additional US$0.79 (95% CI 0.19–2.31) per admitted child. The estimated annual cost of nationally scaling up the full intervention was US$3.6 million, approximately 0.6% of the annual child health budget in Kenya. A “what-if” analysis assuming conservative reductions in mortality suggests the incremental cost per disability adjusted life year (DALY) averted by scaling up would vary between US$39.8 and US$398.3.

Conclusion

Improving quality of care at scale nationally with the full ETAT+ strategy may be affordable for low income countries such as Kenya. Resultant plausible reductions in hospital mortality suggest the intervention could be cost-effective when compared to incremental cost-effectiveness ratios of other priority child health interventions.

Please see later in the article for the Editors' Summary

Editors' Summary

Background

According to latest global estimates from UNICEF, 7.6 million children currently die every year before they reach five years of age. Half of these deaths occur in children in sub-Saharan Africa and tragically, most of these deaths are due to a few treatable and preventable diseases, such as pneumonia, malaria, and diarrhea, for which effective interventions are already available. In order to meet the target of the 4th Millennium Development Goal—which aims to reduce the under-five child mortality rate by two-thirds from 1990 levels by 2015—delivering these interventions is essential.

In Kenya, the under-five child mortality rate must be reduced by half from its 2008 level in order to meet the Millennium Development Goal (MDG) target and so improving the management of serious child illness might help achieve this goal. A study published last year in PLoS Medicine described such an approach and included the development and implementation of evidence-based clinical practice guidelines linked to health worker training, follow-up supervision, performance feedback, and facilitation in eight district hospitals in Kenya.

Why Was This Study Done?

In the study mentioned above, the researchers compared the implementation of various processes of care in intervention and control hospitals at baseline and 18 months later and found that performance improved more in the intervention hospitals than in the control hospitals. However, while this strategy was effective at improving the quality of health care, it is unclear whether scaling up the approach would be a good use of limited resources. So in this study, the same researchers performed a cost-effectiveness analysis (which they conducted alongside the original trial) of their quality improvement intervention and estimated the costs and effects of scaling up this approach to cover the entire population of Kenya.

What Did the Researchers Do and Find?

In order to perform the cost part of the analysis, the researchers collected the relevant information on costs by using clinical and accounting record reviews and interviews with those involved in developing and implementing the intervention. The researchers evaluated the effectiveness part of the analysis by comparing the implementation of their improved quality of care strategy as delivered in the intervention hospitals with the partial intervention as delivered in the control hospitals by calculating the mean percentage improvement in the 14 process of care indicators at 18 months. Finally, the researchers calculated the costs of scaling up the intervention by applying their results to the whole of Kenya—121 hospital facilities with an estimated annual child admission rate of 2,000 per facility.

The researchers found that the quality of care (as measured by the process of care indicators) was 25% higher in intervention hospitals than in control hospitals, while the cost per child admission was US$50.74 in intervention hospitals compared to US$31.1 in control hospitals. The researchers calculated that each percentage improvement in the average quality of care was achieved at an additional cost of US$0.79 per admitted child. Extrapolating these results to all of Kenya, the estimated annual cost of scaling up the intervention nationally was US$3.6 million, about 0.6% of the annual child health budget in Kenya.

What Do These Findings Mean?

The findings of this cost-effectiveness analysis suggests that a comprehensive quality improvement intervention is effective at improving standards of care but at an additional cost, which may be relatively cost effective compared with basic care if the improvements observed are associated with decreases in child inpatient mortality. The absolute costs for scaling up are comparable to, or even lower than, costs of other, major child health interventions. As the international community is giving an increasing focus to strengthening health systems, these findings provide a strong case for scaling up this intervention, which improves quality of care and service provision for the major causes of child mortality, in rural hospitals throughout Kenya and other district hospitals in sub-Saharan Africa.

Additional Information

Please access these Web sites via the online version of this summary at http://dx.doi.org/10.1371/journal.pmed.1001238.

The researchers' original article appeared in PLoS Medicine in 2011: Ayieko P, Ntoburi S, Wagai J, Opondo C, Opiyo N, et al. (2011) A Multifaceted Intervention to Implement Guidelines and Improve Admission Paediatric Care in Kenyan District Hospitals: A Cluster Randomised Trial. PLoS Med 8(4): e1001018. doi:10.1371/journal.pmed.1001018

The IDOC Africa provides further information on the ETAT+ strategy

The World Health Organization (WHO) provides information on MDG 4, including strategies to reduce global child mortality) and the WHO pocket-book “Hospital care for children” includes guidelines for the management of common but serious childhood illnesses in resource-limited settings

UNICEF www.unicef.org also publishes information on global child mortality rates and the Countdown to 2015 website tracks coverage levels for health interventions proven to reduce child mortality

Background

Given the high mortality associated with neonatal illnesses and severe malnutrition and the development of packages of interventions that provide similar challenges for service delivery mechanisms we set out to explore how well such services are provided in Kenya.

Methods

As a sub-component of a larger study we evaluated care during surveys conducted in 8 rural district hospitals using convenience samples of case records. After baseline hospitals received either a full multifaceted intervention (intervention hospitals) or a partial intervention (control hospitals) aimed largely at improving inpatient paediatric care for malaria, pneumonia and diarrhea/dehydration. Additional data were collected to: i) examine the availability of routine information at baseline and their value for morbidity, mortality and quality of care reporting, and ii) compare the care received against national guidelines disseminated to all hospitals.

Results

Clinical documentation for neonatal and malnutrition admissions was often very poor at baseline with case records often entirely missing. Introducing a standard newborn admission record (NAR) form was associated with an increase in median assessment (IQR) score to 25/28 (22-27) from 2/28 (1-4) at baseline. Inadequate and incorrect prescribing of penicillin and gentamicin were common at baseline. For newborns considerable improvements in prescribing in the post baseline period were seen for penicillin but potentially serious errors persisted when prescribing gentamicin, particularly to low-birth weight newborns in the first week of life. Prescribing essential feeds appeared almost universally inadequate at baseline and showed limited improvement after guideline dissemination.

Conclusion

Routine records are inadequate to assess newborn care and thus for monitoring newborn survival interventions. Quality of documented inpatient care for neonates and severely malnourished children is poor with limited improvement after the dissemination of clinical practice guidelines. Further research evaluating approaches to improving care for these vulnerable groups is urgently needed. We also suggest pre-service training curricula should be better aligned to help improve newborn survival particularly.

Background

Antibiotic surveillance is mandatory to optimise antibiotic therapy. Our objectives were to evaluate antibiotic use in our pediatric intensive care unit (PICU) and to implement a simple achievable intervention aimed at improving antibiotic therapy.

Method

Prospective, 3 months surveillance of antibiotic use on PICU (phase I) and evaluation according to the CDC 12-step campaign with development of an attainable intervention. 3 months surveillance (phase II) after implementation of intervention with comparison of antibiotic use.

Results

Appropriate antibiotic use for culture-negative infection-like symptoms and targeted therapy for proven infections were the main areas for potential improvement. The intervention was a mandatory checklist requiring indication and recording likelihood of infection at start of antibiotic therapy and a review of the continuing need for therapy at 48 h and 5 days, reasons for continuation and possible target pathogen. The percentage of appropriate empiric antibiotic therapy courses for culture-negative infection-like symptoms increased from 18% (10/53) to 74% (42/57; p<0.0001), duration of therapy <3 days increased from 18% (10/53) to 35% (20/57; p=0.05) and correct targeting of pathogen increased from 58% (7/12) to 83% (20/24; p=0.21).

Conclusions

Antibiotic surveillance using the CDC 12-step campaign can help to evaluate institutional antibiotic therapy. Development of an attainable intervention using a checklist can show improved antibiotic use with minimal expense.

Background

To evaluate a delivery strategy for newborn interventions in rural Bangladesh.

Methods

A cluster-randomized controlled trial was conducted in Mirzapur, Bangladesh. Twelve unions were randomized to intervention or comparison arm. All women of reproductive age were eligible to participate. In the intervention arm, community health workers identified pregnant women; made two antenatal home visits to promote birth and newborn care preparedness; made four postnatal home visits to negotiate preventive care practices and to assess newborns for illness; and referred sick neonates to a hospital and facilitated compliance. Primary outcome measures were antenatal and immediate newborn care behaviours, knowledge of danger signs, care seeking for neonatal complications, and neonatal mortality.

Findings

A total of 4616 and 5241 live births were recorded from 9987 and 11153 participants in the intervention and comparison arm, respectively. High coverage of antenatal (91% visited twice) and postnatal (69% visited on days 0 or 1) home visitations was achieved. Indicators of care practices and knowledge of maternal and neonatal danger signs improved. Adjusted mortality hazard ratio in the intervention arm, compared to the comparison arm, was 1.02 (95% CI: 0.80–1.30) at baseline and 0.87 (95% CI: 0.68–1.12) at endline. Primary causes of death were birth asphyxia (49%) and prematurity (26%). No adverse events associated with interventions were reported.

Conclusion

Lack of evidence for mortality impact despite high program coverage and quality assurance of implementation, and improvements in targeted newborn care practices suggests the intervention did not adequately address risk factors for mortality. The level and cause-structure of neonatal mortality in the local population must be considered in developing interventions. Programs must ensure skilled care during childbirth, including management of birth asphyxia and prematurity, and curative postnatal care during the first two days of life, in addition to essential newborn care and infection prevention and management.

Trial Registration

Clinicaltrials.gov NCT00198627

Background

Based on the realization that Uganda is not on track to achieving Millennium Development Goals 4 and 5, Makerere University School of Public Health in collaboration with other partners proposed to conduct two community based maternal/newborn care interventions aimed at increasing access to health facility care through transport vouchers and use of community health workers to promote ideal family care practices. Prior to the implementation, a stakeholder analysis was undertaken to assess and map stakeholders’ interests, influence/power and position in relation to the interventions; their views regarding the success and sustainability; and how this research can influence policy formulation in the country.

Methods

A stakeholder analysis was carried out in March 2011 at national level and in four districts of Eastern Uganda where the proposed interventions would be conducted. At the national level, four key informant interviews were conducted with the ministry of health representative, Member of Parliament, and development partners. District health team members were interviewed and also engaged in a workshop; and at community level, twelve focus group discussions were conducted among women, men and motorcycle transporters.

Results

This analysis revealed that district and community level stakeholders were high level supporters of the proposed interventions but not drivers. At community level the mothers, their spouses and transporters were of low influence due to the limited funds they possessed. National level and district stakeholders believed that the intervention is costly and cannot be affordably scaled up. They advised the study team to mobilize and sensitize the communities to contribute financially from the start in order to enhance sustainability beyond the study period. Stakeholders believed that the proposed interventions will influence policy through modeling on how to improve the quality of maternal/newborn health services, male involvement, and improved accessibility of services.

Conclusion

Most of the stakeholders interviewed were supporters of the proposed maternal and newborn care intervention because of the positive benefits of the intervention. The analysis highlighted stakeholder concerns that will be included in the final project design and that could also be useful in countries of similar setting that are planning to set up programmes geared at increasing access to maternal and new born interventions. Key among these concerns was the need to use both human and financial resources that are locally available in the community, to address supply side barriers that influence access to maternal and child healthcare. Research to policy translation, therefore, will require mutual trust, continued dialogue and engagement of the researchers, implementers and policy makers to enable scale up.

Introduction

There has been a lot of attention on the role of human resource management interventions to improve delivery of health services in low- and middle-income countries. However, studies on this subject are few due to limited research on implementation of programmes and methodological difficulties in conducting experimental studies on human resource interventions. The authors present the protocol of an evaluation of a district-level capacity-building intervention to identify the determinants of performance of health workers in managerial positions and to understand how changes (if any) are brought about.

Methods and analysis

The aim of this study is to understand how capacity building works. The authors will use realist evaluation to evaluate an intervention in Karnataka, India. The intervention is a capacity-building programme that seeks to improve management capacities of health managers at district and subdistrict levels through periodic classroom-based teaching and mentoring support at the workplace. The authors conducted interviews and reviewed literature on capacity building in health to draw out the programme theory of the intervention. Based on this, the authors formulated hypothetical pathways connecting the expected outcomes of the intervention (planning and supervision) to the inputs (contact classes and mentoring). The authors prepared a questionnaire to assess elements of the programme theory—organisational culture, self-efficacy and supervision. The authors shall conduct a survey among health managers as well as collect qualitative data through interviews with participants and non-participants selected purposively based on their planning and supervision performance. The authors will construct explanations in the form of context–mechanism–outcome configurations from the results. This will be iterative and the authors will use a realist evaluation framework to refine the explanatory theories that are based on the findings to explain and validate an improved theory on ‘what works for whom and under what conditions’.

Discussion

The scope for applying realist evaluation to study human resource management interventions in health are discussed.

Article summary

Article focus

Despite a lot of focus on capacity building of health workforce to improve health outcomes in developing countries, there are very few studies on how capacity building brings about better performance.

Methodological difficulties and complexity of health systems impose restrictions on evaluating human resource management interventions.

Key messages

Health worker practices are complex behaviours that are determined by various individual, institutional and systemic factors.

It is possible to use a theory-driven evaluation approach such as realist evaluation to understand the mechanisms through which capacity-building programmes improve (or do not improve) performance of health workers.

Strengths and limitations of this study

This is the first application of realist evaluation to a local health system intervention (capacity building).

Instead of ‘does it work or not?’, the study seeks to understand ‘what works for whom and under what conditions’.

Background

A cluster randomised controlled trial (RCT) of a national Kenyan mental health primary care training programme demonstrated a significant impact on the health, disability and quality of life of clients, despite a severe shortage of medicines in the clinics (Jenkins et al. Submitted 2012). As focus group methodology has been found to be a useful method of obtaining a detailed understanding of client and health worker perspectives within health systems (Sharfritz and Roberts. Health Transit Rev 4:81–85, 1994), the experiences of the participating clients were explored through qualitative focus group discussions in order to better understand the potential reasons for the improved outcomes in the intervention group.

Methods

Two ninety minute focus groups were conducted in Nyanza province, a poor agricultural region of Kenya, with 10 clients from the intervention group clinics where staff had received the training programme, and 10 clients from the control group where staff had not received the training during the earlier randomised controlled trial.

Results

These focus group discussions suggest that the clients in the intervention group noticed and appreciated enhanced communication, diagnostic and counselling skills in their respective health workers, whereas clients in the control group were aware of the lack of these skills. Confidentiality emerged from the discussions as a significant client concern in relation to the volunteer cadre of community health workers, whose only training comes from their respective primary care health workers.

Conclusion

Enhanced health worker skills conferred by the mental health training programme may be responsible for the significant improvement in outcomes for clients in the intervention clinics found in the randomised controlled trial, despite the general shortage of medicines and other health system weaknesses. These findings suggest that strengthening mental health training for primary care staff is worthwhile even where health systems are not strong and where the medicine supply cannot be guaranteed.

Trial registration

ISRCTN 53515024.

Background

Of all workers in Dutch construction industry, 20% has an elevated risk of cardiovascular disease (CVD). A major risk factor for CVD risk is an unhealthy lifestyle. The aim of our study is to design a lifestyle intervention for construction workers with an elevated CVD risk, and to evaluate its (cost-) effectiveness.

Methods/Design

In a RCT, 692 participants will be randomised to either the control or the intervention group. The control group will receive usual care. For the intervention group, a lifestyle intervention has been designed based on interviews and current literature. The intervention will last 6 months and will comprise 3 face-to-face and 4 telephone contacts, consisting of individual counselling aimed at increasing daily physical activity (PA) and improving dietary behaviour, and/or smoking cessation. Counselling will take place at the Occupational Health Service (OHS), and will be done according to motivational interviewing (MI). Additional written information about healthy lifestyle will also be provided to those in the intervention group. At baseline, after 6 and after 12 months, measurements will take place. Primary outcome variables will be the lifestyle behaviours of concern, i.e. daily PA, dietary intake, and smoking status. Secondary outcome variables will be body mass index (BMI), systolic and diastolic blood pressure, total and HDL blood cholesterol, Hba1c and cardio-respiratory fitness (CRF). Sickness absenteeism and cost-effectiveness will be assessed as well. Multilevel analysis will be performed to compare all outcome measures between the intervention group and the control group.

Discussion

By improving lifestyle, CVD risk may be lowered, yielding benefits for both employee and employer. If proven effective, this lifestyle intervention will be implemented on a larger scale within the Occupational Health Services in construction industry.

Trial registration

Current Controlled Trials ISRCTN60545588

Background

The structured admission form is an apparently simple measure to improve data quality. Poor motivation, lack of supervision, lack of resources and other factors are conceivably major barriers to their successful use in a Kenyan public hospital setting. Here we have examined the feasibility and acceptability of a structured paediatric admission record (PAR) for district hospitals as a means of improving documentation of illness.

Methods

The PAR was primarily based on symptoms and signs included in the Integrated Management of Childhood Illness (IMCI) diagnostic algorithms. It was introduced with a three-hour training session, repeated subsequently for those absent, aiming for complete coverage of admitting clinical staff. Data from consecutive records before (n = 163) and from a 60% random sample of dates after intervention (n = 705) were then collected to evaluate record quality. The post-intervention period was further divided into four 2-month blocks by open, feedback meetings for hospital staff on the uptake and completeness of the PAR.

Results

The frequency of use of the PAR increased from 50% in the first 2 months to 84% in the final 2 months, although there was significant variation in use among clinicians. The quality of documentation also improved considerably over time. For example documentation of skin turgor in cases of diarrhoea improved from 2% pre-intervention to 83% in the final 2 months of observation. Even in the area of preventive care documentation of immunization status improved from 1% of children before intervention to 21% in the final 2 months.

Conclusion

The PAR was well accepted by most clinicians and greatly improved documentation of features recommended by IMCI for identifying and classifying severity of common diseases. The PAR could provide a useful platform for implementing standard referral care treatment guidelines.

Background

Organizational factors are considered to be an important influence on health workers' uptake of interventions that improve their practices. These are additionally influenced by factors operating at individual and broader health system levels. We sought to explore contextual influences on worker motivation, a factor that may modify the effect of an intervention aimed at changing clinical practices in Kenyan hospitals.

Methods

Franco LM, et al's (Health sector reform and public sector health worker motivation: a conceptual framework. Soc Sci Med. 2002, 54: 1255–66) model of motivational influences was used to frame the study Qualitative methods including individual in-depth interviews, small-group interviews and focus group discussions were used to gather data from 185 health workers during one-week visits to each of eight district hospitals. Data were collected prior to a planned intervention aiming to implement new practice guidelines and improve quality of care. Additionally, on-site observations of routine health worker behaviour in the study sites were used to inform analyses.

Results

Study settings are likely to have important influences on worker motivation. Effective management at hospital level may create an enabling working environment modifying the impact of resource shortfalls. Supportive leadership may foster good working relationships between cadres, improve motivation through provision of local incentives and appropriately handle workers' expectations in terms of promotions, performance appraisal processes, and good communication. Such organisational attributes may counteract de-motivating factors at a national level, such as poor schemes of service, and enhance personally motivating factors such as the desire to maintain professional standards.

Conclusion

Motivation is likely to influence powerfully any attempts to change or improve health worker and hospital practices. Some factors influencing motivation may themselves be influenced by the processes chosen to implement change.

Background

To evaluate the process of a job-specific workers’ health surveillance (WHS) in improving occupational health care for construction workers.

Methods

From January to July 2012 were 899 bricklayers and supervisors invited for the job-specific WHS at three locations of one occupational health service throughout the Netherlands. The intervention aimed at detecting signs of work-related health problems, reduced work capacity and/or reduced work functioning. Measurements were obtained using a recruitment record and questionnaires at baseline and follow-up. The process evaluation included the following: reach (attendance rate), intervention dose delivered (provision of written recommendations and follow-up appointments), intervention dose received (intention to follow-up on advice directly after WHS and remembrance of advice three months later), and fidelity (protocol adherence). The workers scored their increase in knowledge from 0–10 with regard to health status and work ability, their satisfaction with the intervention and the perceived (future) effect of such an intervention. Program implementation was defined as the mean score of reach, fidelity, and intervention dose delivered and received.

Results

Reach was 9% (77 workers participated), fidelity was 67%, the intervention dose delivered was 92 and 63%, and the intervention dose received was 68 and 49%. The total programme implementation was 58%. The increases in knowledge regarding the health status and work ability of the workers after the WHS were graded as 7.0 and 5.9, respectively. The satisfaction of the workers with the entire intervention was graded as 7.5. The perceived (future) effects on health status were graded as 6.3, and the effects on work ability were graded with a 5.2. The economic recession affected the workers as well as the occupational health service that enacted the implementation.

Conclusions

Programme implementation was acceptable. Low reach, limited protocol adherence and modest engagement of the workers with respect to the intervention were the most prominent aspects that influenced the intervention process. The increase in the workers’ knowledge about their health status and work ability was substantial, and the workers’ satisfaction with the intervention was good. The perceived effect of the advised preventive actions on health status was sufficient.

Trial registration

Netherlands Trial Register: http://NTR3012

Objective

To describe the quality of outpatient paediatric malaria case-management approximately 4–6 months after artemether–lumefantrine (AL) replaced sulfadoxine–pyrimethamine (SP) as the nationally recommended first-line therapy in Kenya.

Methods

Cross-sectional survey at all government facilities in four Kenyan districts. Main outcome measures were health facility and health worker readiness to implement AL policy; quality of antimalarial prescribing, counselling and drug dispensing in comparison with national guidelines; and factors influencing AL prescribing for treatment of uncomplicated malaria in under-fives.

Results

We evaluated 193 facilities, 227 health workers and 1533 sick-child consultations. Health facility and health worker readiness was variable: 89% of facilities stocked AL, 55% of health workers had access to guidelines, 46% received in-service training on AL and only 1% of facilities had AL wall charts. Of 940 children who needed AL treatment, AL was prescribed for 26%, amodiaquine for 39%, SP for 4%, various other antimalarials for 8% and 23% of children left the facility without any antimalarial prescribed. When AL was prescribed, 92% of children were prescribed correct weight-specific dose. AL dispensing and counselling tasks were variably performed. Higher health worker's cadre, in-service training including AL use, positive malaria test, main complaint of fever and high temperature were associated with better prescribing.

Conclusions

Changes in clinical practices at the point of care might take longer than anticipated. Delivery of successful interventions and their scaling up to increase coverage are important during this process; however, this should be accompanied by rigorous research evaluations, corrective actions on existing interventions and testing cost-effectiveness of novel interventions capable of improving and maintaining health worker performance and health systems to deliver artemisinin-based combination therapy in Africa.