|Home | About | Journals | Submit | Contact Us | Français|
This systematic review evaluated the effectiveness of professional and organizational interventions aimed at improving medical processes, such as documentation or clinical assessments by health care providers, in the care of pediatric emergency department (ED) patients where abuse was suspected.
A search of electronic databases, references, key journals/conference proceedings was conducted and primary authors contacted. Studies whose purpose was to evaluate a strategy aimed at improving ED clinical care of suspected abuse were included. Study methodological quality was assessed by two independent reviewers. One reviewer extracted the data and a second checked for completeness and accuracy.
Six studies met the inclusion criteria: one randomized (RCT) and one quasi-randomized trial (qRCT), and four observational studies. Study quality ranged from modest (observational studies) to good (trials). Variation in study interventions and outcomes limited between study comparisons. One qRCT supported self-instructional education kits as a means to improve physician knowledge for both physical abuse (mean pre-test score: 13.12, SD 2.36; mean post-test score: 18.16, SD 1.64) and sexual abuse (mean pre-test score: 10.81, SD 3.20; mean post-test score: 18.45, SD 1.79). Modest quality observational studies evaluated reminder systems for physician documentation with similar results across studies. Compared to standard practice, chart checklists paired with an educational program increased physician consideration of non-accidental burns in burn cases (59% increase), documentation of time of injury (36% increase), as well as documentation of consistency (53% increase) and compatibility (55% increase) of reported histories. Decisional flowcharts for suspected physical abuse also increased documentation of non-accidental physical injury (69.5% increase; p<0.0001) and had a similar significant impact as checklists on increasing documentation of history consistency and compatibility (69.5% and 70.0% increases, respectively; p<0.0001) when compared to standard practice. No improvements were noted in these studies for documentation of consultations or current status with child protective services. The introduction of a specialized team and crisis center to standardize practice had little effect on physician documentation, but did increase documentation of child protective services involvement (22.7% increase; p<0.005) and discharge status (23.7% increase; p<0.02). Referral to social services increased in one study following the introduction of a chart checklist (8.6% increase; p=0.018). A recently conducted multi-site RCT did not support observational findings, reporting no significant effect of educational sessions and/or a chart checklist on ED practices.
The small number of studies identified in this review highlights the need for future studies that address care of a vulnerable clinical population. While moderate quality observational studies suggested education and reminder systems increased clinical knowledge and documentation, these findings were not supported by a single randomized trial. The limited theoretical base for conceptualizing change in health care providers and the influence of the ED environment on clinical practice are limitations to this current evidence base.
In 2006, almost 3.6 million American children and youth received a child protection service investigation or assessment, with 28.6% of the investigations determining abuse or neglect.1 Canadian data indicate that, in 2003, just under a quarter of a million investigations of alleged child abuse and neglect were conducted with 50% of those investigations being substantiated.2–3 These statistics have been critiqued as under representative of the magnitude of abuse,1–2,4–5 and there remains an opportunity to not only improve the detection of abuse, but also to initiate protective services. Abuse often consists of a series of incidents rather than a single event,6 and cases are reported after several medical evaluations. There is evidence to suggest that 20–30% of children and youth who die from abuse have been seen by health care providers for abusive sequelae before abuse was formally identified.7–8
The emergency department (ED) represents a critical system entry point for crisis-based health care visits. For children and youth who present to the hospital for accidents related to neglect or inflicted injuries, ED personnel may be their first contact and opportunity for initiating protective services.9–13 Abused children and youth have higher rates of ED use before diagnosis compared to other patients, but do not necessarily have distinguishable features to identify them as a ‘high risk’ population for screening.14 Health care providers in the ED are therefore required to rely on their clinical skills and judgment to identify signs and symptoms consistent with abuse. Prior hospital visits which often meet criteria for suspicion of abuse, however, can go unreported or unidentified, or be poorly documented.6,15–17 Factors known to facilitate or impede the assessment and management of child abuse include perceived time demands and obstacles to an effective health care system response, training gaps, as well as concerns with clinical abilities.18–20 Strategies aimed at improving ED clinical practices have been evaluated, but to date, have not been synthesized to formulate quality improvement recommendations. This systematic review evaluated the effectiveness of professional (e.g., educational initiatives) and organizational (e.g., implementation of specialized teams) interventions aimed at improving medical processes, such as documentation or clinical assessments by health care providers, in the care of pediatric ED patients where abuse was suspected.
A research librarian, with input from the clinical research team, developed and implemented systematic search strategies using language (English and French) and year (1985 to 2009) restrictions. The search was conducted in 14 high yield electronic databases: MEDLINE®, Ovid MEDLINE® In-Process & Other Non-Indexed Citations, EMBASE, Cochrane Central Register of Controlled Trials, OVID HealthStar, Cochrane Database of Systematic Reviews, Health Technology Assessment Database, Database of Abstracts of Reviews of Effects, ACP Journal Club, PsycINFO®, CINAHL®, SocIndex, ProQuest Theses and Dissertations, and Child Welfare Information Gateway. Reference lists, key journals, and conference proceedings (Canadian Association of Emergency Physicians, Society for Academic Emergency Medicine, American College of Emergency Physicians, Canadian Paediatric Society) were also reviewed. The initial search was conducted in January 2008 and was updated in January 2009. Search terms included abuse, violence, risk or risk factors or risk assessment, emergency health service, “emergency care” or “emergency treatment”; comprehensive strategies used in each database are available from the corresponding author on request.
Two reviewers independently screened the search results. Experimental and observational studies were eligible for review inclusion.21 Studies with the primary purpose of evaluating a strategy aimed at improving processes involved in the clinical care of suspected physical or sexual abuse in the ED were included. Health care providers targeted in the studies had to work with children and youth (≤18 years), and a clinically relevant outcome (i.e., identification of abuse, documentation, referral/involvement of protective services) needed to be reported.
The Jadad 5-point scale was used to assess control of bias in randomized controlled trials (RCTs). Points were allocated as follows: randomization (0–2 points), double blinding (0–2 points), and withdrawals/dropouts (0–1point).22 Concealment of allocation was assessed as adequate, inadequate, or unclear using criteria from Schulz et al.23 All other study designs were assessed using methodological criteria developed by Downs and Black.24 Quality was measured by study reporting, external and internal validity, and power, with an assigned maximal quality index (QI) of 29. QI scores of >20 were considered good, 11 to 20 moderate, and <11 poor. Two reviewers (BZ, MH) independently analysed each study’s methodological quality and agreement was quantified with the Kappa statistic.25
Data from the final six studies were extracted using a standardized form that encompassed elements of study characteristics (e.g., language of publication, country), characteristics of the study population; study setting; description of the intervention and comparisons; outcome measures and measurements tools; and, results. Data were extracted by one reviewer and checked for accuracy and completeness by a second reviewer. In the case of unclear or unreported information in the original studies, primary authors were contacted. Heterogeneity in the interventions, outcomes, and unit of analysis precluded the use of meta-analysis to pool and interpret study results; therefore, a descriptive analysis of our findings based on the quality assessment is presented.
The search strategy identified 282 studies as potentially relevant to the review, all of which were assessed for study inclusion by two independent reviewers. As shown in Figure 1, six studies were confirmed for inclusion: one randomized controlled trial (RCT),14 one quasi-randomized trial,26 and four observational studies of various design.27–30 One study was excluded because we could not confirm if it targeted ED-based health care providers.31
Study characteristics and overall study quality are outlined in Table 1. All studies examined the impact of physician oriented strategies to improve assessment and/or care. Studies took place in either the United States (n=4) or United Kingdom (n=2). The age of children and youth cared for by study participants varied. All studies focused on the assessment and treatment of physical abuse; one study also focused on sexual abuse. Studies reviewed the clinical care of children and youth with: (i) any non-accidental injury,28,30 (ii) non-accidental burns27,29 or (iii) unknown.14
Methodological quality of the studies ranged (Table 1). Overall, reviewer agreement on quality was good (κ=.76)29 with reviewer disagreements resolved through consensus. While a randomized design was an overt strength in reducing risk of bias in the review’s one RCT,14 because it is impossible to double-blind studies of provider based interventions, the quality score was limited to a modest maximum of three on the Jadad scale (rather than five; range=one to three). Concealment of allocation was assessed as unclear.23 Using the maximal quality index, scores for the remaining studies on the Downs and Black checklist ranged from 16 (moderate quality) to 21 (good quality) out of a possible 29.26–30 Key methodological strengths for higher quality studies included detailed descriptions of the study design and analyses, inclusion of a representative sample, and adjustment for confounding factors. Lower scores were the result of insufficient study population description, small sample sizes, lack of a control group, and minimal reporting of statistical analyses (Table 1).
An equal number of studies evaluated the impact of a single intervention in changing behaviors27–28 and knowledge26 for physicians compared to a multi-modal intervention.14,29–30 Studies that evaluated educational interventions for physicians examined their impact on knowledge26 and documentation.14,29 While modality differed between these studies, the educational content provided was comparable and included abuse recognition and management (Table 2). The impact on physician behaviors was evaluated using chart appended reminder systems, which included checklists14,27,29 and a decisional flowchart.28 The impact of specialized services (Child Abuse and Neglect Team, Child Crisis Centre) and resident training (a workshop and optional elective) was examined in one study30 with a similar focus on practice behaviors. No studies evaluated the impact of interventions on clinical outcomes (i.e., psychosocial, physical) in children, youth or their families (Table 2).
Educational programs with14,29 and without26 reminder systems for patient charts demonstrated a mixed impact (Table 2). In their evaluation of the effectiveness of two self-instructional programs, Showers and Laird reported a significant increase in mean scores of physician knowledge of both physical and sexual abuse (p<0.0001).26 Benger and McCabe reported significant improvements to physician documentation through the introduction of an educational program and chart reminder for pediatric burn and scald patients.29 The intervention resulted in a 59% increase in the consideration of non-accidental injury, and also improved physician documentation of time of injury (36% increase in documentation), as well as consistency of history and compatibility of injury (53% and 55% increase in documentation, respectively). A recently published RCT16 by Guenther et al. contradicts both the Showers23 and Benger26 study findings. This multi-site trial found that multiple educational interventions along with a reminder chart did not significantly improve ED documentation practices of cases of suspected physical abuse.14
Studies evaluating the impact of reminder systems on physician documentation28 and referral to social services27 also yielded mixed results (Table 2). Benger and Pearce found that, compared to standard practice, flowcharts placed on patient charts consistently increased physician documentation on all expected assessment facets,28 while Clark’s use of a checklist improved physician reporting of non-accidental burns to social services.27
As shown in Table 2, only one study, of modest quality, examined the impact of the introduction of specialized, in-house services and training on physician documentation.30 Significant improvements were noted in documentation of time of arrival to the ED, the involvement of child protective services and disposition status. Improvements were not observed in the documentation of history and physical examinations.30
One study reported the response rate to their mail-out educational intervention, and three studies evaluated adherence to their study’s ED-based intervention. Showers and Laird reported a 22% and 30% response rate for return of test booklets on physical and sexual abuse, respectively.26 Adherence to ED-based interventions was similar across the three reporting studies: 60% inclusion of a documentation reminder in eligible charts,29 72% inclusion of a documentation reminder in eligible charts,28 and 78% inclusion of a chart checklist in eligible charts.27
This systematic review evaluated the effectiveness of interventions aimed at improving medical processes, such as documentation or clinical assessments, in the care of pediatric ED patients where abuse was suspected. While moderate quality studies suggested education and reminder systems increased various aspects of clinical knowledge and physician documentation,26–30 these findings were not supported by a higher quality randomized trial which reported limited to no effect on clinical practice trends.14 The lack of theoretical base for conceptualizing the change in health care provider behaviour and the impact of the ED environment on clinical practice makes it difficult to determine what influenced the medical process changes in the studies. Noted variability in documentation (termed ‘sporadic’) in the Guenther trial14 highlights the possibility of unaccounted for factors that influence clinical practices.
There are no gold standards for developing and evaluating educational, behavioral or organizational interventions designed to improve health care professional practices. Modest and mixed effects of these interventions are noted in comprehensive reviews31–37 and objectives remain for ED researchers to better understand factors which facilitate or impede physician behavior change, and the ideal form interventions and health care systems need to take to effectively address these factors and influence clinical practices.38–41 While the scope of this discussion is beyond the purpose of this paper, systematic reviews have demonstrated that a range of professional interventions such as audit and feedback,37 continuing medical education (CME),42–45 printed educational materials (including clinical practice guidelines and peer reviewed publications),33 and educational outreach31 have varied effects in different clinical practice settings and emphasizes the need for theoretically-driven and -evaluated interventions. This tenet also holds true for improving medical processes for suspected child abuse in the ED. Socolar et al., for example, used an audit and feedback strategy with physicians in the Child Medical Evaluation Program to target physician documentation and knowledge of child sexual abuse.46 While discrete changes were noted in physician history taking, tailored feedback with directed educational resources did not improve documentation or knowledge in this large RCT. Results indicated that CME for child sexual abuse, use of structured record for documentation, and physician gender (female) were significantly associated with better history documentation. Factors such as physician attitudes and beliefs were not measured,46 and the use of a theoretical framework to link intervention strategies with evaluation outcomes would have strengthened interpretation of study results.
In future studies of ED medical processes for suspected child abuse, we propose exploring a comprehensive set of effect mediators and moderators across practice behaviors and to measure barriers and facilitators to ED practice change. As a first step, study designs must acknowledge the complexity of health care provider focused interventions and should include the following: (1) increased sample sizes to detect the small to modest effects health care provider interventions can have, (2) independent comparison groups (trial designs versus observational designs) and (3) measurement of the extent to which interventions were implemented and adhered to in the ED. The nature of behaviors targeted for improvement also requires consideration. An asset of the studies in this review, irrespective of their quality, was the inclusion of evidence-based criteria from which to evaluate documentation practices. However, these behaviors are complex and likely require a number of cognitive and behavioral steps before a change in documentation behavior is realized (such as a cascade of decision-making). Future studies must be theoretically-driven to rigorously conceptualize these steps.
To optimize practice change, future interventions aimed at improving medical processes need to account for the complexity of the ED clinical practice setting and how this can affect care and documentation practices. Effect mediators and moderators of various practice behaviors need to be identified and explored as they can account for how different interventions impact ED care quality and delivery. Factors known to facilitate or impede the assessment and management of child abuse also need to be measured (e.g., concerns with personal abilities, time demands, perceived obstacles to system responses, training gaps, human resources). Factors inherent in the ED practice setting also need to be accounted for including the high patient volume, high levels of diagnostic uncertainty, limited clinician performance feedback, the multitude of interruptions and distractions for clinicians, and the complexity of care—all of which contribute to a clinical practice environment that is prone to clinical error and high stress.47 Accounting for the impact of these factors when measuring practice change will assist in the detection of true intervention effects.
Accounting for the multiple roles involved in delivering care may help to better delineate processes required to improve care and patient outcomes. The evaluation of professional interventions targeting other emergency health care providers such as nurses and social workers would be of value as would the evaluation of organizational strategies aimed at optimizing the role of multidisciplinary teams, and in particular, the role of physicians alongside child protection services and the police. Such studies would help answer questions of how best to carry out comprehensive child protection in the ED.
This review is limited by the small number of studies available meeting our review objective. Other limitations in this review stem primarily from the studies themselves. Confidence in the observational studies’ findings is tempered by the studies’ modest methodological quality as well as inconsistent results among the included studies. Heterogeneity in the interventions, outcomes, and unit of analysis limited our ability to make between study comparisons. The studies also did not assess clinical outcomes in patients or families (e.g., emotional distress) associated with efforts such as charting/documentation.
These limitations are compounded by the lack of theoretical base for conceptualizing behavior change and the influence of clinical environment on clinical practice across all studies. To date, one of the major limitations of provider-focused intervention studies has been the lack of underlying theory to support choice of intervention.32 Research in this area has been “an expensive version of trial and error”.48 The limitations of atheoretical interventions have been duly noted by Grimshaw and colleagues,49–50 which have resulted in a number of well-executed, theoretically-driven studies.51–56 Social cognitive theories, such as the theory of planned behavior, for example, have aided interventions to account for the cognitive mechanisms that underline behavior change in provider-oriented interventions. Attention to the ED provider-specific barriers and facilitators while considering theoretical perspectives to develop and evaluate future provider-focused interventions may result in more effective (and tailored) interventions in emergency medicine41 and may improve the modest adherence noted in several studies.26–29
The small number of studies identified in this review highlights the need for future studies aimed at improving child abuse care in the ED setting. While moderate quality observational studies suggested education and reminder systems increased clinical knowledge and documentation, these findings were not supported by a single randomized trial. More theory-based studies with a rigorous design and well defined clinical scope are needed to better evaluate the impact of professional and organizational interventions on clinical practice and patient and family outcomes.
The authors would like to acknowledge the important contributions from Ms. Lisa Tjosvold (Alberta Research Centre for Child Health Evidence, University of Alberta) for conducting the initial literature search; and Mses. Justina Tuli and Carla Farnesi (Department of Pediatrics, University of Alberta) for assisting with initial screening and data extraction, respectively. Funding for this project was partially provided by a Knowledge Synthesis grant awarded to the principal author from the Canadian Institutes of Health Research (CIHR). Dr. Newton and Ms. Curran are salary supported by the Canadian Child Health Clinician Scientist Program, in partnership with SickKids Foundation, Child & Family Research Institute (British Columbia), Women & Children’s Health Research Institute (Alberta), Manitoba Institute of Child Health.