Search tips
Search criteria 


Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
Arch Phys Med Rehabil. Author manuscript; available in PMC 2013 August 1.
Published in final edited form as:
PMCID: PMC3408840

Variation in adherence to new quality of care indicators for the acute rehabilitation of children with traumatic brain injury

Frederick P. Rivara, MD, MPH,1,2,3,4 Stephanie K. Ennis, M.S., CCC-SLP,1,5 Rita Mangione-Smith, M.D., M.P.H.,2,4 Ellen J. MacKenzie, Ph.D.,6,7 and Kenneth M. Jaffe, M.D.1,2,5



To determine variations in care provided by nine inpatient rehabilitation units for children with TBI using newly developed quality indicators.


Retrospective cohort study


Study conducted in 9 inpatient rehabilitation units.


174 children 0–17 years admitted for the inpatient rehabilitation of moderate to severe TBI.


Not applicable

Main Outcome Measures

Adherence to 119 newly developed quality of care indicators in seven different domains: general care, family-centered care, cognitive-communication, motor, neuropsychological, school and community integration.


There was substantial variation both within and between institutions in the percent of patients receiving recommended care in the seven domains. The lowest scores were found for the school domain. Only five institutions scored above 50% for all quality indicators and only one institution scored above 70% overall. Greater adherence to quality indicators was found for facilities with a higher proportion of therapists with pediatric training and for facilities that only admitted children. Patient volume was not associated with adherence to quality indicators.


The results indicate a tremendous variability and opportunity for improvement in the care children with TBI.

Keywords: Traumatic brain injury, children and adolescents, inpatient rehabilitation, quality of care

An enduring paradigm in measuring quality of care is the structure-process-outcome model developed by Donabedian more than four decades ago.1 While there have been numerous important advances in the science of quality measurement, the Donabedian model has continued to prove valid and useful. Accreditation organizations such as The Joint Commission and the Commission on the Accreditation of Rehabilitation Facilities (CARF) over time have moved along this continuum and increasingly have focused their efforts on the process of care delivery and its effects upon outcomes.

The increased attention to quality of care has occurred against the backdrop of (and would not have been possible without) an increased focus on evidence-based medicine. As a result, tools to measure quality of care have more recently been based on rigorous studies identified through systematic reviews of the literature and less on expert opinion.24 The acute treatment of children with TBI has followed this evolution, resulting in the publication of evidence-based guidelines for their care.5 A key component of the acute care of children with moderate to severe TBI is inpatient rehabilitation.

Previously, we reported on the development of quality indicators for measuring the structural and organizational aspects of acute inpatient rehabilitation care for children with TBI and the variation in these quality measures in a sample of institutions delivering such care in the United States (U.S).6 We also reported on the development of a set of quality measures for examining the process of delivering acute inpatient rehabilitation care.7 In this current study, we tested the ability of those new process measures to determine variations in the quality of care in nine institutions providing inpatient rehabilitation for children with TBI.


Recruitment of Participating Sites

We identified institutions in the US which delivered acute rehabilitation care to children with TBI from lists of hospitals enrolled in the Uniform Data System for Medical Rehabilitation (Amherst, New York), CARF, Tucson, Arizona and/or the National Association of Children’s Hospitals and Related Institutions (Alexandria, Virginia). The 34 facilities that reported average annual volume of at least 10 children with TBI during the calendar years 2007 through 2009 were stratified by status as being part of a pediatric hospital (n=24) or part of a general hospital or free-standing rehabilitation facility (n=10). Facilities were selected and approached for study participation based on ensuring that we included both pediatric and general inpatient rehabilitation units, geographic diversity across all four regions of the US, and interest in participating in the research project. Ten agreed to participate; one hospital was delayed in obtaining Institutional Review Board (IRB) approval and was thus dropped. The IRBs for each of the remaining nine participating sites approved the study. Of these nine institutions, 2 were not CARF certified.

Chart Abstractor Training

At each participating site, a staff member with a clinical and/or research background in rehabilitation medicine was identified to perform chart abstractions; abstractors were not blinded to the study hypotheses. Each abstractor was trained using a written training manual to abstract data from a single training chart, followed by a one-hour individual phone discussion with the study coordinator to review inaccurate responses and discuss difficulties interpreting specific variables. The study coordinator was accessible to respond to individual questions throughout the chart abstraction process, and any decisions reached by the investigators were disseminated to every abstractor via a periodic email digest.

Case Selection

Criteria for selection of a case included: (1) admitted to the inpatient rehabilitation facility with a primary diagnosis of traumatic brain injury; (2) minimum length of stay on the inpatient rehabilitation unit of 7 days; (3) admitted between the dates of March 1, 2008 and September 1, 2010; (4) initial Glasgow Coma Scale (GCS) score of 3–12 and; (5) age 0–17 when admitted. Twenty cases were randomly selected for abstraction from each participating site using computer generated random numbers, with the exception of one site that was only able to identify 18 eligible patients, and another that only identified 16 patients. When the initial GCS was not documented in the chart, severity of injury was confirmed by documentation of unresponsiveness, intubation and/or posturing at the scene of the injury, or by documentation of a GCS of 12 or less when admitted to the inpatient rehabilitation unit. We excluded children with spinal cord injury, those with a prior inpatient rehabilitation admission, and those for whom their rehabilitation stay was interrupted by a transfer to a more intensive level of care. Medical records were defined as any documentation from the inpatient rehabilitation admission, including therapist “shadow” charts and administrative data such as functional outcome measures when available.

Variable Definitions and Abstraction Processes

We a priori defined seven domains of acute rehabilitation care for pediatric TBI, shown in Table 1. As described previously,68 119 quality indicators were developed based on systematic reviews of the literature and the RAND-UCLA modified Delphi method. General elements of care included indicators on the general medical assessment of the patient, including pre-injury functioning, and management of general medical care during the rehabilitation hospital stay. The family-centered care domain focused on the functioning of the family and its needs to support the injured child post-discharge. The domain of cognitive-communication, speech, language and swallowing impairments included indicators for assessment and minimization of aspiration risk and interventions for the development of cognitive-communication skills. Range of motion, mobility and ability to perform activities of daily living were the focus of the gross and fine motor skills domain. The neuropsychological assessment and social and behavioral impairments domain indicators focused on assessment of function and impairment, and interventions for behavioral deficits. The schoolre-entry and community integration domains focused on assessing the needs of the injured child to participate in school and community activities.

Table 1
Number of process of care quality indicators for each domain

We defined pediatric specific training of rehabilitation personnel as certification of clinical specialty in pediatrics, neurology and/or rehabilitation; a fellowship in pediatric rehabilitation for a minimum of one year; and/or a minimum of two years’ experience in providing rehabilitation services for patients typically seen in CARF-accredited pediatric inpatient rehabilitation programs

A secure online database permitted direct data entry and data collection monitoring. Each element of multi-variable indicators was abstracted separately to facilitate interpretation, and scoring guidelines were developed to reconstruct composite indicators during the analysis process. When possible, the database automatically determined whether a child was eligible for the care represented by each indicator based on abstracted data such as age or relevant impairments. For example, indicators related to school re-entry would not be applicable to pre-school children; some indicators would not be relevant for children with severe disability from the TBI. For eligible children, a “passing score” was awarded when the provision of care was documented or documentation indicated that an attempt to provide care was made but could not be completed for a justifiable reason (e.g., the child’s functional level was a barrier to delivering the prescribed care).

Data analysis

A composite score representing the total number of children who received an element of care divided by the total number of children who were eligible to receive that care was calculated for each of the 119 indicators and averaged for each institution. Next, each institution’s composite score for each indicator was adjusted based on the ability of all institutions in the sample to deliver an individual item of care. To adjust the composite scores for this ‘observed difficulty of delivery’ (ODD), the numerator in the calculation was the institutional sum of the “passes” for an indicator minus the sum of the average pass rate for that indicator across all institutions; the denominator was the number of eligible patients for that indicator.8 For each institution, ODD adjusted scores for all quality indicators within a domain were averaged in order to calculate a summary score for that domain of care.


Study population

There were 174 patients studied across the 9 institutions. The characteristics of the patient sample are shown in Table 2. The majority of children were male. Some institutions treated predominantly adolescents while others had a broader age range of patients. The initial GCS at first presentation to the emergency department was not documented in the rehabilitation hospital chart for 11.5% of patients. There was a wide variation between the time of injury and time to first admission for acute rehabilitation, depending on each institution’s criteria for rehabilitation admission (i.e. insurance preauthorization, bed availability, medical co-morbidities and stability, and patient’s ability to actively participate, benefit, and tolerate a multidisciplinary program). There was also substantial variation in the length of the rehabilitation hospital stay.

Table 2
Characteristics of the study sample

Variation in quality of care

As shown in Table 3 and Figure 1, there was substantial variation both within and between institutions in the percent of patients receiving recommended care for each of the seven domains. All institutions scored relatively highly on general elements of care. About 76% of the recommended elements of care for the cognitive and gross and fine motor domains were received by patients, but there was substantial variation from one institution to another. For the cognitive domain, rates of adherence to recommended care ranged from 51% to 94% and for the motor domain, ranged from 61% to 88%. The lowest scores were found for the school re-entry domain, but a number of institutions scored poorly on the domains of neuropsychological assessment, family-centered care and community integration processes of care as well. Overall, only about one-half of the recommended care elements in the neuropsychological, school re-entry and family-centered care domains were received by patients. Only five institutions scored above 50% for all quality indicators and only one institution scored above 70% overall. Four institutions had scores on three or more domains that were below 50%.

Figure 1Figure 1Figure 1Figure 1Figure 1Figure 1Figure 1
Average percent adherence for each domain by hospital. The horizontal line denotes the mean for each domain. Panel 1: General elements of care. Panel 2: Cognitive-communication, speech, language and swallowing. Panel 3: Gross and fine motor skills. Panel ...
Table 3
ODD Adjusted Quality Scores (95% CI) Expressed as a Percentage For Each Institution

Institutional predictors of domain scores

When 75% or more of all staff physical therapists, occupational therapists and speech language pathologists had specific pediatric training, rehabilitation units had higher adherence to motor, neuropsychological and community quality indicators d compared with centers in which fewer therapists were pediatric trained (Figure 2a). Units that only admitted children scored higher on the delivery of care in the cognitive, neuropsychological and school re-entry domains than those units which cared for both children and adults (Figure 2b). The annual volume of pediatric patients with TBI was not associated with the quality scores on any domains (Figure 2c).

Figure 2Figure 2Figure 2
Figure 2a. Association of adherence with pediatric training of therapists.

Seven of the nine institutions were CARF certified. CARF certification was significantly associated only with adherence on indicators in the school re-entry domain (55% with certification vs. 31% without, p=0.0074); there was no association of CARF certification with any of the other domain quality scores.


This study examined variation in the quality of acute inpatient rehabilitation care for children with TBI across a sample of institutions providing such care in the US. We found that there was substantial variation in the provision of different elements of care across hospitals, within hospitals and across domains of care, using newly developed quality of care indicators. Only one institution provided 70% of the recommended care to these injured children, and four of the nine institutions provided less than half the recommended elements of care in three or more domains.

This is the first time that we are aware of that care for the rehabilitation of children with TBI has been examined in this way and publicly reported. CARF does survey hospitals every three years as part of the accreditation process but does not publicly report the results. We thus do not know if the degree of variability in the elements of care examined by CARF was similar to what we found.

This examination of 174 patients from nine institutions suggests that there was considerable opportunity for improvement in the quality of care delivered in this sample. While these institutions do not represent a random sample of inpatient rehabilitation facilities treating children with TBI, we do not believe that they represent a group at the bottom end of the quality of care spectrum. These institutions had higher volumes of pediatric TBI patients than many others in the US, seven of the nine were CARF certified, and the two that were not are both part of prestigious academic medical centers. A study of children with TBI treated at four trauma centers revealed that 3 and 12 months after injury, 26% and 31% of children, respectively, had unmet health care needs, the most frequent of which was for cognitive services (defined by Slomine much broader than the services included in our cognitive domain).9 A recent study in Australia of adults who received inpatient rehabilitation for a TBI reported that many did not receive specialists services recommended in clinical guidelines.10

The variability in adherence to some quality indicators may reflect a lack of standardization in personnel employed to provide the different elements of care, as well as lack of standardization in care delivered. This variability also may reflect a lack of prior consensus on elements of recommended care for TBI patients. This is the first attempt to develop a more standardized approach.

Study limitations

There are limitations to this study. The 119 indicators were rigorously developed7 but the association between the indicators and patient outcomes has not been evaluated beyond the data used for their selection. These indicators were based on literature reviews and a nine member Delphi panel; other experts may choose different indicators. However, pediatric physiatrists from six of the nine hospitals in this study were members of the Delphi panel and scored the quality indicators included in the seven domains as scientifically valid. The data were abstracted from the medical record; care may have been delivered that was not documented in the record. However, documentation of care is essential for delivering quality of care, and the absence of documentation itself would be a marker of poor quality. The sample size of 174 patients from nine institutions is small and may not be representative. The choice of hospitals was also not random, and was based on achieving geographic diversity, including both general and pediatric facilities, and willingness to participate in the research project. The facilities may not be wholly representative of the universe of inpatient rehabilitation units treating children with TBI.


Variations in providing recommended care have been demonstrated across fields in medicine and are recognized as detriments to quality, cost-control and optimal patient outcomes.12, 13 This study represents the first attempt to examine variation in the quality of inpatient rehabilitation for children with TBI. The results indicate a tremendous opportunity for improvement in the care of these children and, we hope, in their outcomes.


We wish to thank the individuals at the institutions who participated in the study for their help.

Funding support: This work was supported by Grant Number R21 HD059049-01A1 from the Eunice Kennedy Shriver National Institute of Child Health & Human Development.

Role of the sponsor: The sponsor had not role in the design and conduct of the study; in the collection, analysis and interpretation of data; or in the preparation, review, or approval of the manuscript.


Traumatic brain injury
Commission on the Accreditation of Rehabilitation Facilities
Institutional review board
Glasgow Coma Scale
Observed difficulty of delivery


Reprints are not available

Suppliers: no devices are referenced in this manuscript

Conflict of interest disclosures

We certify that no party having a direct interest in the results of the research supporting this article has or will confer a benefit on us or on any organization with which we are associated AND, if applicable, we certify that all financial and material support for this research and work are clearly identified in the title page of the manuscript.

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.


1. Donabedian A. Evaluating the quality of medical care. Milbank Mem Fund Q. 1966;44(Suppl):166–206. [PubMed]
2. Fountain NB, Van Ness PC, Swain-Eng R, Tonn S, Bever CT., Jr Quality improvement in neurology: AAN epilepsy quality measures: Report of the Quality Measurement and Reporting Subcommittee of the American Academy of Neurology. Neurology. 2011;76:94–9. [PubMed]
3. Brook RH, McGlynn EA, Shekelle PG. Defining and measuring quality of care: a perspective from US researchers. Int J Qual Health Care. 2000;12:281–95. [PubMed]
4. Brook RH, McGlynn EA, Cleary PD. Quality of health care. Part 2: measuring quality of care. N Engl J Med. 1996;335:966–70. [PubMed]
5. Adelson PD, Bratton SL, Carney NA, et al. Guidelines for the acute medical management of severe traumatic brain injury in infants, children, and adolescents. Chapter 1: Introduction. Pediatr Crit Care Med. 2003;4:S2–4. [PubMed]
6. Zumsteg JM, Ennis SK, jaffe KM, Mangione-Smith R, MacKenzie EJ, Rivara FP. Quality of care indicators for the structure and organization of inpatient rehabilitation care of children with traumatic brain injury. Achives of Physical Medicine and Rehabilitation. 2011 In Press. [PMC free article] [PubMed]
7. Rivara FP, Ennis SK, Mangione-Smith R, MacKenzie EJ, Jaffe KM. Quality of care indicators for the rehabilitation of children with traumatic brain injury. 2011 Submitted for publication. [PMC free article] [PubMed]
8. Mangione-Smith R, DeCristofaro AH, Setodji CM, et al. The quality of ambulatory care delivered to children in the United States. N Engl J Med. 2007;357:1515–23. [PubMed]
9. Slomine BS, McCarthy ML, Ding R, et al. Healthcare utilization and needs following pediatric traumatic brain injury. Pediatrics. 2006;117:e663–74. [PubMed]
10. O’Callaghan AM, McAllister L, Wilson L. Experiences of care reported by adults with traumatic brain injury. Int J Speech Lang Pathol. 2010;12:107–23. [PubMed]
11. White RF, James KE, Vasterling JJ, et al. Interrater reliability of neuropsychological diagnoses: a Department of Veterans Affairs cooperative study. J Int Neuropsychol Soc. 2002;8:555–65. [PubMed]
12. Asch SM, McGlynn EA, Hiatt L, et al. Quality of care for hypertension in the United States. BMC Cardiovasc Disord. 2005;5:1. [PMC free article] [PubMed]
13. Mangione-Smith R, Schonlau M, Chan KS, et al. Measuring the effectiveness of a collaborative for quality improvement in pediatric asthma care: does implementing the chronic care model improve processes and outcomes of care? Ambul Pediatr. 2005;5:75–82. [PubMed]