|Home | About | Journals | Submit | Contact Us | Français|
To develop evidence-based and expert-driven quality indicators for measuring variations in the structure and organization of acute inpatient rehabilitation for children after traumatic brain injury (TBI) and to survey centers across the United States to determine the degree of variation in care.
Quality indicators were developed using the RAND/UCLA modified Delphi method. Adherence to these indicators was determined from a survey of rehabilitation facilities.
Inpatient rehabilitation units in the United States.
A sample of rehabilitation programs identified using data from the National Association of Children’s Hospitals and Related Institutions, Uniform Data System for Medical Rehabilitation, and the Commission on Accreditation of Rehabilitation Facilities yielded 74 inpatient units treating children with TBI. Survey respondents comprised 31 pediatric and 28 all-age units.
Variations in structure and organization of care among institutions providing acute inpatient rehabilitation for children with TBI.
Twelve indicators were developed. Pediatric inpatient rehabilitation units and units with higher volumes of children with TBI were more likely to have: a census of at least one child admitted with a TBI for at least 90% of the time; adequate specialized equipment; a classroom; a pediatric subspecialty trained medical director; and greater than 75% of therapists with pediatric training.
There were clinically and statistically significant variations in the structure and organization of acute pediatric rehabilitation based on the pediatric focus of the unit and volume of children with TBI.
Traumatic brain injury (TBI) is the most common cause of death from trauma in children and adolescents and one of the most common causes of acquired disability.1 Rehabilitation has been viewed as critical to achieving optimal recovery after serious trauma, especially TBI.2 While advancing rehabilitation research has many considerations, analysis of service delivery and organization is an important part of advancing care.3, 4 This approach is congruent with Donabedian’s structure-process-outcome model that has been essential to the understanding of concepts such as care delivery systems, patient outcomes and quality.5 In the delivery of rehabilitation care the overall structure and organization of care delivered are reflected in resources such as the physical space and equipment as well as the number, type and focus of personnel and services available for patient care.6 However, there has been limited progress in determining how the organization of rehabilitation and the processes constituting that care are related to functional outcomes of patients, especially children and adolescents with TBI. An initial critical barrier to advancement in this area is a lack of standardized measurement of the structure, organization and process of care, including limited availability of valid measurement criteria for quality of care in the acute inpatient rehabilitation setting.7, 8
Advancements in understanding the process and organization of care are important steps in understanding how to further optimize patient outcomes. The organization of medical care has been shown to have a major impact on outcomes in other areas, including persons who have sustained trauma.9–11 By better understanding the structure and processes of care we can move toward recommendations regarding systems that work to optimize patient outcomes. In this study, we sought to develop feasible, evidence-based and expert-driven measures for documenting the structure and organization of care among institutions providing inpatient rehabilitation services for the population of children with TBI and examining current variations in this structure and organization among a sample of such institutions in the United States. The development of quality of care indicators for the process of acute inpatient pediatric rehabilitation is reported in an accompanying paper.
A comprehensive literature review was conducted to identify evidence for specific factors related to the structure and organization of rehabilitation using the online databases PubMed (1948–2009), CINAHL Plus (Cumulative Index to Nursing and Allied Health Literature;1937–2009) and HSRProj (Health Services Research Projects in Progress; 1974 – 2010) without any search limitations. Searches were performed to select articles indexed under the major medical subject headings “Rehabilitation” and “Brain Injury” combined with any of the search terms listed in Table 1. The search strategy was designed with a broad scope to include all relevant articles, and therefore initial search results were not confined to a pediatric focus. Current international clinical guidelines for pediatric brain injury rehabilitation known to the authors were also examined.12–16 Articles retrieved from these search strategies were reviewed by a single research coordinator to extract quality indicators and to establish the highest level of evidence supporting each quality indicator using the criteria of the Oxford Centre for Evidence-Based Medicine.17 Initially 18 quality indicators were drafted addressing factors related to the structure and organization of pediatric rehabilitation care.
A panel composed of nine national experts was selected from a list of nominations in an appropriate consistent with the RAND/UCLA Appropriateness Method (RAM); this approach includes a Delphi process18, 19. Selection criteria were the panelists’ national recognition for clinical and/or research expertise in traumatic brain injury, leadership in the field of pediatric physical medicine and rehabilitation, and overall balance of diversity in geographic and practice settings. Panelists received electronic versions of the quality indicators, the supporting literature review, rating sheets, and instructions for completing the rating process. Each quality indicator was presented with its appraised quality of evidence, relevant literature citations, benefits associated with its use, and a brief summary of current practice guidelines or supporting evidence.
Standard RAM methods were employed to determine the appropriateness of the quality indicators using a two-round process.18, 19 In the first round, the expert panel members independently scored the validity and feasibility of each quality indicator on an ordinal nine-point scale (with nine being the highest). Validity was assessed by the panelists based on: (1) the scientific evidence or, when lacking, the collective expert opinion of the appropriateness of the indicator; and (2) the perceived health benefits to patients.20 Previous work has shown this method of rating indicators to be reliable and to have strong construct, content, and predictive validity.21–23
Measures of central tendency (median) and dispersion (mean absolute deviation from the median) subsequently were calculated for each indicator, and panelists were sent a summary sheet comparing their individual ratings to the anonymous distribution of ratings from the entire panel. In accordance with RAM guidelines, indicators with median ratings in the top third of the validity scale (score of seven to nine) were classified as appropriate, those with median ratings in the bottom third (score of one to three) were classified as inappropriate and omitted, and those with intermediate median ratings (score of four to six) were considered uncertain.18 In advance of the second rating round, panelists convened by phone to discuss any quality indicators classified as uncertain or with a dispersion measure greater than one on either scale. The goal of this phone discussion was not to develop a consensus, but rather to allow panelists to share opinions, identify any confusing wording, and suggest additions, omissions or modifications to the reviewed indicators. Modifications recommended by the panelists were incorporated, and panelists independently re-rated the validity and feasibility of the entire set of quality of care indicators immediately following the phone discussion using the same nine-point scale.
In the second round, an indicator was considered to be an appropriate measure of quality care if it received a median validity rating between seven and nine and a median feasibility rating between four and nine.18 After the second round of expert panel ratings, six of the original quality indicators were omitted, seven were modified, and five remained unchanged. One of the original panelists dropped out after the first round of ratings; thus, the final set of indicators was based on eight sets of ratings.
The final set of indicators was selected using RAM procedures for conducting tests of hypotheses about the distribution of ratings in a hypothetical population of repeated ratings. Ratings were classified as “with agreement” or “with disagreement” using standard definitions developed for use with panels of fewer than nine members.18, 19 In order to be classified as “with agreement” no more than two panelist ratings could fall outside the tertile containing the median score (seven to nine). An indicator was classified as “with disagreement” when three or more panelist ratings fell between one and three on the rating scale and three or more fell between a score of seven and nine. Any indicator classified as “with disagreement” was rejected from the final set.
A twelve-question survey was developed from the final set of quality indicators and piloted on a convenience sample of three rehabilitation facilities of different patient volumes and structures (pediatric focus and all ages units) to test feasibility of use. Feedback confirmed that the survey could be completed within 15 minutes, that the requested information was readily available to the inpatient rehabilitation or therapy manager on the unit, and that sufficient response categories were available to capture all possible responses. The final survey questions based on the quality of care indicators for the structure and organization of rehabilitation of children with TBI are available in Appendix A online.
Institutional Review Board approval was obtained to conduct the survey. A list of 144 rehabilitation institutions identified as admitting children was obtained from the Uniform Data System for Medical Rehabilitation (UDS), Amherst, New York, the Commission on Accreditation of Rehabilitation Facilities (CARF), Tucson, Arizona and the National Association of Children’s Hospitals and Related Institutions (NACHRI), Alexandria, Virginia. The above three sources of rehabilitation institutions were utilized to ensure a comprehensive list of possible participating programs. The medical director of inpatient rehabilitation at each institution was notified via mail to explain the purpose of the study and to solicit data on the number of children (defined as age 18 years and under) admitted to the institution for inpatient rehabilitation after TBI in each of the three calendar years preceding the survey (2007–2009). Twenty-two institutions were excluded for failing to offer inpatient rehabilitation services, for not admitting individuals under age 18, or for having no admissions of children with TBI in the last three years. We obtained volume data from 74 of the remaining sites. Institutions were divided into tertiles based on patient volume data and categorized by pediatric focus (40 pediatric versus 34 all age units) to create six strata. Pediatric focus was defined as a pediatric-specialty rehabilitation unit either within a children’s hospital or within a freestanding rehabilitation hospital. The full structure of care surveys were distributed via email to a rehabilitation administrator or manager identified by the medical director of each institution. Non-responders were contacted by phone to achieve a survey response rate of 81%, considered to be acceptable based on previous survey research.24, 25
Descriptive statistics (proportion or mean, and standard deviation where appropriate) were utilized to summarize the responses to each survey question and the general characteristics of the rehabilitation units. Statistical tests were performed using SAS 9.2 software (SAS Institute Inc., Cary, North Carolina). Chi-squared tests were applied to look for statistically significant differences between quality indicators based on pediatric focus and volume tertiles, and a Poisson regression was performed to examine the independent effects of these two variables on the outcome of the total number of indicators met.
The final indicators and criteria for meeting each indicator are listed in Table 2. A total of 12 indicators were developed encompassing the pediatric focus of a program, volume, family accommodations, equipment and services, and availability and training of personnel. Scores range from 0–12, with some indicators allowing partial credit.
Quality care of children with TBI was felt to best occur in a pediatric setting where this clinical diagnosis was not uncommon. While the ideal frequency is unclear, in the survey described above the threshold of not admitting a pediatric patient with TBI in the last three years was used as exclusion criteria. As with all pediatric care, accommodations for family members should be available. Age appropriate equipment should be available including a classroom for educational programs. Personnel should be trained specifically in pediatric rehabilitation, and appropriate ancillary services should be available.
We examined differences between all-ages units and pediatric units in relation to the quality indicators. As can be seen in Table 3, the majority of pediatric specialty units treated 20 or more children with TBI per year and had at least one child with TBI on the unit greater than 90% of the time, compared to relatively few of the all age units (p=0.0002). Pediatric specialty units were more likely to have a policy governing the ratio of nurses to patients (p=0.02), but had similar numbers of primary and ancillary services as the all age units. All of the latter admitted adult patients as well as children, while 26% of the pediatric specialty units also admitted persons older than 18. Data about the upper age limit of patients admitted to pediatric specialty units was not collected.
All units provided accommodations for family. There was more children-specific specialized equipment available in pediatric specialty units (p<0.0001). There were differences in the availability of a classroom, as only one pediatric unit did not have a classroom, while only two of the all age units had a classroom.
There were marked differences in staff expertise between the different types of units, as shown in Table 3, with pediatric specialty units having more pediatric trained staff (p=0.04). All age units, however, were more likely to have a certified rehabilitation registered nurse (CCRN) on staff than were pediatric units (p=0.014). All age units were also more likely to be accredited by CARF than pediatric units (p=0.03).
There were significant differences in the number of quality indicators satisfied by units with different patient volumes (Table 4). For nearly all indicators, centers with a higher annual volume of children with TBI had more resources, including classrooms and ancillary services, and more pediatric trained professional staff than did lower volume centers. Higher volume centers were more likely to be based in children’s hospitals, and less likely to admit those over age 18. There were no differences by volume in the likelihood of being a free-standing rehabilitation facility.
Looking first at broad groups, the mean number of indicators met for pediatric units was 9.1 (SD=1.9), while the mean number of indicators met for all ages units was 5.0 (SD=1.8). Dividing results based on percentage of criteria met, of the 31 pediatric units surveyed, two met all 12 criteria, 19 met at least 75% of the criteria, and only one unit did not pass at least half of the quality indicators. Comparatively, of the 28 all ages units surveyed, none passed all quality indicators, only one met at least 75% of the indicators, and 9 units met at least 50% of the criteria.
Using Poisson regression, having a pediatric focus and admitting a higher volume of children with TBI were independently associated with the total number of quality indicators met (Table 5). All ages units met an average of 5.0 indicators compared with 9.1 reported by pediatric specialty units, adjusted for volume (RR, 0.68, 95% CI: 0.53,0.88). Hospitals admitting an intermediate volume (8–19 per year) of children with TBI met an average of 7.2 indicators compared with 5.0 reported by low volume institutions (1–7 patients per year) (RR 1.19, 95% CI: 0.88, 1.60); hospitals admitting the highest volume (20 or more children per year) of children with TBI met 9.5 indicators compared with the 5.0 met by low volume hospitals, adjusted for the children’s focus of unit (RR 1.56, 95% CI: 1.15, 2.11).
The goals of the current study were (1) to develop valid and feasible quality indicators of care for children with TBI and (2) to survey a sample of rehabilitation units in the United States to measure the quality of the structure and organization of acute inpatient rehabilitation being delivered currently to children with TBI. Substantial variations in the quality of structure and organization of rehabilitation delivered to children after TBI were identified, with volume of children with TBI and the pediatric focus of the unit emerging as important determinants of the total number of quality indicators met.
Structure of care, encompassing material and human resources as well as the organization of these resources, is one of the key elements of the structure-process-outcome framework proposed by Donabedian26 and is recognized as a useful quality assessment system for establishing, monitoring and improving the current state of post acute care.6 Hoenig and colleagues applied this model to evaluate post-stroke rehabilitation, demonstrating that structural characteristics independently predicted processes of care (e.g., family involvement or multidisciplinary care coordination), which in turn influenced six-month functional outcomes.27 The findings from our study provide a measure of the current structure of rehabilitation for children, a critical step toward understanding the complex interactions between structure and processes of care for this population.
The finding that pediatric specialty units were more likely to meet a higher number of quality indicators, such as having a higher percentage of staff with pediatric-specific training, is consistent with extant literature on the importance of children’s-specific care structures for the acute treatment of traumatic injuries and critical illness in children. 28–30 A number of studies suggest that children admitted to pediatric specific trauma centers have lower mortality rates and improved overall functional outcomes compared to those children treated at trauma centers without pediatric specific resources.29, 31 The influence of a pediatric rehabilitation unit on patient outcomes has not been explored, and tools for categorizing and measuring the structure of rehabilitation previously have been lacking.
Pediatric specialty units unexpectedly were found to be less likely to have a certified rehabilitation registered nurse on staff. While the underlying factors for the difference in CRRN representation is not clear, barriers may include exposure to the field, certification and experience requirements.32 The pediatric specialty units in this study also were less likely than all age units to report CARF accreditation, and CRRN certification is strongly encouraged by CARF. On the other hand, pediatric units were more likely to have nursing to patient ratio policies in place, and higher staffing levels have been associated with lower rates of patient mortality and adverse events, improved nursing job satisfaction and higher patient satisfaction in hospital settings.33–36
Units with a pediatric focus were also better structured to deliver services to children given the availability of child-specific equipment and classrooms to continue educational services during the inpatient rehabilitation stay. These services may be important indicators of other components of care structure or quality of care. For example, Hoenig and colleagues demonstrated that VA hospitals with rehabilitation-specific structures, including a higher diversity (number and type) of specialized rehabilitation equipment or a simulated home environment, had longer lengths of stay for adults post-stroke.37 The specialized units studied, while having a longer length of stay, also were 91% more likely to have patients discharged to home rather than to a skilled facility. It is likely that such structural characteristics influence functional outcomes indirectly via the interactions between structure and processes of care. For instance, a classroom permits a careful and functional assessment of a child’s needs and a realistic trial of school to be employed in a natural setting, thereby optimizing the likelihood of a successful and timely school reintegration.
Another important finding in our study was that rehabilitation units admitting a high volume of children with TBI met a higher number of quality indicators when compared with hospitals of intermediate and low volumes. For many medical services, including trauma care and surgical procedures, increased volume of patients often is associated with better quality of care.9, 38–40 For example, a strong association has been documented between high volume trauma centers and improvements in mortality and length of stay in patients at high risk for adverse outcomes.38 An association between institutional volume and recovery from brain injury has not been investigated to date, and future work should focus on examining the association of high volume rehabilitation centers with positive functional outcomes in children with TBI.
This multi-step study has potential limitations at each phase of the process, including the literature review, panel, and survey. While the literature review search strategy included multiple databases and a broad scope with the goal of being inclusive, we may still have missed potentially useful references. In the panel phase of the process we need to consider the use of ratings from eight panelists to develop the quality of care indicators rather that the intended nine. However, RAM publications indicate that panels can be of any size that permits sufficient diversity while ensuring that all have a chance to participate (a minimum of seven and a maximum of 15), and it is not uncommon to have at least one individual fail to complete or submit their ratings.19 The RAM process as applied here sought to utilize evidence-based guidelines and panel consensus. Accordingly, there are likely areas related to the quality indicators where readers may disagree with the elements such as phrasing used, category specificity (for example psychology versus neuropsychology), and elements not included in the current analysis.
In the design of this survey we excluded units that had not admitted a child with a TBI in the last three years and acknowledge that data from those units may have been different from the included sample. We did not weigh the importance of each included item because of the lack of adequate data on outcomes. There may have been response bias in the rehabilitation units that elected to participate in or complete the survey that could have resulted in either over- or underestimation of the differences between units. Additionally, given the nature of the survey, facilities self-report of available services lends information on how services may be structured. For example, there is likely variability across units in how and by whom vision is assessed. There are also likely important quality measures that are not reflected in our indicators either because evidence is currently lacking and/or their validity or feasibility for inclusion in a survey tool was not clear. The Donabedian model for examining quality of care by identifying the structural and process elements of care and determining their effects on outcome rests on the availability of valid measures of these elements of care. The structure of care elements described here have face validity by virtue of the way they were developed. Other measures of validity will need to be tested in future studies as will the comprehensiveness and applications of these indicators.
There is significant variation in the structure and organization of care and services available during inpatient rehabilitation for children who have sustained a TBI. These differences appear to be largely driven by patient volume and pediatric focus of the rehabilitation unit. Prior to these findings, there were no available instruments for measuring the quality of rehabilitation care for injured children. The next step is to apply the measures established in this study to determine if rehabilitation programs meeting more quality indicators also demonstrate higher quality processes of care delivery and, directly or indirectly, better outcomes of care.
This work was supported by Grant Number R21 HD059049-01A1 from the Eunice Kennedy Shriver National Institute of Child Health & Human Development.
No reprints will be available.
We certify that we have affiliations with or financial involvement (eg, employment, consultancies, honoraria, stock ownership or options, expert testimony, grants and patents received or pending, royalties) with an organization or entity with a financial interest in, or financial conflict with, the subject matter or materials discussed in the manuscript AND all such affiliations and involvements are disclosed on the title page of the manuscript.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.