|Home | About | Journals | Submit | Contact Us | Français|
To calculate the intracluster correlation coefficient (ICC) for emergency department (ED) shock rate, early trauma death (ie, death during the first 24 h after arrival at hospital), and in‐hospital trauma death rate for multicenter childhood injuries.
The National Trauma Data Bank (5th revision), the largest multicenter trauma registry in the US, was used. Data from 80 trauma centers were used to calculate the ICC for in‐hospital trauma death rate. Thirty three states provided data for calculation of the ICC for ED shock and early trauma death rate.
From 2000 to 2004, 13% of the 952242 patients in the National Trauma Data Bank were <15 years old. Approximately 17000 of these children had injuries with an injury severity score >15, of whom 84% (14095 subjects) were hospitalized at 80 level I or II trauma centers in 33 states. The ICCs for ED shock rate, early trauma death rate, and in‐hospital death rate were 0.005 (95% CI 0.000 to 0.010), 0.014 (95% CI 0.004 to 0.024), and 0.023 (95% CI 0.013 to 0.033), respectively. These ICCs were calculated for boys and girls and also for blunt and penetrating injuries.
Clustered childhood trauma studies that aim to compare different aspects of pre‐hospital and hospital trauma care should incorporate these ICCs for sample calculation. When cluster randomized clinical trials are mounted, if sample sizes are calculated without adjustment for ICC, then the planned trial is likely to be seriously underpowered.
Trauma registries are valuable tools for monitoring quality of care and can be used to evaluate the effectiveness of injury prevention programs.1 For example, Gentilello et al2 used data from a local trauma registry to evaluate the potential influence of brief alcohol intervention on hospitalization for alcohol‐related injuries. Trauma registries have also facilitated design and conduction of multicenter trauma studies. Roudsari et al3 used data from the local or national trauma registry of eight developed countries to evaluate the association between type of pre‐hospital care and patient outcome in developed countries.3
Multicenter studies, in general, have larger sample sizes which make detection of the small but important differences in outcomes more likely. Furthermore, they may have better generalizability, as subjects are often recruited from heterogeneous settings. However, what is often not considered in multicenter studies is that patients cared for in the same centers (or regions) share attributes related to exposures or outcomes that are different from those of patients cared for in other centers or regions. As a result, patient outcomes may correlate strongly within clusters (defined as a center or region).4,5,6,7 This can be attributed to measured and unmeasured characteristics such as demography of the patients, type and mechanism of injury, and quality of pre‐hospital and in‐hospital care.8 As a result of this correlation, there is a lack of statistical independence across subjects. As independence is a critical assumption for sample size calculations and most other statistical analyses, it is likely that, in many cases, this assumption is violated.4,5,6,9 If each cluster comprises exposed and unexposed subjects, stratification for clusters can alleviate the clustering effect.5 However, if clusters are either all exposed or all unexposed, special methods should be adopted for the sample size calculation and data analysis.4,5,6,7 Under these circumstances, standard sample size formulas that assume that observations are independent estimate sample sizes that are artificially low10,11,12,13,14,15 and result in studies that are underpowered for detecting the desired level of difference between exposed and unexposed groups. Furthermore, pooling the data from different centers and using standard statistical methods—that is, ignoring the clustering design—may result in standard errors and p values that are spuriously low.5,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30
To demonstrate the potential influence of clustering design on sample size calculation and analysis in multicenter trauma research, Roudsari et al8 conducted a hypothetical study. They evaluated the association between policy on pre‐hospital intravenous fluid therapy and trauma fatality.8 The exposure of interest was the policy, which varied at the system level. As a result, patients in each emergency medical services (EMS) system were either exposed or not exposed to the policy. Therefore, to accomplish the objectives of the study, data from several EMS systems, some with and some without the policy, were required.8 The authors showed that ignoring the clustering design resulted in a sample size that was about seven times smaller than the actual sample size required to accomplish the objectives of the study. Furthermore, they showed that, in the analysis phase, not taking into consideration the clustering issue resulted in artificially narrow confidence intervals and mistaken rejection of the null hypothesis.
Intracluster correlation coefficient (ICC) is a measurement that captures between‐cluster and within‐cluster variability in outcome and is required for sample size calculation in clustered studies that comprise exposed or unexposed clusters.4,5,6,7 Equation (1) shows the formula for sample size calculation for clustered studies and its relationship to standard sample size formula4:
where n is the number of subjects in the exposed or unexposed group, Z is the Z score, α is the type I error, β is the type II error, P1 and P2 are the expected proportion of outcome in the exposed and non‐exposed group, respectively, and m0 is approximately the average number of subjects per cluster.
Part I in equation (1) is the formula used for sample size calculation in standard studies. Part II is called the “design effect” or “variance inflator factor”.4 As presented, the design effect depends on the average number of subjects per cluster and the ICC. In multicenter studies with a large number of subjects per cluster, even a small ICC can lead to a large design effect.
In this analysis, we used the US National Trauma Data Bank (NTDB)31 to calculate the ICC for three of the most commonly used outcomes of interest in acute trauma research in children <15 years old: emergency department (ED) shock, early trauma fatality (ie, death during the first 24 h after hospital arrival), and in‐hospital trauma fatality (death before discharge). We have hypothesized that, owing to higher variability of in‐hospital care compared with pre‐hospital care, the ICC for in‐hospital trauma fatality should be higher than those for early trauma death and ED shock rate.8
We used the American College of Surgeons' NTDB (version 5.0 released in November 2005).32 The NTDB is the largest trauma registry in the US and is based on the voluntary registry of hospitalized trauma patients admitted to American College of Surgeons‐verified, state‐designated, or self‐designated trauma centers.32 Hospitals submit their de‐identified trauma registry data to the NTDB. NTDB authorities are responsible for controlling the quality of the submitted data. Because participating in this national data registry is voluntary, larger hospitals with greater resources are more likely to contribute data. As a result, the NTDB is not nationally representative of all trauma centers throughout the US. Although hospital and city level identifiers are submitted to the NTDB, this information is not released in order to protect hospitals' identities. However, to allow researchers to differentiate between states and hospitals, the NTDB provides random identification numbers that can be used to cluster patients within hospitals and hospitals within states.
In 2005, 567 trauma care facilities from 43 states and US territories, including 123 level I and 123 level II trauma centers contributed data to the NTDB. Level I and II trauma centers are tertiary trauma care facilities that are capable of providing the highest level of care to trauma patients.33 The main difference between level I and II trauma centers is the training responsibilities that level I trauma centers have. In the presence of a local level I trauma center, level II trauma centers do not have such a responsibility.33
In general, all records of hospitalized trauma patients with ICD‐9CM codes of 800–959 are eligible to be included in the NTDB, except those with late effects of trauma, such as wound complication and late limb amputation, with ICD‐9CM codes of 905–909.32 Comprehensive information on demographic characteristics of patients and pre‐hospital and hospital care is included in the NTDB. For this analysis, we focused on children (<15 years old) with severe injuries (injury severity score34 (ISS) >15) who were directly transported from the scene of injury to a level I or II trauma center. We used the following variables from the NTDB: age, gender, mechanism of injury, ISS, ED blood pressure (the first blood pressure recorded in the ED), mortality status at discharge, random identification numbers for the participating hospitals and states, and trauma center designation level.
Two sets of outcome were used for evaluation of pre‐hospital or hospital trauma care: ED shock rate (blood pressure <90 mm Hg at the time of arrival in the ED) and early trauma death (death during the first 24 h after arrival at the hospital) are reflective of the quality of care that patients receive in the pre‐hospital setting and are less confounded by the quality of hospital care.8 However, hospital fatality after the first 24 h of arrival at the hospital is mostly influenced by the quality of hospital care.
To estimate the ICC for in‐hospital trauma fatality, we treated each trauma center as a separate cluster. This can be explained by the potential differences in the quality of care among trauma centers even within the same geographical area. To increase the precision of our estimates, we used information from level I and II trauma centers that had contributed data from more than 50 severely injured children. Therefore, only 80 of the 246 level I and II trauma centers were eligible to be included in this analysis.
ED shock and early trauma death, however, are affected to some extent by the quality of pre‐hospital care that is provided by the local EMS systems. Hospitals within the same region are often served by the same EMS system, calling for clustering at city level. However, the NTDB does not provide any city level identifier that can be used for clustering trauma centers by cities. The only other variable that could be used for clustering trauma centers based on the region is the state. As a result, we considered hospitals within the same state as a single cluster. Because of this state‐level clustering, we ended up with 33 clusters that provided data for calculation of the ICC for ED shock and early trauma death rate.
We used Stata 9.1 (Stata Corporation, College Station, Texas, USA) for data analysis. We used the “loneway” command in Stata, which calculates ICC for studies with equal and unequal numbers of subjects per cluster. Each ICC is presented with its 95% CI. The mathematical calculations behind the ICC calculation are summarized in the Appendix.
From 2000 to 2004, 13% of the 952242 patients in the NTDB were children <15 years old. About 17000 of these children had severe injuries (ISS>15), of whom 84% (14095 subjects) were hospitalized at a level I or II trauma center.
Sixty percent of the children were male, and the proportion of male patients varied from 0.33 to 0.67 among different centers. The mean ISS varied from 18 to 27 at the state level and 16 to 38 at the hospital level. Blunt trauma comprised 78–100% of the injuries across trauma centers.
Data from 80 level I and II trauma centers were used to calculate the ICC for trauma fatality rate, and 33 states provided data for calculation of the ICCs for ED shock rate and early trauma death rate. Table 11 summarizes the ICCs for male and female patients and blunt and penetrating injuries. As presented, in general the ICC for ED shock rate (0.005, 95% CI 0.000 to 0.010) was less than the ICC for in‐hospital trauma fatality rate (0.023, 95% CI 0.013 to 0.033). A similar pattern was observed when we compared the ICCs of ED shock and in‐hospital trauma fatality for male and female patients, and also for blunt and penetrating injuries, separately. For these four categories, the ICC of ED shock rate was lower than the ICC of in‐hospital trauma fatality. The ICC for early trauma death was between the ICCs for ED shock and in‐hospital fatality, regardless of sex or type of injury (table 11).
ICC reflects the variability in outcome within and between clusters. Ideally, to estimate the sample size for a clustered study, the ICC for the outcome of interest should be calculated on the basis of clusters in the same population. However, owing to the large number of clusters required for ICC calculation, researchers often borrow ICCs from studies in other communities or from studies that comprise data from several communities. Under these circumstances, researchers should be aware of the potential differences in study settings that might influence the size of the ICCs. For instance, if between‐cluster and within‐cluster variability in outcome of interest in community A is smaller than the observed variabilities in other communities, the borrowed ICC is most likely a conservative estimate. As a result, the study in population A will have an adequate sample size to evaluate the association between the exposure and the outcome of interest. However, if the between‐cluster and within‐cluster variability in community A is larger, the borrowed ICC might be an anti‐conservative estimate. Therefore, the study in community A might not have enough samples to accomplish its objectives.
In this study, we provided real estimates of ICCs for three important outcomes of interest in research into acute childhood injuries. Even with the small size of an ICC, in the presence of a large number of subjects per cluster, small ICCs result in large design effects.
Comparing pre‐hospital with in‐hospital trauma care, we expected to observe a lower ICC for outcomes related to pre‐hospital care compared with those related to in‐hospital care. The primary reason for this is the highly protocolized care by EMS regions and the relatively few treatment modalities offered4 compared with the wide variation in hospital care across centers.
The only other study that has reported the ICCs for acute trauma research is one conducted by Roudsari et al8 which calculated the same ICCs for patients 15 years old. Figure 11 compares the ICC of ED shock, early trauma death, and hospital fatality comparing children with patients 15 years old.
First, participation in the NTDB is voluntary for the trauma care facilities. Therefore, it is not nationally representative of all trauma care facilities in the US. Owing to the involvement of trauma centers from different parts of the country in the NTDB, the between‐cluster variability in the outcomes of interest is probably larger than in any other study that uses trauma centers from a single state. As a result, this larger variability in the NTDB inflates the calculated ICCs. Therefore, we expect that the studies that use the calculated ICCs in our study for sample size calculation will have sufficient sample sizes. Secondly, often a single EMS system is responsible for pre‐hospital care in a city, and all trauma patients hospitalized in that city are exposed to a similar level of pre‐hospital care. Therefore, to calculate the ICC for ED shock rate, which is mainly influenced by the quality of pre‐hospital care, we would prefer to cluster trauma centers at the city level. However, to protect hospitals' identities, no city‐level identifier was included in the NTDB. As a result, considering state instead of city as a clustering level increases the variability within clusters, whereas variability among clusters that could inflate or deflate the ICCs of ED shock rate and early trauma fatality might increase or decrease. However, in spite of these limitations, the NTDB provides a unique opportunity to estimate ICCs for the outcomes of interest using a large number of clusters at both the hospital and state level.
We have provided a real estimate of the ICC for three important outcomes of interest in acute childhood injury research. These estimates should be incorporated into sample size calculations for any multicenter trauma research that uses exclusively exposed or unexposed clusters of patients.
ED - emergency department
EMS - emergency medical services
ICC - intracluster correlation coefficient
ISS - injury severity score
NTDB - National Trauma Data Bank
Statistical approaches to calculating ICCs using the NTDB
We used equation (1) to calculate ICCs for the outcomes of interest4:
where denotes the proportion of the observed outcome in the jth cluster; is the average number of subjects in each cluster, and mj is the number of the subjects in cluster j. is the average proportion of the observed outcome in the data and is calculated from the following formula:
If the number of subjects per cluster varies, equation (2) is used to substitute by m0.
Competing interests: None.