|Home | About | Journals | Submit | Contact Us | Français|
The level of parent-child agreement on post-concussive symptoms (PCS) was examined in children following mild traumatic brain injuries (TBI). As part of a larger longitudinal study, 186 children with mild TBI and 99 with orthopedic injuries (OI), from 8 to 15 years of age, were recruited prospectively. Parents and children completed the PCS Interview (PCS-I) and the Health and Behavior Inventory (HBI) at 2 weeks, 1 month, 3 months and 12 months post-injury. Item-level correlations between child and parent ratings on both measures of PCS were significant but modest in both groups. Parent-child correlations for composite scales on the HBI and the total score on the PCS-I were significant in both groups, but somewhat higher in the OI group than in the mild TBI Group. Mean symptom ratings tended to be significantly higher for children as compared to parents, especially for somatic symptoms. Parents and children display modest agreement when reporting PCS; their ratings correlate significantly, but children report higher mean levels of symptoms than parents.
Traumatic brain injury (TBI) affects approximately 500,000 children ages 0–14 annually in the United States (Langlois, Rutland-Brown, Thomas, 2006). The vast majority of TBI are of mild severity, and mild TBI therefore represents a potential public health concern (Bazarian et al., 2005). Mild TBI is frequently associated with cognitive, somatic, and emotional post-concussive symptoms (PCS), even in the absence of abnormalities on neuroimaging (Taylor et al., 2010; Yeates et al., 2009). Common symptoms include headache, fatigue, dizziness, poor concentration, forgetfulness, irritability, and insomnia. The etiology of PCS is a topic of ongoing debate, often framed in terms of psychogenesis versus physiogenesis (Bigler, 2008; Bigler & Snyder, 1995; Chen et al., 2007; Ryan & Warden, 2003; Yeates et al., 2009).
Debate also continues as to whether persistent PCS form a distinct syndrome. The diagnosis of post-concussion syndrome is included in the International Classification of Diseases (ICD-10; World Health Organization, 1992), and research criteria for post-concussional disorder are included in the fourth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV; American Psychiatric Association, 1994). However, the DSM-IV and ICD-10 have different diagnostic criteria, which reflect different assumptions about the etiology of PCS, result in different incidence estimates, yield limited diagnostic agreement, and may not be specific to TBI (Boake et al., 2004, 2005; Yeates & Taylor, 2005).
One criterion for establishing a syndrome is the ability to assess it reliably. Inter-rater agreement in symptom reports is one important aspect of reliability (Gioia, Schneider, Vaughan, & Isquith, 2009). Inter-rater agreement between parent and child symptom reports is often modest at best. In a meta-analysis of 119 studies examining behavioral and emotional problems in children, the average correlation between children and parents was .25 (Achenbach, McConaughy, & Howell, 1987). Parent-child agreement is likely related to a variety of factors that may differentially affect parent and child responses (Choudhury, Pimentel, & Kendall, 2003; Karver, 2006). Certain characteristics of the target behavior may be influential, such as its consistency across settings and saliency to informants. Thus, parent-child agreement is typically higher for externalizing symptoms such as aggression than for internalizing problems such as anxiety and depression (Edelbrock, Costello, Dulcan, Conover, & Kala, 1986; Hodges, Gordon, & Lennon, 1990). Comer and Kendall (2004) found higher parent-child agreement for observable, family-based, non-school based, and socially acceptable symptoms. Characteristics of the reporter are also likely to be important, such as the ability to recall behaviors and willingness to report (Karver, 2006). Lower agreement may result from parents’ unawareness of certain symptoms, or from the unwillingness or inability of children to verbalize particular symptoms (McCrea, Hammeke, Olsen, Leo, & Guskiewicz, 2004).
Distinctions between symptom types may also be relevant to studies of inter-rater agreement in PCS following mild TBI. Ayr, Yeates, Taylor, and Browne (2009) found distinct somatic and cognitive symptom dimensions that were robust across raters and time post injury. Parent and child agreement in their reports of PCS might differ for cognitive versus somatic symptoms. However, parent-child agreement regarding PCS has not been examined in detail in children with mild TBI.
Agreement can be examined in terms of both group means and correlations, which are independent of each other, and at both item and composite scale levels. The limited existing research on parent-child agreement regarding PCS has focused on correlations. In a recent review, parent-child agreement was moderate for composite ratings on the Post-Concussion Symptom Inventory, with correlations between raters (parent, teacher, child) ranging from .4 to .5 (Gioia et al., 2009). On the Health and Behavior Inventory (HBI), Ayr and colleagues (2009) found moderate correlations between parent and child ratings for both cognitive and somatic symptom dimensions at an initial assessment and 3 months post-injury. Neither of these studies examined mean differences between parent and child ratings, and they examined agreement only at the composite scale level, not at an item level.
Thus, the aim of the current study was to examine parent-child agreement on ratings of PCS in children following mild TBI. We used data drawn from the same larger prospective, longitudinal study that served as the basis for Ayr and colleagues’ (2009) recent paper. The study included children with mild TBI and a comparison group of children with orthopedic injuries (OI). We examined correlations between parent and child ratings, both at an item level and at a composite scale level, and compared mean parent and child ratings. Based on the existing literature, we expected correlations between parent and child reports of PCS to be significant in both groups, but to be modest in magnitude. Additionally, we expected children to report higher levels of symptoms than parents, given that some PCS may not be readily observable by parents.
We also compared parent-child agreement in the mild TBI group to that in the OI group, to determine whether agreement differed based on injury status. Agreement might be higher in the mild TBI group, because children in that group would be likely to display more variability in PCS. Finally, we determined whether parent-child agreement differed for somatic versus cognitive PCS. Our expectation, based on literature suggesting less parent-child agreement on symptoms that are less easily observed, was that agreement would be lower for somatic symptoms. This expectation was based on the assumption that cognitive problems would be evident in children’s performance of daily tasks, whereas somatic discomfort would be less directly observable.
Participants were recruited prospectively from two different sites, Nationwide Children’s Hospital in Columbus, Ohio and Rainbow Babies and Children’s Hospital in Cleveland, Ohio. In total, 285 children were recruited, including 186 children with mild TBI and 99 children with OI. The recruitment process involved an initial screening of all children from 8–15 years of age who were evaluated for blunt head trauma or OI in each hospital’s emergency department.
Children were eligible for the mild TBI group if they demonstrated an observed loss of consciousness at the time of injury, or a subsequent Glasgow Coma Scale (GCS; Teasdale & Jennett, 1974) score of 13–14, or at least two symptoms of concussion as reported by Emergency Department physicians. Symptoms of concussion included persistent post-traumatic amnesia, transient neurological deficits, vomiting, nausea, headache, visual disturbances, dizziness, disorientation, and other mental status changes. Children were excluded from the mild TBI group if they experienced a loss of consciousness over 30 minutes, hypoxia or shock in association with the injury, or received any GCS score below 13. Children were not excluded if they were hospitalized or demonstrated intracranial lesions or skull fractures on acute computerized tomography. Thus, the mild TBI group included injuries often described as “complicated” (e.g., those with intracranial abnormalities) but excluded injuries that would typically be defined as moderate in severity (Williams, Levin, & Eisenberg, 1990).
To be eligible for the OI group, children had to demonstrate an upper or lower extremity fracture with an Abbreviated Injury Scale (AIS) score of 3 or less (American Association for Automotive Medicine, 1990). The AIS is a widely-used scoring system that assesses the severity of injuries to specific anatomic regions on a scale of 1 to 6 (de Vries et al., 1999). Children were excluded from the OI group if they had any evidence of head injury, including external trauma or symptoms of concussion.
Exclusion criteria for both groups included injury-related surgery; any previous head injury requiring hospitalization; premorbid neurological disorders or severe psychiatric disorders requiring hospitalization; any other injuries with an AIS greater than 3; injuries that would hinder neuropsychological assessment; and injuries related to child abuse or drug or alcohol use. Children were not excluded for premorbid learning difficulties or attention problems. The mild TBI and OI groups did not differ on retrospective parent ratings of premorbid school performance or attention problems.
Participation rates (i.e., number of participants/number of individuals eligible and invited to participate) were 35% in the OI group and 48% in the mild TBI group. Participants and non-participants did not significantly differ in terms of age, sex, race/ethnicity, or census tract measures of socioeconomic status (i.e., mean family income, percentage of minority heads of household, and percentage of households below the poverty line). Table 1 presents demographic and injury information about the participants. The two groups did not differ on age, sex, race, socioeconomic status, or number of premorbid PCS as assessed by retrospective parent report. The mild TBI group displayed greater overall injury severity as measured by the Modified Injury Severity Scale, which is an overall index derived from AIS scores, calculated as the sum of the squares of the three most severely injured body areas (Mayer, Matlak, Johnson, & Walker, 1980). Recreational and sports-related injuries were the most common cause of injury (57% of mild TBI, 62% of OI), with falls the second leading cause (20% of mild TBI, 21% of OI). Transportation-related injuries accounted for significantly more mild TBI (17%) than OI (3%).
The research was approved by the appropriate institutional review boards, and informed parental consent and child assent were obtained in writing prior to participation. Participants completed an initial assessment no later than 3 weeks post injury, with 80% completed between 1 and 2 weeks post-injury (M = 11.35 days, SD = 3.42). As part of the larger study, parents and children reported current PCS at the initial assessment and again at 1, 3, and 12 months post injury.
Of 285 children who completed the initial assessment, 280 (98%) completed the assessment at 1 month post injury (183 or 98% with mild TBI, 97 or 98% with OI), 268 (94%) completed the 3 month assessment (178 or 96% with mild TBI, 90 or 91% with OI), and 253 (89%) completed the 12 month assessment (169 or 91% with mild TBI, 84 or 85% with OI). Children who completed all assessments did not differ from those who did not do so in age, sex, pre-injury symptoms, or early post-injury post-concussive symptoms, but were less likely to be of minority status and had higher socioeconomic status. The analyses reported here are based on all participants available at each assessment.
PCS were assessed using two measures, the Health and Behavior Inventory (HBI) and the Post-Concussive Symptom Interview (PCS-I) (Mittenberg, Wittner, & Miller, 1997; Yeates et al., 1999). The parent and child forms of both measures are worded slightly differently to be age-appropriate and to reflect first- versus third-person perspectives.
The HBI is comprised of 50 items covering somatic, cognitive, emotional, and behavioral symptoms. The frequency of each PCS is rated on a four point scale that ranges from 0 (“never”) to 3 (“often”). The HBI was developed based on prior research and has been used in previous studies of mild TBI (Yeates et al., 1999). Factor analyses have revealed two underlying dimensions of the HBI, representing cognitive and somatic symptoms, that are robust across time and raters (Ayr et al., 2009). Scores for each dimension, as well as the individual items constituting each dimension, were used in analyses to assess parent-child agreement. Composite scores on both dimensions showed good to excellent internal consistency at all occasions and for both parent and child ratings (Cronbach’s α from .83 to .95 for parents’ ratings and from .86 to .91 for children’s ratings).
The PCS-I is administered orally, and asks children and their parents to report the presence or absence of 15 PCS over the past week. The symptoms include cognitive, somatic, and emotional difficulties, similar to those listed in the research diagnostic criteria for postconcussional disorder in the DSM-IV (American Psychiatric Association, 1994) and for postconcussion syndrome in the ICD-10 (World Health Organization, 1993). The total score on the PCS-I equals the number of symptoms that are rated present. The total score, as well as the individual symptoms constituting the PCS-I, were used in analyses to assess parent-child agreement. The PCS-I has shown satisfactory reliability and validity in previous studies (Mittenberg et al., 1997). In the current study, the PCS-I demonstrated good internal consistency across all occasion and for both parent and child ratings (Cronbach’s α range from .78 to .82 for parent ratings; .70 to .77 for child ratings).
To examine parent-child agreement on the two measures of PCS, composite scale correlations and item-level agreement between parent and child reports were computed at each of the four post-injury time points. Composite scale correlations for the HBI and the PCS-I and item-level agreement for the HBI were calculated using Pearson r for the overall group and separately for the mild TBI and OI groups to determine if agreement varied between groups. Conventional guidelines suggest correlations of .50 are large in magnitude, .30 are medium, and .10 are small (Hemphill, 2003).
For the PCS-I, item-level agreement was assessed using the κ statistic for dichotomous data. The κ statistic accounts for chance agreement and is generally a more conservative measure of agreement than the Pearson r (Chambers, Reid, Craig, McGrath, & Finley, 1998). Kappa values above .80 represent “almost perfect” agreement, values from .50 to .60 represent “substantial” agreement, and values from .20 to .30 represent “moderate” agreement (Munoz & Bangdiwala, 1997).
We also tested for significant differences between the mild TBI and OI groups in composite-scale correlations, as well as within each group for differences in correlations for somatic versus cognitive symptoms. The Fisher’s z test was used to compare the mild TBI and OI group correlation coefficients. We used a modified version of the Fisher’s z test, developed by Raghunathan, Rosenthal, and Rubin (1996), to compare cognitive and somatic scale correlations, as well as correlations between adjacent assessments, within groups.
Finally, multivariate repeated measures analyses of variance were used to examine the difference between mean parent and child composite scale ratings longitudinally. Group (mild TBI versus OI), rater (parent versus child), and time post-injury were included as independent variables in the analysis, with the latter two factors treated as repeated measures.
Tables 2 through through99 contain the item-level means and correlations for parent and child ratings derived from the PCS-I and the HBI at each assessment. Agreement between child and parent ratings was significant but modest on both measures of PCS, in both groups. In the mild TBI group, across all four assessments, item-level correlations ranged from −.01 to.40 on the PCS-I, with 37 of 60 correlations reaching statistical significance, and from −.08 to .41 on the HBI, with 41 of 80 significant. Agreement did not appear to vary across occasions. In the OI group, across all four assessments, item-level correlations ranged from −.02 to .56 on the PCS-I, with 24 of 60 significant, and from −.04 to .49 on the HBI, with 60 of 80 significant. In the total sample, across all four assessments, item-level correlations ranged from −.02 to .42 on the PCS-I, with 48 of 60 reaching significance, and from .06 to .38 on the HBI, with 51 of 80 significant. Again, agreement did not appear to vary across occasions.
Mean symptom ratings tended to be significantly higher for children as compared to parents on both measures, in both groups. In the mild TBI group, significant mean differences were obtained for 38 of 60 items on the PCS-I, with children’s ratings higher than parents’ on 36 of 38, and 53 of 80 items on the HBI, with children’s ratings higher than parents’ on 45 of 53. In the OI group, significant mean differences were obtained for 25 of 60 items on the PCS-I, with children’s ratings higher than parents’ on 24 of 25, and 57 of 80 items on the HBI, with children’s ratings higher than parents’ on 52 of 57. In the total group, significant mean differences were obtained for 44 of 60 items on the PCS-I, with children’s ratings higher than parents’ on 41 of 44, and 56 of 80 items on the HBI, with children’s ratings higher than parents’ on 48 of 56.
Table 10 contains means and correlations for the total symptom score on the PCS-I as rated by parents and children. Correlations were significant for total symptom scores in both groups at all occasions. The correlations ranged from .27 to .44 in the mild TBI group and from .24 to .45 in the OI group. Correlations for total symptom scores did not differ significantly between groups. Correlations also did not differ significantly over sequential assessments within the mild TBI or OI group.
Figure 1 depicts the repeated measures analysis for the total PCS-I ratings by rater and assessment, with means adjusted for group membership. Children endorsed more symptoms than parents at all four occasions in both groups on the PCS-I, with the difference between raters becoming larger with time, as reflected in a significant rater X time interaction, F(3, 244) = 10.26, p < .001.
Table 11 contains the means and correlations from the cognitive and somatic symptom scales from the HBI. In the mild TBI group, correlations ranged from .11 to .37, with seven of eight being significant. Agreement was significantly higher on the cognitive scale compared to the somatic scale at 3 months, z(175) = 1.97, p < .05, but no other differences between the cognitive and somatic scales were significant in the mild TBI group. In the OI group, all correlations were significant, and ranged from .20 to .54. The magnitude of correlations for the cognitive and somatic scales did not significantly differ in the OI group. Correlations were generally higher in the OI group than in the mild TBI group, but the group difference was significant only for cognitive symptoms at 12 months, z = 2.04, p = .02 and somatic symptoms at 3 months, z = 2.21, p = .01.
Figures 2 and and33 depict the repeated measures analyses for the cognitive and somatic symptom scales by rater and assessment, with means adjusted for group membership. Children reported higher levels of somatic symptoms than parents at all four occasions in both groups, but reported higher cognitive symptoms only at the initial assessment. The difference between raters was significant across time for somatic symptoms, as reflected in a significant main effect for rater, F(1, 246) = 9.44, p < .05. In contrast, the difference between raters was significant only at the baseline assessment for cognitive symptoms, as reflected in a significant rater X time interaction, F(3, 244) = 12.33, p < .001.
The primary goal of the current study was to examine parent-child agreement on ratings of PCS in children following mild TBI. Ayr and colleagues (2009) previously documented moderate parent-child correlations for the cognitive and somatic dimensions in the mild TBI group on the HBI. In the current study, we expanded on these findings and examined correlations in the OI group as well; we also examined correlations at the 1 month and 12 month assessments, and on the PCS-I as well as the HBI. As expected, in correlational analyses, parent-child agreement was in the low to moderate range on both measures of PCS. In the mild TBI group and the OI group, as well as the pooled group, item-level correlations were largely significant but modest in magnitude. Correlational analyses of the PCS-I total score and the composite scale scores of the HBI also revealed significant correlations in both groups. Furthermore, item and composite level agreement was consistent over time in both groups. Overall, the reported level of agreement is consistent with other childhood psychopathology studies (Comer & Kendall, 2004; Edelbrock et al., 1986; Hodges et al., 1990).
We compared correlations between groups and, contrary to expectations, found somewhat higher parent-child agreement in the OI group than the mild TBI group. Group differences in correlational agreement reached statistical significance at one occasion for somatic symptoms and at one occasion for cognitive symptoms. This finding suggests that the level of parent-child agreement depends somewhat on injury status. Parents may be less aware of their child’s fluctuating symptoms following mild TBI compared to OI, or children with mild TBI may lack awareness of their symptoms.
We also compared the level of agreement on the cognitive and somatic scales within groups. Correlations tended to be higher on the cognitive symptom dimension compared to the somatic dimension, at least in the mild TBI group, for which agreement was significantly higher for cognitive PCS at one occasion. This finding suggests that parent-child agreement about PCS may be affected by symptom type, such that somatic symptoms are less easily observed by parents than cognitive symptoms.
A similar trend occurred when we examined agreement in terms of mean symptom levels. As hypothesized, mean symptom ratings tended to be significantly higher for children than for parents. This trend was apparent in item-level analyses of the PCS-I and the HBI for both groups. The analyses of the PCS-I total score revealed a significant interaction of rater and time post injury. Over time, the difference between child and parent ratings on the PCS-I increased in both groups. This may reflect the inclusion of emotional and behavioral symptoms on the PCS-I that are not included in the HBI. Analyses of the HBI scales showed that mean differences between parent and child reports depended on symptom type. Children had significantly higher ratings of somatic symptoms compared to parents across time. In contrast, parent and child ratings of cognitive symptoms differed only at the initial assessment. These results again suggest that children experience somatic symptoms that parents are unaware of, perhaps because children have a heightened awareness of their internal, visceral sensations or because they are unwilling or unable to share such symptoms with parents (McCrea et al., 2004). In contrast, parents may be more aware cognitive symptoms, which may have a greater impact on children’s overt behavior.
Research on childhood psychopathology has also found that parents and children report different levels of symptoms depending on symptom type. Studies consistently demonstrate that children report more internalizing symptoms, while parents tend to report more externalizing symptoms (Edelbrock et al., 1986; Hodges et al., 1990; Rey, Schrader, & Morris-Yates, 1992). Although the cognitive and somatic dimensions of PCS do not directly correspond to the distinction between internalizing and externalizing symptoms, the findings may be analogous, reflecting the less overt nature of somatic symptoms as compared to cognitive symptoms.
Overall, the results highlight the importance of examining both parent-child correlations and mean differences between parent and child reports as distinct aspects of agreement. As shown in the current study, parent and child ratings can be correlated significantly despite significant differences between their mean ratings. In this instance, parent and child reports are highly consistent internally but only moderately related correlationally and differ in mean levels, suggesting that parents and children judge post-concussive symptoms in somewhat different ways. Whatever the basis of the differences, parents’ and children’s report cannot be reliably substituted for another and both are needed in a thorough clinical assessment because they may be differentially sensitive to mild TBI (Taylor et al., 2010).
Methodological strengths of the current study include clear criteria for defining mild TBI, the use of a control group with other injuries, and the prospective, longitudinal design. The current study is limited by the moderate participation rate and attrition over the course of the study. In the future, additional factors that may contribute to parent-child agreement should be explored. Previous research indicates that child characteristics such as age, sex, race/ethnicity, and parent characteristics such as psychopathology and stress may influence parent-child agreement (Achenbach et al., 1987; Cremeens, Eiser, & Blades, 2006; De Los Reyes & Kazdin, 2005; Martin, Ford, Dyer-Friedman, Tang, & Huffman, 2004).
The current results indicate that PCS can be assessed reliably based on parent and child measures. A variety of scales have been developed to be used to assess PCS following mild TBI in childhood (Gioia et al., 2009). The scales used in the current study have demonstrated satisfactory psychometric properties. They are consistent across raters, showing significant parent-child agreement at the item level and on composite symptom dimensions. Moreover, the level of agreement is consistent with other diagnosed syndromes (Achenbach et al., 1987; Choudhury et al., 2003; Edelbrock et al., 1986; Hodges et al., 1990). Indeed, many of the correlations on composite PCS scales exceed the average agreement reported in studies of childhood psychopathology (Achenbach et al., 1987). Nevertheless, the current findings neither confirm nor disprove the existence of post-concussive disorder as a coherent syndrome. Future research is needed to further validate postconcussive disorder in children. Such research has the potential to improve clinical management of pediatric mild TBI.
The larger study on which the research was based was supported by grants R01 HD39834 and K02 HD44099 from the National Institute of Child Health and Human Development and the National Center for Medical Rehabilitation Research to Keith Owen Yeates. The authors wish to acknowledge the contributions of Lauren Ayr, Anne Birnbaum, Amy Clemens, Taryn Fay, Kalaichelvi Ganesalingham, Amanda Lininger, Melissa Ginn, Katie Pestro, Elizabeth Roth, Elizabeth Shaver, and Heidi Walker in conducting the study. Portions of the research were presented at the annual meeting of the International Neuropsychological Society in Atlanta, GA, February 2009.