PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
J Am Acad Child Adolesc Psychiatry. Author manuscript; available in PMC 2013 December 17.
Published in final edited form as:
PMCID: PMC3865780
NIHMSID: NIHMS393012

Relational Aggression in Children with Preschool Onset (PO) Psychiatric Disorders

Abstract

Objective

The role of preschool onset (PO) psychiatric disorders as correlates and/or risk factors for relational aggression during kindergarten or 1st grade was tested in a sample of N = 146 preschool-age children (3 to 5.11).

Method

Axis-I diagnoses and symptom scores were derived using the Preschool Age Psychiatric Assessment. Children’s roles in relational aggression as aggressor, victim, aggressive-victim, or non-aggressor/non-victim were determined at preschool and again 24 months later at elementary school entry.

Results

Preschoolers diagnosed with PO-psychiatric disorders were 3 times as likely as the healthy preschoolers to be classified aggressors, victims, or aggressive-victims. Children diagnosed with PO-disruptive, depressive, and/or anxiety disorders were at least 6 times as likely as children without PO-psychiatric disorders to become aggressive-victims during elementary school after covarying for other key risk factors.

Conclusions

Findings suggested that PO-psychiatric disorders differentiated preschool and school-age children’s roles in relational aggression based on teacher-report. Recommendations for future research and preventative intervention aimed at minimizing the development of relational aggression in early childhood by identifying and targeting PO-psychiatric disorders are made.

Keywords: Relational Aggression, Preschool Psychiatric Disorders, Aggressive-Victims, Bullying

The experience of peer aggression as perpetrator, victim or both during childhood is one of the strongest social predictors of developmental difficulties and maladjustment in later childhood.16 Historically, investigations of peer aggression have focused on physical forms in males during middle childhood and adolescence.712 Physical aggression involves the intent to hurt, harm, or injure others using physical force, such as hitting, kicking, punching, pushing, and forcibly taking things away from peers.13, 14 To assess non-physical forms of peer aggression, thought to be more characteristic of females, Crick and Grotpeter developed and tested an instrument that reliably measured and differentiated physical and relational forms of peer aggression.15 Relational aggression is defined as the intent to hurt or harm others through non-physical manipulation, threat, or damage to close relationships, friendships and/or social status.16, 17 That is, relational aggression is the expression of aggression and manipulation of others through the use of social inclusion or exclusion.

Crick and others have examined the factor structure of physical versus relational peer aggression.18 Results indicated that despite several overlapping characteristics, physical and relational forms of peer aggression have discrete factor structures. When examined simultaneously, measures of relational aggression accounted for unique portions of variance in social, emotional and cognitive development outcomes above and beyond physical peer aggression.1820 While a rich body of literature has informed patterns of physical aggression in early childhood 21, 22, the pioneering work of Crick and colleagues examining relational aggression resulted in a more comprehensive understanding of physical and social forms of aggression throughout development.23 These findings also provided initial evidence that relational forms of peer aggression occur much earlier in development than originally thought.2427

Relational Aggression Among Preschool-Age Children

Only recently has the study of relational aggression broadened from an almost exclusive focus on middle childhood and adolescence to the preschool-age period of development. Despite a broad understanding of overt physical aggression among preschoolers, it had been widely accepted that relational forms of peer aggression were a relatively uncommon occurrence among preschoolers (for exception see 9). Because researchers had historically focused on measuring overt aggression, it was thought that peer aggression in preschoolers was predominantly exhibited by boys in the form of physical aggression.2830 Findings consistently demonstrated that preschool boys compared to girls were in fact more physically aggressive during peer interactions.31,12 These findings led many researchers to assume that the social lives of preschool-age girls were largely devoid of peer aggression. A second problematic assumption held prior to the early 1990s was that preschoolers did not possess the social, emotional, and cognitive capacities to use more sophisticated forms of peer aggression characteristic of relational aggression. Both assumptions were challenged when empirical studies began examining relational and physical forms of peer aggression simultaneously within samples of preschool children.26, 31

Studies using multiple methods and informants have demonstrated validity, reliability, and short-term stability of relational aggression measured in preschool children.12, 26, 27, 3133 Relationally aggressive behaviors in preschool-age children tend to be overt and direct (and therefore easily observed) as opposed to indirect, discrete or subtle as more often manifest in older children. An example of overt relational aggression used by preschool-age children would be a child putting their hands over their ears indicating they are actively ignoring and rejecting a peer. Another example is when an aggressor directly tells the victim that he/she will not be invited to his/her party unless the victim does what the aggressor demands. Indirect relationally aggressive behaviors more commonly used by school-aged children includes behaviors such as disseminating malicious rumors about victims to peers. While studies examining relational aggression in preschoolers have indicated both genders engaged in this behavior, relational aggression is used more frequently by preschool girls than boys.8, 2932, 3436 Preschool girls engage in and experience more sophisticated, complex, and socially directed forms of peer relational aggression.12, 3739 It has been estimated that 70% of girls’ aggressive behaviors directed at peers are relationally focused and non-physical.32 By measuring children’s use of physical and relational forms peer aggression, several studies have illustrated developmental continuity of peer aggression from early to middle childhood for both boys and girls.4042

Peer Aggression in Preschoolers

Monks et al. reported that 25% of preschool children were aggressors and 22.1% were victims of peer aggresion.43 Preschool children who are persistently aggressive toward peers (“aggressors”) show greater oppositionality, poorer school adjustment, greater emotion dysregulation and more symptoms of inattention and depression; they are also more likely to become antisocial in adolescence.6, 4447 Being persistently victimized during early childhood has been associated with poorer school performance and impaired social adjustment, greater loneliness, increased social withdrawal and isolation, as well as episodic reactive aggression.19, 4850 Of particular importance is a distinct subgroup of children who are aggressors as well as victims of peer aggression, referred to as “aggressive-victims”.

Preschoolers classified by their teachers and or peers as being aggressive-victims differ in several important ways from peers classified as being “pure-aggressors” or “pure-victims”. Aggressive-victims are more likely to show reactive aggression in contrast to pure-aggressors who proactively use aggression to achieve a goal.51 Compared to “pure-aggressors” or “pure-victims”, aggressive-victims are described as being more anxious, physically reactive, and annoying to other children.52 Findings have also indicated that aggressive-victims have distinct temperamental characteristics that differ from children classified as being pure-aggressors or pure-victims. Aggressive-victims are more likely than pure victims and pure aggressors to be impulsive, irritable, and impatient during interactions with peers.53 The temperamental and behavioral characteristics of aggressive-victims are often the least socially desirable and are known risk factors for continued involvement in peer aggression. The maladaptive behavioral and temperamental characteristics of aggressive-victims as well as findings that these children often have significantly greater functional and developmental impairments supports the finding of increased risk for and/or rates of mental illness in this group. 54, 55

The prevalence of being an aggressive-victim is estimated to be around 6–8% in young children.56, 57 Results from kindergarten students found 18% of boys and 8% of girls were classified as aggressive-victims using teacher reports.58 In addition to experiencing more severe developmental impairment, aggressive-victims have a greater risk for manifesting psychiatric problems.59, 60 Prior results demonstrated that after controlling for pre-existing adjustment problems at age 5, aggressive-victims compared to aggressors or victims had significantly higher internalizing and externalizing behavior problem mean scores by the age of 7.56 Aggressive-victims are consistently reported as having the highest level of maladjustment among all children involved in peer aggression, exhibiting more symptoms of both internalizing and externalizing problems.54, 55

Preschool Onset (PO)–Psychiatric Disorders as Risk Factors for Relational Aggression in Preschool

Of particular public health concern are findings that school children involved in relational aggression are more likely to manifest an array of mental health problems that often continue into adolescence and adulthood.59, 6163 Findings from older children suggest that correlations between relational aggression and mental disorders may exist before children enter elementary school.5, 6, 64 To date, studies that have examined emotional and behavioral problems associated with relational aggression in preschool children have predominantly used more general dimensional measures of internalizing or externalizing symptoms but have not examined more specific categorical DSM-IV psychiatric disorders.25 A mounting body of literature has established that reliable and valid DSM-IV Axis-I psychiatric disorders can be identified in children as young as 3 (for review see65). The importance of early identification of psychiatric disorders during the preschool period continues to gain attention based on new findings demonstrating the developmental continuity of these disorders from preschool to early adolsecence.66 The preschool period represents a unique phase of life during which rapid social development takes place and patterns of social interactions begin to form in the context of the “semi-structured” environment of the preschool classroom. Therefore, the social milieu of the preschool classroom, playground, and lunchroom provide the stage for the emergence of aggressor and victim related social behavior. This preschool classroom also represents a unique opportunity for observations of these behaviors, as result of minimal self-monitoring, limited cognitive capacities to anticipate future consequences of misbehavior, and decreased awareness of social norms previously described. Following this, identifying and characterizing associations between early onset mental illness and relational forms of aggression may be key to inform how these early behaviors influence each other developmentally.67 Thus, the current study tested expected associations and group differences between children diagnosed with PO-psychiatric disorders and their roles in relational aggression at preschool and 2 years later in elementary school.

METHOD

Recruitment and Participants

This investigation used data from a National Institute of Mental Health (NIMH) funded study entitled Validation of Preschool Depressive Syndromes (PDS). This ongoing longitudinal multi-method and multi-informant (i.e., parents, children, and teachers) study was designed to examine the nosology, etiology and course of PO–Major Depressive Disorder (MDD) (for additional recruitment details see 68). Between May 2003 and March 2005, caregivers with children between 3.0 and 5.11-years-old were recruited from pediatricians’ offices, daycares, and preschools in a large metropolitan community using the Preschool Feelings Checklist (PFC).69 The PFC is a brief validated screening tool for early-onset emotional disorders. Excluded were children with chronic medical illnesses, neurological problems, pervasive developmental disorders, language and/or cognitive delays, as well as those out of the study age range. It is important to note that the recruitment techniques used in this study were designed to over-sample for preschoolers with or at risk for MDD and or attention-deficit/hyperactivity disorder (ADHD). Therefore, diagnostic data from the present study cannot be used to calculate the prevalence rates of PO-psychiatric disorders in the general population.

A total sample of N = 306 caregiver-child dyads agreed to participate and completed their baseline assessment in a laboratory. Out of the total sample of N =306 children at baseline a subsample of n = 202 children had completed teacher data. Children who stayed at home with a primary caregiver accounted for n = 80 of the total n = 104 children with missing teacher data on the MacArthur Health and Behavior Questionnaire–Teacher Version (HBQ-T; see description in measures section). The remaining n = 24 children with missing teacher data was the result of caregivers refusing to consent for us to contact children’s preschool teachers or teachers failing to return a completed HBQ-T. For the cross-sectional analyses of at baseline, children had to be enrolled in formal (i.e., home daycares were not included) preschool or pre-kindergarten program. Of the n = 202 children with completed HBQ-T data at baseline n = 42 children were attending home daycares. A total of n = 14 children were already enrolled in kindergarten at baseline (i.e., did not have preschool data available). Thus, the final sample size of preschoolers eligible for analysis was n =146. Children attending preschool or pre-kindergarten programs at baseline and who were enrolled in Kindergarten or 1st grade 24 months following their baseline assessment were examined in the longitudinal analyses (n =121). The n = 25 children with missing data at school age had either dropped out of the study or had teachers that did not complete the HBQ-T. Descriptive data for demographic and diagnostic variables used in the analyses are included in Table 1.

Table 1
Demographic and Diagnostic Characteristics at Baseline

Procedure and Measures

Parent-child dyads participated in a 3–4 hour annual assessment. During this time primary caregivers (94% mothers) were interviewed about their children’s behaviors, emotions and age-adjusted manifestations of psychiatric symptoms. Caregivers were also asked for permission to contact children’s current or most recent teacher. Teachers of consenting families were contacted within 7–10 days of the annual assessment and sent a brief study description, directions for participating, and questionnaires to be completed.

Ratings of Childhood Aggressors and/or Victims

Children were classified as aggressors, victims, aggressive-victims, or non-aggressor/non victims using preschool and elementary school teachers’ report on the MacArthur Health and Behavior Questionnaire-Teacher Version (HBQ-T 1.0).70 Aggressors (i.e., perpetrators of relational aggression) were assessed using 6 items from the HBQ-T: 1. When mad at peer, keeps that peer from being in the playgroup, 2. Tries to get others to dislike a peer, 3. Tells others not to play with or be a peer’s friend, 4. Tells peers that he/she won’t play with peers or be peers’ friend unless peers do what he/she asks, 5. Verbally threatens to keep a peer out of the playgroup if the peer doesn’t do what he/she wants, and 6. Tells a peer that they won’t be invited to their birthday party unless that peer does what he/she wants. For each item teachers rank (0 = Never; 1 = Sometimes; 2 = Often) children’s engagement in relational aggression as aggressors. The six items are averaged together to create a relational-aggressor mean score for each child.

Victims of relational aggression were identified using four items from the peer victimization subscale of the HBQ-T: 1. Other children refuse to let him/her play with them, 2. Actively disliked by other children, who reject him/her from their playgroup, 3. Is picked on by other children, and 4. Is teased and ridiculed by other children. For each item, teachers rated children as a 1 = not at all like this; 2 = very little like; 3 somewhat like; 4 = very much like. Mean scores were computed to provide continuous victim scores.

In order to create a categorical aggressor variable, at preschool and again during elementary school, children with an aggressor mean score in the top 20th percentile and a victim score in the bottom 80th percentile were classified as aggressors. Children scoring in the top 20th percentile of the victim subscale and in the bottom 80th percentile of the aggressor subscale were classified as victims. Children who scored in the top 20% of the aggressor and victim scales were classified as aggressive-victims. Children who scored in the bottom 80th percentile on both the aggressor and victim subscales of the HBQ-T were in the non-aggressor/non-victim group. This method resulted in 4 mutually exclusive groups at preschool and elementary school: aggressor, victim, aggressive-victim, or non-aggressor/non-victim. Similar classification methods for differentiating aggressors, victims, and aggressive-victims have been used and validated in several publications from independent studies.31, 61

The functional impairment subscale of the HBQ-T was also used as a covariate in the final set of analyses. Impairment was included as a covariate to test whether the hypothesized effect of PO-psychiatric disorder on school age relational aggression behaviors remained significant after controlling for the potential effects of social impairment associated with children’s psychiatric disorder present at school age. The HBQ-T impairment subscale uses teacher report to measure functional impairment children are exhibiting in the classroom. This subscale of the HBQ-T includes seven items that are rated using a likert scale (0 = none, 1 = a little, and 2 = a lot). Studies reporting the psychometric properties of this subscale suggest it has moderate to strong internal consistency and acceptable test retest reliability across reporters and age ranges.

PO DSM-IV Psychiatric Disorders

The Preschool Age Psychiatric Assessment (PAPA) is an interviewer based semi-structured diagnostic interview with established test re-test reliability designed for use in caregivers of children aged 2.0–6.0.65 Although psychometric properties of the PAPA have only been published in children up to age 6, it is important to note that the PAPA has been successfully used in children up to age 8.0 by a number of research groups. The PAPA includes all relevant DSM-IV criteria and their age appropriate manifestations. Diagnoses are derived by computer algorithms that apply all of the DSM-IV criteria (with the exception of duration criteria for MDD). The PAPA rates the intensity of symptoms, their frequency and duration as well as impairment from symptoms in three separate contexts (i.e., at home, school, and elsewhere). Interviewers undergo 5 to 7 day training and practice assessments are done until proficiency is achieved. Interviews were audio taped for later quality control and interviewer calibration. A master coder reviewed 20% of each interviewer’s PAPA assessments and when discrepancies arose items were re-coded in consultation with a senior child psychiatrist. To maintain high levels of interviewer reliability weekly coding meetings were conducted with a “master” rater as recommended by the authors of the measure.

Data Analyses

Chi-square analyses, one-way univariate analysis of variance tests, and correlation analyses were conducted to examine variation and or differences in relational aggression during preschool and elementary school associated with demographic variables. To examine the stability of children’s mean aggressor and victim scores at preschool and school-age Pearson correlation matrices were calculated. Multinomial logistic regression analyses were conducted to test whether children diagnosed with PO-psychiatric disorders were significantly more or less likely as healthy peers to be classified as aggressors, victims, aggressive-victims, or non-aggressors/non-victims during preschool and/or elementary school. For the final set of analyses multinomial logistic regression analyses were conducted using covariates previously found to influence children’s involvement in relational aggression during elementary school. The following covariates were tested: children’s mean aggressor scores and victim scores obtained at preschool, schoolchildren’s current level of functional impairment (teacher report on the HBQ-T), as well as the total number of disruptive, anxiety, depression symptoms experienced at school-age. The criterion variable for each model was children’s role in relational aggression as an aggressive-victim versus non-aggressor/non-victim during elementary school.

RESULTS

Demographic and Diagnostic Characteristics at Baseline and Preschoolers’ Relational Aggression

Children’s roles in relational aggression (i.e., aggressor, victim, aggressive-victim or non-aggressor/non-victim) during preschool and/or elementary school did not differ in relation to children’s gender, age, ethnic origin, family gross income or their primary caregivers’ highest level of education achieved. At baseline n = 74 children had a diagnosis of 1 or more preschool onset psychiatric disorder (n = 42 children had a disruptive disorder which for the current study included ADHD as well as oppositional defiant disorder [ODD] and/or conduct disorder [CD]), (n =37 children had an anxiety disorder which included generalized anxiety disorder [GAD], social anxiety disorder [SAD], and/or post-traumatic stress disorder [PTSD]), (n = 41 children had been diagnosed with MDD). The remaining n = 72 preschoolers had no psychiatric disorders and comprised the healthy comparison group for the following analyses (see Table 1 for a further breakdown of diagnostic group and comorbidity).

Descriptive Analyses Examining the Stability of Children’s Involvement in Relational Aggression

Pearson correlations indicated that children’s aggressor scores at preschool were significantly associated with their aggressor scores at school age, r = .34; p < .001. Similarly, children’s mean scores for the victim subscale of the HBQ-T at preschool were significantly associated with their mean scores on the victim subscale 2 years later at school age, r = .31; p < .01. Descriptive analyses revealed that approximately 60% of preschool non-aggressor/non-victims continued to be classified this way by teachers when measured again 24 months later. The remaining 40% of preschoolers classified as non-aggressor/non-victim had a different classification once they were school age, 17% became aggressors; 8% were classified as victims at school age, and 15% were aggressive-victims at school age. Forty-five percent of children classified as aggressors during preschool were classified as non-aggressor/non-victim when measured at school age. Only 20% of children classified as aggressors during preschool went on to be classified as aggressors once they were school age. The remaining 35% of preschool aggressors were classified as aggressive-victims when measured at school age. Results indicated that 36% of children classified as victims during preschool were classified as non-aggressor/non-victims at school age. Of the remaining children classified as victims during preschool 14% became aggressors, 22% retained their victim status, and 28% were classified as aggressive-victims when measured at school age. Thirty-six percent of children classified as aggressive-victims during preschool were also classified as aggressive victims at school age. Of the remaining 64% of children who were aggressive-victims during preschool their classifications changed as follows once they were school age: 29% were non-aggressor/non-victim, 21% were aggressors, and 14% were classified as victims based on teacher-report at school age.

Presence vs. Absence of PO-Psychiatric Disorders and Children’s Roles in Relational Aggression during Preschool and at School Age

Preschoolers’ roles in relational aggression differed significantly between diagnostic groups at preschool, χ2(3, n=146) = 6.68,p< .05. Compared to healthy peers, preschoolers diagnosed with one or more PO-psychiatric disorders were significantly more likely to be classified into one of the following mutually exclusive groups during preschool: aggressive-victim (odds ratio [OR] = 3.75, CI 95%: 1.22–11.23, p < .05), aggressor (OR = 3.61, CI 95%: 1.06–12.87, p < .05), or victim (OR = 3.64, CI 95%: 1.04–12.87, p < .05) when using non-aggressor/non-victim as the reference group. Most notably, findings indicated that among all preschoolers identified as being aggressive-victims (n = 21), 76% had PO-psychiatric disorder(s). In sum, children with PO-psychiatric disorders were on average at least 3 times as likely as healthy same age peers to be classified by teachers as being either aggressors, victims, or aggressive-victims during preschool.

Children’s relational aggression roles were measured again 2-years after their baseline diagnostic assessment when they were in elementary school. Schoolchildren’s relational aggression roles during elementary school differed significantly in relation to their history of PO-psychiatric disorder(s), χ2(3, n=121) = 10.46, p < .01. Schoolchildren diagnosed with PO-psychiatric disorders compared to schoolmates with no history of PO-psychiatric disorders were twice as likely to be classified as aggressors than as non-aggressors/non-victims, OR = 2.88, CI 95%: 1.01–7.80, p < .05. Schoolchildren with PO-psychiatric disorder(s) were 4 times as likely as healthy peers to be classified as aggressive-victims compared to a non-aggressors/non-victims, OR = 4.21, CI 95%: 1.60–11.09, p < .01. Schoolchildren’s risk for being classified as victims of relational aggression did not differ significantly between diagnostic groups.

PO Disruptive, Anxiety, or Depressive Disorders and Children’s Relational Aggression Roles during Preschool and Elementary school

PO-Disruptive Disorders

Preschoolers’ roles in relational aggression differed between healthy and disruptive disordered groups, χ2(3, n=116) = 10.94, p < .01. Compared to healthy peers, preschoolers with PO-disruptive disorder(s) were significantly more likely to be classified as aggressive-victims than aggressors (OR = 4.51, CI 95%: 1.17–17.37, p < .05), victims (OR = 3.20, CI 95%: .89 – 12.02, p < .05), or non-aggressor/non-victims. It is important to note that disruptive preschoolers were equally as likely as healthy preschoolers to be classified by their teachers as “pure aggressors” or “pure victims” when non-aggressors/non-victims status was used at the reference group (Figure 1).

Figure 1
Preschool Onset (PO) Disruptive Disorders and Children’s Relational Aggression Status at Preschool and School Age. Note: NS = not significant; OR = odds ratio. * p < .05, ** p < .01, *** p < .001

As seen in Figure 1, when measured again during elementary school, schoolchildren previously diagnosed with PO-disruptive disorder(s) versus schoolchildren who were healthy throughout preschool differed significantly in their relational aggression roles, χ2(3, n=100) = 17.89, p < .001. Schoolchildren diagnosed with PO-disruptive disorders were more than 8 times as likely as children in the healthy comparison group to be classified as aggressive-victims. Schoolchildren with PO-disruptive disorders were more than 5 times as likely as healthy peers to be identified by teachers as aggressors. In contrast, schoolchildren with PO-disruptive disorders were 4 times less likely than healthy peers to be classified as victims when using non-aggressor/non-victim as the reference group.

PO-Anxiety Disorders

Preschoolers’ relational aggression roles did not differ significantly between anxiety disordered and healthy preschoolers (Figure 2). When testing PO-anxiety disorders as predictors of relational aggression roles at school age the overall chi-square testing for proportional differences in the relational aggression roles of schoolchildren who were healthy during preschool versus classmates with a history of PO-anxiety disorders was non-significant, χ2(3, n=89) = 5.47, p = .07. However, one pairwise result was significant and is worth noting. That is, schoolchildren diagnosed with PO-anxiety disorders were 4 times as likely as schoolmates who were healthy preschoolers to be classified as aggressive-victims than non-aggressors/non-victims (Figure 2).

Figure 2
Preschool Onset (PO) Anxiety Disorders and Children’s Relational Aggression Status at Preschool and School Age. Note: NS = not significant; OR = odds ratio. * p < .05

PO-MDD

Healthy and PO-MDD preschoolers did not differ in their relational aggression roles during preschool (Figure 3). Roles in relational aggression at school-age differed between schoolchildren who were healthy as preschoolers compared to schoolmates previously diagnosed with PO-MDD, χ2(3, n=90) = 8.55, p < .05. School-age children with PO-MDD were 5 times more likely than the healthy comparison group to be classified as aggressive-victims than non-aggressor/non-victims during elementary school, see Figure 3.

Figure 3
Preschool Onset (PO)–Major Depressive Disorder (MDD) and Children’s Relational Aggression Status at Preschool and School Age. Note: NS = not significant; OR = odds ratio. * p < .05

PO-Psychiatric Disorders as Predictors of Later Aggressive-Victim Status: Controlling for Relational Aggression Roles at Preschool, Psychopathology and Functional Impairment at School Age

The above findings indicated that schoolchildren with a history of PO-disruptive, PO-anxiety, and PO-MDD were significantly more likely than their healthy peers to be classified as aggressive-victims than non-aggressor/non-victims. To further explore this finding, a series of multinomial logistic regression analyses were conducted to test whether PO-psychiatric disorders continued to predict schoolchildren’s relational aggression roles when covarying for children’s relational aggression behaviors during preschool as well as schoolchildren’s current experience of psychiatric symptoms and associated functional impairments. The following covariates were tested: schoolchildren’s mean aggressor and victim scores obtained at preschool, schoolchildren’s current level of functional impairment (teacher report), as well as the total number of disruptive, anxiety, depression symptoms experienced at school-age. The criterion variable for each model was children’s role in relational aggression as an aggressive-victim versus non-aggressor/non-victim during elementary school.

Results indicated that PO-disruptive disorders continued to be a significant predictor of the aggressive-victim classification during elementary school, p < .01. Specifically, schoolchildren with PO-disruptive disorder were 8 times as likely as schoolchildren who were healthy preschoolers to be classified as aggressive-victims after including covariates in the model (Figure 4). Similarly, schoolchildren with a history of PO-MDD were significantly more likely than the healthy comparison group to be classified as aggressive-victims, χ2(df21, n=84) = 37.53, p < .01. Schoolchildren diagnosed with PO-MDD were 6 times as likely as schoolchildren who were healthy as preschoolers to be classified as aggressive-victims after covariates were included in the model (Figure 4). Schoolchildren with a history of PO-anxiety disorders compared to schoolchildren who were healthy preschoolers were also significantly more likely to be classified by teachers as aggressive-victims during elementary school, χ 2(df21, n=76) = 42.81, p < .01. Compared to schoolchildren who were healthy in preschool, schoolchildren diagnosed with PO-anxiety disorders were 9 times more likely to be aggressive-victims than non-aggressor/non-victims after including covariates.

Figure 4
Preschool Onset (PO) Psychiatric Disorders and Likelihood of being an Aggressive-Victim at School Age after Covarying for School Age Psychiatric Disorder Severity, Functional Impairment, and Mean Aggression and Victim Scores at Preschool. Note: Results ...

DISCUSSION

The association between childhood onset psychiatric symptoms/disorders and involvement in overt or physical forms of peer aggression (e.g., bullying) throughout development has been well established.59, 61 However, a growing body of literature suggests that covert and non-physical forms of aggression, such as relational aggression, have equally as deleterious effects on children’s development, occur at relatively high frequencies, and start as early as the preschool period of development. With a few exceptions, a relatively limited number of studies have examined whether PO-psychiatric disorders demonstrate associations with non physical forms of peer aggression consistent with findings examining physical forms of peer aggression (e.g., bullying). The aim of the present study was to examine whether preschool onset psychiatric disorders were concurrently related to preschoolers’ involvement in relational aggression and or predicted their later involvement in relational aggression at school age.

Despite overlap between peer focused aggressive behaviors and ODD symptoms (e.g., spiteful and vindictive) as well as CD symptoms (e.g., bullying), findings have illustrated that as few as 6% of children with ODD and or CD also had high relational aggression scores (i.e., 1 SD above the sample mean). In contrast, the same study found that 14% of children without a diagnosis of ODD and/or CD had high relational aggression scores.71 Despite children diagnosed with ODD and/or CD being more likely to be involved in relational aggression the majority of youth engaged in high levels of relational aggression do not meet symptom criteria for ODD and/or CD.

In the current study preschoolers diagnosed with ADHD, ODD, and/or CD were no more likely that healthy same age peers to be classified as “pure-aggressors or victims” of relational aggression during preschool. However, disruptive disordered preschoolers were 6 times as likely as healthy same age peers to classified as aggressive-victims. This suggests that preschoolers with disruptive disorders are frequently the perpetrator and victims of relational aggression. Once disruptive disordered preschoolers were schoolchildren they were significantly more likely than schoolmates without a history of preschool onset disruptive disorders to be “pure-aggressors” and aggressive-victims but significantly less likely to be victims of relational aggression. This finding is consistent with results from older children, which have demonstrated that as aggressors grow in physical strength and join social groups with other aggressors their likelihood for becoming victimized decreases. This may be the result of other aggressors becoming more fearful and avoidant of confrontation with known aggressors.72

Arguably the most interesting and novel findings to emerge in the current study were children diagnosed with preschool onset anxiety and/or depressive disorders were no more likely than healthy preschoolers to be involved in relational aggression as aggressors or victims during preschool or at school age. However, this same group of children (with preschool onset anxiety and/or depressive disorders) were more than 6 times as likely to be classified aggressive-victims at school age compared to healthy preschoolers. These results emerged after controlling for children’s involvement in relational aggression as aggressors and victims during preschool as well as their current disruptive, anxiety, and depressive symptoms as well as functional impairment scores as school age. The high risk for this unique outcome (aggressive-victim) is also consistent with prior findings that have demonstrated during the beginning years of elementary school, aggressive-victims show significantly greater internalizing symptoms than aggressors and victims of relational aggression. In contrast to these findings, studies that measure aggressor and victim scores/status only (omitting an aggressive-victim score/group) in relation to children’s concurrent anxiety and depressive symptoms typically demonstrate that increases in anxiety and depression are positively correlated with children’s victim scores. These findings related to early onset anxiety and depression in the context of the extant literature raise several interesting questions for future research. Of particular interest is the need for futures studies examining the possibility of differing trajectories for children’s involvement in relational aggression as function of varying onsets as well as current severity of specific psychiatric disorders.

Findings examining gender differences in relational aggression during preschool have been mixed. 9, 17, 23, 29 Crick found that preschool girls were more likely than boys to be involved in relational aggression.26 In contrast, other studies including the current study did not detect gender differences.39, 64 When examining gender differences in a sample of youth 9 to 17, Keenan and colleagues concluded that gender similarities and not differences in levels of relational aggression were the norm. 71 Although speculative, the lack of gender differences in the present study may have been related to high percentage of children with preschool onset psychiatric disorders in the sample studied. It is possible that when including children with disruptive and other preschool onset psychiatric disorders boys and girls’ use of relational forms of aggression may be more equally distributed.

The present study has several limitations. First the HBQ-T has no standardized cut-points for determining children’s involvement in relational aggression as either perpetrator and or victim. Consistent with prior literature a 20% cutoff was used at each time point. Thus, children were assigned aggressor/victim status based on observed levels of aggression in the existing sample. Nonetheless, children’s aggressor/victim status at time 1 was significantly predictive of their aggressor/victim status when measured 24 months later and was associated with later mental health problems. Second, there were high rates of psychiatric disorder comorbidity in the present sample. Given the relatively small sample size examining co-occurring disorders in relation to children’s aggressor/victim status resulted in group sizes too small for statistical comparisons. As a result, the current findings should be interpreted with a degree of caution. Third, children’s DSM-IV diagnostic group status (based on primary caregiver reports) as well as their aggressor/victim status (based on teacher report) was measured using a single informant. Although a multi-informant method is preferred for both constructs, caregiver report for assessing research based diagnostic status in preschool-age children remains the current standard in the field. Along these same lines, peer ratings and observational measures, in conjunction with teacher ratings would have been ideal for assessing preschoolers and schoolchildren’s involvement in relational aggression. Given the young age of the sample there are numerous challenges when using peer-based nominations of relationally aggressive behaviors. Arguably, teacher report of preschoolers’ involvement in relational aggression may provide the most reliable source of information related to these forms of peer aggression in very young children. Last, the recruitment methods used to obtain the present sample limits the generalizability of the current findings to the general population of children. However, findings from the current study warrant future studies in community based and or clinical samples to test the generalizability of these findings to both healthy and clinical populations of young children.

Findings suggest that increased attention to the detection of psychiatric disorders in preschool populations, which currently remain undetected in the vast majority of affected preschoolers, may be a promising strategy for identifying those at high risk for later involvement in relational aggression as well as providing a target for preventative intervention for schoolchildren’s involvement in relational aggression. These findings underscore two potentially key public health principles. The first is the importance of identifying and treating psychiatric disorders during the preschool period given the established association to poorer peer relationship outcomes. 30, 73, 74 Second is the importance of evaluating relational aggression behaviors as early as the preschool period given their clear manifestation at this early juncture and the possibility of more effective intervention during this time of rapid social and emotional development. 40 In addition to this, and relevant to the prevention of school-age relational aggression, is the need to account for history of early onset mental disorders in preschool populations as a possible mechanism to prevent later aggressive-victim behaviors. That is, the current findings suggest that the manifestation of psychiatric symptoms in preschool children may provide an observable and targetable antecedent to more severe forms of relational aggression (i.e., bullying) behaviors in schoolchildren. Based on the current findings we conclude that efforts to reduce relational aggression in schools should focus on the earliest possible detection of risk for or early onset psychopathology. Interventions designed to specifically target these subgroups, and focus on relieving psychiatric symptoms, appear to be indicated. Such strategies may in turn minimize the occurrence of relational aggression at school age, an increasingly serious public health concern. 20

Acknowledgments

This study was funded by the National Institute of Mental Health (NIMH) R01 grants # R01 MH64769-01 (J.L.) and K01MH090515 (A.B.).

We gratefully acknowledge Edward Spitznagel, Ph.D., of Washington University–St. Louis for his statistical consultation and Marilyn Essex, Ph.D., of the University of Wisconsin for her assistance with earlier versions of this manuscript. We are also grateful to the EEDP staff, our preschool participants and their parents, and community recruiting sites whose participation and cooperation made this research possible.

Footnotes

Disclosure: Dr. Luby has received grant or research support from the NIMH, the National Alliance for Research on Schizophrenia and Depression, the Communities Healing Adolescent Depression and Suicide (CHADS) Coalition, and the Sidney R. Baer Foundation. She has served as a consultant to the Food and Drug Administration (FDA) Advisory Board. Drs. Belden and Gaffrey report no biomedical financial interests or potential conflicts of interest.

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

1. Berkowitz L. Pain and Aggression - Some Findings and Implications. Motivation and Emotion. 1993 Sep;17(3):277–293.
2. Loeber R. Development and Risk-Factors of Juvenile Antisocial-Behavior and Delinquency. Clinical Psychology Review. 1990;10(1):1–41.
3. Parker JG, Asher SR. Peer Relations And Later Personal Adjustment - Are Low-Accepted Children At Risk. Psychological Bulletin. 1987 Nov;102(3):357–389. [PubMed]
4. Hanish LD, Guerra NG. Aggressive Victims, Passive Victims, and Bullies: Developmental Continuity or Developmental Change? Merrill-Palmer Quarterly. 2004 Feb;50(1):17–38.
5. Barker EDP, Boivin MP, Brendgen MP, et al. Predictive Validity and Early Predictors of Peer-Victimization Trajectories in Preschool. Archives of General Psychiatry. 2008;65(10):1185–1192. [PubMed]
6. Burk LR, Armstrong JM, Park J-h, Zahn-Waxler C, Klein MH, Essex MJ. Stability of Early Identified Aggressive Victim Status in Elementary School and Associations with Later Mental Health Problems and Functional Impairments. Journal of Abnormal Child Psychology. 2010 Sep 01;39(2):225–238. [PMC free article] [PubMed]
7. Block JH. Differential Premises Arising From Differential Socialization Of The Sexes - Some Conjectures. Child Development. 1983;54(6):1335–1354. [PubMed]
8. Monks CE, Smith PK, Swettenham J. Aggressors, victims, and defenders in preschool: Peer, self-, and teacher reports. Merrill-Palmer Quarterly-Journal of Developmental Psychology. 2003 Oct;49(4):453–469.
9. Bjorkqvist K, Lagerspetz KMJ, Kaukiainen A. Do Girls Manipulate And Boys Fight - Developmental-Trends In Regard To Direct And Indirect Aggression. Aggressive Behavior. 1992;18(2):117–127.
10. Crick NR, Bigbee MA, Howes C. Gender differences in children’s normative beliefs about aggression: How do I hurt thee? Let me count the ways. Child Development. 1996 Jun;67(3):1003–1014. [PubMed]
11. Baillargeon RH, Zoccolillo M, Keenan K, et al. Gender differences in physical aggression: A prospective population-based survey of children before and after 2 years of age. Developmental Psychology. 2007 Jan;43(1):13–26. [PubMed]
12. Ostrov JM, Keating CF. Gender differences in preschool aggression during free play and structured interactions: An observational study.
Social Development 2004. 2004;13(2):255–277.
13. Archer J, Coyne SM. An Integrated Review of Indirect, Relational, and Social Aggression. Personality and Social Psychology Review. 2005 Aug;9(3):212–230. [PubMed]
14. Ostrov J, Crick N. How Recent Developments In The Study Of Relational Aggression And Close Relationships In Early Childhood Advance.
The Field. Journal of Applied Developmental Psychology. 2006 Jun;27(3):189–192.
15. Crick NR, Grotpeter JK. Relational Aggression, Gender, And Social-Psychological Adjustment. Child Development. 1995 Jun;66(3):710–722. [PubMed]
16. Crick NR. Relational Aggression - The Role Of Intent Attributions, Feelings Of Distress, And Provocation Type. Development and Psychopathology. 1995 Spr;7(2):313–322.
17. Ostrov JM, Godleski SA. Toward an Integrated Gender-Linked Model of Aggression Subtypes in Early and Middle Childhood. Psychological Review. 2010 Jan;117(1):233–242. [PubMed]
18. Crick NR. The role of overt aggression, relational aggression, and prosocial behavior in the prediction of. Child Development. 1996;67(5):2317–2327. [PubMed]
19. Crick NR, Ostrov JM, Werner NE. A longitudinal study of relational aggression, physical aggression, and children’s social-psychological adjustment. Journal of Abnormal Child Psychology. 2006 Apr;34(2):131–142. [PubMed]
20. Fite PJ, Stoppelbein L, Greening L, Preddy TM. Associations Between Relational Aggression, Depression, and Suicidal Ideation in a Child Psychiatric Inpatient Sample. Child Psychiatry and Human Development. 2011 Dec;42(6):666–678. [PubMed]
21. Keenan K, Shaw DS. THE DEVELOPMENT OF AGGRESSION IN TODDLERS - A STUDY OF LOW-INCOME FAMILIES. Journal of Abnormal Child Psychology. 1994 Feb;22(1):53–77. [PubMed]
22. Wakschlag LS, Briggs-Gowan MJ, Carter AS, et al. A developmental framework for distinguishing disruptive behavior from normative misbehavior in preschool children. Journal of Child Psychology and Psychiatry. 2007 Oct;48(10):976–987. [PubMed]
23. Card NA, Stucky BD, Sawalani GM, Little TD. Direct and indirect aggression during childhood and adolescence: A meta-analytic review of gender differences, intercorrelations, and relations to maladjustment. Child Development. 2008 Sep-Oct;79(5):1185–1229. [PubMed]
24. Crick NR, Grotpeter JK. Children’s treatment by peers: Victims of relational and overt aggression. Development and Psychopathology. 1996 Spr;8(2):367–380.
25. Crick NR, Casas JF, Mosher M. Relational and overt aggression in preschool. Developmental Psychology. 1997 Jul;33(4):579–588. [PubMed]
26. Crick NR, Casas JF, Ku HC. Relational and physical forms of peer victimization in preschool. Developmental Psychology. 1999 Mar;35(2):376–385. [PubMed]
27. Ostrov JM, Crick NR, Keating CF. Gender-biased perceptions of preschoolers’ behavior: How much is aggression and prosocial behavior in the eye of the beholder? Sex Roles. 2005 Mar;52(5–6):393–398.
28. Schwartz D, Dodge KA, Coie JD, et al. Social-cognitive and behavioral correlates of aggression and victimization in boys’ play groups. Journal of Abnormal Child Psychology. 1998 Dec;26(6):431–440. [PubMed]
29. Pellegrini A, Roseth C. Relational Aggression And Relationships In Preschoolers: A Discussion Of Methods, Gender Differences, And Function. Journal of Applied Developmental Psychology. 2006 Jun;27(3):269–276.
30. Sebanc AM. The friendship features of preschool children: Links with prosocial behavior and aggression. Social Development. 2003;12(2):249–268.
31. Crick NR, Ostrov JM, Burr JE, Cullerton-Sen C, Jansen-Yeh E, Ralston P. A longitudinal study of relational and physical aggression in preschool. Journal of Applied Developmental Psychology. 2006 May-Jun;27(3):254–268.
32. McEvoy MA, Estrem TL, Rodriguez M, Olson ML. Assessing relational and physical aggression among preschool children: Intermethod agreement. Topics in Early Childhood Special Education. 2003 Sum;23(2):53–63.
33. Bonica C, Arnold DH, Fisher PH, Zeljo A, Yershova K. Relational aggression, relational victimization, and language development in preschoolers. Social Development. 2003;12(4):551–562.
34. Ostrov JM. Deception and subtypes of aggression during early childhood. Journal of Experimental Child Psychology. 2006 Apr;93(4):322–336. [PubMed]
35. Ostrov JM, Crick NR. Forms and functions of aggression during early childhood: A short-term longitudinal study. School Psychology Review. 2007;36(1):22–43.
36. Garner PW, Lemerise EA. The roles of behavioral adjustment and conceptions of peers and emotions in preschool children’s peer victimization. Development and Psychopathology. 2007 Win;19(1):57–71. [PubMed]
37. Casas JF, Weigel SM, Crick NR, et al. Early parenting and children’s relational and physical aggression in the preschool and home contexts. Journal of Applied Developmental Psychology. 2006 May-Jun;27(3):209–227.
38. Ostrov JM, Woods KE, Jansen EA, Casas JF, Crick NR. An observational study of delivered and received aggression, gender, and social-psychological adjustment in preschool: “This White Crayon Doesn’t Work…” Early Childhood Research Quarterly. 2004;19(2):355–371.
39. Hart CH, Nelson DA, Robinson CC, Olsen SF, McNeilly-Choque MK. Overt and relational aggression in Russian nursery-school-age children: Parenting style and marital linkages. Developmental Psychology. 1998 Jul;34(4):687–697. [PubMed]
40. Crick NR. The role of overt aggression, relational aggression, and prosocial behavior in the prediction of children’s future social adjustment. Child Development. 1996 Oct;67(5):2317–2327. [PubMed]
41. Vaillancourt T, Brendgen M, Boivin M, Tremblay RE. A longitudinal confirmatory factor analysis of indirect and physical aggression: Evidence of two factors over time? Child Development. 2003 Nov-Dec;74(6):1628–1638. [PubMed]
42. Herrenkohl TI, McMorris BJ, Catalano RF, Abbott RD, Hemphill SA, Toumbourou JW. Risk factors for violence and relational aggression in adolescence. Journal of Interpersonal Violence. 2007 Apr;22(4):386–405. [PubMed]
43. Monks CP, Smith PK, Swettenham J. Psychological correlates of peer victimisation in preschool: social cognitive skills, executive function and attachment profiles. Aggressive Behavior. 2005;31(6):571–588.
44. van Lier PAC, Crijnen AAM. Trajectories of peer-nominated aggression: Risk status, predictors and outcomes. Journal of Abnormal Child Psychology. 2005 Feb;33(1):99–112. [PubMed]
45. Campbell SB, Spieker S, Burchinal M, Poe MD, Nichd Trajectories of aggression from toddlerhood to age 9 predict academic and social functioning through age 12. Journal of Child Psychology and Psychiatry. 2006 Aug;47(8):791–800. [PubMed]
46. Kumpulainen K. Children involved in bullying at elementary school age: their psychiatric symptoms and deviance in adolescence An epidemiological sample. Child Abuse and Neglect. 2000;24(12):1567. [PubMed]
47. Scholte RHJ, Engels RCME, Overbeek G, de Kemp RAT, Haselager GJT. Stability in bullying and victimization and its association with social adjustment in childhood and adolescence. Journal of Abnormal Child Psychology. 2007 Apr;35(2):217–228. [PMC free article] [PubMed]
48. Murray-Close D, Ostrov JM. A Longitudinal Study of Forms and Functions of Aggressive Behavior in Early Childhood. Child Development. 2009 May-Jun;80(3):828–842. [PubMed]
49. Dodge KA, Lochman JE, Harnish JD, Bates JE, Pettit GS. Reactive and proactive aggression in school children and psychiatrically impaired chronically assaultive youth. Journal of Abnormal Psychology. 1997 Feb;106(1):37–51. [PubMed]
50. Mathieson LC, Crick NR. Reactive and Proactive Subtypes of Relational and Physical Aggression in Middle Childhood: Links to Concurrent and Longitudinal Adjustment. School Psychology Review. 2010 Dec;39(4):601–611.
51. Pellegrini AD. Bullies and victims in school: A review and call for research. Journal of Applied Developmental Psychology. 1998 Apr-Jun;19(2):165–176.
52. Griffin RSG, AM Childhood bullying: Current empirical findings and future directions for research. Aggression and Violent Behavior. 2004;9:379–400.
53. Orpinas P, Horne AM, Staniszewski D. School bullying: Changing the problem by changing the school. School Psychology Review. 2003;32(3):431–444.
54. Nansel TR, Overpeck M, Pilla RS, Ruan WJ, Simons-Morton B, Scheidt P. Bullying behaviors among US youth: prevalence and association with psychosocial adjustment. JAMA. 2001 Apr 25;285(16):2094–2100. [PMC free article] [PubMed]
55. Juvonen J, Graham S, Schuster MA. Bullying among young adolescents: The strong, the weak, and the troubled. Pediatrics. 2003 Dec;112(6):1231–1237. [PubMed]
56. Arseneault L, Walsh E, Trzesniewski K, Newcombe R, Caspi A, Moffitt TE. Bullying Victimization Uniquely Contributes to Adjustment Problems in Young Children: A Nationally Representative Cohort Study. Pediatrics. 2006;118(1):130–138. [PubMed]
57. Juvonen J, Graham S, Schuster MA. Bullying Among Young Adolescents: The Strong, the Weak, and the Troubled. Pediatrics. 2011 Mar 25;112(6):1231–1237. [PubMed]
58. Perren S, Alsaker FD. Social behavior and peer relationships of victims, bully-victims, and bullies in kindergarten. Journal of Child Psychology and Psychiatry. 2006 Feb;47(1):45–57. [PubMed]
59. Kumpulainen K, Rasanen E. Children involved in bullying at elementary school age: Their psychiatric symptoms and deviance in adolescence - An epidemiological sample. Child Abuse and Neglect. 2000 Dec;24(12):1567–1577. [PubMed]
60. Swearer SM, Song SY, Cary PT, Eagle JW, Mickelson WT. Psychosocial Correlates in Bullying and Victimization: The Relationship Between Depression, Anxiety, and Bully/Victim Status. Journal of Emotional Abuse. 2001;2(2/3):95–121.
61. Burk LR, Armstrong JM, Park J-H, Zahn-Waxler C, Klein MH, Essex MJ. Stability of Early Identified Aggressive Victim Status in Elementary School and Associations with Later Mental Health Problems and Functional Impairments. Journal of Abnormal Child Psychology. 2011 Feb;39(2):225–238. [PMC free article] [PubMed]
62. Haynie DDL. Bullies, Victims, and Bully/Victims:: Distinct Groups of At-Risk Youth. The Journal of early adolescence. 2001;21(1):29–49.
63. Solberg ME, Olweus D. Prevalence estimation of school bullying with the Olweus Bully/Victim Questionnaire. Aggressive Behavior. 2003;29(3):239–268.
64. Burk LR, Park J-h, Armstrong JM, et al. Identification of Early Child and Family Risk Factors for Aggressive Victim Status in First Grade. Journal of Abnormal Child Psychology. 2008;36(4):513–526. [PubMed]
65. Egger HL, Erkanli A, Keeler G, Potts E, Walter BK, Angold A. Test-Retest Reliability of the Preschool Age Psychiatric Assessment (PAPA) Journal of the American Academy of Child and Adolescent Psychiatry. 2006;45(5):538–549. [PubMed]
66. Luby JL, Si X, Belden AC, Tandon M, Spitznagel E. Preschool Depression Homotypic Continuity and Course Over 24 Months. Archives of General Psychiatry. 2009 Aug;66(8):897–905. [PMC free article] [PubMed]
67. Merrell KW, Gueldner BA, Ross SW, Isava DM. How effective are school bullying intervention programs? A meta-analysis of intervention research. School Psychology Quarterly. 2008;23(1):26–42.
68. Luby JL, Belden AC, Pautsch J, Si X, Spitznagel E. The clinical significance of preschool depression: Impairment in functioning and clinical markers of the disorder. Journal of Affective Disorders. 2009;112(1–3):111–119. [PMC free article] [PubMed]
69. Luby J, Heffelfinger A, Mrakotsky C, Hildebrand T. Preschool Feelings Checklist. St. Louis, Missouri: Washington University; 1999.
70. Essex MJ, Boyce WT, Goldstein LH, Armstrong JM, Kraemer HC, Kupfer DJ. The confluence of mental, physical, social, and academic difficulties in middle childhood II: Developing the MacArthur Health and Behavior Questionnaire. Journal of the American Academy of Child and Adolescent Psychiatry. 2002 May;41(5):588–603. [PubMed]
71. Keenan K, Coyne C, Lahey BB. Should relational aggression be included in DSM-V? Journal of the American Academy of Child and Adolescent Psychiatry. 2008 Jan;47(1):86–93. [PubMed]
72. Boivin M, Petitclerc A, Feng B, Barker ED. The Developmental Trajectories of Peer Victimization in Middle to Late Childhood and the Changing Nature of Their Behavioral Correlates. Merrill-Palmer Quarterly. 2010;56(3):231–260.
73. Werner NE, Crick NR. Maladaptive peer relationships and the development of relational and physical aggression during middle childhood. Social Development. 2004;13(4):495–514.
74. Mesman J, Koot HM. Early preschool predictors of preadolescent internalizing and externalizing DSM-IV diagnoses. Journal of the American Academy of Child and Adolescent Psychiatry. 2001 Sep;40(9):1029–1036. [PubMed]