PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
 
J Abnorm Child Psychol. Apr 2007; 35(2): 217–228.
Published online Feb 13, 2007. doi:  10.1007/s10802-006-9074-3
PMCID: PMC1915632
Stability in Bullying and Victimization and its Association with Social Adjustment in Childhood and Adolescence
Ron H. J. Scholte,corresponding author1 Rutger C. M. E. Engels,1 Geertjan Overbeek,1 Raymond A. T. de Kemp,1 and Gerbert J. T. Haselager2
1Institute of Family and Child Care Studies, University of Nijmegen, P.O. Box 9104, 6500 HE Nijmegen, The Netherlands
2Department of Developmental Psychology, University of Nijmegen, Nijmegen, The Netherlands
Ron H. J. Scholte, r.scholte/at/pwo.ru.nl.
corresponding authorCorresponding author.
Received November 14, 2005; Accepted October 6, 2006.
This study examined the concurrent and longitudinal associations between stability in bullying and victimization, and social adjustment in childhood and adolescence. Participants were 189 girls and 328 boys who were studied in primary school and in secondary school. The mean age of the participants was 11.1 years in primary school and 14.1 years in secondary school. The measures consisted of peer reported social and personal characteristics. Children who bullied in childhood and adolescence were less liked and more disliked in childhood, and more aggressive and disruptive both in childhood and adolescence, than children who bullied only in childhood or adolescence. Children who bullied or who were victimized only in childhood did not differ largely in adolescence from the children that were never bullies or victims. Children who were victimized in adolescence closely resembled those who were victimized in childhood and adolescence in terms of being liked or disliked, being nominated as a friend, and shyness. The study stresses the need to distinguish between stable and transient bullies and victims.
Keywords: Bullying, Victimization, Stability, Social adjustment
Bullying in school classes refers to negative physical or social actions that are repeated over time by one or more other persons towards a person that can not easily defend (Olweus, 1991). Bulling involvement seems relatively stable over time (Boulton & Smith, 1994; Kumpulainen, Räsänen, & Henttonen, 1999) and has been related to various psychosocial adjustment problems in childhood and adolescence. Much research on bullying involvement, especially on victims, has focused on internalizing indicators of adjustment (see for a review Hawker & Boulton, 2000). Little is known about the overt, interpersonal behavioral characteristics of bullies and victims, and even less is known about the association between stability in bullying and victimization and these social characteristics. The aim of the present study was to examine the link between stability in bullying and victimization, and individual differences in social behaviors that are salient to the peer environment. More specifically, we were interested in differences in childhood and adolescent social adjustment of transient and stable bullies and victims.
Bullies, victims, and adjustment
In primary schools between 20% and 30% of the children are victims of bullying, while between 10% and 20% of the children are bullies (Smith et al., 1999), both in Western and non-Western countries (Eslea et al., 2003). Cross-sectional research on the adjustment of bullies revealed that bullies are more rejected and less popular (Boulton & Smith, 1994), and display more antisocial, aggressive and disruptive behavior than non-involved children (Pellegrini, Bartini, & Brooks, 1999; Rigby & Cox, 1996). Furthermore, longitudinal studies suggest that childhood bullying is associated with social maladjustment in adolescence (Kumpulainen & Räsänen, 2000). Research on the adjustment of victims showed that these children are socially isolated and rejected, and have fewer friends (Hodges, Malone, & Perry, 1997; Hodges & Perry, 1997). In addition, victims tend to be more submissive in their interactions with peers (Schwartz, Dodge, & Coie, 1993) and show overt signs of helplessness and distress (Perry, Williard, & Perry, 1990).
Longitudinal research on bullies, victims and adjustment usually linked being a bully or a victim in childhood to adjustment measures in adolescence, without accounting for the chronicity in bullying or victimization. For example, Kumpulainen and Räsänen (2000), reported that children who bullied at age 8 and 12 displayed more externalizing behavior, hyperactivity, and relationship difficulties when they were 15 years old, while children who were victimized at age 12 were more likely to experience relationship problems such as being irritable, isolated, and rejected three years later. Camodeca, Goossens, Meerum Terwogt, and Schuengel (2002) found that stable victims were more reactively aggressive than transient victims. Neither of these studies, however, examined whether the participants were still bullies or victims at the follow-up assessment. Consequently, the longitudinal associations that have been reported may have been confounded by concurrent associations between adolescent bullying involvement and adolescent adjustment.
Adequate descriptions of longitudinal associations between childhood bullying involvement and adolescent social adjustment may require distinguishing between children who remain involved in bullying from childhood to adolescence (i.e. stable bullies and victims) and those whose involvement is restricted to childhood (i.e., transient or unstable bullies and victims). Various studies showed that a substantial number of children display a stable pattern of bullying or victimization (e.g., Boulton & Smith, 1994; Camodeca et al., 2002; Salmivalli, Lappalainen, & Lagerspetz, 1998). Boulton and Smith (1994) found correlations of around.60 between bullying at age 8 and one year later, while Kumpulainen, Räsänen, and Henttonen (1999) reported that 25% of 8-year old bullies had turned into stable bullies by the time they were 12, and 15% of the victims had become stable victims. However, very few studies have examined the associations between stability in bullying involvement and interpersonal behavioral characteristics.
Stability in being a bully or a victim
Stability in being a bully or a victim in school can be caused by two mechanisms: continuity in social environment and continuity in children’s interactional styles. According to Caspi, Elder, and Bem’s cumulative continuity model (1987; Caspi, Bem, & Elder, 1989) and to Scarr’s genotype environment correlations model (Scarr & McCartney, 1983; Scarr, 1985; Scarr, 1992), children’s social maladaptive behaviors may direct them into social environments that perpetuate these behaviors. The idea is that children with specific behaviors select and create environments that promote and maintain their behavior. This may hold for bullies as well as victims. It is well known that antisocial boys affiliate with boys who are similarly aggressive and deviant (Cairns, Cairns, Neckerman, Gest, & Gariepy, 1988), and that victims are more likely to have friends who are less accepted by peers or who themselves are victimized (Hodges et al., 1997; Salmivalli, Huttunen, & Lagerspetz, 1997), thus reinforcing their behavior. Additionally, peers may also reinforce bullying and victimization by conferring reputations that may lock bullies and victims into their specific roles (DeRosier, Cillessen, Coie, & Dodge, 1994). These reputations are difficult to change as long as the school class constellation remains unchanged (Hymel, Wagner, & Butler, 1990).
The social environment may reinforce bullies’ behaviors yet in another way. Observational studies (Craig & Pepler, 1997; Pepler & Craig, 1995; Pepler, Craig, & Roberts, 1998) showed that peers often do not intervene in bullying. Bullies may perceive this lack of intervention as a signal that peers condone their bullying behavior. Furthermore, studies on participant roles reveal that some peers may even actively reinforce the bullies by encouraging gestures or by laughing during bullying episodes (Salmivalli, Lagerspetz, Björkqvist, Osterman, & Kaukiainen, 1996).
Stability in being a bully or a victim may also be due to continuity in bullies’ or victims’ interactional styles (i.e., interactional continuity, Caspi et al., 1987, 1989) that place them at risk for prolonged bullying or victimization, even after changes in the social environment. Being stably involved in bullying in the formative years of primary school may deprive children from positive social experiences, inhibiting the acquisition of prosocial skills and fostering social skills deficits. Because these children do not learn how to adequately react in social interactions, they may develop dysfunctional interactional styles which may make them prone to social adjustment problems later in life. In this way, a vicious cycle between bully’s or victims’ interactional styles and their bullying or victimization may be established (cf. Kochenderfer-Ladd & Wardrop, 2001). For example, Snyder et al. (2003) showed that for some children victimization was situational, while for other children victimization became a trait like status, possibly because of this dysfunctional interactional style.
Stability in being a bully or a victim, and adjustment
Although the processes contributing to stability in bullying or victimization are well described, very few studies have examined the behavioral profiles of different bully or victim trajectory types over time. Research has indicated that most bullies desist after some time, but a small group persists (Broidy et al., 2003; Loeber & Hay, 1997; Le Blanc & Loeber, 1993). Because of their aggressive interactional styles and their lack of opportunities to develop social skills, these children are at risk for higher levels and more serious forms of aggression, poor peer relations and social maladjustment later in life (Pope & Bierman, 1999). This behavioral style reflects Olweus’ (1991) notion that bullying is not an isolated phenomenon but rather a component of more general antisocial and rule-breaking behavioral patterns.
Regarding victims, only two studies seem to have longitudinally examined the adjustment of stable and transient victim groups. Juvonen, Nishina, and Graham (2000) found that stable victims were lonelier and reported a lower self-worth than transient victims (ie. those who were victimized one year earlier but were no longer victims). Noteworthy, transient victims did not differ from stable non-victims on loneliness, self-worth or depressive symptoms. Covering a two-year period, Smith, Talamelli, Cowie, Naylor, and Chauhan (2004) found that stable victims had fewer friends at school and scored higher on self-reported and teacher-reported emotional problems and peer problems. As was found by Juvonen et al. (2000), in general, the transient victims did not differ largely from the stable non-victims. The latter seems to indicate that psychosocial problems only occur at the time when the victimization takes place. It also supports the cessation hypothesis (Kochenderfer-Ladd & Wardrop, 2001) which states that the psychosocial problems are likely to disappear as soon as the victimization ends.
The present study
The present study investigated whether stable bullies or victims showed specific behavioral patterns that distinguished them from children who were involved in bullying only in childhood or adolescence. Whereas previous studies on bullying and victimization often focused on psychological adjustment in terms internal characteristics such as self-esteem, loneliness, or emotional stability, our aim was to examine the interpersonal, social adjustment as it was reported by the peers. The bully groups and victim groups were compared on social characteristics that have been identified in previous research as highly relevant correlates for describing bullies or victims in childhood or adolescence. These characteristics included being liked and being disliked (Hodges & Perry, 1999; Pellegrini et al., 1999), friendships (Hodges et al., 1997; Hodges & Perry, 1999), aggressive and disturbing behaviors (Boulton & Smith, 1994; Snyder et al., 2003), help seeking, shyness (Boulton & Smith, 1994) and offering help and cooperation (Boulton & Smith, 1994). We used peer reports to assess these constructs for children may have distorted views of their own social competence (Lemerise & Arsenio, 2000) and self-reports may thus not always be the most reliable means of obtaining information about children’s social functioning in peer groups.
The longitudinal group design allowed us to test a number of hypotheses. The hypotheses concerning bullies were in part guided by the work of Olweus (1991) and Loeber (Loeber & Hay, 1997; LeBlanc & Loeber, 1993) and the cumulative continuity hypothesis (Caspi et al., 1987) and genotype-environment correlations model (Scarr, 1983, 1992; Scarr & McCartney, 1983). We hypothesized that due to the accumulative effects of their negative peer interactions stable bullies would exhibit a pattern of more severe, negative and hostile behaviors in childhood and adolescence, and would have poorer adjustment, compared to bullies who bullied for a restricted period of time (i.e., only in childhood or adolescence). Because we assumed that adolescents who first started to bully in adolescence may have done so in order to obtain dominance (Pellegrini et al., 1999) and not because of a specific stable behavioral pattern, we expected that they would show specific maladaptive features in adolescence but not in childhood.
Concerning the victims, we hypothesized that stable victims would show higher levels of peer perceived social problems in adolescence compared to childhood or adolescence-only victims. This was based on the idea that stable victims were exposed to victimization for a substantially longer period and thus have had fewer opportunities to acquire positive social skills and experiences. In line with the cessation hypothesis (Kochenderfer-Ladd & Wardrop, 2001) and the findings provided by Juvonen et al. (2000) and Smith et al. (2004) we expected that childhood-only victims would not differ from the stable non-involved children in adolescence. Although we did not have theoretically based hypotheses, we tested whether it would be possible to already predict in childhood which children would become new victims in adolescence (i.e., adolescence-only victims). Finally, we tested whether gender moderated the associations between stability and adjustment. Because we did not have theoretically derived hypotheses about the interaction the interaction effects of gender, testing these effects was entirely exploratory, and no research question specifically addressed this issue.
Participants
Participants were 517 children (189 girls, 328 boys) who took part in the third and fourth wave of a longitudinal study that started in 1985. In the first two waves (1985 and 1986), 231 boys were examined (see for detailed information Cillessen, van Yzendoorn, van Lieshout, & Hartup, 1992). In waves 3 (1991) and 4 (1994), the initial boys were again assessed, but now the assessment also included the boys’ classmates, resulting in total samples of 2521 and 3361 children, for Wave 3 and 4, respectively. The sample of the present study consisted of 517 children who were present both in Wave 3 (childhood data) as well as in Wave 4 (early adolescence data). Because the 231 initial longitudinal participants were boys, some of whom attended schools with a predominantly male population in secondary education (e.g., technical education), there were more boys than girls in the present study. Attrition bias checks using t-tests comparing the present sample with the larger childhood and adolescence samples that were dropped from this study (i.e., the Wave 3 and Wave 4 samples minus the participants who were in the longitudinal sample) revealed that the present study’s participants did not differ on bullying or victimization, nor on any other study variable, from the larger cross-sectional childhood or adolescence sample. They did also not differ on ethnicity or education, nor on socioeconomic status of the parents. In 1991 (Wave 3), participants’ age was 11 years, 1 month (SD = 11 months) and ranged from 10 to 13. The children came from 100 elementary school classes in the Nijmegen-Arnhem area in The Netherlands. In 1994 (Wave 4), all participants were enrolled in secondary education, and were distributed across 131 school classes. Their mean age was 14 years and one month (SD = 11 months), and ranged from 13 to 16. Ninety-five percent of the participants were of Dutch origin, while five percent of the adolescents were ethnic minorities: 1.5% came from Surinam, the Netherlands Antilles, and the Molucca Islands; 2% from Mediterranean countries; and 1.5% from elsewhere. Socio-economic status was based on the classification of the parents’ occupations. The lower socio-economic status contained occupations such as factory workers, while middle socio-economic status consisted of occupations like for example teachers and nurses. The higher socio-economic status included occupations such as lawyers, and physicians. It showed that 22% of all children had a lower, 43% a middle, and 35% a higher socio-economic status. Overall, at the two waves the participants appeared to be representative of the Dutch school population in the geographic area in which the children lived and at the time the assessments were made.
Procedure
Both the childhood data and adolescence data were obtained by classroom data collections, arranged separately for each school class (for detailed descriptions, see Haselager, Hartup, van Lieshout, & Riksen-Walraven, 1998, and Scholte, van Lieshout, & van Aken, 2001, for childhood and adolescent data assessment, respectively). In order to ensue that school staff and parents obtained as much information as they needed before the study was conducted, letters were sent to schools and children took home these letters for their parents to read. In these letters parents were informed about the study and were asked for passive consent. As in other large scale studies that we conducted in the Netherlands, there were no parents in the present study who did not allow their children to participate. Consent was also obtained from the children and adolescents themselves and from school authorities. Participants were guaranteed confidentiality in the collection and maintenance of the data. In both assessment waves the data collection session started with a brief introduction and class instruction, given by a trained examiner. During the one-hour assessment, the children were asked to fill out a questionnaire booklet. Before answering the bullying involvement questions, the children were provided a definition of “bullying other children” (Olweus, 1989) by the examiner.
Measures
Childhood measures (peer nominations)
We developed a sociometric questionnaire containing 11 items on peer reported social behavior. This measure was similar to the widely used instrument developed by Coie, Dodge, and Coppotelli (1982). Children were asked to nominate a maximum of three classmates who best fitted the descriptions, in a rank ordered way such that the classmate who best fitted a description was nominated first, followed by the second and third best fitting classmate. Same- as well as cross-sex nominations were allowed on each description but self-nominations were not. The children were presented a roster of their classmates to use as a reference in making their nominations on the sociometric questions. Bullying other children was assessed by the question “Which children in class often bully other children, or pick on them.?” Being bullied (victimization) was assessed by the question “Which classmates are often bullied and picked on by other children.?” Being liked and being disliked were based on the questions “Which children in your class do you like most?” and “Which children in your class do you like least,?” respectively. Aggression referred to starting fights in class, while ‘Disruption’ assessed disturbing and disruptive behavior. Cooperation assessed being considerate and cooperative, while ‘Offering help’ indicated offering help to others. Seeking help assessed which children sought help often, while Shyness referred to being shy. Being nominated as a friend, finally, was based on the question “Which three children in you class are your friends.” This variable was used as an indicator of a child’s social integration in the class. Being nominated as a friend was not synonym with being liked because it was possible that children liked classmates without being friends with them.
Adolescence measures (peer nominations)
Adolescents were also presented a roster with the names of their classmates. In grades 1 through 3 of secondary education, adolescents were in the same group all year, and this group served as the nomination reference group. The reference group of the adolescents who were in grade 4 of secondary education consisted of their root class, which consists of those classmates with whom they spent most of the classes, and with whom they share the same mentor. The sociometric questionnaire administered in the adolescent sample contained 25 questions referring to liking and disliking, bullying and victimization, number of friends, and peer group behavior (Scholte et al., 1997). Nine of the 25 items were similar to those asked in childhood (see below), while 16 new items assessed peer-perceived self-confidence, emotional stability, and school achievement. In the present study we only used items in adolescence that were similar to the items in childhood, added with the item on “insecurity’ which in a way reflected the childhood question on seeking help.
Participants could nominate up to five classmates on each of the questions, and were asked to nominate first the classmate who fitted an item best, followed by the classmate who fitted an item second best et cetera. In the present study, only the first three nominations on each item were used in the analyses, which was similar to the number of nominations on each item in the childhood data. Same and cross-sex nominations were permitted, self-nominations were not allowed. Assistance was provided if needed. Bullying others and Being bullied were assessed by the questions “Which classmates bully others,” and “Which classmates are being bullied,” respectively. Being liked and being disliked, being nominated as a friend, aggression, disruption, cooperation, and shyness were assessed by similar questions as used in the childhood assessment. Insecurity was assessed by the item ‘Which classmates are insecure and seem to lack self-confidence.’ The intercorrelations among these indicators of social adjustment are given in Table Table11.
Table 1
Table 1
Intercorrelations among social adjustment measures in childhood
Sociometric nominations were processed as follows. For each participant, scores on each item were determined by summing all received nominations from classmates on that item. These two raw scores were transformed into within-class probability scores (p-scores) assuming a generalized binomial distribution, thus correcting for unequal numbers of nominations made among children and differences in class size (Newcomb & Bukowski, 1983). The p-scores were then z-standardized across all participants.
Each peer nomination scale in childhood and adolescence consisted of one item. Because peer nominations involve aggregating across multiple respondents, - in our study on average 23 per class- peer nominations based on one item tend to be reliable (Coie, Dodge, & Kupersmidt, 1990), because, in contrast to self-reports, they are based on the reports of many informants which may decrease the influence of individual bias (Boulton & Smith, 1994). In order to be able to describe the differences between the different groups of victims and bullies more clearly, the childhood and adolescence social adjustment variables were standardized within the total sample and these standardized scores were subsequently analyzed.
The correlations between the childhood measures and adolescence measures are presented in Table Table2.2. As can be seen, correlations between the childhood and adolescence data were low to moderate, with the highest correlations between childhood aggression and disruption and adolescence aggression and disruption (r’s between .28 and .40).
Table 2
Table 2
Correlations between social adjustment measures in childhood and adolescence
Classification of bullies, victims and non-involved children
To determine which children and adolescents were bullies or victims, we used the peer nomination scores on Bullying others and Being bullied. Children and adolescents who scored 1 standard deviation above the mean on Bullying others were considered bullies, and children and adolescents who scored one standard deviation above the mean on Being bullied were regarded as victims. This procedure has been used in other studies (e.g., Pellegrini et al., 1999; Solberg & Olweus, 2003).
Of all children in childhood, 9 % were victims (n = 49; 17 girls of the total sample of girls, 32 boys of the total sample of boys), 19% were bullies (n = 100; 6 girls), and 71% were not involved in bullying (n = 368; 163 girls). In adolescence these percentages were 11% (n = 55; 14 girls) for victims, 20% (n = 104; 12 girls) for bullies, and 69% (n = 358; 160 girls) for non-involved children. Chi-square analyses indicated that while boys and girls were equally likely to be victims, boys were more likely to be bullies in childhood (χ2 (2) = 44.57, p < .001) and in adolescence (χ2 (2) = 37.34, p < .001). Because the focus of this study was on bullies or victims, children who scored one standard deviation above the mean on being bullied and bullying either in childhood or adolescence (i.e., bully-victims, n = 9) were not included in the analyses.
To examine the longitudinal involvement in bullying, a chi-square analysis was conducted with bully status in childhood and adolescence as factors. The chi-square analysis for the total sample (χ2(4, N = 517) = 111.30, p < .001) and the subsequent test of standardized residuals (Haberman, 1973) showed that victimization and bullying were relatively stable across this three year period. Forty-six percent (n = 46) of the childhood bullies persisted into adolescence (i.e., Stable Bullies), whereas the others had either stopped being involved (i.e., Childhood Bullies, 45%, n = 45) or had turned into victims (9%, n = 9). Forty-three percent (n = 21) of the children who were victims in childhood were still victims in adolescence (i.e. Stable Victims), 51% of the childhood victims (n = 25) were not involved in bullying in adolescence (i.e. Childhood Only Victims), while 6% (n = 3) had turned into bullies. Of all the children not involved in bullying in childhood, 15% (n = 55) started bullying in adolescence (i.e., Adolescence Only Bully), and 7% (n = 25) became victims (i.e., Adolescence Only Victim). The children who were not involved in childhood nor adolescence were considered Non-involved, and served as a reference group. Gender differences in bullying and victimization continuity indicated that male and female childhood victims were equally likely to become a stable victim, but that the continuity of bullying other children was low in girls and high in boys (χ2 (6, N = 517) = 66,13, p < .001). Except for one girl, all stable bullies were boys.
To examine the association between the stability in bullying and victimization, and social adjustment, the three groups of bullies as well as the three groups of victims were compared. For reasons of conceptual clarity, children who bullied in childhood and had turned into victim in adolescence and those who were victims in childhood but became bullies in adolescence were dropped from further analyses. Bullying group by gender interactions were examined, as well as gender main effects. The latter were significant in childhood (Wilks Λ = .93, F (9, 343) = 2.94, p < .01) and adolescence (Wilks Λ = .93), F (8, 342) = 2.99, p < .01) and indicated that both in childhood and adolescence, boys were more disliked, more aggressive and disruptive, less cooperative, and less shy than girls. In addition, boys were less often nominated as a friend in childhood than girls did, and were less insecure in adolescence. Because the Stable Bullies included only one girl, gender by bully group interactions could not be examined in analyses that included the Stable Bullies group.
Social adjustment of bullies in childhood
To examine whether the Stable Bullies experienced more social problems in childhood than the other bullies and non-involved children, and whether Adolescence Only Bullies showed signs of maladjustment already in childhood, a MANOVA was conducted with bullying groups (Childhood Only Bullies, Adolescence Only Bullies, Stable Bullies, and Non-involved) and gender as independent variables and the childhood adjustment scores as dependent variables.1 A significant multivariate relation was found, Wilk’s Λ = .32, approx F (27, 1230) = 21.81, p < .001. Except for offering help, all univariate tests were significant (see Table Table3).3). Childhood Only Bullies scored higher on help seeking behavior than the Stable Bullies. However, Stable Bullies were less liked and more disliked by their peers, were less often nominated as a friend, and displayed more aggression and disruption than all other children. Adolescence Only Bullies scored lower on being disliked, aggression, disruption, and help seeking, and higher on cooperation than Childhood Only Bullies.
Table 3
Table 3
Childhood and adolescent adjustment of childhood only, adolescence only, and stable bullies, and non-involved children
Social adjustment of bullies in adolescence
Next, we wanted to answer the question whether the Stable Bullies differed from the other bullies and non-involved children in adolescence. To address this question, a MANOVA was conducted in which the three bullying groups and gender served as the independent variables, and the adolescent measures as dependent variables. A significant multivariate effect emerged for bully group (Wilk’s Λ = .49), approx. F (24, 1218) = 14.14, p < .001. The results of the univariate and post hoc analyses (Student Newman-Keuls, p < .05) are presented in Table Table3,3, and indicated no differences between the groups on being liked, number of friends, and insecurity. Childhood Only Bullies scored more positive on the other measures that did the other bullies, and in fact did not differ largely from the Non-involved children, except for their lower scores on shyness, indicating that they turned out to be normally functioning adolescents who did not display problematic social behavior. Stable Bullies turned out to be distinctive from the Adolescence Only Bullies in that they displayed higher levels of aggression and disruption. As the Stable Bullies consisted only of boys while the other groups also contained girls, the analyses were repeated comparing the Stable Bullies with only their male counterparts from the other groups. These analyses resulted in similar findings.
Social adjustment of victims in childhood
The next research goal was to examine whether the Stable Victims displayed lower social adjustment in childhood than the other victim groups and non-involved children and whether Adolescence Only Victims could already be identified in childhood. To address this goal, a MANOVA with the victim groups (Stable Victims, Childhood Only, and Adolescence Only Victims and non-involved children) and gender as independent, and childhood social adjustment as dependent variable were conducted. Results indicated a significant multivariate effect for group (Wilk’s Λ = .68), approx. F (27, 1002) = 5.33, p < .001. Follow-up univariate analysis of variance were significant for all variables except for disturbance and shyness (Table (Table4).4). Compared to the Childhood Only Victims, Stable Victims scored significantly higher on being disliked and help seeking behaviors. In contrast to the other victims, Adolescent Only Victims did not differ from the Non-involved children on any of the behavioral characteristics. There was no significant multivariate gender by group interaction.
Table 4
Table 4
Childhood and adolescent adjustment of childhood only, adolescence only, and stable victims, and non-involved children
Social adjustment of victims in adolescence
The last question that we addressed was whether Stable Bullies showed more signs of maladjustment in adolescence than the other victim groups and whether those who were victimized only in childhood would show social maladjustment problems in adolescence. A MANOVA was run with the victim and noninvolved groups, and gender as independent variables, and the adolescent measures as dependent variables. Findings revealed a significant multivariate effect for victim group (Wilk’s Λ = .74) approx. F(24, 992) = 4.58, p < .001. The results of the univariate tests and post hoc comparisons are given in Table Table4.4. As can be seen, Childhood Only Victims scored more positively than the Stable and Adolescence Only Victims on being liked and being disliked, received friendship nominations, shyness, and insecurity. In fact, except on being liked, they did not differ from the Non-involved children, which indicated that being victimized only in childhood was not reflected in specific social behavior three years later. As Table Table44 further shows, Stable Victims did not significantly differ from the Adolescence Only Victims on any of the peer nominated indicators of social adjustment. A multivariate victim group by gender interaction was found, (Wilk’s Λ = .88), approx. F (24, 992) = 1.86, p < .01. Univariate analyses showed a significant group by gender interaction for shyness (F (3, 349) = 5.07, p < .01 which revealed that in the Stable Victim group boys scored substantially higher on peer reported shyness than girls, whereas in the other three groups, boys scored lower.
The findings of our study give insight into the peer perceived behavioral profiles of children who were bullies or victims for a restricted period of time and those who were involved over a period of three years. They show that stable bullies and victims displayed a behavioral pattern in childhood and adolescence that clearly distinguished them from the children whose bullying or victimization was restricted to childhood. The latter did not show social adjustment problems later in adolescence, while the former did. Children who were victimized only in adolescence showed similar signs of social maladjustment in adolescence as children who were consistently victimized.
Our study shows that half of the childhood bullies turned into stable bullies, and half of the childhood victims into stable victims. In contrast to stability in victimization, continuity in bullying was very gender specific: Whereas only one out of 12 girls continued to bully in adolescence, this held true for almost half of the boys. An explanation for this gender specificity in stability may be that bullying behavior in our study referred more to overt aggression, which is more typical for boys, than to relational aggression (cf. Crick & Bigbee, 1998), which is more typical for girls. It is also possible that for girls, in contrast to boys, bullying may be more related to specific social situations rather than to an individual characteristic reflecting an underlying antisocial and aggressive personality pattern (Salmivally et al., 1998).
With respect to the behavioral profile of bullies, this investigation revealed that, compared to Childhood Only Bullies, the Stable Bullies were less liked and more disliked, were less often nominated as a friend, were more aggressive and disruptive, and scored lower on help seeking in childhood. This finding supports our hypothesis which states that, because of their socially deviant behavioral profile, children who will continue to bully over time could already be distinguished in childhood from those who will desist after some time (Loeber & Hay, 1997). This negative behavioral pattern may result in accumulation of negative social consequences such as continued peer rejection and fewer opportunities to acquire adequate coping skills. Through this interactive continuity (Caspi et al., 1987) the deviant behavioral pattern is likely to be maintained into adolescence. This was reflected in finding that the Stable Bullies showed signs of peer perceived social maladjustment that distinguished them from Adolescence Only Bullies, which was in line with our hypothesis. Olweus’ (1991) notion that bullying reflects a stable aggressive and antisocial, rule-breaking personality pattern predisposing children to social maladjustment and delinquency in adolescence only matches the behavioral profile of the Stable Bullies. Given that Childhood Only Bullies did not differ much from the Non-involved children in adolescence in their peer perceived social adjustment, an important conclusion of our study is that half of the children who bully in childhood, may not constitute a group at risk for later social problems.
The possible role of friends in relation to bullying deserves closer attention. In childhood, Stable Bullies were less often nominated as a friend than the Childhood Only Bullies and Non-involved children. Although we do not know whether these nominations were reciprocated, they might suggest that Stable Bullies have fewer reciprocal friends or at least fewer peers who consider them to be friends. The explanation may be that because of their behavioral profile, these bullies are less attractive as a friend, which is in line with studies showing a negative association between socially deviant behavior and positive peer relations (Rose & Asher, 1999). However, caution must be taken in interpreting these findings, because the differences between the childhood only and stable bullies in the number of received friendship nominations were no longer present after the male only classes were removed. This finding might indicate that in male only classes stable bullies receive fewer friendship nominations than childhood only bullies, while this may not be true for classes where boys and girls are equally present. Because it is not yet clear why this is the case, further study is warranted.
In adolescence, although Stable Bullies (and Adolescence Only Bullies) were still more disliked than all other adolescents, they had similar number of peers who thought of them as friends, given that no differences existed between them and other adolescents in number of received friendship nominations. Recently, Cillessen and Mayeux (2004) showed that in adolescence antisocial behaviors including bullying are increasingly linked to social status, suggesting that bullying may become more accepted. Consequently, associating with and becoming friends with bullies may also become more accepted. Since friends may passively or actively encourage bullying (cf Salmivalli et al., 1996) bullies may feel reinforced and continue with their behavior (cumulative continuity, Caspi et al., 1987). The fact that Stable Bullies are as often nominated as a friend as other adolescents may reflect bistrategic orientations as described by Hawley (2003; Hawley, Little, & Pasupathi, 2002). That is, they display antisocial behavior but at the same time seem to be able to somehow convey to certain peers that they are their “friends” even if the bully him or herself does not necessarily consider that person a friend. This suggests that these bullies are more likely to be skilled manipulators rather than to be socially inadequate (Sutton et al., 1999). The environmental influences, combined with the Stable Bullies’ behavioral pattern, may make them relatively resistant to behavior change.
Regarding the victims, Stable Victims showed a pattern of peer perceived social maladjustment in childhood that clearly distinguished them from the Childhood Only Victims. They were more disliked by their peers and were more likely to be perceived as seeking help from others. As the study by Boulton and Smith (1994) suggests, a pattern of consistently seeking help may signal that these children lack social self confidence. This, rather than the lack of prosocial behaviors such as cooperation or offering help may predispose children to remain victimized from childhood to adolescence (Boulton & Smith, 1994; Egan & Perry, 1998). This feature may prevent them from successfully interacting with peers in childhood and adolescence (interactive continuity, Caspi et al., 1987), depriving them of positive peer experiences. It may also affect the way they create their own environment (Caspi et al., 1987; Scarr, 1985, 1992; Scarr & McCartney, 1983) in that they may be more inclined to withdraw from social interactions and make them prone to be targeted by bullies in the group throughout their school life.
We found that Stable Victims did not have a more problematic social behavioral profile in adolescence than Adolescence Only Victims did. This is in contrast to our expectation that the longer the victims experienced victimization, the more impaired their social behaviors would be (Kochenderfer-Ladd & Wardrop, 2001), and thus that Stable Victims would show the most problematic social adjustment. Because adolescents who are victimized only in adolescence show the same social adjustment problems as the adolescents who have been victimized for a long period of time, this finding might indicate that the duration of victimization is relatively unrelated to the severity or magnitude of peer reported social adjustment problems in adolescence. Our findings showed that in general, victimized or noninvolved boys were less shy than girls, however, those who were stably victimized scored higher on shyness in adolescence. This is consistent with other findings of negative social implications of shyness for boys (Kerr, 2000) and suggests that being shy in adolescence may place boys at particular risk for being victimized for a prolonged period.
One of the positive findings of the present study seems to be that Childhood Only Victims may become normally adjusted children in adolescence, at least in terms of their peer reported social adjustment. This suggests that victimization experiences that are restricted to childhood do not necessarily translate into impaired social functioning observed by the peer context. This finding does not support the widely held assumption that being a victim of bullying in childhood is related to social adjustment problems in adolescence (e.g., Kumpulainen & Räsänen, 2000; see also Parker & Asher, 1987). It extends more recent studies because it shows that victimization is not only concurrently associated with psychological adjustment (Juvonen et al., 2000; Smith et al., 2004) but also with social adjustment. Like the findings in these other two studies, our findings may lend support for the cessation hypothesis (Kochenderfer-Ladd & Wardrop, 2001) and might indicate that social problems can disappear once the victimization is over. Nevertheless, it should be remembered that we assessed social adjustment as perceived by the peer group, and that findings might have been differed when more subjective experiences of feelings would have been used. Finally, the Adolescence Only Victims did not differ from the Non-involved children in their peer reported childhood social adjustment. This implies that children who will become victims later in their school career do not necessarily show specific social behavioral patterns that are salient to their peers and on the basis of which they could already be identified in childhood. An alternative explanation might be that becoming a victim in adolescence may actually have little to do with social behavior per se. Rather than because of their specific social behavior, adolescents may become victimized because of how they dress, how they look etc, which becomes more salient at this age. As the present study showed, these victims do have social adjustment problems in adolescence, but these problems may have resulted from being victimized rather than caused it.
The present study has a number of positive features. It is among the first to longitudinally examine stability in bullying and victimization in relation to peer perceived social adjustment, during the transition from primary to secondary education. Bullying and victimization have been studied in a cultural context (i.e., the Netherlands) that has not been reported on extensively. This issue may bear some importance given that dimensions of peer relations can have differing connotations by culture (e.g., Schneider, 2000). For example, a recent study revealed that large variations existed between western counties such as England, Spain, Italy, and Ireland not only in the prevalence of bullies and victims but also in how bullying and victimization were related to social relations (Eslea et al., 2003). This indicates that findings from one western country may not be generalized to another. Our findings revealed that in general the behavioral profiles of bullies and victims in the Netherlands were similar to those reported in studies in other western cultures such as the USA (Hodges & Perry, 1999; Pellegrini & Long, 2002) and Great Britain (Boulton & Smith, 1994).
Nevertheless, several caveats should be kept in mind. First, we only examined bullies and victims, but not bully-victims. This latter group turned out to be very small in our sample (ie., 2%, n = 9), comparable to other studies (Boulton & Smith, 1994; Olafsen & Viemerö, 2000; Solberg & Olweus, 2003). Additional exploratory analyses that we conducted on the bully-victims showed that the only feature that distinguished them from the other victims was their aggressive and disturbing behavior in class, on which they scored higher than all other victims, both in childhood and adolescence. In fact they were as aggressive as the Stable Bullies in childhood and adolescence. Nevertheless, due to the small sample size these results are only exploratory, and more research seems warranted to further describe the social correlates of bully-victims. Second, because the Stable Bullies contained only one girl, gender interactions could not be examined in analyses comparing this group of bullies with all other groups of bullies and the findings regarding Stable Bullies may thus not generalize to female bullies. Third, although this study was longitudinal in nature, causality in terms of victims’ social behavior triggering bullies bullying behavior or vice versa was not implied. Fourth, this study used peer reports to classify children and adolescents, and to assess the dependent variables. Using one source of information increases the risk of shared-method variance and inflated associations between independent and dependent variables. In addition, several behaviors assessed in our study were based on single items, which may raise some concerns about the validity of these behaviors. Finally, we have focused on the social adjustment of the bullies and victims as it is perceived and reported by their peer environment. Even though the use of peer reported social adjustment measures was informative, we could not identify which of the children who were not victimized in childhood became victims in adolescence. We did not examine children’s subjective experiences such as self-esteem, social insecurity, and loneliness. Exploring these individual subjective experiences may prove more valuable in predicting who is at risk to become victimized in adolescence and may add to our understanding of the consequences of being a bully or victim.
Despite these caveats the present investigation shows that it is highly relevant for future scientific research on bullying, victimization, and adjustment to distinguish between children who are only involved in childhood or adolescence, and those who are chronically involved from childhood into adolescence. The positive message of our study is that many of the childhood victims and bullies did not seem to show social adjustment problems in adolescence, as perceived by their peers. The more troublesome message is that between 40 and 50% of the childhood bullies and victims will continue to be involved in bullying in adolescence. These children are the ones who are likely to display peer perceived social behavioral problems in adolescence. However, while this study is among the first to examine the behavioral profiles of stable and transient bullies and victims, this suggestion may be premature, and more research on the stability of bullying and victimization is needed.
Footnotes
1Some male participants were in classes that only consisted of boys. To examine whether being in a male class might have affected the results, we reran the analyses without the boys from male classes. The results of the additional analyses on bullies’ and victim’s adjustment in childhood and adolescence were similar to those of the analyses on the total sample. The only exception was that in the additional analyses there were no differences between the bully groups on being liked and being nominated as a friend in childhood.
Ron H. J. Scholte: Area of interest: bullying and victimization, social relationships in adolescence, adolescent development and adjustment.
Rutger C. M. E. Engels: Area of interest: adolescent adjustment, substance use.
Geertjan Overbeek: Area of interest: adolescent attachment, social and romantic relationships.
Raymond A. T. de Kemp: Area of interest: social relationships in adolescence, adolescent development and adjustment.
Gerbert J. T. Haselager: Area of interest: social relationships in childhood and adolescence.
  • Boulton, M. J., & Smith, P. K. (1994). Bully/victim problems in middle-school children: Stability, self-perceived competence, peer perceptions, and peer acceptance. British Journal of Developmental Psychology, 12, 315–329.
  • Broidy, L. M., Nagin, D. S., Tremblay, R. E., Bates, J. E., Brame, B., Dodge, K. A., Fergusson, D., Horwood, J. L., Loeber, L., Laird, R., Lynam, D. R., et al. (2003). Developmental trajectories of childhood disruptive behaviors and adolescent delinquency: A six-site, cross-national study. Developmental Psychology, 39, 222–245. [PMC free article] [PubMed]
  • Camodeca, M., Goossens, F. A., Meerum Terwogt, M., & Schuengel, C. (2002). Bullying and victimization among school-age children: Stability and links to proactive and reactive aggression. Social Development, 11, 332–345.
  • Cairns, R. B., Cairns, B. D., Neckerman, H. J., Gest, S.D., & Gariepy, J. L. (1988). Social networks and aggressive behavior: Peer support or peer rejection? Developmental Psychology, 24, 815–823.
  • Caspi, A., Elder, G. H., & Bem, D. J. (1987). Moving against the world: Life-course patterns of explosive children. Developmental Psychology, 23, 308–313.
  • Cillessen, A. H. N., & Mayeax, L. (2004). From censure to reinforcement: Developmental changes in the association between aggression and social status. Child Development, 75, 147–163. [PubMed]
  • Cillessen, A. H. N., van Yzendoorn, H. W., van Lieshout, C. F. M., & Hartup, W. W. (1992). Heterogeneity among peer-rejected boys: Subtypes and stabilities. Child Development, 63, 893–905.
  • Coie, J. D., Dodge, K. A., & Coppotelli, H. (1982). Dimensions and types of social status: A cross-age perspective. Developmental Psychology, 18, 557–570.
  • Coie, J. D., Dodge, K. A., & Kupersmidt, J. B. (1990). Peer group behavior and social status. In S. R. Asher & J. D. Coie (Eds.), Peer rejection in childhood (pp. 17–59). New York: Cambridge University Press.
  • Craig, D., & Pepler, W. (1997). Observations of bullying and victimization in the schoolyard. Canadian Journal of School Psychology, 2, 41–60.
  • Crick, N. R., & Bigbee, M. A. (1998). Relational and overt victimization: A multi-informant approach. Journal of Consulting and Clinical Psychology, 66, 367–380.
  • DeRosier, M. E., Cillessen, A. H. N., Coie, J. D., & Dodge, K. A. (1994). Group social context and children’s aggressive behavior. Child Development, 65, 1068–1079. [PubMed]
  • Egan, S. K., & Perry, D. G. (1998). Does low self-regard invite victimization? Developmental Psychology, 34, 299–309. [PubMed]
  • Eslea, M., Menesini, E., Morita, Y., O’Moore, M., Mora-Merchán, J. A., Pereira, B., & Smith, P. K. (2003). Friendship and loneliness among bullies and victims: Data from seven countries. Aggressive Behavior, 30, 71–83.
  • Haberman, S. J. (1973). The analysis of residuals in cross-classified tables. Biometrics, 29, 205–220.
  • Haselager, G. J. T., Hartup, W. W., van Lieshout, C. F. M., & Riksen-Walraven, J. M. A. (1998). Similarities between friends and nonfriends in middle childhood. Child Development, 69, 1198–1208.
  • Hawker, D. S. J., & Boulton, M.,J. (2000). Twenty years’ research on peer victimization and psychosocial maladjustment: A meta-analytic review of cross-sectional studies. Journal of Child Psychology and Psychiatry and Allied Disciplines, 41, 441–455.
  • Hawley, P. H. (2003). Prosocial and coercive configurations of resource contrl in early adolescence: A case of the well-adapted Machiavellian. Merrill-Palmer Quarterly, 49, 279–309.
  • Hawley, P. H., Little, T. D., & Pasupathi, M. (2002). Winning friends and and influencing peers: Strategies of peer influence in late childhood. International Journal of Behavioral Development, 26, 466–474.
  • Hodges, E. V. E., Malone, M. J., & Perry, D. G. (1997). Individual risk and social risk as interacting determinants of victimization in the peer group. Developmental Psychology, 33, 1032–1039. [PubMed]
  • Hodges, E. V. E., & Perry, D. G. (1999). Personal and interpersonal antecedents and consequences of victimization by peers. Journal of Personality and Social Psychology, 76, 677–685. [PubMed]
  • Hymel, S., Wagner, E., & Butler, L. J. (1990). Reputational bias: View from the peer group. In S. R. Asher & J. D. Coie (Eds.), Peer rejection in childhood (pp. 156–186). New York: Cambridge University Press.
  • Kerr, M. (2000). Childhood and adolescent shyness in long-term perspective: Does it matter? In W. R. Crozier (Ed.), Shyness: Development, consolidation and change (pp. 64–87). London: Routledge.
  • Kochenderfer, B. J., & Ladd, G. W. (1996). Peer victimization: Cause or consequence of school maladjustment? Child Development, 67, 1305–1317. [PubMed]
  • Kochenderfer-Ladd, B., & Wardrop, J. L. (2001). Chronicity and instability of children’s peer victimization experiences as predictors of loneliness and social satisfaction trajectories. Child Development, 72, 134–151. [PubMed]
  • Juvonen, J., Nishina, A., & Graham, S. (2000). Peer harassment, psychological adjustment, and school functioning in early adolescence. Journal of Educational Psychology, 92, 349–359.
  • Kumpulainen, K., & Räsänen, E. (2000). Children involved in bullying at elementary school age: Their psychiatric symptoms and deviance in adolescence. Child Abuse & Neglect, 24, 1567–1577. [PubMed]
  • LeBlanc, M., & Loeber, R. (1993). Precursors, causes and the development of criminal offending. In A. Angold & H. F. Dale (Eds.), Precursors and causes in development and psychopathology (pp. 233–263). Oxford, England: John Wiley & Sons.
  • Loeber, R., & Hay, D. (1997). Key issues in the development of aggression and violence from childhood to early adulthood. Annual Review of Psychology, 48, 371–410. [PubMed]
  • Newcomb, A. F., & Bukowski, W. M. (1983). Social impact and social preference as determinants of children’s peer group status. Developmental Psychology, 19, 856–867.
  • Olweus, D. (1989). The Olweus bully/victim questionnaire. Bergen, Norway: Mimeo.
  • Olweus, D. (1991). Bully/victim problems among school children: Basic facts and effects of a school based intervention program. In: D. Pepler & K. Rubin (Eds.), The development and treatment of childhood aggression (pp. 411–448). Hillsdale, NJ: Erlbaum.
  • Parker, J. G., & Asher, S. R. (1987). Peer relations and later personal adjustment: Are low-accepted children at risk? Psychological Bulletin, 102, 357–389. [PubMed]
  • Pellegrini, A. D., Bartini, M., & Brooks, F. (1999). School bullies, victims, and aggressive victims: Factors relating to group affiliation and victimization in early adolescence. Journal of Educational Psychology, 91, 216–224.
  • Pellegrini, A. D., & Long, J. A. (2002). A longitudinal study of bullying, dominance, and victimization during the transition from primary to secondary school. British Journal of Developmental Psychology, 20, 259–2280.
  • Pepler, D., & Craig, W. (1995). A peek behind the fence: Naturalistic observations of aggressive children with remote audiovisual recording. Developmental Psychology, 31, 548–553.
  • Pepler, D. J., Craig, W. M., & Roberts, W. L. (1998). Observations of aggressive and nonaggressive children on the school playground. Merrill-Palmer Quarterly, 44, 55–76.
  • Perry, D. G., Williard, J. C., & Perry, L. C. (1990). Peers’ perceptions of the consequences that victimized children provide aggressors. Child Development, 61, 1310–1325. [PubMed]
  • Pope, A. W., & Bierman, K. L. (1999). Predicting adolescent peer problems and antisocial activities: The relative roles of aggression and dysregulation. Developmental Psychology, 35, 335–346. [PubMed]
  • Rigby, K., & Cox, I. (1996). The contribution of bullying at school and low self-esteem to acts of delinquency among Australian teenagers. Personality and Individual Differences, 21, 609–612.
  • Rose, A. J., & Asher, S. R. (1999). Children’s goals and strategies in response to conflicts within a friendship. Developmental Psychology, 35, 69–79. [PubMed]
  • Salmivalli, C., Huttunen, A., Lagerspetz, K. M. J. (1997). Peer networks and bullying in schools. Scandinavian Journal of Psychology, 38, 305–312.
  • Salmivalli, C., Lagerspetz, K., Björkqvist, K., Osterman, K., & Kaukiainen, A. (1996). Bullying as a group process: Participant roles and their relations to social status within the group. Aggressive Behavior, 22, 1–15.
  • Salmivally, C., Lappalainen, M., & Lagerspetz, K. M. J. (1998). Stability and change of behavior in connection with bullying in schools: A two-year follow-up. Aggressive Behavior, 24, 205–218.
  • Scarr, S. (1985). Constructing psychology: Making facts and fables for our times. American Psychologist, 40, 499–512.
  • Scarr, S. (1992). Developmental theories for the 1990s: Developmental and individual differences. Child Development, 63, 1–19. [PubMed]
  • Scarr, S., & McCartney, K. (1983). How people make their own environments: A theory of genotype à environment effects. Child Development, 54, 424–435. [PubMed]
  • Schneider, B. H. (2000). Friends and enemies: Peer relations in childhood. London: Arnold.
  • Scholte, R. H. J., van Aken, M. A. G., & van Lieshout, C. F. M. (1997). Adolescent personality factors in self -ratings and peer nominations and their prediction of peer acceptance and peer rejection. Journal of Personality Assessment, 69, 534–554. [PubMed]
  • Scholte, R. H. J., van Lieshout, C. F. M., & van Aken, M. A. G. (2001). Perceived relational support in adolescence: Dimensions, configurations, and adolescent adjustment. Journal of Research on Adolescence, 11, 71–94.
  • Schwartz, D., Dodge, K. A., & Coie, J. D. (1993). The emergence of chronic peer victimization in boys’ play groups. Child Development, 64, 1755–1772. [PubMed]
  • Smith, P. K., Morita, Y., Junger-Tas, J., Olweus, D., Catalano, R., & Slee, P. (1999). The nature of school bullying: A cross-national perspective. London, UK: Routledge.
  • Smith, P. K., Talamelli, L., Cowie, H., Naylor, P., & Chauhan, P. (2004). Profiles of non-victims, escaped victims, continuing victims and new victims of school bullying. British Journal of Educational Psychology, 74, 565–581. [PubMed]
  • Solberg, M. E., & Olweus, D. (2003). Prevalence estimations of school bullying with the Olweus Bully/victim Questionnaire. Aggressive Behavior, 29, 239–268.
  • Snyder, J., Brooker, M., Patrick, M. R., Snyder, A., Schrepferman, L., & Stoolmiller, M. (2003). Observed peer victimization during elementary school: Continuity, growth, and relation to risk for child antisocial and depressive behavior. Child Development, 74, 1881–1898. [PubMed]
  • Sutton, J., Smith, P. K., & Swettenham, J. (1999). Social cognition and bullying: Social inadequacy or skilled manipulation? British Journal of Developmental Psychology, 17, 435–450.
Articles from Springer Open Choice are provided here courtesy of
Springer