|Home | About | Journals | Submit | Contact Us | Français|
The Revised Attribution Questionnaire (r-AQ) measures mental illness stigma. This study’s purpose is to evaluate the factor structure of the (r-AQ) and examine the validity of the factor structure in adolescents. A convenience sample (n=210) of adolescents completed the r-AQ and these data were used in exploratory (EFA) and confirmatory factor analyses (CFA). The EFA established a five item single factor structure, which we called the modified r-AQ and captures the negative emotional reactions to people with mental illness, a domain of mental illness stigma. The CFA established the validity of the factor structure (χ2=2.4, df=4, p=.659, TLI=1.042, CFI=1.00, RMSEA=.000). Internal consistency reliability for the scale was acceptable (α= .70). The modified r-AQ is a reliable and valid measure of the emotional reaction to people with mental illness.
Mental illness stigma is a critical barrier to mental health treatment access and the activation of mental health self-management behavior. Specifically, the reduction of mental illness stigma has been identified as a priority area for adolescents by the Institute of Medicine, President’s New Freedom Commission on Mental Health, and U.S. Surgeon General. In response to this pervasive problem, these national groups have recommended the implementation of anti-stigma interventions for adolescents in community settings. However, before evidence based anti-stigma interventions can be implemented and evaluated with confidence, it is essential to establish the reliability and validity of instruments that measure mental illness stigma among adolescents. Thus, the purposes of this research are to evaluate the factor structure of the Revised Attribution Questionnaire (r-AQ) (Logsdon, 2009; Services, 1999; Watson, Otey, Westbrook, & al., 2004) and examine the validity of the factor structure in adolescents.
Untreated mental illness during adolescence carries severe and long-lasting consequences that extend into adulthood. Specifically, adolescents with untreated mental illness have lower academic achievement, higher rates of suicide, more unintended pregnancies, lower levels of functioning, and they are more likely, than adolescents without mental illness, to use substances like tobacco, alcohol, and drugs (Kuehn, 2005; McCarty et al., 2008). The onset of mental illness during adolescence is associated with physical problems, such as difficulty maintaining a healthy body weight, increased susceptibility to infectious diseases, and respiratory problems (Aarons et al., 2008; Schinke, Fang, & Cole, 2008). Adolescents who do not receive needed mental health treatment as teens have higher healthcare resource utilization and more physical impairments as young adults, which contributes to morbidity of individuals living with mental illness (Kennan-Miller, Hammen, & Brennan, 2007). Despite these profound consequences, adolescents will continue to forgo beneficial and lifesaving mental health treatment until mental illness stigma is effectively addressed (Goldsmith, Pellmar, Kleinman, & Bunney, 2002).
Mental illness stigma research focused on adolescents is sparse and accounts for only 4% of all research on mental illness stigma (Link, Yang, Phelan, & Collins, 2004). Among adults, there are several well-known instruments available to measure mental illness stigma, e.g., Devaluation-Discrimination Scale (Link, Cullen, Frank, & Wozniak, 1987), Stigma Scale for Receiving Psychological Help (SSRPH) (Komiya, Good, & Sherrod, 2000), and the Attribution Questionnaire (Corrigan, Markowitz, Watson, Rowan, & Kubiak, 2003; Corrigan et al., 2002). However, instruments to measure mental illness stigma have limited evidence to support reliability and validity in adolescents (Link et al., 2004). The r-AQ was selected for modification in the adolescent population because it is one of the few instruments that has been used in a sample of adolescents and found to be internally consistent (Watson et al., 2004). Thus, use of the instrument among adolescents warrants further investigation to support construct validity. Although it is assumed that the r-AQ has a single factor structure with each item measuring one of 9 domains of mental illness stigma, the factor structure has not been supported among adolescents. The purpose of this was to evaluate the factor structure of r-AQ and examine the validity of the factor structure in adolescents.
The r-AQ is a measure of mental illness stigma and was developed based on Weiner’s attribution theory (Weiner, 1995). Attribution theory posits that people respond to another’s health condition in a negative manner if they believe the individual has a high degree of responsibility and control over the health condition. Individuals with this perspective will often express feelings of anger, conduct acts of avoidance, and resist helping those with a mental illness. On the other hand, people will respond in a more positive manner if they believe the individual has little responsibility and low control over their health condition. Individuals with this perspective often feel pity for and will offer support to a person with mental illness. Mental illness stigma is defined as the culmination of negative attitudes and beliefs that motivate the general public to fear, reject, avoid, and enact behaviors of discrimination against people with mental illness (Corrigan & Penn, 1999). Mental illness stigma exists across cultures and age groups, including adolescents (Logsdon, 2009; Pescosolido, February, 2009; Services, 1999; Watson et al., 2004). Link and colleagues (Link et al., 2004), who are pioneers in mental illness stigma research, have identified 8 components of mental illness stigma that include: behavior, labeling, stereotyping, cognitive separating, emotional reactions, status loss/discrimination (expectations and experiences), structural discrimination, and behavioral responses to stigma. No single instrument measures all components of mental illness stigma proposed by (Link et al., 2004). There is no gold standard measure of mental illness stigma, and only few studies have examined mental illness stigma among adolescents (Pinto-Foltz & Logsdon, 2009). These studies provide beginning evidence to support reliability of measures of mental illness stigma. However, evidence to support construct validity of these instruments have not been examined among adolescents, but is needed to advance the science of measurement of mental illness stigma in this population (Link et al., 2004).
Based on the premise of Attribution Theory (Weiner, 1995) and the work of Reisenzein (1986), Corrigan initially developed a 27-item instrument, the Attribution Questionnaire (AQ-27), to measure the domains of blame, anger, pity, help, dangerousness, fear, avoidance, segregation, and coercion (Corrigan, et al., 2003). A 9 item short form of the AQ-27, the AQ-9, was later developed by extracting 9 items from the AQ-27 with the highest factor loadings (Corrigan et al., 2003). The AQ-9 items include the domains of blame, anger, pity, help, dangerousness, fear, avoidance, segregation, and coercion. Associations among the 9 domains have been established by a path analysis with excellent model fit in adults (Corrigan et al., 2002). The AQ-9 has since been modified by Watson and colleagues (Watson et al., 2004) for middle school adolescents into the Revised Attribution Questionnaire (r-AQ). These modifications to the instrument involved revising language so it was consistent with the setting that adolescents frequently encounter peers with mental illness, in the school setting. The r-AQ was shown to be internally consistent among middle school adolescents (Watson et al., 2004).
The Revised Attribution Questionnaire (r-AQ) for adolescents (Watson et al., 2004) is a 9-item self-administered, paper and pencil scale designed to measure two common attributes of mental illness stigma−dangerousness and attribution−in children and adolescents. The first eight items correspond to the domains of blame, anger, pity, help, dangerousness, fear, avoidance, and coercion (Watson et al., 2004). Item 9 captures attitude toward seeking mental health treatment (Watson et al., 2004). Participants are asked to respond to each item, on a Likert scale from 1 (strongly disagree) to 7 (strongly agree), after reflecting on the following scenario: “There is a new student in your class who just came from another school. You have heard that this student has a mental illness.” From these 9 items, a total score is calculated, and scores range from seven to 63, with a higher total score indicating a higher magnitude of mental illness stigma.
This study is a secondary analysis of data collected for a larger longitudinal study that examined the impact of a mental health intervention among adolescents (Pinto-Foltz, Logsdon, & Myers, 2011). This study analyzed cross-sectional data from baseline interviews, prior to the administration of the intervention.
A convenience sample of 210 adolescents who were: English speaking adolescents, 13–18 years of age, and attending one of the three high schools. Data were collected from two public high schools and one private high school in a southern urban area of the United States in December, 2008 through May, 2009.
On average participants were 15 years of age, mostly female, Caucasian, enrolled in 10th grade, and of moderate to high socioeconomic level. Sociodemographic characteristics of each analytic subsample of adolescents are presented in Table 1. There were no significant differences in sociodemographics between the EFA and CFA analytic subsamples. The sample sizes for each analytic subsample exceeded the minimum criteria for subjects per item (10:1) ratio needed to conduct rigorous exploratory and confirmatory factor analyses of the r-AQ (Kline, 2004; Nunnally, 1978).
Prior to data collection and recruitment, this study was approved by the institutional review boards of the principal investigator’s university and each school system involved with this research. To recruit subjects, the investigator visited all adolescents during class to discuss the study and distribute a consent/assent form packet. Interested adolescents were given consent/assent form packets for they and their parents to review, sign, and return to a designated school counselor within a week. The investigator retrieved the consent form packet a week after recruitment. On the day of data collection, adolescent participants were dismissed from class and sequestered in a private room within the school to complete the battery of instruments, which included the r-AQ. On average, adolescents completed the battery of instruments in less than 15 minutes time. Adolescents received a $5 gift card to a retail discount store for study participation.
The statistical analyses were conducted in two stages. First, an EFA was conducted to specify the most parsimonious factor structure of the 9-item r-AQ. Second, a CFA was performed to evaluate the construct validity of the factor structure derived from this EFA. The following criteria were used to guide the exploratory and confirmatory factor analyses:
A series of EFAs were performed using SPSS (Version 17, Chicago, IL) to determine the most parsimonious factor structure for the r-AQ. To assess the latent factor structure of the r-AQ, principal axis factoring (PAF) was used as the extraction method to reveal the r-AQ’s factor structure. The selection of extraction and rotational methods were guided by Watson et al. whom proposed the r-AQ as a single factor structure in adolescents. Since the r-AQ has been constructed as a unidimensional measure of mental illness stigma an orthogonal rotation, varimax, was used to extract a single factor structure with inter-related items. Costello and Osborne (2005) recommend the use of PAF because it accounts for the unique contribution of each item and can robustly analyze data that violate the statistical assumption of normality. Prior to the EFA, indicators of sampling adequacy were verified by Kaiser-Meyer-Olkin (KMO) .642 and Bartlett’s Test of Sphericity (X2=77.84, df=10, p<.001).
The specification of the most parsimonious factor structure was guided by the evaluation of scree plot and eigenvalue of each factor. Interpretation of the scree plot is the recommended criterion of Costello and Osborne (Costello & Osborne, 2005) and the assessment of the eigenvalues was directed by the Kaiser criterion, which recommends an eigenvalue ≥ 1 for factor retention (Hatcher, 1994). The interpretation of the scree plot and eigenvalue of each factor determined the number of factors captured by the r-AQ.
Items of the r-AQ that were retained met the following a priori criteria: had primary factor loadings > .40, secondary factor loadings <.30, and did not have primary factor loadings on more than one factor (Pai et al., 2007). Items that did not meet these retention criteria were removed individually, and the EFA was repeated until all items retained met the retention criteria. Item 9 of the r-AQ, “If I thought that I had a mental illness, I would talk with my parents about taking me to a doctor or counselor” was the only item excluded from this factor analysis because the item was originally included to determine if an adolescent would seek help for a mental health problem (Watson et al., 2004).
Once a parsimonious factor structure was derived from the EFA, the factors were labeled based on the content of the items retained on each factor. Evaluation of the internal reliability consistency of each factor was performed to assess the contribution of each item to the factor or total score of the r-AQ. Factors that had a Cronbach’s alpha ≥ .70 were considered reliable (Cortina, 1993).
The evaluation of the validity of the factor structure of the r-AQ derived from EFA was conducted in the remaining 105 participants randomly sorted to this analytic sample. A first-order CFA using Analysis of Moment Structures (AMOS, Version 17, Chicago, IL) was used to determine the validity of these two factor structures in this sample of adolescents. The following goodness-of-fit indices were used to assess the model fit: χ2, Tucker Lewis Index (TLI: >.90 acceptable, >.95 excellent), the Comparative Fit Index (CFI: >.90 acceptable, >.95 excellent), and Root Mean Square Error of Approximation (RMSEA: <.08 acceptable, <.05 excellent) (Bentler, 1990; Bentler & Bonnett, 1980). Based on the interpretation of modification indices, paths between error terms were considered added to enhance the goodness-of-fit of the model to these data (Kline, 2004).
In this sample, the most parsimonious factor structure for the r-AQ consisted of five-items and a single factor. Although Corrigan (Corrigan et al., 2002) originally conceptualized the adult version of the AQ-9 as a 9-item multidimensional measure of mental illness stigma and Watson, et al. (2004) later modified the instrument for adolescents, called the r-AQ for adolescents.
This EFA does not support the originally proposed factor structure. When the investigators attempted to examine a multidimensional factor structure, the interpretation of the scree plot and eigenvalues for each factor indicated a unidemsional structure that included all items. Based on these findings, the investigators departed from an oblique rotational method because an oblique rotational method requires at least two factors for rotation (Costello & Osborne, 2005). Instead, an orthogonal method (varimax) was used to assess the single factor structure of the r-AQ. Items 2 “I feel sorry for the new student”, 3 “The new students should be locked in a mental hospital,” and 5 “It is not the student’s fault if he or she has a mental illness” were individually removed based on our item reduction criterion. In sum, five-items (3, 4, 6, 7, and 8) were retained on a single factor, which accounted for 31% of the explained variance of the construct. See Table 2 for factor loadings.
Based on the content of five-items retained on the single factor and the magnitude of these items’ factor loading, the single factor was labeled “emotional reaction to people with mental illness”, as conceptualized by Link, Yang, Phelan, and Collins (Link et al., 2004) and Angermeyer and Matschinger (1996). This five-item measure, which we will now call modified r-AQ was reliable (α = .70) in this analytic sample. Cronbach’s alpha coefficient could not be improved by deleting any item (Table 2). The homogeneity of the instrument was supported by statistically significant item-to-total correlations between .50–.76 and inter-item correlations .18–.51 (Table 4).
The modified r-AQ was a single factor structure and examined for validity in the remaining 105 adolescents. Table 3 provides the goodness-of-fit indices for the five-item single factor structure derived from the EFA. Model 1 is the single-factor model with all 5 items identified in the EFA. Model 2 is the single-factor model with paths from items 4 and 7 error terms added. Model 3 is the single-factor with paths between error terms 4 and 7, and between error terms 3 and 4. The final structural model of our modified r-AQ, Model 3, was the best fitting model (χ2=2.4, df=4, p =.659, TLI=1.042, CFI=1.00, RMSEA=.000) and is shown in Figure 1. The modified r-AQ is a valid measure in this sample of adolescents.
In an effort to establish a reliable and valid measure of mental illness stigma among adolescents, the investigators examined the underlying factor structure of the r-AQ and reliability of this factor structure. The EFA yielded a single factor structure that consisted of five items, which the investigators call the modified r-AQ. The modified r-AQ was internally consistent. The CFA confirmed the single factor structure identified in the EFA and supported construct validity of the modified r-AQ; the single factor structure had excellent model fit to these data. The five item modified r-AQ measures a dimension of mental illness stigma, which the investigators called emotional reaction to people with mental illness, and is based on the work of Link, Yang, Phelan, and Collins (Link et al., 2004) and Angermeyer, Matschinger, and Riedel-Heller (1999). Emotional reaction is the affective response of the person who is stigmatizing (Link et al., 2004) and consists of aggressive emotions (anger), pro-social reaction (help, sympathy), and feelings of anxiety (scared) (Angermeyer et al., 1999). Item 8 describes feelings of anxiety, items 3, 4, and 7 describe pro-social reaction, and item 6 describes feelings of aggressive emotions. Emotional reactions to people with mental illness are important because they are one of the strongest predictors of discrimination and social distance from persons with mental illness (Angermeyer, Holzinger, & Matschinger, 2010). Emotional reactions also communicate to the stigmatized person how he or she is perceived by others (Angermeyer et al., 2010). Emotional reactions in interpersonal interactions may verify the misconceptions of mental illness held by the public and shape future behaviors of avoidance and social withdrawal from persons with mental illness (Link et al., 2004).
According to Link, et al., (Link et al., 2004) stigma is a multidimensional construct that includes the domains of behavior, labeling, stereotyping, cognitive separating, emotional reactions, status loss, structural discrimination, and behavioral responses. Although the EFA supports construct validity of the r-AQ, only a third of the variance was captured, which suggests there may be multiple dimensions of the construct. It is possible that items 2 and 5 may demonstrate validity with other samples or when included with more items that enhance the conceptual fit of the items to emotional reaction. However, the EFA did not support the retention of these items in this sample of adolescents. Because this instrument has not been evaluated in previous studies with adolescents, we cannot compare our findings to others. This is first investigation of the r-AQ among adolescents; it is possible that more items are needed to adequately measure feelings of aggressive emotion and pro-social reaction, and the reduction of items to five items is insufficient to fully capture the construct. Since the r-AQ is based on attribution theory, and has evidence to support validity in adults, items 2 and 5 should be considered for inclusion in future studies. The inclusion of items 2 and 5, along with new items that are theoretically consistent with emotional reactions, may better capture the multidimensional construct of stigma and consequently enhance the r-AQ. Further instrument development is needed to encompass these domains in adolescent populations.
Several limitations should be considered in interpretation of these results. First, this study employed a convenience sample from three high schools in the southern area of the United States. Adolescents in this study may not be representative of all adolescents, and future studies should examine the r-AQ in other populations to enhance generalizability of study findings. Second, although the samples size was 20 participants per item and exceeded conventional standards for adequate sample size, (Costello & Osborne, 2005) the majority of participants in this study were Caucasian adolescent girls of moderate socioeconomic level. Future studies should include more adolescent boys and subset populations of adolescents that are of lower socioeconomic level and ethnically diverse and aim to explain more variance with the addition of items. Finally, instrument stability is not known. Longitudinal data is needed to determine if the factor model is invariant over time.
Mental illness stigma is pervasive and a critical barrier for entry into mental health treatment and the self-management of mental illness. Advances in measurement of mental illness stigma hold important health implications for adolescents who suffer from mental illness. Reliable and valid measures of mental illness stigma must be established before scientists can evaluate the effectiveness of interventions designed to combat mental illness stigma among adolescents. The five item measure, modified r-AQ, is a reliable and valid measure that captures an important domain of mental illness stigma, emotional reaction to persons with mental illness. These study findings serve as a foundation to advance the science and evaluation of mental illness stigma. Future studies should consider the further construction and assessment of a multidemisional measure of mental illness stigma that has reliability and validity among diverse groups of adolescents.
This publication was made possible by RR024990 from the National Center for Research Resources (NCRR), a component of the National Institutes of Health (NIH), and NIH Roadmap for Medical Research. Its contents are solely the responsibility of the authors and do not necessarily represent the official view of NCRR or NIH. Additional sources of support were Midwest Nursing Research Society Dissertation Research Grant, Ruth Craddock Research Grant from the University of Louisville School of Nursing, CODRE Graduate Research Grant from the University of Louisville, and Sigma Theta Tau International Iota Zeta Chapter Research Grant.
Melissa D. Pinto, Case Western Reserve University School of Nursing.
Ronald Hickman, Case Western Reserve University School of Nursing.
M. Cynthia Logsdon, University of Louisville School of Nursing.
Christopher Burant, Case Western Reserve University School of Nursing.