|Home | About | Journals | Submit | Contact Us | Français|
Internet addiction is a growing concern; however, both a clear understanding of the mechanisms driving problematic behaviors and a gold standard instrument for assessing symptoms are lacking. The purpose of this study was to perform a psychometric analysis of the most widely used screening instrument, the Young Internet Addiction Test (IAT), using a sample of US college students. 215 (70% response rate) undergraduate college students were recruited from two US universities to complete an online version of the IAT. Using exploratory factor analysis, two factors were extracted, “dependent use” and “excessive use” which together explained 91% of the total variance. Participants who scored into the problematic user category of the IAT scored items on the dependent use scale an average of 0.8±1.5 points higher and items on the excessive use scale an average of 1.4±1.5 points higher than participants who scored in the average user category. Results suggest that Internet addiction symptoms may cluster into two separate components, dependent and excessive use. Further, the IAT appears to be a valid instrument for assessing Internet addiction in US college students.
The pervasive and influential presence of the Internet in today’s society has raised concern over the existence of an Internet addiction disorder. While this view may be controversial, the upcoming inclusion of Internet addiction in the DSM-V as a disorder in need of further study compels further investigation (Holden, 2010). At its core, Internet addiction is characterized by frequent, uncontrolled and harmful use of the Internet. Symptoms of the disorder most frequently observed in clinical settings include preoccupation, withdrawal, loss of control and functional impairment (Tao et al., 2010). Enhancing our understanding of how these symptoms define and drive Internet addiction will help to guide this emerging field of study.
Developing a standardized instrument for assessing Internet addiction symptoms in clinical and research settings is an important step. Of the available instruments, thus far the most widely used has been the Young Internet Addiction Test (IAT). The IAT was adapted from the DSM-IV pathological gambling criteria, and was first published in 1998 (Young, 1998). The instrument covers a variety of Internet use behaviors and common addiction symptoms, with the notable exception of tolerance. Initial investigation into the validity of the IAT has shown strong internal consistency (α = .90–.93) and good test-retest reliability (r = 0.85) (Widyanto and McMurran, 2004; Yang et al., 2005; Chang and Law, 2008; Khazaal et al., 2008; Milani et al., 2009; Korkeila et al., 2010).
Further assessment of the IAT is warranted for both theoretical and psychometric reasons. First, the IAT was designed as a uni-dimensional instrument with each item contributing equally to the overall score. However, prior analyses have suggested an ambiguous factor structure; table 1 provides a summary of these findings. Second, to date, the psychometric properties of the IAT have not been assessed in a US population. As cultural differences may affect the presentation of problematic Internet use behaviors, results obtained in international settings may not be relevant to US populations.
This work seeks to build on the current literature by assessing the construct validity of the IAT in an at-risk, US population. Behavioral addictions often have their onset during adolescence or young adulthood (Grant et al., 2010). College students, the target population for this analysis, are particularly vulnerable to Internet addiction (Kandell, 1998). College students report near universal rates of computer ownership, Internet accessibility and daily use; most spend at least two hours online daily (Cotten and Jelenewicz, 2006; Fortson et al., 2007; Jones et al., 2009). The prevalence of Internet addiction is often reported to be higher among college students than other adolescent populations (Huang et al., 2009; Ko et al., 2009; Tsai et al., 2009; Fu et al., 2010). Among US college students, the prevalence of Internet addiction has been estimated to be between 8–25% (Scherer, 1997; Lavin et al., 1999; Morahan-Martin and Schumacher, 2000; Anderson, 2001; Lavin et al., 2004; Iacovelli and Valenti, 2009), placing it on a comparable scale with substance use disorders and pathological gambling (Shaffer and Hall, 1996; Slutske, 2005). Given the high risk for problematic behavior within this group, the development of a standardized instrument for assessment is greatly needed.
By conducting a factor analysis of the IAT using a college student sample, we aim to (1) explore the behavioral components underlying addictive use of the Internet, and (2) assess the validity of the IAT as a screening instrument within this at-risk population.
This study was conducted between September 1, 2009 and September 15, 2010 and received IRB approval from both the University of Wisconsin and the University of Washington.
Participants for this study were undergraduate students aged 18–20 enrolled at two large public universities in the US. In order to achieve a target population of regular Internet users, we used the social networking site Facebook to identify potential participants. Facebook (www.Facebook.com) was chosen because over 90% of students report maintaining a profile through the site, making it is the most popular social networking site among our target college populations (Pempek et al., 2009; Ross et al., 2009).
Eligible participants were identified by using the Facebook search engine to search for profiles belonging to freshmen, sophomore and junior undergraduate students at our two selected universities. Outside of these identified search criteria, Facebook search results are returned in a random order with respect to other demographic and identifying characteristics. All resulting profiles were assessed sequentially for eligibility until the target sample size was reached. From an initial search result of 3000 profiles, the majority of profiles were excluded because their profile owners were incorrectly listed and were not undergraduates (14.9%), did not meet the age criteria (10.4%) or did not display their age (1.6%), would not be reachable for follow-up as no contact information (phone number or email) was listed on the profile or in the university directory (10.1%), or due to privacy settings (54.3%). The first 307 eligible profiles owners were invited to participate in the study, with the goal of recruiting a minimum of 200 participants.
Eligible participants were contacted over the phone. The study was explained and permission was requested to send an email that contained further information about the study. If the participant consented to receive the email, an email was sent to the participant’s university email account that provided detailed information about the study as well as a link to the online survey. The survey was administered via a Catalyst WebQ online survey engine; respondents were provided a $15 iTunes gift card as compensation.
Survey components included an informed consent statement, demographic questions and the IAT. The IAT contains twenty categories assessing a respondent’s productivity at work, school, or home (3 questions), social behaviors (3 questions), emotional connection to and response from using the internet (7 questions), and general patterns of Internet use (7 questions). Participants responded to the 20 IAT items on a 6-point Likert measure (“does not apply” to “always”), which produced an overall score between 0 and 100. IAT Author guidelines recommend that participants scoring less than 40 be categorized as average users, those scoring 40 or above be categorized as problematic internet users, and those scoring 70 or above be categorized as severely problematic users (Young, 1998).
All statistical analyses were conducted using STATA version 11.0 (Statacorp, College Station, TX). Demographic characteristics and individual IAT item scores were summarized using descriptive statistics. Overall IAT scores were dichotomized into a binary variable coded as “average user” (0 to 39) and “problematic user” (40 to 100).
Exploratory factor analysis was conducted to assess the construct validity of the IAT. The number of factors to be extracted was determined through visual examination of a scree plot in combination with the conventional cut-off of eigenvalues greater than one. In order to distinguish independent underlying constructs, varimax rotation was employed to determine factor loadings. Items were assigned to the factor that produced the highest factor loading. The internal consistency of each factor was confirmed by calculating Cronbach’s alphas. We then assessed the linear correlation coefficient between the two factors to confirm the degree of independence between the factors identified.
Factor subscale scores were calculated as the mean item scores for all items loading to each factor. Because the number of items loading onto each factor differed, this approach was taken to allow a more direct comparison between subscale scores. The relationships between the subscale scores for each factor and the binary “problem user” variable were evaluated using t-tests. Finally, we assessed whether the effect of problem use differed between the sub-scales using mixed effects linear regression in which subscale mean item score was the independent variable and subscale type, problem user, and their interaction were the independent covariates. Each student contributed two observations: one for each factor subscale; thus, the model included a random effect for student to account for this correlation between observations on the same student.
A total of 224 participants responded to the survey (73% response rate). 215 surveys were complete and were included in the analysis. Participants were on average 18.8 years old, 54% were female and 68% were Caucasian. Approximately half of participants were from each university. Table 2 further describes demographic characteristics of the sample.
Overall, the mean IAT score was 28.4±10.3; scores did not differ significantly by age, gender, race or university. The majority of participants (n=190, 88%) were categorized as “average users” (IAT overall score < 40). The remaining participants (n=25, 12%) were categorized as “problematic users” (IAT overall score ≥ 40). Table 3 provides descriptive statistics for individual item scores for the IAT. Mean item scores ranged from 0.7±0.5 to 3.0±1.0 among average users, and 1.1±0.9 to 4.2±1.0 among problematic users.
Two interpretable factors were generated for the IAT; together they explained 91% of the total variance (Table 4). All items loaded at .40 or above; there were no cross-loadings. Cronbach’s alphas for the factors were found to be 0.91 and 0.83, respectively. There was a moderate linear correlation between the two factors (r=0.57).
Factor 1 accounted for 73% of the variance, and grouped twelve questions. Questions loading most strongly on this factor included “feeling depressed, moody or nervous when offline, which goes away once back online,” “snapping, yelling, or acting annoyed if bothered when online,” “feeling preoccupied with the internet when offline,” and “choosing to spend time online over going out with others.” We termed this factor “dependent use.”
Factor 2 included the remaining eight questions and accounted for 17% of the variance. Questions loading most strongly on this factor included “staying online longer than intended,” “saying ‘just a few more minutes’ when online,” “trying to cut down on online time” and “grades or schoolwork suffering from time spent online.” Thus, we described factor two as “excessive use.”
Additional analysis using a three-factor model found that the third factor drew items evenly from both subscales, had an eigenvalue of 0.6 and explained an additional 6% of the variance. However, the two-factor model contained more robust Cronbach’s alphas and factor loadings, as well as greater theoretical consistency. Further, the loading of questions within the three-factor model was not consistent with any previously described three-factor solutions to the IAT; thus, the two-factor model was determined to be the best fit for this data.
As a group, problematic users scored higher on both the “dependent use” (1.7±1.3 versus 0.9±0.4, P<0.0001) and “excessive use” (3.3±1.5 versus 1.9±0.6, P<0.0001) factors as compared to average users. The additional change in the “excessive use” subscale between average and problematic users was also significantly higher (Diff=0.6±1.5, P<0.001). Thus, while those who were considered problem users were on average more likely to report more frequent dependent use behaviors, they were even more likely to report more frequent excessive use behaviors.
Using exploratory factor analysis, we identified a two-factor model for the IAT; the factors, “dependent use” and “excessive use,” together accounted for 91% of the total variance. The high degree of variance explained, strong inter-item correlation and theoretical consistency of our model suggests that the IAT is a valid instrument for assessing Internet addiction in adolescent to young adult college students.
Previous analyses have found between one- and six-factor solutions for the IAT. In spite of these differences, factor arrangements from prior analyses show striking resemblances to our results (Table 5). The items clustered within the first factor of the two-factor solution by Korkeila et al. (2010) and the first and third factors in the three-factor solutions by both Chang and Law (2010) and Widyanto et al. (2010), show broad overlap with our first factor. Similarly, our second factor, “excessive use,” is consistent with the second factors of all three previous analyses. Although there is variation in factor loadings for a small subset of questions between the models, we believe these discrepancies are most likely due to differences inherent to the samples used for analysis such as age or differences in cultural norms surrounding Internet use, rather than differing representations of the overall construct.
Thus, our results suggest that Internet addiction symptoms, as measured by the IAT, cluster into two distinct constructs which we have named, “dependent use” and “excessive use.” The first component, “dependent use,” encompassed the majority of previously established addiction symptoms, such as preoccupation and withdrawal, along with social impairment. The second component, “excessive use” grouped other forms of functional impairment with symptoms of overuse and loss of control. While the intrinsic nature of the two factors suggests that some degree of connection is to be expected, their moderate linear correlation supports the representation of these constructs as unique, distinguishable components.
Our findings also suggest several areas for refinement of the IAT. First, several IAT items produced lower factor loadings. Cultural and technological changes in Internet use since the development of the IAT, and characteristics specific to this population, may account for why these items are less precise in assessing Internet addiction symptoms. In particular, while the item, “do you form new relationships with fellow online users,” (factor loading 0.40) may have represented a problematic behavior in the earlier context of this instrument’s development, the recent rise of social networking sites, blogs, chat rooms and other internet social utilities may have normalized this behavior. Similarly, given the substantial integration of e-mail as a communication utility and increases in Internet availability through wireless connections and mobile devices, “do you check your e-mail before something else that you need to do” (factor loading 0.49) may no longer represent an abnormal behavior. Additionally, the items, “do you prefer the internet to intimacy with your partner” (factor loading 0.44) and “does your job performance suffer because of the internet,” (factor loading 0.44) may refer to lifestyle characteristics uncommon among young, full-time students, and thus may not be appropriate for assessing a disorder in this population. Although these items did produce acceptable factor loadings, re-working or removing them may enhance the performance of the IAT in this population.
A second area of refinement may be the instrument’s scoring system. Currently, in calculating the overall IAT score, equal weight is given to each item. However, several items such as “do you snap, yell or act annoyed when bothered online” and “do you feel depressed, moody, or nervous when offline” showed a reduced range of responses and lower mean scores, despite strong factor loadings (0.83 and 0.85, respectively). Thus, while these items appear to identifying key symptoms of Internet addiction, their contribution to the overall score may be diminished. Applying a weighted scoring scheme to the IAT may be useful in improving the instruments precision. Further, although IAT scores could range between 0 and 100, the limited range of overall scores suggests that narrowing the response scale may enhance the feasibility of categorizing non-problematic, problematic and severely problematic behavior.
There are several potential limitations of our study. First, because we identified potential participants through searching a social networking site database, there is a possible sampling bias in our study. However, the near-universal use of this site within our target population, our strong response rate and the comparable demographic breakdown between our sample and reference populations suggest that our methods are reliable. Second, because we focused on college students, generalizing results to other adolescent or young adult populations may not be warranted. Given that college students are a key population in which intense Internet use is common and potentially consequential, this was our population of choice for this analysis. Our findings should be interpreted with some caution, as results may have been affected by characteristics specific to a campus environment. In particular, participants’ reporting of excessive use symptoms may have been augmented in the context of high standards of Internet use found among students. If this were the case, it would suggest that when developing approaches for screening, diagnosing and treating Internet addiction, careful consideration should be taken to integrate clinical guidelines with the technology use norms relevant to the sample of interest.
In spite of these limitations, there are several ways our findings can inform future research on both the IAT and Internet addiction. First, while these findings support the IAT as a valid assessment of Internet addiction in this population, confirmatory analysis of our two-factor solution is necessary to corroborate these results. The effect of removing or reworking low-loading items on overall scale performance should also be assessed. Second, the concurrent and predictive validity of the overall score cut-offs, item-weighting scheme and sub-scale scores should be evaluate using clinical assessments. These will be challenging tasks, given the paucity of gold-standard measures for Internet dependency and excessive use available at present. Third, additional work is needed to confirm whether Internet addiction symptoms cluster similarly in other relevant populations such as individuals with existing psychiatric comorbidities or those reporting more severe addiction symptoms.
Our results also shed light on the potential significance of the dimensionality of Internet addiction. While increased reporting of both symptom categories was present among those scoring higher on the IAT, the more substantial rise in excessive use symptoms suggests that changes in this subscale may be a better indicator of increasingly addictive behavior. Alternatively, given the modest prevalence of PIU noted in this sample (12%), we may not have fully captured the extent of dependent use symptoms effecting problematic users. Thus, while determining the absolute number of symptoms present has thus far been used to determine the severity of an individual’s problematic use, further investigation into the influence of symptom dimensions may help to refine this practice. Given the growing exposure to Internet use within our society, continuing to develop effective means for identifying both those at risk for and currently suffering from problematic behavior is of the upmost importance.
The authors would like to express gratitude to Allison Grant for her assistance in reviewing a previous version of this manuscript. This work was supported by award R211AA017936 from NIAAA as well as award K12HD055894 from the Eunice Kennedy Shriver National Institute of Child Health and Human Development.