Diagnostic Criteria for Problematic Internet Use among U.S. University Students: A Mixed-Methods Evaluation

Abstract

Empirical studies have identified increasing rates of problematic Internet use worldwide and a host of related negative consequences. However, researchers disagree as to whether problematic Internet use is a subtype of behavioral addiction. Thus, there are not yet widely accepted and validated diagnostic criteria for problematic Internet use. To address this gap, we used mixed-methods to examine the extent to which signs and symptoms of problematic Internet use mirror DSM-5 diagnostic criteria for substance use disorder, gambling disorder, and Internet gaming disorder. A total of 27 university students, who self-identified as intensive Internet users and who reported Internet-use-associated health and/or psychosocial problems were recruited. Students completed two measures that assess problematic Internet use (Young’s Diagnostic Questionnaire and the Compulsive Internet Use Scale) and participated in focus groups exploring their experiences with problematic Internet use. Results of standardized measures and focus group discussions indicated substantial overlap between students’ experiences of problematic Internet use and the signs and symptoms reflected in the DSM-5 criteria for substance use disorder, gambling disorder, and Internet gaming disorder. These signs and symptoms included: a) use Internet longer than intended, b) preoccupation with the Internet, c) withdrawal symptoms when unable to access the Internet, d) unsuccessful attempts to stop or reduce Internet use, e) craving, f) loss of interest in hobbies or activities other than the Internet, g) excessive Internet use despite the knowledge of related problems, g) use of the Internet to escape or relieve a negative mood, and h) lying about Internet use. Tolerance, withdrawal symptoms, and recurrent Internet use in hazardous situations were uniquely manifested in the context of problematic Internet use. Implications for research and practice are discussed.

About the Authors:

Wen Li

* E-mail: [email protected]

Affiliation: School of Social Work, University of North Carolina at Chapel Hill, Chapel Hill, NC, United States of America

Jennifer E. O’Brien

Affiliation: School of Social Work, University of North Carolina at Chapel Hill, Chapel Hill, NC, United States of America

Susan M. Snyder

Affiliation: School of Social Work, Georgia State University, Atlanta, GA, United States of America

Matthew O. Howard

Affiliation: School of Social Work, University of North Carolina at Chapel Hill, Chapel Hill, NC, United States of America

Introduction

Problematic Internet use may lead to serious psychosocial dysfunction [1]. Problematic Internet use is a serious problem for 6% to 11% of Internet users in the United States [2]. Compared to other age groups, college-aged youth and young adults appear to be at greater risk for problematic Internet use given the pervasiveness of Internet access on college campuses and possibly the freedom from parental supervision many college students experience when living away from home for the first time [3–5]. Epidemiological studies indicate that approximately 5% of U.S. university students suffer from problematic Internet use [6–8].

University students with problematic Internet use may exhibit symptoms of impaired physical health (e.g., obesity, sleep disorders) [9, 10], psychological distress (e.g., depression, social anxiety, attention deficit-hyperactivity disorder [ADHD]) [11–14], and behavioral problems (e.g., substance abuse and behavioral addictions, aggression, self-injurious behaviors) [15–17]. Further, students with problematic Internet use often experience more interpersonal problems [18], and worse school and work performance compared to their problematic Internet use-free peers [6].

Although research has documented the prevalence rates and negative consequences of problematic Internet use, the literature does not reflect a consistent conceptualization of problematic Internet use [19–21]. Specifically, it is unclear whether problematic Internet use should be classified as a type of behavioral addiction [21]. Neurobiological studies indicate that problematic Internet use shares many of the same neurobiological underpinnings as substance use disorder [22–25]. Researchers have paralleled the signs and symptoms of problematic Internet use to those of substance use disorder and behavioral addictions including pathological gambling [1, 26–28]. These researchers have argued that characteristics of problematic Internet use include signs and symptoms such as: preoccupation with Internet use; tolerance (i.e., the compulsion to use the Internet for ever increasing amounts of time); repeated but unsuccessful efforts to control, cut back, or stop Internet use; restlessness, irritability, and other signs of withdrawal when unable to use the Internet; having jeopardized or lost a relationship, job, educational or career opportunity because of Internet use; lying to friends, family members, and others to conceal the extent of involvement with the Internet; and using the Internet to escape or palliate dysphoric moods such as depression and anxiety [1, 27, 28]. In addition, some scholars have argued that people are not addicted to Internet; instead, they are addicted to specific activities on the Internet including online gambling, online gaming, online-shopping, and pornography viewing [21]. Internet is an environment or a delivery mechanism [29], and therefore the concept of Internet addiction should refer to pathological and addictive behaviors related to the specific online activities rather than Internet itself.

Many measurement instruments purporting to assess and diagnose problematic Internet use have been developed based on diagnostic criteria for substance use disorder and gambling disorder [1, 26–28, 30–32]. The proposed diagnostic criteria for Internet gaming disorder have been incorporated into Section III of DSM-5 [31]. These instruments assess domains that largely overlap with signs and symptoms of substance use disorder including salience, tolerance, withdrawal symptoms, and use to regulate mood [32]. Although these instruments often demonstrate good psychometric characteristics [27, 32–35], the extent to which the characteristics of problematic Internet use are similar to signs and symptoms of substance use disorder or behavioral addictions remains unclear and controversial [21]. To address this important gap, our research team conducted an exploratory study using mixed-methods (i.e., descriptive and qualitative results) to investigate the degree to which signs and symptoms that are associated with problematic Internet use, as described by university students who self-identified as having problems with Internet overuse, mirror DSM-5 diagnostic criteria for substance use disorder, gambling disorder, and Internet gaming disorder.

Methods

To obtain detailed descriptions of college students’ experiences with problem Internet use, our team employed exploratory in-depth focus groups. Quantitative data regarding participants’ sociodemographic characteristics and Internet usage patterns were also collected. In addition, two standardized measures were included to assess signs and symptoms that are associated with problematic Internet use. The University of North Carolina-Chapel Hill Institutional Review Board approved the research project.

Ethics Statement

This study was approved by the University of North Carolina-Chapel Hill Institutional Review Board and performed in accordance with the Declaration of Helsinki. Written consent was obtained from all participants before the focus groups commenced.

Participants

The study sample was composed of students at a large public university in the southeastern United States. A recruitment email was distributed via the university student listserv to all undergraduate and graduate students once. Eligibility criteria specified that participants (≥ 18 years of age) must spend greater than 25 hours/week on the Internet for non-school or non-work-related purposes, and have experienced one or more physical or psychosocial problems caused by problematic Internet use. Ultimately, 39 students agreed to participate in the focus groups. However, 12 students did not attend scheduled groups. Thus, four groups were held including 27 students. Each student only participated in one focus group discussion. Students were provided with $20 as compensation for their study participation. Participant characteristics are reported in Table 1.

[Figure omitted. See PDF.]

Table 1. Sample Characteristics and Internet Use Patterns of 27 College Students Who Report Problematic Internet Use.

https://doi.org/10.1371/journal.pone.0145981.t001

Focus Group Materials and Measures

Focus group assessment materials consisted of 22 open-ended questions and a set of objective measurement instruments. The group discussion guide was developed and refined by the investigators based on the extant problematic Internet use literature. Major issues explored in the focus groups included patterns, features, and consequences of problematic Internet use, and signs and symptoms of problematic Internet use.

Young’s Diagnostic Questionnaire (YDQ) and the Compulsive Internet Use Scale (CIUS) were employed to assess problematic Internet use. The YDQ consists of 8 questions that assess signs and symptoms of problematic Internet use. Participants answering “yes” to 5 or more questions were identified as having Internet addiction whereas those meeting 3 or 4 criteria were considered to have “potential Internet addiction [1, 36].” The CIUS includes 14 items rated on a 5-point Likert-type scale, ranging from 0 (never) to 4 (very often). The CIUS assesses severity of compulsive Internet use behavior; higher scores indicate greater severity. Prior studies suggest that the CIUS has an internal consistency reliability of ~.90 [30]. A CIUS cutoff score of ≥ 21 for the identification of problematic Internet use has been recommended [37].

Data Collection

Written consent was obtained from all participants before focus groups commenced. Prior to each focus group, participants completed the YDQ, CIUS, and a brief sociodemographic survey. The surveys were administered to the group members at the beginning of each focus group discussion. Participants were sufficiently separated from each other to ensure that they could not view each other’s responses. The surveys were collected by the researcher once participants completed it. Each focus group lasted approximately one hour. Six to eight participants attended each focus group. Facilitator prompts were used to assure that a wide range of ideas and opinions were represented. A member of the research team conducted each focus group while another member took notes. The presence of multiple observers at group sessions allowed for observer triangulation to improve the reliability and validity of findings emerging from group discussions [38].

Data Analysis

Audiotapes of focus group sessions were transcribed verbatim. All authors checked for accuracy of the audiotape transcription. A theory-driven coding approach was used to create an initial list of codes [39]. The key research questions, the interview guide, and the signs and symptoms of substance use disorder informed the preliminary coding scheme. Following this coding plan, three research team members thoroughly reviewed each interview transcript to review and revise the codes in the context of the data [39]. Coding discrepancies among the team were resolved through mutual discussion and agreement. Patterns were identified and the analysts implemented constant comparison procedures. In addition, regular debriefing and consultation among research team members helped guard against research bias [38, 40].

Results

Almost half (48.1%) of the student sample scored five or more on Young’s Diagnostic Questionnaire (YDQ), and therefore scored above the suggested cut-off point for Internet addiction. Another 40.7% scored a three or four on the YDQ, reflecting the suggested cut off for sub-threshold Internet addiction. The internal consistency of the YDQ in this study was .69. Almost the entire sample (96.3%) exceeded the recommended cutoff for problematic Internet use according to the Compulsive Internet Use Scale (CIUS). In this study, the CIUS had an α = .92. Many participants felt their problematic Internet use could validly be described as an addiction. In discussing this, participants likened their Internet use to substance dependence. In the words of one participant, “If you’re addicted to cigarettes, you can’t really go a day without smoking. Probably you can’t go many days without (the) Internet.” In fact, many participants used terms that referred to the signs and symptoms of substance use disorder when discussing their problematic Internet use, including “withdrawal,” “tolerance,” and “craving.” Participant quotes regarding their problematic Internet use are shown in Table 2. These quotes are reflective of the signs and symptoms generally used for substance use disorder and behavioral addictions. These signs and symptoms are: (a) use longer than intended; (b) preoccupation; (c) withdrawal signs/symptoms; (d) tolerance; (e) unsuccessful attempts to stop or reduce Internet use; (f) craving; (g) loss of interest in other hobbies or activities; (h) excessive use despite problems; (i) use the Internet to escape or relieve negative mood; and (j) lying about use.

[Figure omitted. See PDF.]

Table 2. Participants’ Quotes about Signs and Symptoms of their Problematic Internet Use and Participants’ Endorsement Rates on Correspondent Items on YDQ and CIUS.

https://doi.org/10.1371/journal.pone.0145981.t002

Use Longer than Intended

Taking a substance in larger amounts or over longer periods than was intended is a sign of substance use disorder [31]. This sign is not assessed by DSM-5 criteria for gambling disorder and Internet gaming disorder [31]. Focus group participants reported similar signs in relation to their Internet use. Many group members had experienced being on the Internet longer than they had initially intended. Participants noted that they often lost track of time while on the Internet for recreational purposes, resulting in loss of sleep, less social interaction, and reduced academic work productivity (see Table 2 for direct quotes from focus group participants and participants’ endorsement rates on the correspondent items on the YDQ and CIUS). YDQ results showed that 96.3% of participants reported having stayed on the Internet longer than they intended. Similarly, 80.8% of participants reported that they often/very often continued to use the Internet despite their intention to stop.

Preoccupation

Preoccupation is characterized by individuals devoting a great deal of time to obtaining and using and/or recovering from the effects of substances [31]. This criterion is also used to assess gambling and Internet gaming disorder in DSM-5 [31], and refers to persistent thoughts of previous gambling/gaming activity, anticipating and planning the next gambling/gaming venture, and thinking of ways to get money for gambling. Preoccupation refers to when substance use, gambling, or Internet gaming has become the dominant activity in an individual’s daily life. Focus group participants reported similar signs with respect to their problematic Internet use. Participants noted spending substantial amounts of time thinking about activities on the Internet, not only while using the Internet but also when not using or anticipating the next session of use (see Table 2 for direct quotes from focus group participants and participants’ endorsement rates on the correspondent items on the YDQ and CIUS). YDQ results showed that 81.5% of participants felt preoccupied with the Internet. According to CIUS, 29.6% of participants frequently thought of the Internet (even when not online), and 33.3% of participants often/very often reported looking forward to their next Internet session.

Withdrawal Signs/Symptoms

Withdrawal refers to a characteristic syndrome of signs and symptoms that follow abstinence from a substance in a person dependent on that substance [31]. Withdrawal signs and symptoms assessed in the DSM-5 diagnostic criteria for gambling and Internet gaming disorder only include psychological dependence [31]. Such psychological dependence is characterized by feeling restless, irritable, or sad when attempting to cut down or stop gambling or gaming, or when one cannot access games. Similarly, focus group participants reported experiencing psychological withdrawal symptoms when unable to use the Internet. Participants noted negative mood states such as “frustration,” “irritation,” and “anxiety” when they were unable to access the Internet, or had attempted to reduce or stop their Internet use (see Table 2 for direct quotes from focus group participants and participants’ endorsement rates on the correspondent items on the YDQ and CIUS). YDQ indicated that 44.5% of participants had experienced feeling restless, moody, depressed, or irritable when attempting to cut down or stop their Internet use. CIUS results showed that 44.4% of participants frequently experienced feeling restless, frustrated, or irritated when they could not use the Internet.

Tolerance

Tolerance is characterized by individuals needing increasing amounts of a substance over time to achieve intoxication or desired effects [31]. Tolerance is also a criterion included in the DSM-5 criteria for gambling and Internet gaming disorder [31]. It parallels the criterion for substance use disorder and refers to needs to gamble with increasing amounts of money, or to spend increasing amounts of time engaged in Internet gaming in order to achieve the desired excitement. Focus group participants reported using the Internet in greater amounts due to its accessibility. Some participants noted using the Internet for the entire time they are awake. Participants also indicated that they could use the Internet more when they have smartphones with unlimited data (see Table 2 for direct quotes from focus group participants and participants’ endorsement rates on the correspondent items on the YDQ). YDQ results showed that 55.6% of participants reported feeling the need to use the Internet for increasing amounts of time in order to achieve satisfaction. Tolerance was not examined on the CIUS.

Unsuccessful Attempts to Stop or Reduce Internet Use

This sign is characterized by individuals having made unsuccessful efforts to stop or cut back on use of a substance [31]. This criterion is also assessed in DSM-5 criteria for gambling and Internet gaming disorder. It refers to a desire to stop or cut back on pathological gambling or gaming behaviors, but being unable to do it [31]. Focus group participants reported similar signs in relation to their Internet use. Participants noted a desire to reduce their Internet use, followed by unsuccessful attempts to stop or reduce their Internet use (see Table 2 for direct quotes from focus group participants and participants’ endorsement rates on the correspondent items on the YDQ and CIUS). YDQ results showed that the majority of participants had repeatedly made unsuccessful efforts to control, cut back, or stop Internet use (74.1%). CIUS results suggested that more than 80% of participants (84.6%) frequently found it difficult to stop using the Internet once they were online, and almost half of the participants (48.3%) reported they had frequently unsuccessfully tried to spend less time on the Internet.

Craving

Craving refers to strong desires or urges to use a substance [31]. However, craving for gambling or playing Internet games is not assessed in DSM-5 criteria for gambling and Internet gaming disorder. Focus group participants reported craving in regard to their Internet use. Participants noted urges or a strong desire to engage in activities on the Internet, specifically, when Internet access is available to them (see Table 2 for direct quotes from focus group participants). Neither the YDQ nor the CIUS included items related to craving.

Loss of Interest in Other Hobbies or Activities

A loss of interest in other hobbies or activities is a sign of substance use disorder and Internet gaming disorder [31]. This criteria is not included in the DSM-5 criteria for gambling disorder. Participants noted having lost interest in (or having participated less often in) activities they had previously found enjoyable including “socializing with friends or family,” “going-out,” and “exercising” due to Internet use (see Table 2 for direct quotes from focus group participants and participants’ endorsement rates on the correspondent items on the YDQ and CIUS). CIUS responses indicated that 22.2% of participants often/very often preferred to use the Internet instead of spending time with others. The YDQ did not include questions regarding loss of interest in other hobbies or activities.

Excessive Use despite Problems

This sign is characterized by continued use of a substance despite a persistent physical or psychological problem associated with substance use, or playing video games on the Internet [31]. This criteria is not included in the DSM-5 criteria for gambling disorder. Focus group participants reported similar behavior vis-a-vis their problem Internet use. Participants noted continued excessive Internet use despite problems such as academic under-achievement, conflict with others about Internet overuse, negative physical outcomes (e.g., inadequate amount of sleep), and Internet use while driving (see Table 2 for direct quotes from focus group participants and participants’ endorsement rates on the correspondent items on the YDQ and CIUS). Another related criterion in the DSM-5 criteria for substance use, gambling and Internet gaming disorder assesses the actual negative consequences (e.g., jeopardizing or actually losing important relationships or work/educational opportunities) of substance use, pathological gambling or Internet gaming. Focus group participants reported adverse health and/or psychosocial consequences due to their problematic Internet use behaviors. The negative consequences related to problematic Internet use have been reported in the previous study by the authors [4]. YDQ results showed that 33.3% of participants had jeopardized or risked the loss of a significant relationship, job, educational, or career opportunity because of their Internet use. CIUS responses indicated that 62.9% of participants frequently experienced being short of sleep because of their excessive Internet use, 38.5% frequently rushed through their homework in order to get on the Internet, and 44.4% frequently neglected their daily obligations because they preferred to access the Internet.

Use of the Internet to Escape or Relieve a Negative Mood

This sign is characterized by individuals using a substance to cope with negative moods such as depression, guilt, or anxiety; or gambling or playing Internet games when feeling distressed (e.g., helpless, guilty, anxious, or depressed) [31]. However, this criterion is not included in the DSM-5 criteria for substance use disorder. Focus group participants noted engaging in excessive Internet use to escape from, or cope with, negative moods or feelings such as “sadness,” “annoyance,” or “boredom” (see Table 2 for direct quotes from focus group participants and participants’ endorsement rates on the correspondent items on the YDQ and CIUS). YDQ results showed that 63.0% of participants had used the Internet as a way to escape from problems or relieve a dysphoric mood. CIUS findings indicated that half (50.0%) of participants often/very often used the Internet when they were feeling “down,” and almost half (42.3%) frequently used the Internet to escape from their sorrows or get relief from negative feelings.

Lying about Use

Individuals having lied to family members, a therapist, or others to conceal the extent of their involvement with gambling or gaming on the Internet characterize this theme [31]. However, this criterion is not included in the DSM-5 criteria for substance use disorder. Focus group participants also reported deceit in association with their Internet use. Some participants noted having lied to cover up the extent of their Internet use, such as the amount of time spent on the Internet or the specific activities performed online (see Table 2 for direct quotes from focus group participants and participants’ endorsement rates on the correspondent item on the YDQ). YDQ results showed that 25.9% of participants had lied to family members, a therapist, or others to conceal the extent of their involvement with the Internet. The CIUS did not ask questions regarding this theme.

Discussion

This study explored the extent to which problematic Internet use behaviors described by university students mirror the DSM-5 criteria for substance use disorder, gambling disorder, and Internet gaming disorder. Overall, signs and symptoms associated with problematic Internet use described by students in this study were similar to those of substance use disorder, gambling disorder, and Internet gaming disorder [31]. Importantly, participant quotes provided detailed descriptions of signs and symptoms of problematic Internet use, and contextualized the related quantitative findings. Although clearly, more rigorous studies are needed, our findings supported the previous evidence suggesting that problematic Internet use could be a type of behavioral addiction [1–2,20,22]. In a recently published qualitative study using the same sample [4], the authors explored the natural history of problematic Internet use; common affective, interpersonal, and situational triggers of Internet overuse; patterns of Internet use; and negative consequence of problematic Internet use among university students. The findings from the recent published study suggested that students’ self-reports of problematic Internet use were consistent with results of standardized measure which were developed based on diagnostic criteria for substance use disorder and gambling disorder, including Young’s Diagnostic Criteria and Compulsive Internet Use Scale [4]. Building upon previous work, the findings of this study further suggest that the signs and symptoms “volunteered” by university students who self-identified as having problems with Internet overuse mirrored signs and symptoms of substance use disorder, gambling disorder, and Internet gaming disorder assessed by DSM-5 criteria.

In line with extant literature, results indicated that there was overlap between the signs and symptoms of problematic Internet use and substance use disorder as well as behavioral addictions [2, 27, 28]; however, the specific context and symptom manifestation for problematic Internet use was distinct. Specifically, the Internet is widely available on college campuses and is normalized within college life. Students can use the Internet anytime and anywhere due to the wide adoption of campus-wide Internet access, smartphones, and data coverage [4]. This may limit the visible manifestation of symptoms such as tolerance in that there is a “ceiling effect” [41]—it is impossible for students to spend more than 24 hours in one day on the Internet. In addition, students in the focus groups appeared to be able to access the Internet whenever they have a desire to engage in Internet-related activities. Therefore students in this study may not notice the development of tolerance symptoms. Although more than half (55.6%) of participants responded positively to the YDQ item that assesses tolerance, very few students in the focus group reported using the Internet increasingly to achieve the same level of satisfaction. Measurement of such a construct among populations with problematic Internet use may require researchers and service providers to develop new and distinct diagnostic criteria. Tolerance refers to feeling the need to use the Internet longer periods of time to experience the same amount of satisfaction. Tolerance may also involve the need for more exciting activities on the Internet. When assessing tolerance among students, other information regarding the history and current patterns of Internet use may also need to be assessed in addition to the YDQ item “a need to increasee the amount of time spent on the Internet to achieve same amount of satisfaction.”

Similarly, withdrawal symptoms may also manifest uniquely among populations with problematic Internet use behaviors. While substance use withdrawal generally includes physical symptoms [31], students in this study did not report physiological symptoms of withdrawal. Instead, students reported a myriad of psychological withdrawal symptoms including negative mood states (e.g., depression), and anxiety symptoms (e.g., restlessness). These psychological symptoms have been linked to other substance, gambling, and Internet gaming disorder [1–2, 27,28, 31], and therefore may need to be more closely examined by researchers and service providers when assessing for problematic Internet use among college-aged youth. Further, the results suggest that the “withdrawal like symptoms” could partially related to interpersonal conflict due to Internet overuse or problematic Internet use. Therefore, withdrawal symptoms associated with problematic Internet use should be distinguished from psychological distress that arise in response to other related factors, including stop using the Internet because of conflicts with parents or significant others regarding problematic Internet use.

Finally, Internet use while driving emerged as a key finding under the theme “Excessive Use Despite Problems.” Students reported using the Internet on their phones/tablets to chat with friends, play games, and post on social media sites while driving, despite knowledge that such behaviors are life threatening and illegal in all states. These behaviors strongly mirror substance use behaviors such as drinking and driving. However, as with tolerance and withdrawal symptoms, the context of Internet use while driving is often normalized among young people and so the serious repercussions are minimized by the youth themselves and the service providers who work with them [42].

These findings have implications for future research. Specifically, studies using larger and more representative sample and rigorous designs are needed to further investigate the extent to which the characteristics of problematic Internet use mirror the signs and symptoms of substance use and behavioral addictions. Further studies need to be conducted to closely examine the extent to which the existing instruments and diagnostic criteria adequately assess the domains of problematic Internet use, and to establish the validity of the cutoff points of existing instruments that measure problematic Internet use behaviors. Without validated and sensitive/specific instruments and diagnostic criteria, it is difficult to reliably distinguish pathological from normal Internet users.

In addition, Internet is an environment or a medium, and the addiction or the addictive behavior would pertain to the corresponding activities on the Internet and not to the Internet itself [21]. Thus, the further studies need to focus on the addiction-like symptoms that are associated with specific problematic behaviors related to Internet-use (e.g., Internet gaming disorder, online-gambling, online-shopping, and online pornography viewing). Specific criteria should be developed to assess subtypes of problematic Internet use (such as compulsive pornography viewing and Internet gaming addiction), because each subtype may have different signs, symptoms, and adverse effects.

Such knowledge would be especially important for service providers working with college-aged youth suffering from problematic Internet use and its associated psychosocial problems. While the Internet is necessary for academic work, job performance, and socialization, problematic Internet use has serious adverse effects on psychosocial well-being. Thus, service providers should screen and assess for problematic Internet use among at-risk college-aged youth. Our findings show that tolerance may not be a sensitive criterion; instead, other signs and symptoms, such as psychological withdrawal symptoms, craving, and excessive use despite problems should be included when assessing problematic Internet use.

Study limitations include the small sample size, single site location of the investigation, and exploratory nature of the findings. These factors all may limit the generalizability of results. However, the university where this research was conducted is similar to many other large public universities and the sample was diverse with respect to ethnicity and gender. Further, focus group discussion rather than individual assessment may lead to social desirability biases and influence the validity of the qualitative findings. The data collected were rich and informed by quantitative and qualitative assessments. Perhaps most importantly, given the paucity of research on U.S. college students with problematic Internet use, we hope our findings will stimulate further investigation in this important emerging area.

Supporting Information

[Figure omitted. See PDF.]

S1 Document. Survey Questions for Sociodemographic and Characteristics of Problematic Internet Use, and Focus Group Discussion Guidance.

https://doi.org/10.1371/journal.pone.0145981.s001

(DOCX)

S2 Document. Codebook.

https://doi.org/10.1371/journal.pone.0145981.s002

(DOCX)

S1 Table. Data Set for Sample Characteristics of 27 Participants Who Self-Reported Intensive Internet Use.

https://doi.org/10.1371/journal.pone.0145981.s003

(DOCX)

S2 Table. Data Set for Young’s Diagnostic Questionnaire (N = 27).

https://doi.org/10.1371/journal.pone.0145981.s004

(DOCX)

S3 Table. Data set for the Compulsive Internet Use Scale (N = 27).

https://doi.org/10.1371/journal.pone.0145981.s005

(DOCX)

Author Contributions

Conceived and designed the experiments: WL MOH. Performed the experiments: WL MOH. Analyzed the data: WL JEO SMS. Contributed reagents/materials/analysis tools: WL JEO MOH. Wrote the paper: WL JEO SMS MOH.

Citation: Li W, O’Brien JE, Snyder SM, Howard MO (2016) Diagnostic Criteria for Problematic Internet Use among U.S. University Students: A Mixed-Methods Evaluation. PLoS ONE 11(1): e0145981. https://doi.org/10.1371/journal.pone.0145981

References

1. Young KS (1998) Internet addiction: The emergence of a new clinical disorder. Cyberpsychol Behav 1:237–244.

2. Weinstein A, Lejoyeux M (2010) Internet addiction or excessive internet use. Am J Drug Alcohol Abuse 36: 277–283. pmid:20545603

3. Li W, Garland EL, Howard MO (2014) Family factors in Internet addiction among Chinese youth: A review of English- and Chinese-language studies. Comput Human Behav 31: 393–411.

4. Li W, O’Brien JE, Snyder SM, Howard MO (2015) Characteristics of internet addiction/pathological internet use in U.S. University students: A qualitative-method investigation. PLOS ONE 10: e0117372. pmid:25647224

5. Liu T, Potenza MN (2007) Problematic Internet use: Clinical implications. CNS Spectr 12: 453–466. pmid:17545956

6. Derbyshire KL, Lust KA, Schreiber LRN, Odlaug BL, Christenson GA, Golden DJ, et al. (2013) Problematic Internet use and associated risks in a college sample. Compr Psychiatry 54: 415–422. pmid:23312879

7. Christakis DA, Moreno MM, Jelenchick L, Myaing MT, Zhou C (2011) Problematic internet usage in US college students: A pilot study. BMC Med 9: 77. pmid:21696582

8. Moreno MA, Jelenchick L, Cox E, Young H, Christakis DA (2011) Problematic Internet use among US youth: A systematic review. Arch Pediatr Adolesc Med 165: 797–805. pmid:21536950

9. Lam LT (2014) Internet gaming addiction, problematic use of the Internet, and sleep problems: A systematic review. Curr Psychiatry Rep 16: 444. pmid:24619594

10. Vandelanotte C, Sugiyama T, Gardiner P, Owen N (2009) Associations of leisure-time internet and computer use with overweight and obesity, physical activity and sedentary behaviors: Cross-sectional study. J Med Internet Res 11: e28. pmid:19666455

11. Dong GG, Lu Q, Zhou H, Zhao X (2011) Precursor or sequela: Pathological disorders in people with Internet addiction disorder. PlOS ONE 6: e14703. pmid:21358822

12. Ko CH, Yen JY, Yen CF, Chen CS, Chen CC (2010) The association between Internet addiction and psychiatric disorder: A review of the literature. Eur Psychiatry 27: 1–8.

13. Park S, Hong KEM, Park EJ, Ha KS, Yoo HJ (2013) The association between problematic internet use and depression, suicidal ideation and bipolar disorder symptoms in Korean adolescents. Aust N Z J Psychiatry 47: 153–159. pmid:23047959

14. Yen JY, Yen CF, Chen CS, Tang TC, Ko CH (2009) The association between adult ADHD symptoms and Internet addiction among college students: The gender difference. Cyberpsychol Behav 12: 187–191. pmid:19072077

15. Sun P, Johnson CA, Palmer P, Arpawong TE, Unger JB, Xie B, et al. (2012) Concurrent and predictive relationships between compulsive Internet use and substance use: Findings from vocational high school students in China and the USA. International Journal of Environmental Research and Public Health 9: 660–673. pmid:22690154

16. Lam LT, Peng Z, Mai J, Jing J (2009) The association between internet addiction and self-injurious behaviour among adolescents. Inj Prev 15: 403–408. pmid:19959733

17. Ko CH, Yen JY, Liu SC, Huang CF, Yen CF (2009) The associations between aggressive behaviors and Internet addiction and online activities in adolescents. J Adolescent Health 44:598–605.

18. Kerkhof P, Finkenauer C, Muusses LD (2011) Relational consequences of compulsive Internet use: A longitudinal study among newlyweds. Hum Commun Res 37: 147–173.

19. Kardefelt-Winther D (2014) A conceptual and methodological critique of Internet addiction research: Towards a model of compensatory Internet use. Comput Human Behav 31: 351–354.

20. Weinstein A, Feder LC, Rosenberg KP, Dannon P Internet addiction disorder: Overview and controsersies. In: Rosenberg KP, Feder LC, editors. Behavioral addictions: Criteria, evidence, and treatment. Academic Press; 2014. pp. 99–118.

21. Starcevic V (2013) Is Internet addiction a useful concept? Aust N Z J Psychiatry 47: 16–19. pmid:23293309

22. Lee HW, Choi JS, Shin YC, Lee JY, Jung HY, Kwon JS (2012) Impulsivity in Internet addiction: A comparison with pathological gambling. Cyberpsychology, Behav Soc Netw 15: 373–377.

23. Yen J, Ko C, Yen C, Chen C, Chen C (2009) The association between harmful alcohol use and Internet addiction among college students: Comparison of personality. Psychiat Clin Neuros 63: 218–224.

24. Kim SH, Baik SH, Park CS, Kim SJ, Choi SW, Kim SE (2011) Reduced striatal dopamine D2 receptors in people with Internet addiction. Neuroreport 22: 407–411. pmid:21499141

25. Kühn S, Gallinat J (2014) Brains online: Structural and functional correlates of habitual Internet use. Addict Biol.

26. Scherer K (1997) College life on-line: Healthy and unhealthy Internet use. J Coll Stud Dev 38: 655–665.

27. Tao R, Huang X, Wang J, Zhang H, Zhang Y, Li M (2010) Proposed diagnostic criteria for Internet addiction. Addiction 105: 556–564. pmid:20403001

28. Petry NM, Rehbein F, Gentile DA, Lemmens JS, Rumpf HJ, Mößle T, et al. (2014) An international consensus for assessing internet gaming disorder using the new DSM‐5 approach. Addiction 109:1399–1406. pmid:24456155

29. Sim T, Gentile DA, Bricolo F, Serpelloni G, Gulamoydeen F (2012) A conceptual review of research on the pathological use of computers, video games, and the Internet. International Journal of Mental Health and Addiction 10: 748–769.

30. Meerkerk GJ, Van Den Eijnden RJJM, Vermulst AA, Garretsen HFL (2009) The compulsive Internet use scale (CIUS): Some psychometric properties. Cyberpsychol Behav 12: 1–6. pmid:19072079

31. American Psychiatric Association (2013) Diagnostic and Statistical Manual of Mental Disorders (5th ed.). Arlington, VA: American Psychiatric Publishing.

32. Lortie C, Guitton MJ (2013) Internet addiction assessment tools: Dimensional structure and methodological status. Addiction 108: 1207–1216. pmid:23651255

33. Jelenchick LA, Becker T, Moreno MA (2012) Assessing the psychometric properties of the Internet Addiction Test (IAT) in US college students. Psychiatry Res 196: 296–301. pmid:22386568

34. Canan F, Ataoglu A, Nichols LA, Yildirim T, Ozturk O (2010) Evaluation of psychometric properties of the Internet addiction scale in a sample of Turkish high school students. Cyberpsychology, Behav Soc Netw 13:317–20.

35. Widyanto L, Griffiths MD, Brunsden VB (2011) A psychometric comparison of the Internet addiction test, the Internet-related problem scale, and self-diagnosis. Cyberpsychology, Behav Soc Netw 14:141–9.

36. Dowling NA, Quirk KL (2009) Screening for Internet dependence: Do the proposed diagnostic criteria differentiate normal from dependent Internet use? Cyberpsychol Behav 12: 21–27. pmid:19196045

37. Guertler D, Rumpf HJ, Bischof A, Kastirke N, Petersen KU, John U, et al. (2014) Assessment of problematic internet use by the compulsive internet use scale and the internet addiction test: A sample of problematic and pathological gamblers. Eur Addict Res 20: 75–81. pmid:24080838

38. Padgett DK (1998) Qualitative methods in social work research: Challenges and rewards. Thousand Oaks: Sage Publications.

39. DeCuir-Gunby JT, Marshall PL, McCulloch AW (2011) Developing and using a codebook for the analysis of interview data: An example from a professional development research project. Field Methods 23: 136–155.

40. Patton MQ (2001) Qualitative Research and Evaluation Methods (3rd edition). Thousand Oaks, CA: Sage.

41. Gallagher KE, Lisco CG, Parrott DJ, Giancola PR (2014) Effects of thought suppression on provoked men’s alcohol-related physical aggression in the laboratory. Psychology of Violence 4:78–89. pmid:25337430

42. National Organization for Youth Safety (2009) Young drivers understand risky behavior but tempt fate behind the wheel. Available: http://allstatenewsroom.com/categories/7-news-release-archives/releases/4510-young-drivers-understand-risky Accessed 15 June 2015.