Predicting Psychological State Among Chinese Undergraduate Students in the COVID-19 Epidemic: A Longitudinal Study Using a Machine Learning

Abstract

Background: The outbreak of the 2019 novel coronavirus disease (COVID-19) not only caused physical abnormalities, but also caused psychological distress, especially for undergraduate students who are facing the pressure of academic study and work. We aimed to explore the prevalence rate of probable anxiety and probable insomnia and to find the risk factors among a longitudinal study of undergraduate students using the approach of machine learning.

Methods: The baseline data (T1) were collected from freshmen who underwent psychological evaluation at two months after entering the university. At T2 stage (February 10th to 13th, 2020), we used a convenience cluster sampling to assess psychological state (probable anxiety was assessed by general anxiety disorder-7 and probable insomnia was assessed by insomnia severity index-7) based on a web survey. We integrated information attained at T1 stage to predict probable anxiety and probable insomnia at T2 stage using a machine learning algorithm (XGBoost).

Results: Finally, we included 2009 students (response rate: 80.36%). The prevalence rate of probable anxiety and probable insomnia was 12.49% and 16.87%, respectively. The XGBoost algorithm predicted 1954 out of 2009 students (translated into 97.3% accuracy) and 1932 out of 2009 students (translated into 96.2% accuracy) who suffered anxiety and insomnia symptoms, respectively. The most relevant variables in predicting probable anxiety included romantic relationship, suicidal ideation, sleep symptoms, and a history of anxiety symptoms. The most relevant variables in predicting probable insomnia included aggression, psychotic experiences, suicidal ideation, and romantic relationship.

Conclusion: Risks for probable anxiety and probable insomnia among undergraduate students can be identified at an individual level by baseline data. Thus, timely psychological intervention for anxiety and insomnia symptoms among undergraduate students is needed considering the above factors.

Introduction

The 2019 novel coronavirus disease (COVID-19) is caused by a variety of coronavirus (SARS-CoV-2). In March 2020, the World Health Organization (WHO) declared COVID-19 as a global pandemic.¹ The symptoms of COVID-19 are usually non-specific (eg, fever, cough, and dyspnea).² Up to now, COVID-19 is significantly contagious and no effective treatments or vaccines are available.³ As of 19 June 2020 (10:00 am CET), over 8.45 million cases have been diagnosed globally with more than 453,000 fatalities.⁴ Many unprecedented strategies have been taken to cut off the spread of the virus in many countries (eg, China, England, and the United States). For example, the Chinese government released some guidance for the general population to self-isolate. Specifically, lockdown (eg, in Wuhan), temporary closure of schools/factories, and restriction of individuals' activities Isolation and quarantine measures have been effective at preventing the spread of COVID-19. However, consensus has arisen about their potential mental health.^5,6 In the severe acute respiratory syndrome epidemic, confirmed cases suffered higher stress levels, poor sleep quality, and depressed mood.⁷ Recently, Brooks‘ review suggested that the mental health impact of quarantine is wide-ranging and substantial.⁵ In the present epidemic of COVID-19, higher anxiety level (23.04%) was reported among medical staff.⁸ And Zhang‘s study found that more than one-third of medical staff suffered insomnia symptoms.⁹ In addition to special groups (eg, medical staff and confirmed cases), Wang's study found that about 33.33% of the general population reported moderate-to-severe anxiety.¹⁰ Isolated people may experience stressful conditions (eg, social activities and face to face communication were restricted).¹¹ Thus, negative emotions experienced by people may be compounded.^11,12 Undergraduate education is a special stage that is highly specialized, knowledgeable, and continuous. Theory teaching and practical teaching are two profiles as well as two taches in the same teaching life-cycle. The epidemic prevented undergraduate students from returning to university. Thus, undergraduate students' psychological state may fluctuate.^13,14

We found that previous studies have some limitations. Firstly, most research focuses on the mental health of medical students, and ignores the mental health of students of other disciplines (eg, humanities, engineering, and agricultural). Although medical students are a special group, they only constitute a very small part of undergraduate students. Secondly, most of the studies have a cross-sectional design, only exploring the psychological state and risk factors of undergraduate students during the COVID-19 epidemic.

In summary, we aimed to investigate the prevalence rate of probable anxiety and probable insomnia and to confirm the risk factors among undergraduate students during the COVID-19 outbreak. Finally, we ranked risk factors in the model based on "feature importance."

Methods Participants and Procedure

We have obtained the Ethics Committee of the Ocean University of China (2,020,001). An online version of an informed consent form is provided to students before starting the survey. In the form, we explained to the students that participation was voluntary, and refusals would have no negative consequences. We also guaranteed data confidentiality and that only the researchers could access the information. Undergraduate students can choose to participate in or reject the survey. If they choose to participate in the survey, they are evaluated via an online platform. If they refuse to take part in the survey, they withdraw from the online platform. We have obtained informed consent from students who accomplished the survey. The target population of the research was undergraduate students at the Ocean University of China. The Ocean University of China is a government university located in Shandong Province. The baseline survey (T1) was collected from freshmen. Specifically, all freshmen underwent psychological evaluation at two months after entering the Ocean University of China. In the presence of the COVID-19 epidemic (T2, February 10th to 13th, 2020), we used convenience cluster sampling and invited undergraduate students (n=2500) from four grades (freshman, sophomore, junior and senior) to participate in the survey. Finally, 2009 participants completed the web-based survey (response rate: 80.36%). We used the students' ID numbers to match the data.

Measures

Social Demographic Information Basic demographic information was collected at T1 stage. We collected basic characteristics using a self-constructed questionnaire. The questionnaire included gender (female=1, male=2), year of education (first=1, second=2, third=3, fourth=4), family economic (low family economic=1, high family economic=2), upbringing place (metropolitan=1, medium and small cities=2, town=3, country=4) and single-child families (yes=1, no=2).

College Students Mental Health Screening Scale The Ministry of Education of the People's Republic of China recommended the College Students Mental Health Screening Scale (CSMHSS) as a reliable and valid measurement tool to evaluate the mental health of undergraduate students. The scale of CSMHSS consists of 22 dimensions. Specifically, it includes psychotic experiences (4 items), suicidal ideation (3 items), the history of anxiety symptoms (4 items), the history of depression symptoms (5 items), paranoia (4 items), inferiority (5 items), sensitivity (4 items), social phobia (4 items), somatization (4 items), dependence (4 items), aggression (4 items), impulsive (4 items), obsession and compulsion (4 items), Internet addiction (5 items), self-injury (4 items), eating problems (4 items), sleeping problem (4 items), school adjustment difficulties (4 items), interpersonal distress (4 items), academic pressure (4 items), employment pressure (4 items) and romantic relationship problems (4 items). The scale uses 4-point Likert-scaled items ranging from 1 (not at all like me) to 4 (very like me). The scores in each dimension are added then standardized. The standard score of the total score in each dimension can be categorized at a cutoff of 3 (Fang, Yuan, Hu, Deng, and Lin, 2018).

General Anxiety Disorder-7 The scale of General Anxiety Disorder (GAD-7) was used in this research. It was a tool to assess anxiety symptoms. A score ≥7 indicated clinically significant anxiety symptoms.^15,16 GAD-7 was well validated and sensitive to the general population.^17–20 Studies had shown that GAD-7 has good reliability and validity in China.^21,22 The Cronbach's alpha value was 0.90 for GAD-7 in this research.

Insomnia Severity Index-7 We used the Insomnia Severity Index (ISI-7) which is a 7‐item instrument that evaluates subjective sleep symptoms. Each item is scored on a 5‐point scale, with higher scores representing more severe insomnia symptoms. A score >14 indicates clinically significant insomnia symptoms.²³ The Cronbach's alpha value was 0.86 for ISI-7 in this research.

Statistics Analysis

Descriptive data analysis was implemented in SPSS 21.0 for windows. XGBoost (Extreme Gradient Boosting), which is a machine learning algorithm, was implemented in Python 3.70. XGBoost is a method for regression and classification problems according to the Gradient Boosting Decision Tree. This method has been widely used in all kinds of data fields for regression and classification.²⁴ The algorithm of XGBoost can utilize a cross-validation approach to divide data into a model "training set" and "testing set." In the current research, we used a 5-fold cross-validation method. Classification performance was scored with the area under the receiver-operation Curve (AUC), sensitivity (Sen), specificity (Spe), and accuracy (ACC).

Results Participant Characteristics

Finally, there were 2009 undergraduate students who were included in our research. Of the 2009 participants, 50.97% were female, 79.99% were from high family economic background, and 25.35% were from metropolitan areas. In the present epidemic of COVID-19, the prevalence rate of probable anxiety and probable insomnia symptoms was 12.49% (GAD-7≥7) and 16.87% (ISI-7>14), respectively. The detailed basic characteristics are shown in Table 1.

Table 1 Basic Characteristics of Undergraduate Students

Predicting Probable Anxiety and Probable Insomnia

We integrated the data collected at T1 stage to predict probable anxiety and probable insomnia during the COVID-19 epidemic (at T2 stage). The AUC of probable anxiety and probable insomnia is 99.00% and 98.00%, respectively. Figure 1 shows the AUCs for probable anxiety and probable insomnia.

Figure 1 The AUCs for the prediction of probable anxiety A and insomnia B.

"Feature Importance" in Predicting Probable Anxiety and Insomnia

According to the AUCs and the confusion matrix, we calculated the sensitivity (Sen), specificity (Spe), and accuracy (ACC). The machine learning of XGBoost predicted 1954 out of 2009 as either anxiety or no-anxiety and this translated into 97.3% accuracy (97.3% sensitivity and 96.3% specificity). The machine learning of XGBoost predicted 1932 out of 2009 as either insomnia or no-insomnia and this translated into 96.2% accuracy (95.5% sensitivity and 100.0% specificity). Detailed information is demonstrated in Table 2. Feature importance assigned positive coefficients via XGBoost, indicating that an increase in probable anxiety included 1) romantic relationship problems, 2) suicidal ideation, 3) the history of anxiety symptoms, and 4) sleep symptoms. It was also indicated that an increase in probable insomnia included 1) aggression, 2) psychotic experiences, 3) suicide ideation, and 4) romantic relationship problems. The confusion matrix and "feature importance" are shown in Figures 2A and B and 3A and B, respectively.

Table 2 Confusion Matrix of Probable Anxiety and Insomnia

Figure 2 Confusion matrix (A) anxiety; (B) insomnia.

Figure 3 Bar graph showing 'feature importance' in predicting probable anxiety and insomnia, respectively (A) anxiety; (B) insomnia.

Discussion Prevalence of Probable Anxiety and Insomnia

In the current research, the prevalence rate of probable anxiety and probable insomnia among undergraduate students was 12.49% and 16.87%, respectively. The prevalence rate of probable anxiety is higher than the Zhang's study (7.5%) and lower than the Cao's study. Cao's study found that 24.9% of medical students suffered from anxiety symptoms.²⁵ The variability of prevalence rates could be explained by medical students being a special group who face more academic and employment pressure. And previous studies found that medical students are more likely to have psychological problems.^26,27 The prevalence rate of probable insomnia is the similarity to Huang's study (18.2%)²⁸ and lower than the 36.1% reported by Zhang's study.⁹ This variability of prevalence rates could be explained by the participants, questionnaires, and regions.

Prediction Factors of Probable Anxiety

Most relevant variables predicting probable anxiety included romantic relationship problems, suicidal ideation, history of anxiety symptoms, and sleep symptoms. Falling in love is a universal behavior among undergraduate students. Studies indicate that youths experience romantic relationships of joy and happiness. However, a romantic relationship is not entirely a happy period of life. Bajoghli's study found that for youths, falling in love may be also associated with anxiety symptoms.²⁹ Consistent with Asselmann's study, we found that the history of anxiety symptoms prior to/at baseline predicted a recurrence of probable anxiety at the time of follow-up.³⁰ Narmandakh's study found that sleep disturbance may precede anxiety symptoms. And anxiety symptoms might be prevented by alleviating sleep disturbance.³¹ Previous results suggested that the presence of "any anxiety disorder" increases the risk for suicidal ideation among the general population, even after controlling for confounding factors (Wilcox et al, 2010). In the current study, we found that suicidal ideation can be used to predict probable anxiety. The results may indicate that there is a bidirectional relationship between suicidal ideation and anxiety symptoms among youths.

Prediction Factors of Probable Insomnia

Most relevant variables predicting probable insomnia included aggression, psychotic experiences, suicidal ideation, and romantic relationship problems. Consistent with previous studies, we also found that insomnia is a consequence of psychotic symptoms.³² Recent studies demonstrate that insomnia also contributed to the development of psychotic symptoms.³³ Insomnia symptoms may be one of the top warning signs of suicide in a clinical outpatient setting.³⁴ Suh's study also found that insomnia symptoms were related to concurrent and future ideations of suicide in a population-based longitudinal study.³⁵ And a meta-analysis showed that sleep disturbances in general, as well as insomnia individually, appear to represent a risk factor for suicidal ideation and behavior.³⁶ Namely, there may be a bidirectional association of insomnia symptoms with psychotic experiences and insomnia symptoms with suicide ideation. Falling in love is an emotional occurrence at any age, but for undergraduate students, the feelings might be overwhelming.³⁷ In addition to being a positive feeling (eg, joy and happiness), a romantic relationship may cause stress and negative effect, especially if the feeling is not reciprocal.³⁸ Kuula's study revealed that romantic relationship is one reason for sleep disturbance in girls and may be associated with symptoms of anxiety in both boys and girls.³⁹

Clinical Implications

We have found relatively reliable and accurate predictive models during the COVID-19 epidemic. And our models provide useful information about the most relevant variables to predict probable anxiety and insomnia among college students. The stage of university education is an important period of life development, and it is very necessary to carry out psychological assessment of freshmen who have just entered the university. Intervening with students with psychological problems in a timely and effective manner would not only help them recover their mental health, but also help them adjust their state when facing emergencies. Stoessel's study found besides being positive feelings, romantic relationships may cause stress and negative effect, especially if the love is not reciprocal.³⁸ Thus, in addition to resolving regular psychological problems, it is necessary to help college students to establish healthy romantic relationships, one of the principal developmental tasks of emerging adulthood. How to effectively organize the mental health services for those undergraduates who have present anxiety and/or insomnia symptoms due to the COVID-19 pandemic is also very important. Community-based and school-based mental health services care be combined into the national health system.⁴⁰

Strengths and Limitations

There are some strengths in this research, including 1) this is a longitudinal study and we use the data at the time of enrollment to predict college students' anxiety and insomnia during the outbreak; 2) we integrate data from multiple dimensions; 3) we calculate models for individual classification using machine learning. However, the current research has some limitations, including 1) our participants are from a specific university located in Shandong province. And this university does not include medical students. Thus, the results cannot be generalized to all Chinese undergraduates. 2) We used self-reported questionnaires in this research, so response bias and recall bias may exist considering that undergraduate students may have underreported or overreported their anxiety and/or sleep symptoms. We took some steps to reduce this by keeping uniformity of data collection approach. It is worth mentioning that we found that romantic relationship trouble is an important factor in predicting anxiety and insomnia. 3) We used different questionnaires at T1 stage and T2 stage. Thus, it is difficult to directly compare the prevalence rate of anxiety and insomnia at two stages. 4) Temperament is stable across the lifespan and mediate adaptive functioning to some extent. And the attachment system may be activated in stressful situations. Recently, Moccia's research found that some specific affective temperament (eg, cyclothymic and anxious temperaments) and attachment features (eg, need for approval) can be used to predict the burden of mental health.⁴¹ However, information on temperamental and attachment was not collected in our study. Thus, it is necessary for researchers to consider temperament and attachment in future studies.

Conclusion

This longitudinal research contributes to our understanding of the psychological state of undergraduate students who suffered a sudden public health event. And we found a reliable model to predict anxiety and insomnia during the sudden public health. Thus, timely psychological intervention is necessary, not only to help undergraduate students recover their mental health but also to help them face some emergency events.

Disclosure

Fenfen Ge and Di Zhang are co-first authors. The authors report no conflicts of interest in this work.

References

1. Mahase E, Humphreys K, Cullen MR, Basu S. China coronavirus: WHO declares international emergency as death toll exceeds 200. BMJ. 2020;368:m408. doi:10.1136/bmj.l6968

2. Wang C, Horby PW, Hayden FG, Gao GF. A novel coronavirus outbreak of global health concern. Lancet. 2020;395(10223):470–473. doi:10.1016/S0140-6736(20)30185-9

3. Gallagher TH, Schleyer AM. “We signed up for this!” - student and trainee responses to the Covid-19 pandemic. N Engl J Med. 2020;382:e96. doi:10.1056/NEJMp2005234

4. GOV.UK. Coronavirus (COVID-19): what you need to do. 2020. Available from: https://www.gov.uk/government/publications/wuhan-novel-coronavirus-background-information/wuhan-novel-coronavirus-epidemiology-virology-and-clinical-features.AccessedAugust22, 2020 .

5. Brooks SK, Webster RK, Smith LE, et al. The psychological impact of quarantine and how to reduce it: rapid review of the evidence. Lancet. 2020;395:912–920. doi:10.1016/S0140-6736(20)30460-8

6. Fiorillo A, Gorwood P. The consequences of the COVID-19 pandemic on mental health and implications for clinical practice. European Psychiatry. 2020;63(1):e32. doi:10.1192/j.eurpsy.2020.35

7. Lee DT, Wing YK, Leung HC, et al. Factors associated with psychosis among patients with severe acute respiratory syndrome: a case-control study. Clin Infect Dis. 2004;39(8):1247–1249. doi:10.1086/424016

8. Huang JZ, Han MF, Luo TD, Ren AK, Zhou XP. [Mental health survey of 230 medical staff in a tertiary infectious disease hospital for COVID-19]. Zhonghua Lao Dong Wei Sheng Zhi Ye Bing Za Zhi. 2020;38:E001.

9. Zhang C, Yang L, Liu S, et al. Survey of insomnia and related social psychological factors among medical staff involved in the 2019 novel coronavirus disease outbreak. Front Psychiatry. 2020;11:306. doi:10.3389/fpsyt.2020.00306

10. Wang C, Pan R, Wan X, et al. Immediate psychological responses and associated factors during the initial stage of the 2019 coronavirus disease (COVID-19) epidemic among the general population in China. Int J Environ Res Public Health. 2020;17:5.

11. Zhang J, Wu W, Zhao X, Zhang W. Recommended psychological crisis intervention response to the 2019 novel coronavirus pneumonia outbreak in China: a model of West China Hospital. Precision Clinical Med. 2020;3(1):3–8. doi:10.1093/pcmedi/pbaa006

12. Van Bortel T, Basnayake A, Wurie F, et al. Psychosocial effects of an Ebola outbreak at individual, community and international levels. Bull World Health Organ. 2016;94(3):210–214. doi:10.2471/BLT.15.158543

13. AlShamlan NA, AlShamlan RA, AlShamlan AA, et al. Prevalence of depression and its associated factors among clinical-year medical students in Eastern Province, Saudi Arabia. Postgrad Med J. 2020;96:343–348. doi:10.1136/postgradmedj-2020-137578

14. Soled D, Goel S, Barry D, et al. Medical student mobilization during a crisis: lessons from a COVID-19 medical student response team. Acad Med. 2020. doi:10.1097/ACM.0000000000003401

15. Blenkiron P, Goldsmith L. Patient-reported outcome measures in community mental health teams: pragmatic evaluation of PHQ-9, GAD-7 and SWEMWBS. BJPsych Bull. 2019;43(5):221–227. doi:10.1192/bjb.2019.20

16. Spitzer RL, Kroenke K, Williams JBW, Loewe B. A brief measure for assessing generalized anxiety disorder - The GAD-7. Arch Intern Med. 2006;166(10):1092–1097. doi:10.1001/archinte.166.10.1092

17. Andreas JB, Brunborg GS. Depressive symptomatology among norwegian adolescent boys and girls: the patient health questionnaire-9 (PHQ-9) psychometric properties and correlates. Front Psychol. 2017;8:11.

18. Tiirikainen K, Haravuori H, Ranta K, Kaltiala-Heino R, Marttunen M. Psychometric properties of the 7-item generalized anxiety disorder scale (GAD-7) in a large representative sample of Finnish adolescents. Psychiatry Res. 2019;272:30–35. doi:10.1016/j.psychres.2018.12.004

19. Horton M, Perry AE. Screening for depression in primary care: a Rasch analysis of the PHQ-9. BJPsych Bull. 2016;40(5):237–243. doi:10.1192/pb.bp.114.050294

20. Kroenke K, Spitzer RL, Williams JBW, Monahan PO, Lowe B. Anxiety disorders in primary care: prevalence, impairment, comorbidity, and detection. Ann Intern Med. 2007;146(5):317–325. doi:10.7326/0003-4819-146-5-200703060-00004

21. Wang W, Bian Q, Zhao Y, et al. Reliability and validity of the Chinese version of the Patient Health Questionnaire (PHQ-9) in the general population. Gen Hosp Psychiatry. 2014;36(5):539–544. doi:10.1016/j.genhosppsych.2014.05.021

22. Zeng Q-Z, He Y-L, Liu H, et al. Reliability and validity of Chinese version of the generalized anxiety disorder 7-item (GAD-7) scale in screening anxiety disorders in outpatients from traditional Chinese internal department. [Reliability and validity of Chinese version of the Generalized Anxiety Disorder 7-item (GAD-7) scale in screening anxiety disorders in outpatients from traditional Chinese internal department.]. Chinese Mental Health J. 2013;27(3):163–168.

23. Savard MH, Savard J, Simard S, Ivers H. Empirical validation of the insomnia severity index in cancer patients. Psychooncology. 2005;14(6):429–441. doi:10.1002/pon.860

24. Chen T, Guestrin C. XGBoost: a scalable tree boosting system. 2016:785–794.

25. Cao W, Fang Z, Hou G, et al. The psychological impact of the COVID-19 epidemic on college students in China. Psychiatry Res. 2020;287:112934. doi:10.1016/j.psychres.2020.112934

26. Dahlin M, Joneborg N, Runeson B. Stress and depression among medical students: a cross-sectional study. Med Edu. 2005;39(6):594–604. doi:10.1111/j.1365-2929.2005.02176.x

27. Dyrbye LN, Thomas MR, Shanafelt TD. Systematic review of depression, anxiety, and other indicators of psychological distress among U.S. and Canadian medical students. Acad Med. 2006;81(4):354–373. doi:10.1097/00001888-200604000-00009

28. Huang Y, Zhao N. Mental health burden for the public affected by the COVID-19 outbreak in China: who will be the high-risk group? Psychol Health Med. 2020:1–12.

29. Bajoghli H, Keshavarzi Z, Mohammadi MR, et al. “I love you more than I can stand!” - romantic love, symptoms of depression and anxiety, and sleep complaints are related among young adults. Int J Psychiatry Clin Pract. 2014;18(3):169–174. doi:10.3109/13651501.2014.902072

30. Asselmann E, Beesdo-Baum K. Predictors of the course of anxiety disorders in adolescents and young adults. Curr Psychiatry Rep. 2015;17(2):7. doi:10.1007/s11920-014-0543-z

31. Narmandakh A, Roest AM, Jonge P, Oldehinkel AJ. The bidirectional association between sleep problems and anxiety symptoms in adolescents: a TRAILS report. Sleep Med. 2020;67:39–46. doi:10.1016/j.sleep.2019.10.018

32. Reeve S, Sheaves B, Freeman D. The role of sleep dysfunction in the occurrence of delusions and hallucinations: a systematic review. Clin Psychol Rev. 2015;42:96–115. doi:10.1016/j.cpr.2015.09.001

33. Reeve S, Nickless A, Sheaves B, Freeman D. Insomnia, negative affect, and psychotic experiences: modelling pathways over time in a clinical observational study. Psychiatry Res. 2018;269:673–680. doi:10.1016/j.psychres.2018.08.090

34. Bernert RA, Joiner TE, Cukrowicz KC, Schmidt NB, Krakow B. Suicidality and sleep disturbances. Sleep. 2005;28(9):1135–1141. doi:10.1093/sleep/28.9.1135

35. Suh S, Kim H, Yang H-C, Cho ER, Lee SK, Shin C. Longitudinal course of depression scores with and without insomnia in non-depressed individuals: a 6-year follow-up longitudinal study in a Korean cohort. Sleep. 2013;36(3):369–376. doi:10.5665/sleep.2452

36. Pigeon WR, Pinquart M, Conner K. Meta-analysis of sleep disturbance and suicidal thoughts and behaviors. J Clin Psychiatry. 2012;73(9):e1160–1167. doi:10.4088/JCP.11r07586

37. Collins WA, Welsh DP, Furman W. Adolescent romantic relationships. Ann Rev Psychol. 2008;60(1):631–652. doi:10.1146/annurev.psych.60.110707.163459

38. Stoessel C, Stiller J, Bleich S, et al. Differences and similarities on neuronal activities of people being happily and unhappily in love: a functional magnetic resonance imaging study. Neuropsychobiology. 2011;64(1):52–60. doi:10.1159/000325076

39. Kuula L, Partonen T, Pesonen AK. Emotions relating to romantic love-further disruptors of adolescent sleep. Sleep Health. 2020;6(2):159–165. doi:10.1016/j.sleh.2020.01.006

40. Sani G, Janiri D, Di Nicola M, Janiri L, Ferretti S, Chieffo D. Mental health during and after the COVID-19 emergency in Italy. Psychiatry Clin Neurosci. 2020;74(6):372. doi:10.1111/pcn.13004

41. Moccia L, Janiri D, Pepe M, et al. Affective temperament, attachment style, and the psychological impact of the COVID-19 outbreak: an early report on the Italian general population. Brain Behav Immun. 2020;87:75–79. doi:10.1016/j.bbi.2020.04.048