Application of eye trackers for understanding mental disorders: Cases for schizophrenia and autism spectrum disorder

Abstract

Studies of eye movement have become an essential tool of basic neuroscience research. Measures of eye movement have been applied to higher brain functions such as cognition, social behavior, and higher‐level decision‐making. With the development of eye trackers, a growing body of research has described eye movements in relation to mental disorders, reporting that the basic oculomotor properties of patients with mental disorders differ from those of healthy controls. Using discrimination analysis, several independent research groups have used eye movements to differentiate patients with schizophrenia from a mixed population of patients and controls. Recently, in addition to traditional oculomotor measures, several new techniques have been applied to measure and analyze eye movement data. One research group investigated eye movements in relation to the risk of autism spectrum disorder several years prior to the emergence of verbal‐behavioral abnormalities. Research on eye movement in humans in social communication is therefore considered important, but has not been well explored. Since eye movement patterns vary between patients with mental disorders and healthy controls, it is necessary to collect a large amount of eye movement data from various populations and age groups. The application of eye trackers in the clinical setting could contribute to the early treatment of mental disorders.

INTRODUCTION

As has been postulated, eyes are windows to the brain as well as to the mind. Studies of eye movement have become an essential tool of basic neuroscience research. Almost every area of the brain plays some role in relation to ocular motor control. Eye movements are used to evaluate brain function in disorders affecting mostly the brain stem and cerebellum. Recently, measures of eye movements have been applied to higher brain functions such as cognition, social behavior, and higher‐level decision. Four noticeable features regarding the utility of eye movements have been observed. First, in many reports on schizophrenia and autism spectrum disorder (ASD), eye movement components are altered in the patient groups. Second, measures of eye movements may contribute to the early diagnosis of mental disorders. Third, eye movements may help group current diagnoses into new categories with or without certain types of eye control problems. Fourth, although social behavior including mutual gaze interaction is crucial for communication, eye movements in social situations have not been well explored. In this review, we briefly examine the current situation of eye tracking in mental disorders and discuss the possible application of eye trackers in the clinical setting.

METHODS

We performed a PubMed search for articles published between 1955 and 2017. We used (eye movement & psychiatric disorders), (eye movement & autism), and (eye movement & schizophrenia) as key words. About 90% of the initial search results for (eye movement & psychiatric disorders) were published after 1980; therefore, we examined studies from this period in regard to the following five aspects: (a) use of basic oculomotor movement measures; (b) introduction of a new method for measuring eye movements and/or conducting data analysis; (c) intention to perform a discrimination analysis of patients and controls from a mixed population; (d) provision of new information regarding measurements and data analysis; and (e) discussions regarding the possible advantages of using eye trackers in the clinical setting. All evaluations were conducted carefully while taking the number of patients and controls described in each report into account.

RESULTS

Figure  shows the results of the PubMed literature search. As listed in Table , eye movements serve as useful biomarkers in various types of psychiatric and cognitive disorders. There is a large variety of tests using eye trackers. Part of the reason for this variation is the different types of images shown on the monitor. The other part of the reason is the data analysis. In some studies, the analyzed data were simply the density of the gaze point in a picture, while in others, a temporal analysis of eye movement was conducted. A few studies included additional interpretations of eye movement data, such as a discrimination analysis of patients and an interpretation of eye movement in combination with the participant's genetic background.

Examples of eye movement research involving individuals with various mental disorders and healthy controls

Study	Subject	Type of eye tracker	Contents
Original articles
Kojima et al	SCZ	Display type	When viewing geometric figures, the patient group had fewer eye fixations
Takarae et al	ASD	Display type	Comparison of pursuit eye movements between patients with high‐functioning autism and healthy controls
Nakano et al	ASD	Display type	Examined interpersonal synchrony using eye blinks
Matsumoto et al	Parkinson's disease	Display type	When viewing still images, the scanned area was smaller in patients with Parkinson's disease
Benson et al	SCZ	Display type	Discrimination analysis of schizophrenia cases and controls with high accuracy; application of a neural network model
Jones et al	ASD (infants)	Display type	Longitudinal study of infants. Infants later diagnosed with ASD exhibited a mean decline in eye fixation
Wang et al	ASD	Display type	Used a saliency model for quantification of visual attention in patients with ASD and matched controls
Fujioka et al	ASD	Display type	Percentage of eye fixation time was depicted in movies such as eyes and mouth in human face movies, upright and inverted biological motion
Constantino et al	ASD (infants)	Display type	Examined twin‐twin concordance of monozygotes and dizygotes. Variation in viewing of social scenes was strongly influenced by genetic factors. Social visual engagement was impaired in children with ASD
Morita et al	SCZ	Display type	Linear classifier differentiated cases from controls. Medication affected eye movement
Sumner et al	DCD (not comorbid with another diagnosis)	Display type	Low‐level oculomotor processes appear intact in children with DCD. Top‐down cognitive control of eye movement was difficult in children with DCD
Reviews
Sweeney et al	ASD, ADHD, Tourette's syndrome	Display type	Emphasis on visually guided saccade, antisaccade, and memory‐guided saccade tasks
Karatekin	Various disorders, children	Multiple	Thorough review of the history of eye tracking studies. Some limitations of eye tracking are pointed out
Rommelse et al	ADHD, ASD, SLD, SCZ, OCD	Display type	Thorough review of eye movements in mental disorders during childhood and adolescence
Schilbach	Various disorders and healthy subjects	Multiple	Review of reciprocal and social gaze, including various setups for measurements
Thakkar et al	Various disorders	Display type	Focused on the failure to predict sensory consequences of one's own actions and psychosis

ADHD, attention‐deficit/hyperactivity disorder; ASD, autism spectrum disorder; DCD, developmental coordination disorder; OCD, obsessive‐compulsive disorder; SCZ, schizophrenia; SLD, specific learning disorder.

Eye movements in schizophrenia

The first study of eye movements in schizophrenia was published in the early 1900s. Since then, numerous studies have investigated eye movements in schizophrenia (Figure B). Strong evidence of abnormal scanning patterns have been reported among patients with schizophrenia. The study of exploratory eye movements has been the focus of large‐scale research on scanning patterns in schizophrenia. Another form of eye movements called pursuit (tracking movements of the eyes toward the moving target) was found to be abnormal in some schizophrenic patients. Saccade, a rapid type of eye movement, has also been reported to differ in such patients compared with healthy controls.

Using a combination of these differences, Benson et al conducted a discrimination analysis of schizophrenia. They used a neural network model for classification that differentiated patients and controls from a mixed population. Another independent research group conducted a discrimination analysis of schizophrenia with a linear classifier. In both these reports, the accuracy of the discrimination analysis was 80% or higher; however, this may have been affected by the optimistic nature of their classifier.

One important factor concerning schizophrenia is the effect of antipsychotic agents. The effect of dopamine blockade associated with antipsychotic agents is known to affect muscle movements, causing extrapyramidal symptoms such as parkinsonism, dystonia, and dyskinesia. Although eye movement has been reported to be significantly related to chlorpromazine‐equivalent doses of antipsychotic agents, the antipsychotic effects of muscular abnormalities may vary depending on the individual. Antipsychotic therapy is commonly used in the treatment of schizophrenia, and the daily antipsychotic dose has been reported to have significant positive and negative associations with left globus pallidus volume and right hippocampus volume, respectively. Therefore, the effect of antipsychotic agents on eye movement must be carefully evaluated. Further research is needed to clarify the exact role of antipsychotic agents on eye movement.

Eye movements in ASD

Eye movement research in ASD has increased sharply in the twenty‐first century (Figure C). Among mental disorders, ASD is distinctive because its diagnosis is based on early childhood behavior. It is hard to obtain verbal‐behavioral information from a child during the limited time of a hospital visit. In 2015, a research group described a novel early detection method that used eye movements to assess the risk of developing ASD several years prior to verbal‐behavioral abnormalities. Detection methods using an eye tracker can assist clinicians in early identification of ASD. It is worth noting that 2‐ to 6‐month‐old infants who were later diagnosed with autism initially moved their eyes in a way that was indistinguishable from healthy controls; however, their eye movements declined over this period. This decline occurred in a narrow developmental time window. The research group recently published a paper on the social gaze of infants, reporting that eye movement traits have a genetic component. This finding is consistent with the strong tendency of heritability in autism. The gaze deficit in social contexts may begin as a neurological sign of their genetic background.

The results regarding ASD in early childhood are adequately consistent and sufficient for a possible diagnosis prior to the emergence of the behavioral characteristics applied as diagnostic criteria. It may therefore be possible to use eye trackers in screening for ASD in children who are amenable to early treatment. Measurement using eye trackers is a safe and noninvasive method for examining childhood development that may help examiners identify early signs of ASD in tandem with information from parents and caregivers.

As stated above, ASD is diagnosed in early childhood, but the majority of research experiments involving eye movement has been conducted in adults or adolescents. Some reports have described oculomotor behavior in school‐age children with ASD and compared it with age‐matched controls. The results in those reports are not as prominent as those reported in studies on schizophrenia. In many of these published reports, group differences were found between participants with ASD and healthy controls; however, these are not without major discrepancies. This may be partly because there were often less than 20 participants with ASD in each study, so some of the results may not accurately reflect the total ASD population.

In 2016, a study group conducted a discrimination analysis using an eye tracker on adolescents and adults with ASD. They measured the eye movements of 26 ASD patients and 35 age‐matched controls and differentiated between both groups efficiently. Wang et al found atypical visual attention in people with ASD using a saliency map, which is a form of image processing that uses human vision as a model, with a dataset of 700 pictures of natural scenes. These two studies utilized high‐context images (such as faces and multiple people in a single image) and low‐context images (such as geometrical shapes and letters). The key feature of eye tracking in ASD is its ability to compare eye movements between two different image contexts and incorporate them into the analysis.

In regard to the specificity and overlap of eye movement abnormalities found across mental disorders, eye movements may help subdivide current diagnoses into subpopulations with or without a certain type of eye control problem. ASD is occasionally associated with intellectual disabilities, learning disorders, and attention‐deficit/hyperactivity disorder (ADHD), so study populations should contain multiple subgroups with or without comorbid disorders. For example, hyperactivity is relatively common during childhood in high‐functioning ASD, and children with high‐functioning ASD are often hyperactive. This comorbidity may cause the heterogeneity of eye movements in ASD. In people with ADHD, altered eye movements have been reported by a number of research groups. In addition, saccade timing has been shown to be altered in people with ADHD.

The discrepancies between the published papers on ASD may reflect the heterogeneity of the patients in each report. Therefore, subtype‐specific descriptions of eye behavior in people with ASD may be important for better understanding of its association with eye movements.

DISCUSSION

Recent developments in eye trackers

The history of eye tracker hardware has been described in several studies. In 1991, a US company, SR Research, released a prototypical eye tracker. In 2001, another leading company, Tobii Technology, was founded in Sweden. In the twenty‐first century, the number of eye tracker manufacturers has been increasing with advances in the field. This increase in the number of manufacturers has coincided with the increase in the number of published papers (Figure ). The most widely used method of estimating gaze using an eye tracker is based on the pupil‐corneal reflection technique, in which a camera captures time‐lapse images of the eye under appropriate illumination (Figure A). The positional data of parts of the eye (Figure B) are acquired and converted to indicate the geometrical location of the gaze on a monitor.

In the literature, the majority of the research has been conducted with the images shown on a display (Figure A). Typically, participants have been asked to sit on a chair and look at a visual stimulus on a monitor that includes graphical images, daily objects, faces, and scenes. Sometimes, participants are asked to follow a moving target. A setup such as that shown in Figure A provides easier control of spatial conditions with limited head movement. Usually, basic oculomotor behavior and eye movements have been measured by having a person look at a monitor screen. In the setup shown in Figure B, the gaze of two persons can be simultaneously monitored using eyeglasses equipped with an eye tracker. Although the interaction in terms of gaze between the two subjects can be monitored in this setup, the data analysis is more complicated than that for the setup shown in Figure A because it involves more variables and additional steps of analysis. Head movement is extensive and should be considered in the setup shown in Figure B.

Traditionally, basic oculomotor measures consist of those from three eye movement types: fixations, saccades, and smooth pursuit. Fixation centers the eyes on a stationary target; saccade is rapid eye movement between fixations, and smooth pursuit is seen when a person is tracking a moving target. In healthy persons, personal differences in oculomotor behavior are observed when looking at a monitor. Differences in eye movements in healthy individuals are referred as their “oculomotor signature.” Variations in gaze patterns during face‐to‐face communication have also recently been described.

Role of eye movements in social interaction

Despite the development of eye trackers, it remains unclear why eye‐to‐eye contact is necessary in communication. A situation in which two individuals are looking at each other is referred to as “mutual gaze”. Mutual gaze has a large impact on cognition and emotion, and the information obtained from another person's eyes is essential for mutual interaction. Extensive analysis conducted to date has yet to elucidate the mechanism that underlies this phenomenon. This may be partly because individual differences in gaze patterns are substantial. Everyone has a unique eye movement “fingerprint” when having a conversation. If someone has a defect in their eye movement, it is likely to affect how well they understand the behaviors of others, which can cause difficulties in his/her social life. Eye movement is among the endophenotypes necessary for a better understanding of the pathogenic mechanisms underlying mental disorders.

Genetic factors also play a substantial role in the etiology of mental disorders. It has been suggested that ASD and schizophrenia share common biological pathways based on the recent genomic studies. Several groups have examined heritability estimates of eye movement phenotypes in patients with schizophrenia. In those studies, two eye movement tasks were associated with moderate to high heritability. Ladea and Prelipceanu concluded that such eye movement tasks are insufficient as diagnostic tests for schizophrenia. This is partly because the abnormalities observed in the eye movement tasks were frequently found in the patients’ unaffected relatives, as well as in a minor population of healthy controls. However, eye movement tasks may be effective to test for biological markers of the associated risk of developing schizophrenia. Participating in a test involving such tasks may increase the chance of receiving an early intervention.

The application of eye trackers in the clinical setting to screen for mental disorders seems promising. At the same time, eye movement patterns vary considerably among both patients and healthy controls. Therefore, future studies should include larger numbers of eye movement examples from various populations and age groups. As reported in the case of ASD, the longitudinal design of eye movement research could help achieve a more comprehensive understanding of the relationship between eye movements and mental disorders.

ACKNOWLEDGMENTS

We wish to thank Drs T. Koike and N. Sadato for their helpful discussions. This research was supported by AMED under grant No. JP18dm0207005, JSPS KAKENHI grant No. 26461778, and a Research Fellowship by the Astellas Foundation for Research on Metabolic Disorders.

DISCLOSURE

Dr. N. Ozaki has received research support or speakers’ honoraria from or has served as a consultant to Astellas, Eisai, Eli Lilly, Janssen, Meiji Seika Pharma, Mochida, MSD, Nihon Medi‐Physics, Novartis, Ono, Kyowa Hakko Kirin, Novartis, Otsuka, Pfizer, Takeda, Taisho, Mitsubishi Tanabe, Tsumura, and KAITEKI. The remaining author have no conflict of interest to disclose.

AUTHOR CONTRIBUTION

E.S. wrote the initial draft of the manuscript. All other authors contributed to the data interpretation and critically reviewed the manuscript.

DATA REPOSITORY

Supplementary material for figure 1 can be found in timeline.xlsx.