When Watching Video, Many Saccades Are Curved and Deviate From a Velocity Profile Model

Abstract

Commonly, saccades are thought to be a ballistic eye movement, not modified during flight, with a straight path and a well-described velocity profile, even though saccade curvature has been reported previously. In a prior study, we developed a real-time, saccade-trajectory prediction algorithm to improve the updating of gaze-contingent displays. We found that saccades with a curved path or that deviated from the expected velocity profile were not well fit by our saccade-prediction algorithm, and thus had larger updating errors than saccades that had a straight path and a conventional velocity profile. Further, we felt that the curved and non-conventional velocity-profile saccades were more common than we had expected when participants performed a natural-viewing task. Since those saccades caused larger display updating errors, we sought a better understanding of them. Here we examine factors that could affect curvature and velocity profile of saccades using a pool of 218,744 saccades from 74 participants watching ‘Hollywood’ video clips. Those factors included characteristics of the participants (e.g. age), of the videos (importance of faces for following the story, genre), of the saccade (e.g. magnitude, direction), time during the session (e.g. fatigue) and presence and timing of scene cuts. While viewing the video clips, saccades were most likely horizontal or vertical over obliques. Measured curvature and velocity-profile error had continuous, skewed frequency distributions. We used mixed-effects multiple regression models that included restricted cubic spline fits and found a complex relationship between curvature, velocity-profile error and saccade duration (or magnitude). Curvature and velocity-profile error were related to some video-dependent features such as lighting, face presence, or nature and human figure content, but there were no effects of subject factors such as age or gender. Time during the session was a predictor for saccadic curvatures. Saccades characteristics vary with a variety of factors, which suggests complex interactions between oculomotor control and scene content that could be explored further.

Introduction

It seems a common misperception that saccades are ballistic eye movements, displacing in straight lines, not modified during flight, and with a well-described velocity profile. In fact, plenty of recent research has investigated the deviations of saccades from a straight line, saccade curvature. In a previous study (Han et al., 2013), as a real-time, saccade-trajectory prediction algorithm was developed, it was noted that some saccades caused the algorithm to have greater errors. Those saccades did not fit the common perception that saccades are ballistic eye movements, not modified during flight, with a straight path and a well-described velocity profile. The saccade prediction algorithm was developed for use in a gaze-contingent visual display system. All systems have an updating latency of at least 10 ms (Santini et al., 2007) to 12 ms (Saunders et al., 2014) and most systems have a considerably longer updating latency (Saunders et al., 2014). That latency can become a problem when the viewer is making saccades, as the eye is no longer at the measured location by the time the display is updated, and thus there is an updating error. When implemented in a gaze-contingent system, the Han et al. (2013) saccade prediction did improve the gaze-contingent updating, but saccades with a curved path and or that had a velocity profile that was not typical (and thus not well fit with that model or other saccade models) had larger updating errors.

Gaze-contingent systems have been used in a variety of settings. Perhaps the settings in which gaze-contingent systems are most likely to be widely used are in real-world settings. This includes control or interaction based on eye movements or modifying the view during the conduct of activities of daily living. Most studies of saccades have been conducted in experimental settings with very constrained conditions such as moving fixation from one defined location to another defined location, often repeatedly, and usually without other stimuli present and a blank background. However, the real world is much more complex, so it is not clear that all that we have learnt from such studies applies when viewing natural stimuli, such as while walking (scene change and object motion created by self-motion) or viewing video, when there are complex scenes with objects moving within the scene. For example, Blackmon et al. (1999) tested very simple, synthetic dynamic scenes and reported some evidence for certain aspects of the “scanpath theory” (Noton and Stark, 1971). Modeling saliency, i.e., the contribution of low-level features to gaze control, was also important in a number of studies (e.g., Itti, 2005; Le Meur et al., 2007) that found that temporal change and motion predict eye movements strongly. As reported by Dorr et al. (2010), there are significant differences in gaze patterns between natural static scenes and dynamic scenes. Eye movements on static images are initially driven by stimulus onset effects and later by subject-specific idiosyncrasies. Conversation scenes are another example in which classical models of visual attention dramatically fail to predict eye positions. Coutrot and Guyader (2014) showed that faces, and particularly talking faces, are the features that best explain the gazes recorded during viewing of conversation scenes. Other authors have also studied gaze behavior in real-world tasks, such as driving (Land and Lee, 1994), food preparation (Land and Hayhoe, 2001), and walking around indoors (Munn et al., 2008) and outdoors (Cristino and Baddeley, 2009). While eye movements, gaze behavior and scanpath are of interest, very few studies have looked at the characteristics of the saccades made during such behavior. Here, we are interested in saccades made during a naturalistic task, watching Hollywood movies, which have not been studied before.

The path of a saccade is not always straight (Lamasky, 1869). Yarbus (1967) showed curved saccades made while scanning a two dimensional shape. Sheliga et al. (1994) were the first to study these curved trajectories, showing that directing covert attention to a spatial location influences eye movement trajectories (Sheliga et al., 1994, 1995a,b). These findings and subsequent studies on the activation of the oculomotor system in relation to spatial attention allocation (Van der Stigchel and Theeuwes, 2007) concluded that the strength of saccade deviation is a measure of the amount of attention allocated to any particular location in space. Interestingly, saccade deviations are also observed for reflexive shifts of visual attention without the presentation of a peripheral cue (Nummenmaa et al., 2009).

The time course of exogenous and endogenous activation in the saccade trajectory planning can explain a variety of eye movement data, including endpoints, latencies, and trajectories of saccades (Godijn and Theeuwes, 2002). Arai and Keller (2005) presented a model of saccade generation that accounted for curved saccades found during visual search tasks through a proposed delayed parallel input from the cerebellum to the saccadic burst generators in addition to that provided by the distributed input by the superior colliculus. Schreiber et al. (2006) reported that the curved trajectory of a significant proportion of catch-up saccades in the double step-ramp paradigm could be explained by an asynchrony between position and motion signals in saccade programming.

As indicated in the extensive review by Van der Stigchel (2010), many investigations of the curvature of saccade trajectories have studied the effects of distractors, including visual, auditory, and somatosensory cues (Doyle and Walker, 2001, 2002). Curvature can be toward or away from the distractor, may increase when the target is more predictable (Walker et al., 2006), and the amount of curvature may not systematically vary with the number of distractors (Arai et al., 2004). The trajectory of an erroneous initial saccade to a distractor is curved toward the goal of the subsequent saccade, suggesting that both movements are processed concurrently (McPeek and Keller, 2001). Curvature back toward the saccade goal may be attributed to a feedback system, with a separate representation of the visual target location, that enables an on-line correction of the saccade during mid-flight (McSorley et al., 2004). All of these studies have used very constrained, non-natural settings.

Deviations away from an element are observed in situations in which top-down preparation can influence the target selection process (Van der Stigchel et al., 2006). The salience of a second visual goal can be simultaneously maintained in the superior colliculus (McPeek and Keller, 2001) as it is presumably involved in the competitive neural interactions underlying saccade target selection (McPeek et al., 2003). Saccade curvature can provide a measure of processes that control and influence our behavior, during reading (Inhoff et al., 2010) or attention allocation to locations over time (Van der Stigchel and Theeuwes, 2007; Van der Stigchel, 2010; van Zoest et al., 2012). The only one of those studies that used a natural setting and task was the measurement of curved saccades during reading by Inhoff et al. (2010), where they found that curvatures increased with increases in the size of forward-directed saccades, regressions, and return sweeps. We are not aware of any studies that have investigated saccade curvature while viewing a dynamic scene.

Saccade velocity profiles are usually symmetrical at least for small and medium size saccades. An irregular velocity profile occurs when there is not a smooth acceleration or (more commonly) deceleration and may be determined by comparison to a typical saccade or the fit for a typical saccade (e.g., from a model). Overshoot, undershoot and glissades, which occur at the end of saccades, will not fit most models of saccades, and, consequently, display a velocity-profile mismatch (Bahill et al., 1975a). There are few reports of irregular velocity profiles, although they have been shown in eye gaze traces from monkeys (Figure 1 in Straube et al., 1997) and in a patient with Stiff-Person Syndrome with cerebellar degeneration (Zivotofsky et al., 2006).

Since we are using a gaze-contingent system during studies in which participants perform a natural-viewing task (watching videos), we wanted to explore the prevalence of these difficult-to-predict saccades and the circumstances in which they occur, or are more likely to occur. This would allow us to design studies that would reduce the prevalence of such saccades and, possibly, provide information that would increase our knowledge about the control and generation of saccades. To our knowledge, no prior studies have examined the patterns of saccadic curvatures or velocity-profile deviations while performing a natural-viewing task such as watching TV or movies. We suspected that the dynamic nature of video may elicit a high frequency of the difficult-to-predict saccades due to changing of visual elements on-screen during a saccade. Such changes, whether from action within the scene or from a scene cut, could act as attractor or repellor, by triggering adjustments from the oculomotor control system. This happens with hand and arm movements (i.e., if target is moved, the path of the hand will alter during its travel to the target; Bédard and Sanes, 2009). We hypothesized that the visual transitions introduced by scene cuts will serve as distractors, thus generating saccades with curvature and, perhaps, irregular velocity profiles (due to mid-flight alterations in the saccade path).

Further, we wanted to explore whether curvature and irregular-velocity profiles may vary between individuals (subject dependent), might vary according to characteristics of video content (video dependent), and might vary with characteristics of the saccade (saccade related, such as saccade direction) and might vary with time during the experimental session (e.g., fatigue). Thus, we conducted an observational study with hypotheses that examined certain features of the participant, eye movement and viewed material, to provide a first look at the prevalence and characteristics of saccade curvature and irregular-velocity profiles while watching videos.

Materials and Methods

Participants

Seventy-one normally-sighted participants (median age: 49.3 years) participated in two related experiments of a project that was approved by the Institutional Review Board of the Schepens Eye Research Institute in accordance with the Code of Ethics of the World Medical Association (Declaration of Helsinki). Informed consent was obtained both written and verbally from each participant prior to data collection.

Preliminary screening of the participants included self-report of ocular health, measures of visual acuity, and contrast sensitivity for a 2.5 degree-high letter target and evaluation of fixation and central retinal health using retinal photography (Nidek MP-1, Nidek Technologies, Vigonza, Italy or Optos OCT/SLO, Marlborough, MA, United States). All the participants had visual acuity of 20/25 or better, letter contrast sensitivity of 1.675 log units or better and steady central fixation with no evidence of retinal defects. For each participant we included gender, age, and education level (categorized within seven levels based on the International Standard Classification of Education, ISCED 2011) as demographic variables within our model. Education is a surrogate for socio-economic status which can be related to health status. The demographics of the participants are summarized in Table 1. Education information was not available for four of the 74 participants (22,984 saccades). Race-ethnicity information was not available for two participants.

TABLE 1

Table 1. Self-reported demographic characteristics of participants.

Studies

We tracked the gaze of each subject as they viewed a 27” display (60 × 34 cm) from 1 m for a 33 × 19° potential viewing area with a table-mounted, video-based EyeLink 1,000 system (SR Research Ltd., Mississauga, Ontario, Canada). Subjects' head movements were restrained for the duration of the experiment using an SR Research head and chin rest. The clips were displayed and data collected with a MATLAB program using the Psychophysics (Brainard, 1997), Video (Pelli, 1997), and EyeLink Toolboxes. At the beginning of the experiment, the eye tracker was calibrated using a nine-point calibration procedure. If the average deviation exceeded 0.5° during this step, the calibration was performed again. In the first study, participants watched 40 to 46 of 206 30-s “Hollywood” video clips. At the beginning of each trial, participants were instructed to watch the stimulus “normally, as you would watch television or a movie program at home.” At the end of each clip, the participant was asked to describe the contents of the clip (Saunders et al., 2013). These participants (N=61) contributed a total of 108,043 saccades to the dataset. Certain outcomes from this study were previously published (Costela et al., 2017). In the second study, fourteen participants (one was also in the first study) watched at least two of five different 30-min movie clips (Bambi, Inside Job, Juno, Kpax, and Flash of Genius), with calibration repeated every 5 min, contributing 110,695 saccades. A total of 218,744 saccades were identified from the two studies.

Video Clips

For the first study, two hundred and six Hollywood video clips were chosen to represent a range of genres and types of depicted activities. The genres included nature documentaries (e.g., BBC's Deep Blue, The March of the Penguins), cartoons (e.g., Shrek, Mulan), dramas (e.g., Shakespeare in Love, Pay it Forward), comedies (e.g., Julie and Julia, Adventureland) and action (e.g., Batman Forever, Eastern Promises). The clips were 30 s long and were selected from parts of the films that had relatively few scene cuts, which was reflected in the average number of cuts per minute in our clips being 9, as compared to approximately 12 per minute in contemporary films (Cutting et al., 2010). The clips included conversation, indoor and outdoor scenes, action sequences, and wordless scenes where the relevant content was primarily the facial expressions and body language of one or more actors. For the second study, the five 30-min clips were from movies with content similar to the 30-s clips.

We hypothesized that video content might affect saccades. For example, videos that contain more man-made objects might be more visually cluttered than natural environments, and thus have more distractors that might increase saccade curvature. In a study of image enhancement that measured preference, participants preferred less enhancement (mainly of local contrast) in video clips when human faces were important, but not when human figures, man-made objects or nature were important (Satgunam et al., 2013). “Importance” because objects that are important for comprehension are likely to be attended and to be saccade targets. To investigate this hypothesis, the video content of all 206 30-s clips and for each 30-s segment of the five 30-min clips was rated for eight categories: (1) human faces; (2) human figures; (3) man-made objects; (4) nature; (5) auditory information; (6) lighting; (7) environment type; and (8) movie genre. Three observers who did not participate in the studies in which gaze was tracked were asked to count the number of scene cuts and rate each clip based on the importance of the categories human faces, human figures, man-made objects, nature and auditory information for understanding of the video content. Each rating scale ranged from 0 to 5, with 0 being absent and 5 being always present (or important), except for environment type that was binary (outdoor/indoor). To include video content in our model, video clips were considered to have high content (e.g., presence of faces was important) if the average rating was 4 or greater, and to have low content if the average rating was 1 or less, otherwise it was medium (rated 2 or 3). Time-on-task, or the time of occurrence of the saccade during the viewing session (order in intervals of 30 s) ranged from 1 to 60. Finally, to evaluate the effect of movie genre, clips were classified as dramas (N = 60), comedies (N = 49), action movies (N = 22), documentaries (N = 42), or animation (N = 42).

Scene Cut (Shot Change) Detection

With a scene cut (to a new scene or a different viewpoint of the same scene) there is a sudden, unexpected change in the visual stimulus. For any saccade that was planned or in progress at the time of the scene cut, the scene changes during the saccade (sudden onset distractors) and the planned landing point no longer contains the object of interest (saccade target). Scene cuts in videos could act in the same way that distractors have been shown to influence in-flight saccade dynamics in research studies. To investigate this hypothesis, we detected scene cuts and compared those saccades that occurred over the scene cut to other saccades.

We implemented a version of the shot transition detection (SAD) method. Frames within each clip were converted to grayscale and we calculated the log-between-frame-pointwise difference in pixels, after normalizing by the size of the clip. We used a threshold based on the average of this metric plus one standard deviation to discriminate transitions that corresponded to scene cuts. This threshold returned a better sensitivity (0.7) than using two (0.55) or more standard deviations when compared with the number of cuts reported by the observers who counted scene cuts in 200 30-s clips.

For 58 clips, where the number of scene cuts calculated by our algorithm was lower than the reported number of cuts by observers, we added the timesample of the missing cut scenes by manually tracking the clips with QuickTime Player 7. The algorithm found some additional events in 43 clips. Some of these related to fast transitions (i.e., revolving doors or fast-motion panning) that we kept, as these would be expected to act as dynamic distractors. We removed those few cases where transitions were due to slow fading between scenes. This algorithm to detect scene cuts was applied to all 30-s and 30-min clips. In total, we identified 941 saccades (0.4%) that were in transition when a scene cut occurred.

Next, we examined the profiles of in-flight saccades, where the scene cut onset occurred within the saccade timesample, as compared with saccades happening away from the scene cut onset (i.e., saccades finished before the cut scene onset or triggered later in time).

Saccade Detection

Many studies of saccades used non-natural viewing conditions in which there are few stimuli, the saccade target is stationary and easily identified and the interval between saccades is controlled and “long” (in the order of seconds), or natural tasks such as reading in which the stimuli are stationary and the background is simple (e.g., plain). In videos, saccade targets may not be obvious or unique, they may be moving (relative to the viewer), the inter-saccadic interval is often short (<200 ms; Otero-Millan et al., 2008), there may be many other objects within the scene creating visual clutter, and those may also be moving. The visual scene for many other natural tasks such as visual search, walking and driving are similar in their visual complexity. While watching videos, saccades may be preceded by a fixation or a pursuit and may end with a fixation or a pursuit, and that might vary depending on the conditions (e.g., orientation of saccade). Thus, detecting saccade initiation and completion is more difficult when using data from watching videos (and other natural viewing tasks). The most common method of saccade detection employs velocity or acceleration thresholds. Besides our own saccade detection algorithm (see details below), we employed an alternative algorithm, the Eye-Markup analysis tool (Berg et al., 2009), which provided comparable results (see Supplementary Material), although was found to be less efficient than ours by examining the main sequence of the detected saccades. We were unable to get access to the LNS algorithm (described as one of the most robust in a recent review on saccade detection algorithms Andersson et al., 2017). For a visual search task (static images), Dandekar et al. (2012) employed peak velocities and the acceleration window to identify saccade start and end. Konig and Buffalo (2014) used a novel method based on k-means cluster analysis to analyze visual search data (static images). We employed velocity criteria and additional criteria to remove eye-position overshoots and handle saccades that did not initiate or end with a fixation (including glissades).

Sampling frequency was 1 kHz for all examples. Blinks were identified and removed using Eyelink's online data parser. Periods preceding and following these missing data were removed if they exceeded a speed threshold of 30°/s. Then, we interpolated over the removed blink data by applying cubic splines. For saccade detection in both data sets, the raw data was smoothed by applying a 3rd-order Savinsky-Sgolay filter with a frame size of 15. Without this smoothing, saccade detection was much less reliable. Speed was calculated as the norm of the velocity vector (speed), which is the first derivative of the eye position with respect to time. The beginning of a saccade was signaled when speed exceeded 30°/s for at least 10 ms. The end of a saccade was signaled when speed went below 30°/s. For analysis, the saccades were restricted to saccades (1) smaller than 40° as this was approximately the maximum diagonal dimension of the display; and (2) larger than 1° and 15 ms in duration to exclude microsaccades. We imposed additional restrictions regarding the initial (< 0.075°/ms) and terminal velocity (<0.3°/ms), as well as the removal of saccades with a velocity at first quartile of duration lower than 0.15 peak velocity. This threshold removed those eye movements with uniform but unrealistic low velocity profiles during their initial phase, and which may have been pursuit eye movements. This pool of saccades followed the saccadic peak velocity-magnitude relationship (Bahill et al., 1975b; Figure 1A) with a skewed distribution of magnitudes (median 4.13°) and durations (median 31 ms; Figure 4C). The main-sequence in our data when plotted on bi-logarithmic axes (Figure 1A) was not as linear as expected. Hamel et al. (2013) also showed curved saccades that did not fit the main sequence. The main sequence has been reported to vary between individuals (Bollen et al., 1993; Bargary et al., 2017) and between sessions (Bollen et al., 1993) and with factors such as saccade orientation (van Beers, 2007), saccade start location (van Beers, 2007), age (Dowiasch et al., 2015) and mental workload (Di Stasi et al., 2011). For each saccade, based on start location and landing point, we categorized saccade orientation as: Horizontal if path was 0 ± 5° or 180 ± 5° (Duke Elder and Wybar, 1973); Vertical if path was 90 ± 22° or 270 ± 22°; otherwise, the saccade orientation was considered Oblique. Saccades were predominantly horizontal and vertical, with fewer saccades being oblique (Figure 1B). The frequency distribution of saccade magnitudes is shown in Figure 4C.

FIGURE 1

Figure 1. Saccadic features for all 74 subjects and 218,744 saccades. (A) Saccadic peak velocity–magnitude main sequence. The distribution is plotted on logarithmic scale where peak velocity is indicated on the y-axis and magnitude indicated on the x-axis. (B) Polar frequency distribution of saccade directions in polar co-ordinate orientations.

Curvature

There have been different methods in the literature to define saccade curvatures (Van der Stigchel, 2010). Ludwig and Gilchrist (2002) reviewed prior methods and proposed using the area under the curve of a quadratic polynomial fit. Their model measures curvature as the deviation of a point from a line. But in their approach, the line is drawn between the start and end points of the saccade and curvature is quantified by whichever third point is farthest away from the line. It treats the saccade as if it contains only three measured points. This will yield an accurate representation of curvature only when the saccade has a perfect quadratic form, i.e., when the second derivative (the actual point-wise curvature) is a constant. If the saccade is not a perfect arc (specifically, one that can be fit with a quadratic) their method will not represent its actual curvature but the maximum deviation under the assumption that curvature is constant. The second derivative method is a standard mathematical measure of curvature (Coolidge, 1952; Casey, 2012) and is essentially equivalent to the method of Ludwig and Gilchrist (2002) when curvature is constant. However, when saccade curvature varies along the saccade, the second-derivative measure will vary from that found using a quadratic polynomial fit. When comparing curvature measures, Van der Stigchel et al. (2006) stated that “conclusions usually do not hinge on the exact measure chosen.” Here we used the median pointwise curvature as our saccade curvature metric, which it is based on the second derivative method, and was reported to be a reliable measure by Flynn and Jain (1989).

We defined curvature, k, as the median pointwise curvature, kt:

kt=(xt+1−2xt+xt−1)2+(yt+1−2yt+yt−1)2∑t(xt−xt−1)2+(yt−yt−1)2 t={2,3,...,T−2,T−1} (1)

kt =(Δxt+1−Δxt)2+(Δyt+1−Δyt)2∑t=2T−1Δx2+Δy2 (2)

The calculation of kt is illustrated in Figure 2. If the line is straight, then kt = 0. Saccades with low and high curvature are illustrated in Figures 3A,B, respectively. The frequency distribution of saccade curvature is shown in Figure 4A.

FIGURE 2

Figure 2. Schematic illustration of the calculation of saccade curvature. As detailed in Equation 1, saccade curvature compares the change in the vertical and horizontal dimensions between the change at sample t with the change at sample t + 1.

FIGURE 3

Figure 3. Examples of normal (left) and difficult-to-predict (right) saccades (as defined in the Methods section). (A) Saccades with “low” curvature; k < 0.21. (B) Saccades with “high” curvature; k > 0.21. (C) Saccades with “low” velocity-profile deviation; velocity-profile deviation < 4 × 10−3. (D) Saccades with “high” velocity-profile deviation; velocity-profile deviation > 4 × 10−3. The solid red lines in (C,D) show the fit using a compressed exponential model (Han et al., 2013).

FIGURE 4

Figure 4. Frequency histograms of: (A) Saccade curvature that ranged from 0.0023 to 0.563 units (median: 0.075 units). (B) Saccade velocity-profile deviation that ranged between 4.87 × 10–5 and 0.078°/ms (median: 0.0023°/ms). (C) Saccade duration that ranged between 16 and 99° (median: 31°). All are shown with a log10 scale and the bins have been equally spaced logarithmically as the raw distributions were skewed.

Velocity-Profile Deviation

Earlier studies have noted the skewness in the shape of the velocity profile (Baloh et al., 1975; van Opstal and van Gisbergen, 1987). Baloh et al. (1975) fitted an exponential curve to magnitude-velocity plots to find that skewness increased with magnitude. van Opstal and van Gisbergen (1987) proposed five different measures of the degree of asymmetry in the saccadic velocity profile with the density function of the gamma distribution returning the best correlation between magnitude and skewness (r = 0.71). They also found a stronger correlation between saccade duration and skewness (r = 0.88), where saccades with large differences in magnitudes, but fixed duration, appear to keep the same skewness. Similarly, Takagi et al. (1993) found a close relationship between duration and skewness ratio (the proportion of acceleration time to total saccadic duration) where the ratios of temporal saccades were smaller than nasal ones. Chen et al. (2002) also applied a gamma function to the velocity profile to determine that at the same magnitude, skewness can be predicted from the peak velocity. We used a compressed exponential model, where one parameter in the generalized exponential model (p3 >1 in Equations 3, 4), accounts for the slower deceleration than acceleration phase (Han et al., 2013). We defined velocity-profile deviation as the mean-square error between the measured velocity and the best fit to the compressed exponential model.

For this study, we fit the saccadic eye movement trajectory with a compressed exponential model (Han et al., 2013):

f(t)=p1[1−exp[−(tp2)p3]] (3)

where p1, p2, and p3 are three parameters that need to be determined in data fitting, and t is the time since saccade onset. The parameter p1 corresponds to the amplitude of saccade. The parameter p2 can be used to rescale the saccade duration, and parameter p3 is used to control the decay trend This function is called the generalized exponential function because of the additional parameter p3. With p3 = 1, the generalized exponential function becomes the standard exponential function.

For our study, as deviations were most obvious in the velocity profile, though we fit in the position domain using Equation 3, we used errors in velocity instead of position. Given that the fit is the same, the errors would be linearly related to the fit in the position domain, so we would not expect any substantive difference in our subsequent analyses if we had used positional error. Saccades with low and high velocity-profile deviations are illustrated in Figures 3C,D, respectively

The velocity profile of the saccadic trajectory is the differential of Equation 3 (Han et al., 2013):

f′(t)=p1p2−1p3(tp2)p3−1exp[−(tp2)p3] (4)

Equation 4 is very similar to the probability density function of the Gamma distribution used by van Opstal and van Gisbergen (1987) to investigate the skewness of saccadic velocity profiles. We defined velocity-profile deviation as the root-mean-square error of the fit in Equation 4. See frequency distribution of velocity-profile deviations in Figure 4.

Statistical Analyses

In a preliminary analysis, we did not find any substantive differences between the saccade data collected from the two studies (30-s clips vs. 30-min clips), so we merged the data into a single dataset and considered all of the available data without a term to differentiate between the two studies. For the analyses, the five 30-min clips were treated as 300 30-s segments.

The distributions of velocity-profile deviation, and saccade duration (or magnitude) were highly positively skewed, while curvature was mildly skewed, with all having a very small number of high values. Those distributions became approximately normal (Gaussian) when logarithmic values were used (frequency distributions are shown in Figure 4). For analyses, we used log10 values of curvature, velocity profile deviation, and saccade duration. The relationships between curvature, velocity profile deviation, and saccade duration (or magnitude) were non-linear. To examine the relationships, we investigated a variety of non-linear approaches including restricted cubic splines, and settled on the inclusion of square and cubic terms (of the log10 transformed values) for velocity profile deviation and saccade duration when predicting curvature and curvature and saccade duration when predicting velocity-profile deviation, as the residuals were not patterned. Figures 5–7 show the univariate relationships between curvature, velocity profile deviation, and saccade duration, when square and cubic terms were included (cubic polynomial fits in log10-log10).

FIGURE 5

Figure 5. Saccadic curvature as a function of saccadic duration, showing the frequency scale on the right. A fit of the saccade curvatures (log10) using a cubic polynomial (uni-variable relationship, Table 2) is shown as the solid line.

We conducted crossed-random, mixed-effects regression analyses (Stata v14, College Station, TX) to determine which parameters had effects on the curvature and velocity-profile deviations. We present (1) “uni-variable” models in which a single predictor (independent) variable (fixed effect) was included in the model (forty models in total); and (2) “multi-variable” models in which all potential predictors were initially included and then predictors that were not significant (p > 0.01) were removed sequentially (one at a time) until all predictors in the model were significant (p ≤ 0.01; backwards regression method, two models in total). In the uni-variable models for factors curvature, velocity-profile error and duration, square and cubic terms were included in each model. As we had many models, we chose the p = 0.01 criterion to reduce the risk of false-positive findings (as compared to the commonly used criterion of p = 0.05). Backwards regression in the multi-variable models can remove variables that are highly correlated with another predictor (collinear), even when that predictor is significant in the uni-variable model. The fixed-effect factors were saccade-dependent (magnitude or duration, orientation); subject-dependent (age, gender, maximum education level); video-dependent (number of cuts, lighting level, environment, nature importance, auditory information presence, face importance, man-made object importance, human figure importance, movie genre); and time-dependent (time of saccade occurrence within the experimental session, time of in-flight saccade occurrence during scene cut). There were many strong correlations between the video-dependent factors, such as lighting level and man-made-object importance (r = -0.50) or human-figure importance (r = −0.45) and between man-made-object importance and human-figure importance (r = 0.72). In the multi-variable models, all factors should be independently predictive, and are corrected for the other predictors. There was no obvious multi-collinearity in the reported models.

van Opstal and van Gisbergen (1987) found a stronger correlation of saccade duration with skewness than with saccade magnitude. In our sample, the correlation between saccade duration and saccade magnitude was r = 0.74. We report models of both curvature and velocity-profile deviations using the duration of the saccade instead of the magnitude. Models using saccade magnitude were similar to those found using duration (slightly higher Akaike information criterion). The Kolmogorov-Smirnov two-sample test was used to examine differences between distributions of saccades. This tests for both location and shape of the two distributions.

Participant and video clip were crossed-at-random factors in all models. For all models, random slopes for duration, and saccade curvature or velocity-profile error were included if the variable was a fixed effect rather than the dependent variable, for both participant and video clip. These random slopes were included as these were the most strongly-associated fixed factors (independent variables). Inclusion of random slopes for all variables was not possible with the computers available to us, as the models became too large and complex when including those additional 19 fixed factors in the full model. Inclusion of those random slopes improved the models over random intercepts only (Likelihood ratio test).

Results

Uni-Variable Relationships

Saccade curvature and velocity-profile deviation were related to each other, and both were related to saccade duration (or length) and orientation (Table 2). Figures 5–7 show the uni-variable relationships between saccade duration, saccade curvature and velocity-profile deviation. The relationships were non-linear. Cubic polynomial fits to the curvature and velocity-profile deviation were used within the crossed-random, mixed-effects regression models. As shown in Table 2, many of the potential predictor (fixed-factor) variables were related to saccade curvature and velocity-profile deviation in the uni-variable models. The strongest relationships with curvature and velocity-profile deviation were with saccade-dependent predictors (saccade duration and direction). There were some weaker relationships with time-dependent predictors (time within the session, in-flight saccade during scene cut), and a few video-dependent predictors (e.g., man-made object importance, human-figure importance). Age and gender were related to curvature.

TABLE 2

Table 2. Summary of the regression coefficients of factors in “uni-variable” models of log10(curvature) and log10(velocity-profile deviation).

Multi-Variable Relationships

As shown in Table 3, many of the potential predictor (fixed-factor) variables were related to saccade curvature and velocity-profile deviation in the two multi-variable models that included cubic polynomial fits, as shown. Many, but not all of the predictors found to be related to saccade curvature and velocity-profile deviation in the uni-variable models were included in the multi-variable models, and thus were independent predictors. Similarly, some factors that were significant in uni-variable models (e.g., scene cut during saccade) or even in the initial multi-variable model (e.g., importance of nature content), were not included in the final multi-variable model. Factors that were not included in the final model are shown with the significance from the full (all factors) model. Within the multi-variable models, the strongest effects were saccade-dependent variables (duration, orientation).

TABLE 3

Table 3. Summary of the outcomes of the “multi-variable,” backwards-regression models of curvature and with velocity profile deviation.

Scene cut (shot change) during a saccade: We predicted that when a scene cut occurred during a saccade, that saccade would have more curvature and greater velocity-profile deviations than saccades that did not have an in-flight scene cut. In the uni-variable analyses (Table 2) saccades with an in-flight scene cut had lower curvature (p < 0.001) and greater velocity-profile deviation (p < 0.001). However, those effects were not present when corrected for other variables (as reported in Table 3; multi-variable models).

Saccade curvature: was greatest when saccade duration was about 30 ms (or magnitude about 5 degrees; see Figure 5), and greater when saccades were oblique compared to horizontal (p < 0.001), and was greater when saccades were to the right (p < 0.001) or down (p < 0.001) than to the left or up, respectively (Table 3). Also, one video-dependent variable was significant, human-figure importance (p < 0.001), and curvature was greater with increasing age (subject-dependent factor) (p < 0.001; Table 3).

Velocity-profile deviation: was greatest when saccade duration (or magnitude) was longer (Figure 6). Velocity-profile deviation was lower with saccades that were vertical (p < 0.001) or oblique (p < 0.001) compared to horizontal and with saccades that were upward (p < 0.001) as compared to downward, respectively. Velocity-profile deviation was also related to some video-content factors, environment type (p = 0.001), and lighting (p = 0.001; Table 3). In the multi-variable model, we did not find significant effects in any of the subject-dependent factors.

FIGURE 6

Figure 6. Saccadic velocity profile deviation as a function of saccadic duration showing the frequency scale on the right. A fit of the saccade velocity profile deviations (log10) using a cubic polynomial (uni-variable relationship, Table 2) is shown as the solid line.

FIGURE 7

Figure 7. Saccadic velocity-profile deviation as a function of saccadic curvature showing the frequency (scale on the right). A fit of the saccade velocity-profile deviations (log10) using a cubic polynomial (uni-variable relationship, Table 2) is shown as the solid line. While the fit line does not pass through what appear to be the most-dense regions of data (A), the residuals were well distributed, in the log-log domain of the fit (B). Note that the distribution has a long tail to higher velocity-profile deviation values when plotted in the linear-linear domain, as here. (B) has the three axes in logarithmic domain.

Saccadic curvature and velocity-profile deviation were interrelated. As shown in Tables 2, 3, we modeled the velocity-profile deviation as a cubic polynomial of saccadic curvature (Figure 6) and saccadic curvature as a cubic polynomial of velocity-profile deviation. As shown in Figure 6, as curvature increases, velocity-profile deviation becomes slightly greater. Conversely, as velocity-profile deviation increased, curvature increased to about 0.07 units at about 0.0025 units of velocity-profile deviation, after which curvature was approximately stable with increasing velocity-profile deviation (not shown). Though the fits were significant (Tables 2, 3), the fits were not reversible and produced substantively different fits through the data, which reduces confidence in their meaning. This comes from the nature of linear regression in which the errors are minimized in the dimension of the predicted variable (plotted as the ordinate).

Discussion

In our large sample of saccades made while watching movie and TV video clips, saccade curvature, velocity-profile deviation, and saccade duration (or magnitude) were inter-related. Also, saccade curvature and velocity-profile deviation varied strongly with saccade orientation. Interestingly, some aspects of the video content were related to curvature and velocity-profile deviation (as discussed individually below), including the lighting conditions, and human figures being important for understanding clip content, even when corrected for other factors (Table 3). Contrary to our hypothesis scene cuts were not related to saccade curvature or velocity-profile deviation, when corrected for other factors (Table 3).

Curved saccades were shown in very early studies (Lamasky, 1869; Yarbus, 1967) and can be elicited by elements or events in the scene, particularly when those occur during saccade planning or saccade flight. The mechanism for curved saccades may lie in the timing and relationships between motor-control pathways (McPeek et al., 2003; Arai and Keller, 2005). Saccade curvature has been mainly studied in the context of distracting, attracting and repelling objects that often have been made to occur during saccade flight. Given this, we hypothesized that the dynamic nature of movies might cause saccade curvature, since on-screen elements may change during a saccade and thus act as attractors or repellors, or the saccade target might move location during the saccade flight, triggering adjustments from the oculomotor control system. Deviation of saccade trajectory has been used as a measure of the oculomotor competition evoked by a distractor (McPeek and Keller, 2001). In our sample, saccades that were in flight during cut scenes, in the uni-variable models (Table 2), had lower saccade curvature (p < 0.001) and higher velocity-profile deviations (p < 0.001). However, in-flight scene cuts was not significantly related to either saccade curvature (p = 0.37) or velocity-profile deviations (p = 0.82), when corrected for other factors (multi-variable models; Table 3). Thus, scene cuts were not related to saccadic curvature or velocity profile deviation.

Some video-dependent factors were related to saccade curvature (Table 2). However, when corrected for other factors (e.g., saccade duration, orientation; Table 3), only the importance of human figures (e.g., people running) presented in the video clips was related to curvature, with lower curvature when human figures were important for understanding (p < 0.001). Perhaps, if human-figure stimuli were tracked during a saccade, their nature did not elicit large in-flight modifications of the saccade path (which would appear as higher curvature) or their movement could be predicted in saccade planning. The effect of distractors (stimuli that appeared during a saccade) on saccade curvature were greater when the distractors were salient (van Zoest et al., 2012) or had greater social relevance (Laidlaw et al., 2015), and varied with high-level (semantic) information related to the distractors (Weaver et al., 2011). Videos contain high-level information and salient and socially-relevant stimuli. The greater curvature with human-figure importance could indicate that the human figures had high salience. The importance of faces, man-made objects (e.g., cars) or nature were not related to the amount of curvature, even in the uni-variables analyses (Table 2), even though these were considered important for understanding of the video content. Curvature was lower with comedy videos than documentary (p = 0.004) or animations (p = 0.007; Table 3). It is possible that our documentary and animation clips had faster movements that could have induced curvature, but the clips labeled as action, did not have more curvature than the comedy clips (p = 0.15).

Similarly, in uni-variable analyses (Table 2), some video-dependent factors including when nature (p < 0.001), faces (p = 0.001), man-made objects (e.g., vehicles; p = 0.008) and human figures (p < 0.001) were related to velocity-profile deviations, but not when corrected for other factors (Table 3). Velocity-profile deviations were greater in outdoor than indoor scenes in the multi-variate analysis (p < 0.001), though, were not significant in the uni-variate analyses (Table 2).

A well-illuminated background enables the visual system to calculate target position faster and more accurately than when the target moves in the dark (Lemij and Collewijn, 1989). This is consistent with the finding that targeting saccades made by monkeys in dim background light had a shorter latency and were less hypometric than those in the dark (Straube et al., 1997). On that basis, we expected that curvature and velocity-profile deviation would be less when viewing brighter scenes. Contrary to that expectation, in the multi-variable models (thus corrected for other factors; Table 3), velocity-profile deviation was greater when the lighting conditions in the video clip were brighter (p = 0.001). Similarly, outdoor scenes, which were associated with higher lighting (z = 115, p < 0.001), was independently related to greater velocity-profile deviations (p < 0.001).

Wu et al. (2015) found that the number of shifts of the eyes to the center of the screen after presenting a scene change increased with the level of scene change, suggesting that the detection of a global context change may be required to trigger a re-evaluation of the scene. A higher number of cuts might be related to a greater number of saccades to the display center, and those might be more subject to curvature or velocity-profile deviation as they were not planned (driven by a new object of interest). In the uni-variable model, the number of cuts was not associated with curvature or velocity-profile deviation (Table 2), but was initially associated with velocity-profile deviation when all factors were included in the model (p < 0.001), but was not significant (p = 0.02) as other non-significant factors were removed (Table 3).

Di Stasi et al. (2013, 2014) found that time-on-task decreased saccadic velocity, presumably from fatigue. While fatigue impairs motor planning, our task of watching 10 to 30 min of video was not particularly fatiguing. Time-on-task was related to velocity-profile deviation (p < 0.001) in the uni-variable models (Table 2), but in the multivariable models (Table 3) time-on-task was not related to curvature (p = 0.02) or velocity-profile deviations (p = 0.12), when corrected for other factors.

Saccade curvature increased with increasing age (univariable and multi-variate models), which suggests that some aspect of the oculo-motor system is less able to control the eye movements. However, Pratt et al. (2006) found no differences in saccade kinematics between younger and older subjects.

Previously, we had noted that some saccades elicited while viewing movies had a velocity profile that was not well fit with the compressed exponential model (Han et al., 2013). That model accounts for the skewness of velocity profiles that has been studied previously (van Opstal and van Gisbergen, 1987; Takagi et al., 1993). We are aware of only one report of discrepancies in the velocity profile which was related to a cerebellar degeneration (Zivotofsky et al., 2006), though velocity-profile deviations are apparent in some previous publications. The lack of reports of velocity-profile deviations may be because such saccades are removed from off-line analyses because of their failure to comply with known models. When working with a real-time gaze-contingent system with saccade prediction, a response is required from the system for each saccade, whether the saccade looks normal or not. We had hypothesized that velocity-profile deviations might be related to curvature, with the discrepancies in the velocity profile being related to mid-flight changes in motor commands. Indeed, generally, velocity-profile deviations were larger when curvature was greater, but the relationship was not compelling, despite the high level of significance in our models, as the relationship was not strong, in the sense of large changes (Figure 6). Overall, longer saccades had greater velocity-profile deviation, and this was not because there was more opportunity for a discrepancy between the data and the model, since the root-mean-square normalizes for the amount of data (Equation 1).

Saccade curvature was greater for oblique than horizontal and vertical saccades, while velocity-profile deviation was greater for vertical than horizontal or oblique saccades (Table 3). We had hypothesized that horizontal saccades might be less subject to saccade curvature or velocity-profile deviation, there being fewer extraocular muscles involved in horizontal saccades, and thus less opportunity for errors in motor execution. This was based on models of the generation of saccades (Godijn and Theeuwes, 2002; Arai and Keller, 2005; Schreiber et al., 2006,) since there would be fewer opportunities for mistiming. We had hypothesized that mistiming might be a cause of difficult-to-predict saccades, however, our results are not consistent with that hypothesis. It is not clear to us, why rightward and downward saccades were related to greater curvature than rightward or upward saccades while downward saccades were associated with greater velocity-profile deviation (Table 3).

Our purpose in conducting this exploratory study was to examine conditions under which saccade curvature or velocity-profile deviations occurred, as these features created discrepancies between the real-time prediction of eye position during a saccade as we do with our gaze-contingent system. System latency occurs even when using displays with high refresh rates (e.g., 100–120 Hz; Saunders et al., 2014). Previously, Han et al. (2013) showed that real-time saccade prediction reduces errors between location of a stimulus and eye position when calculated off-line. When that algorithm was implemented in our gaze-contingent system (i.e., runs in real time), there was a reduction in stimulus placement errors, as shown in Figure 8. That algorithm does not account for saccade curvature and assumes a conventional velocity profile (i.e., fit by compressed exponential function). If we knew when more-curved saccades or saccade with higher velocity-profile deviations were likely to occur, we could either plan an experiment to reduce the likelihood of occurrence (e.g., avoid conditions related to difficult-to-predict saccades, Table 3) or we could, in real-time, determine that there was a risk and implement a slower prediction method that has less prediction error. Recently, we have developed an algorithm that further reduces errors in prediction of eye position in the presence of saccade curvature and velocity-profile deviation (Wang et al., 2017). Interestingly, that algorithm is faster than the Han et al. (2013) algorithm and needs less data to start making a prediction, so has the potential to provide even better reduction of stimulus presentation location errors.

FIGURE 8

Figure 8. Gaze position deviations in a real-time, gaze-contingent system were smaller when using the Han et al. (2013) saccade-prediction algorithm (blue) than when not using saccade prediction (red).

A major limitation of our study is that it was primarily observational, and therefore, cannot address the causes of curved saccades or causes of saccades with velocity-profile deviations. Another limitation is that we used an infra-red-video-based eye tracking system. Though the data is not presented here, we have observed similar curvature and velocity profile deviations in data collected using a dual-Purkinje-image system (Han et al., 2013) and a scleral-coil system (Costela et al., 2014, 2015; Wang et al., 2017), which suggests that the effects reported here are not an artifact of the video-based eye-tracking system (Hooge et al., 2016). These results suggest that distractors are not essential to elicit saccades with strong curvature and velocity-profile deviations. In future studies, we will compare and analyze these saccadic features during the performance of different visual tasks.

Author Contributions

FC and RW: Designed the experiment, analyzed the data, and wrote and revised the manuscript. FC: Collected the data.

Funding

This work was supported by NIH EY023724 and Core grant P30EY003790.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

We thank John Ackerman for his technical advice on the analysis of saccadic features, Daniel Saunders for the initial implementation of the Han et al. (2013) saccade prediction algorithm in our gaze-contingent visual display system and collection of the data reported in Figure 8 and Jorge Otero-Millan for his comments on an early version of the manuscript.

Supplementary Material

The Supplementary Material for this article can be found online at: https://[see PDF]/articles/10.3389/fnins.2018.00960/full#supplementary-material

References

Andersson, R., Larsson, L., Holmqvist, K., Stridh, M., and Nystrom, M. (2017). One algorithm to rule them all? An evaluation and discussion of ten eye movement event-detection algorithms. Behav. Res. Methods 49, 616–637. doi: 10.3758/s13428-016-0738-9

Arai, K., and Keller, E. L. (2005). A model of the saccade-generating system that accounts for trajectory variations produced by competing visual stimuli. Biol. Cybern. 92, 21–37. doi: 10.1007/s00422-004-0526-y

Arai, K., McPeek, R. M., and Keller, E. L. (2004). Properties of saccadic responses in monkey when multiple competing visual stimuli are present. J. Neurophysiol. 91, 890–900. doi: 10.1152/jn.00818.2003

Bahill, A. T., Clark, M. R., and Stark, L. (1975a). Glissades—eye movements generated by mismatched components of the saccadic motoneuronal control signal. Math. Biosci. 26, 303–318. doi: 10.1016/0025-5564(75)90018-8

Bahill, A. T., Clark, M. R., and Stark, L. (1975b). The main sequence, a tool for studying human eye movements. Math. Biosci. 24, 191–204. doi: 10.1016/0025-5564(75)90075-9

Baloh, R. W., Sills, A. W., Kumley, W. E., and Honrubia, V. (1975). Quantitative measurement of saccade amplitude, duration, and velocity. Neurology 25, 1065–1070. doi: 10.1212/WNL.25.11.1065

Bargary, G., Bosten, J. M., Goodbourn, P. T., Lawrance-Owen, A. J., Hogg, R. E., and Mollon, J. D. (2017). Individual differences in human eye movements: an oculomotor signature? Vision Res. 141, 157–169. doi: 10.1016/j.visres.2017.03.001

Bédard, P., and Sanes, J. (2009). Gaze and hand position effects on finger-movement-related human brain activation. J. Neurophysiol. 101, 834–842. doi: 10.1152/jn.90683.2008

Berg, D., Shen, J., and Itti, L. (2009). The eyeMarkup Eye Movement Analysis Tool. Available online at: http://ilab.usc.edu/cgi-bin/secure/viewcvs.cgi/trunk/saliency/matlab/Eye-Markup/.

Blackmon, T. H., Chernyak, D.A., Azzariti, M., and Stark, L. W., (1999). “Dynamic scanpaths: eye movement analysis methods,” in Human Vision and Electronic Imaging IV SPIE Proceedings, Vol. 3644, (San Jose, CA), 511–519.

Bollen, E., Bax, J., Van Dijk, J. G., Koning, M., Bos, J. E., Kramer, C. G., et al. (1993). Variability of the main sequence. Invest. Ophthalmol. Vis. Sci. 34, 3700–3704.

Brainard, D. H. (1997). The psychophysics toolbox. Spatial Vis. 10, 433–436. doi: 10.1163/156856897X00357

Casey, J. (2012). Exploring Curvature. Springer Science & Business Media. doi: 10.1007/978-3-322-80274-3

Chen, Y.-F., Lin, H.-H., Chen, T., Tsai, T.-T., and Shee, I.-F. (2002). The peak velocity and skewness relationship for the reflexive saccades. Biomed. Eng. 14, 71–80. doi: 10.4015/S1016237202000115

Coolidge, J. L. (1952). The unsatisfactory story of curvature. Am. Math. Mon. 59, 375–379. doi: 10.1080/00029890.1952.11988145

Costela, F. M., Kajtezovic, S., and Woods, R. L. (2017). The preferred retinal locus used to watch videos. Invest. Ophthalmol. Vis. Sci. 58, 6073–6081. doi: 10.1167/iovs.17-21839

Costela, F. M., Otero-Millan, J., Mccamy, M. B., Macknik, S. L., Di Stasi, L. L., Rieiro, H., et al. (2015). Characteristics of spontaneous square-wave jerks in the healthy macaque monkey during visual fixation. PLoS ONE 10:e0126485. doi: 10.1371/journal.pone.0126485

Costela, F. M., Otero-Millan, J., Mccamy, M. B., Macknik, S. L., Troncoso, X. G., Jazi, A. N., et al. (2014). Fixational eye movement correction of blink-induced gaze position errors. PLoS ONE 9:e110889. doi: 10.1371/journal.pone.0110889

Coutrot, A., and Guyader, N. (2014). How saliency, faces, and sound influence gaze in dynamic social scenes. J. Vis. 14:5. doi: 10.1167/14.8.5

Cristino, F., and Baddeley, R. (2009). The nature of the visual representations involved in eye movements when walking down the street. Vis. Cogn. 17, 880–903. doi: 10.1080/13506280902834696

Cutting, J. E., Delong, J. E., and Nothelfer, C. E. (2010). Attention and the evolution of Hollywood film. Psychol. Sci. 21, 432–439. doi: 10.1177/0956797610361679

Dandekar, S., Privitera, C., Carney, T., and Klein, S. A. (2012). Neural saccadic response estimation during natural viewing. J. Neurophysiol. 107, 1776–1790. doi: 10.1152/jn.00237.2011

Di Stasi, L. A., Antolí, A., and Cañas, J. J. (2011). Main sequence: an index for detecting mental workload variation in complex tasks. Appl. Ergon. 42, 807–813. doi: 10.1016/j.apergo.2011.01.003

Di Stasi, L. L., Mccamy, M. B., Catena, A., Macknik, S. L., Canas, J. J., and Martinez-Conde, S. (2013). Microsaccade and drift dynamics reflect mental fatigue. Eur. J. Neurosci. 38, 2389–2398. doi: 10.1111/ejn.12248

Di Stasi, L. L., Mccamy, M. B., Macknik, S. L., Mankin, J. A., Hooft, N., Catena, A., et al. (2014). Saccadic eye movement metrics reflect surgical residents' fatigue. Ann. Surg. 259, 824–829. doi: 10.1097/SLA.0000000000000260

Dorr, M., Martinetz, T., Gegenfurtner, K. R., and Barth, E. (2010). Variability of eye movements when viewing dynamic natural scenes. J. Vis. 10:28. doi: 10.1167/10.10.28

Dowiasch, S., Marx, S., Einhauser, W., and Bremmer, F. (2015). Effects of aging on eye movements in the real world. Front. Hum. Neurosci. 9:46. doi: 10.3389/fnhum.2015.00046

Doyle, M., and Walker, R. (2001). Curved saccade trajectories: voluntary and reflexive saccades curve away from irrelevant distractors. Exp. Brain Res. 139, 333–344. doi: 10.1007/s002210100742

Doyle, M., and Walker, R. (2002). Multisensory interactions in saccade target selection: curved saccade trajectories. Exp. Brain Res. 142, 116–130. doi: 10.1007/s00221-001-0919-2

Duke Elder, S., and Wybar, K. (1973). Ocular Motility and Stabismus. London: Henry Kimpton.

Flynn, P. J., and Jain, A. K. (1989). “On reliable curvature estimation,” in IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1989. Proceedings CVPR (San Diego, CA).

Godijn, R., and Theeuwes, J. (2002). Programming of endogenous and exogenous saccades: evidence for a competitive integration model. J. Exp. Psychol. Hum. Percept. Perform. 28, 1039–1054. doi: 10.1037/0096-1523.28.5.1039

Hamel, J., De Beukelear, S., Kraft, A., Ohl, S., Audebert, H. J., and Brandt, S. A. (2013). Age-related changes in visual exploratory behavior in a natural scene setting. Front. Psychol. 4:339. doi: 10.3389/fpsyg.2013.00339

Han, P., Saunders, D. R., Woods, R. L., and Luo, G. (2013). Trajectory prediction of saccadic eye movements using a compressed exponential model. J. Vis. 13, 8. doi: 10.1167/13.8.27

Hooge, I., Holmqvist, K., and Nystrom, M. (2016). The pupil is faster than the corneal reflection (CR): are video based pupil-CR eye trackers suitable for studying detailed dynamics of eye movements? Vis. Res. 128, 6–18. doi: 10.1016/j.visres.2016.09.002

Inhoff, A. W., Seymour, B. A., Schad, D., and Greenberg, S. (2010). The size and direction of saccadic curvatures during reading. Vis. Res. 50, 1117–1130. doi: 10.1016/j.visres.2010.03.025

Itti, L. (2005). Quantifying the contribution of low-level saliency to human eye movements in dynamic scenes. Vis. Cogn. 12, 1093–1123. doi: 10.1080/13506280444000661

Konig, S. D., and Buffalo, E. A. (2014). A nonparametric method for detecting fixations and saccades using cluster analysis: removing the need for arbitrary thresholds. J. Neurosci. Methods 227, 121–131. doi: 10.1016/j.jneumeth.2014.01.032

Laidlaw, K. E., Badiudeen, T. A., Zhu, M. J., and Kingstone, A. (2015). A fresh look at saccadic trajectories and task irrelevant stimuli: social relevance matters. Vis. Res. 111 (Pt. A), 82–90. doi: 10.1016/j.visres.2015.03.024

Lamasky, S. (1869). Pfluger's Archiv. f. d. gesammte Physiologie II, 418.

Land, M. F., and Hayhoe, M. (2001). In what ways do eye movements contribute to everyday activities? Vis. Res. 41, 3559–3565. doi: 10.1016/S0042-6989(01)00102-X

Land, M. F., and Lee, D. N. (1994). Where we look when we steer. Nature 369, 742. doi: 10.1038/369742a0

Le Meur, O., Le Callet, P., and Barba, D. (2007). Predicting visual fixations on video based on low-level visual features. Vis. Res. 47, 2483–2498. doi: 10.1016/j.visres.2007.06.015

Lemij, H. G., and Collewijn, H. (1989). Differences in accuracy of human saccades between stationary and jumping targets. Vis. Res. 29, 1737–1748. doi: 10.1016/0042-6989(89)90156-9

Ludwig, C. J., and Gilchrist, I. D. (2002). Measuring saccade curvature: a curve-fitting approach. Behav. Res. Methods Instrum. Comput. 34, 618–624. doi: 10.3758/BF03195490

McPeek, R. M., Han, J. H., and Keller, E. L. (2003). Competition between saccade goals in the superior colliculus produces saccade curvature. J. Neurophysiol. 89, 2577–2590. doi: 10.1152/jn.00657.2002

McPeek, R. M., and Keller, E. L. (2001). Short-term priming, concurrent processing, and saccade curvature during a target selection task in the monkey. Vis. Res. 41, 785–800. doi: 10.1016/S0042-6989(00)00287-X

McSorley, E., Haggard, P., and Walker, R. (2004). Distractor modulation of saccade trajectories: spatial separation and symmetry effects. Exp. Brain Res. 155, 320–333. doi: 10.1007/s00221-003-1729-5

Munn, S. M., Stefano, L., and Pelz, J. B. (2008). “Fixation-identification in dynamic scenes: Comparing an automated algorithm to manual coding,” in APGV'08: Proceedings of the 5th Symposium on Applied Perception in Graphics and Visualization, (Los Angeles, CA), 33–42.

Noton, D., and Stark, L. (1971). Eye movements and visual perception. Sci. Am. 224, 35–43.

Nummenmaa, L., Hyona, J., and Calvo, M. G. (2009). Emotional scene content drives the saccade generation system reflexively. J. Exp. Psychol. Hum. Percept. Perform. 35, 305–323. doi: 10.1037/a0013626

Otero-Millan, J., Troncoso, X. G., Macknik, S. L., Serrano-Pedraza, I., and Martinez-Conde, S. (2008). Saccades and microsaccades during visual fixation, exploration, and search: foundations for a common saccadic generator. J. Vis. 8, 21 21–18. doi: 10.1167/8.14.21

Pelli, D. G. (1997). The VideoToolbox software for visual psychophysics: transforming numbers into movies. Spatial Vis. 10, 437–442. doi: 10.1163/156856897X00366

Pratt, J., Dodd, M., and Welsh, T. (2006). Growing older does not always mean moving slower: examining aging and the saccadic motor system. J. Mot. Behav. 38, 373–382. doi: 10.3200/JMBR.38.5.373-382

Santini, F., Redner, G., Iovin, R., and Rucci, M. (2007). EyeRIS: a general-purpose system for eye-movement-contingent display control. Behav. Res. Methods 39, 350–364. doi: 10.3758/BF03193003

Satgunam, P., Woods, R. L., Bronstad, P. M., and Peli, E. (2013). Factors affecting enhanced video quality preferences. IEEE Trans. Image Proc. 22, 5146–5157. doi: 10.1109/TIP.2013.2282120

Saunders, D. R., Bex, P. J., Rose, D. J., and Woods, R. L. (2014). Measuring information acquisition from sensory input using automated scoring of natural-language descriptions. PLoS ONE 9:e93251. doi: 10.1371/journal.pone.0093251

Saunders, D. R., Bex, P. J., and Woods, R. L. (2013). Crowdsourcing a normative natural language dataset: a comparison of Amazon Mechanical Turk and in-lab data collection. J. Med. Internet Res. 15:e100. doi: 10.2196/jmir.2620

Schreiber, C., Missal, M., and Lefevre, P. (2006). Asynchrony between position and motion signals in the saccadic system. J. Neurophysiol. 95, 960–969. doi: 10.1152/jn.00315.2005

Sheliga, B. M., Riggio, L., Craighero, L., and Rizzolatti, G. (1995a). Spatial attention-determined modifications in saccade trajectories. Neuroreport 6, 585–588. doi: 10.1097/00001756-199502000-00044

Sheliga, B. M., Riggio, L., and Rizzolatti, G. (1994). Orienting of attention and eye movements. Exp. Brain Res. 98, 507–522. doi: 10.1007/BF00233988

Sheliga, B. M., Riggio, L., and Rizzolatti, G. (1995b). Spatial attention and eye movements. Exp. Brain Res. 105, 261–275. doi: 10.1007/BF00240962

Straube, A., Robinson, F. R., and Fuchs, A. F. (1997). Decrease in saccadic performance after many visually guided saccadic eye movements in monkeys. Invest. Ophthalmol. Vis. Sci. 38, 2810–2816.

Takagi, M., Abe, H., Hasegawa, S., Yoshizawa, T., and Usui, T. (1993). Velocity profile of human horizontal saccades. Ophthalmologica 206, 169–176. doi: 10.1159/000310386

van Beers, R. (2007). The sources of variability in saccadic eye movements. J. Neurosci. 27, 8757–8770. doi: 10.1523/JNEUROSCI.2311-07.2007

Van der Stigchel, S. (2010). Recent advances in the study of saccade trajectory deviations. Vis. Res. 50, 1619–1627. doi: 10.1016/j.visres.2010.05.028

Van der Stigchel, S., Meeter, M., and Theeuwes, J. (2006). Eye movement trajectories and what they tell us. Neurosci. Biobehav. Rev. 30, 666–679. doi: 10.1016/j.neubiorev.2005.12.001

Van der Stigchel, S., and Theeuwes, J. (2007). The relationship between covert and overt attention in endogenous cuing. Percept. Psychophys. 69, 719–731. doi: 10.3758/BF03193774

van Opstal, A. J., and van Gisbergen, J. A. (1987). Skewness of saccadic velocity profiles: a unifying parameter for normal and slow saccades. Vis. Res. 27, 731–745. doi: 10.1016/0042-6989(87)90071-X

van Zoest, W., Donk, M., and Van der Stigchel, S. (2012). Stimulus-salience and the time-course of saccade trajectory deviations. J. Vis. 12:16. doi: 10.1167/12.8.16

Walker, R., McSorley, E., and Haggard, P. (2006). The control of saccade trajectories: direction of curvature depends on prior knowledge of target location and saccade latency. Percept. Psychophys. 68, 129–138. doi: 10.3758/BF03193663

Wang, S., Woods, R. L., Costela, F. M., and Luo, G. (2017). Dynamic gaze-position prediction of saccadic eye movements using a Taylor series. J. Vis. 17, 13. doi: 10.1167/17.14.3

Weaver, M. D., Lauwereyns, J., and Theeuwes, J. (2011). The effect of semantic information on saccade trajectory deviations. Vis. Res. 51, 1124–1128. doi: 10.1016/j.visres.2011.03.005

Wu, E., Chua, F.-K., and Yen, S.-C. (2015). Lights, camera, action-CUT! How film cuts influence eye movements. J. Vis. 15, 219–219. doi: 10.1167/15.12.219

Yarbus, A. L. (1967). Eye Movements During Perception of Complex Objects. New York, NY: Springer US.

Zivotofsky, A. Z., Siman-Tov, T., Gadoth, N., and Gordon, C. R. (2006). A rare saccade velocity profile in Stiff-Person Syndrome with cerebellar degeneration. Brain Res. 1093, 135–140. doi: 10.1016/j.brainres.2006.03.064