Neurobiological differences in mental rotation and instrument interpretation in airline pilots

Abstract

Airline pilots and similar professions require reliable spatial cognition abilities, such as mental imagery of static and moving three-dimensional objects in space. A well-known task to investigate these skills is the Shepard and Metzler mental rotation task (SMT), which is also frequently used during pre-assessment of pilot candidates. Despite the intuitive relationship between real-life spatial cognition and SMT, several studies have challenged its predictive value. Here we report on a novel instrument interpretation task (IIT) based on a realistic attitude indicator used in modern aircrafts that was designed to bridge the gap between the abstract SMT and a cockpit environment. We investigated 18 professional airline pilots using fMRI. No significant correlation was found between SMT and IIT task accuracies. Contrasting both tasks revealed higher activation in the fusiform gyrus, angular gyrus, and medial precuneus for IIT, whereas SMT elicited significantly stronger activation in pre- and supplementary motor areas, as well as lateral precuneus and superior parietal lobe. Our results show that SMT skills per se are not sufficient to predict task accuracy during (close to) real-life instrument interpretation. While there is a substantial overlap of activation across the task conditions, we found that there are important differences between instrument interpretation and non-aviation based mental rotation.

R

A P

Sladky ka, Tik Lamm,, Niedermeier Field &

per se

One of the central competences of airline pilots is an extraordinary ability in spatial cognition and mental imagery of static and moving three-dimensional objects in space. A well-known task to investigate these skills is the Shepard-Metzler mental rotation task1, which is also frequently used in test batteries for the pre-assessment of pilot candidates. While it has been shown that, e.g., (military) pilots perform better than non-pilots in similar three dimensional tasks2, it seems clear that performance in the mental rotation task alone is insufficient for inferring on general real-life spatial cognitions skills. Instead, spatial cognition is a component of the complex concept of situation awareness (SA)3,4. SA encompasses multiple high-level cognitive processes such as perception, understanding, and projection, which are inuenced by and depend on domain- and profession-specic sub-concepts. In aviation, such a sub-concept of SA is spatial situational awareness (SPASA). A decline or break down in SPASA is thus expected to diminish overall SA and consequently can impact ight crew performance. A decline or loss of SPASA, namely, an inability to correctly interpret aircra attitude, altitude and airspeed in relation to points of reference, can occur in three dierent stages as recently dened5: (1) spatial disorientation that is unrecognized (i.e., unawareness of current state), (2) recognized spatial disorientation which may or may not be identied as such, but with the awareness that the perceived state(s) do not match the current information from instruments or other environmental input (i.e., misinterpretation of current state), and (3) awareness of spatial disorientation including the incapability of reacting to the identied state.

Apparently, a lack of SPASA and the incorrect interpretation of three-dimensional representations of information are well known reasons for accidents, which is why these important skills are tested in pilot assessments. Depending on airline regulations, pilot-assessments include various tests and are carried out by dierent certied authorities. These evaluations are commonly carried out in a multimodal assessment process which usually tests

MR Center of Excellence, Center for Medical Physics and Biomedical Engineering, Medical University of Vienna, Austria. Cognitive Science Research Platform, University of Vienna, Austria. National Aerospace Centre (NLR), Amsterdam, Netherlands. Vienna. German Aerospace Center (DLR), Braunschweig, Germany. Correspondence and requests for materials should be addressed to C.W. (email: [email protected])

SCIENTIFIC REPORTS

1

www.nature.com/scientificreports/

Figure 1. Shepard-Metzler task and three-dimensional instrument interpretation task. Task conditions were presented in alternating blocks (duration 20s). Subjects had to decide by button press if the le and right images are consistent (i.e., if they are the same object and if the attitude indicator matches the three-dimensional orientation of the airplane with respect to pitch and roll). New stimulus appeared aer button press; there was no maximum trial duration. Solutions for the present example sequence are: SMT: correct, incorrect, correct; IIT: incorrect, correct, correct.

various cognitive abilities such as mechanical comprehension and technical knowledge, numerical reasoning, short-term memory for auditory and visual information, speed of perception, attention control, reaction time, psychomotor coordination, multiple task capacity, and spatial orientation6. To test multiple task coordination the Multiple Task Coordination Test is oen applied which requires the simultaneous performance of three dierent tasks: a compensatory tracking task, an acoustic task and an instrument monitoring task.

In addition to these conceptual considerations, a number of empirical dierences between real-life spatial cognition and abstract mental rotation were found. On a behavioral level, dierences in mental rotation (i.e., rotating an external object) and egocentric perspective taking (i.e., realigning oneself) have been observed7, indicating a dierence in the responsible cognitive mechanisms. Furthermore, studies that investigated the relationship between mental rotation and actual spatio-motor performance found such a relationship only in inexperienced or untrained pilots8, drivers9, and surgeons10. Wolf et al.9, for example, have shown that female novice drivers scored lower in parking skills and a mental rotation paradigm, when compared to a group of males matched for driving experience. However, when repeating the experiment at a later time, this gender dierence was only observed for the abstract mental rotation paradigm, yet not for the driving task, where task performance was determined by the level of acquired driving experience, instead.

In combination, these ndings raise not only the question if performance in a Shepard-Metzler-like mental rotation task indeed represents the current skill level of real-life spatio-motor performance in a specic profession but also its function as predictor for the future individual development resulting from training and experience. In this study we have designed and implemented an instrument interpretation (IIT) task that strongly resembles the attitude indicator, the main instrument used by pilots to determine the attitude of the aircra and, thus, is close to a pilots real-life requirements and aordances while still being highly comparable to the Shepard-Metzler task (SMT) for mental rotation with respect to the presentation of visual stimuli and subject responses by means of pressing one of two buttons (Fig.1).

However, setting up a comparative study between scenarios that closely resemble real-life, profession-specic requirements versus classic Shepard-Metzler task stimuli impose experimental challenges, particular when using MR neuroimaging methods. First, the form of interaction with the environment could become so complex that it cannot be implemented within the constrained MR scanner environment and requires the use of elaborated simulator models or actual motion-capable simulators, respectively. Even if a realistic implementation was feasible given the constraints of the setup, a task that requires such complex interaction with the environment would induce additional brain activations that limit the comparability with those associated with an abstract mental rotation task (which only requires the subject to observe and press one or two buttons). Second, to suit the experimental constraints of fMRI, real-life scenarios might have to be simplied or distorted to a degree where they no longer pose a valid experimental proxy for the actual phenomenon of interest. The IIT presented herein was specically designed to combine the basic requirements of the Shepard-Metzler task together with an instrument interpretation referring to the aircras orientation in space.

We investigated the biological substrates recruited for the novel IIT in comparison to the abstract Shepard-Metzler mental rotation task. Further, we compared task performance and task-specic brain activations between both tasks to assess whether the SMT can be regarded a reliable tool to investigate aviation-relevant spatial cognition skills.

There were no signicant behavioural or brain activation dierences associated with individual ight experience (i.e., ight hours, aircra types, and age). On average, participants performed signicantly (paired t-test, T=4.62, p < 0.001) more trials of IIT (mean = 60.33, SD = 16.48) than SMT (mean = 46.50, SD = 12.12) due to faster response times for instrument interpretation (mean = 1.78 s, SD = 0.51s vs. mean = 2.32 s, SD = 0.72 s, paired t-test, T=3.73, p< 0.002). Task accuracy was signicantly lower for IIT (mean=80.08%, SD=9.24%) compared

SCIENTIFIC REPORTS

2

www.nature.com/scientificreports/

Figure 2. Overlapping brain activation related to Shepard-Metzler task (blue) and instrument interpretation task (red). Overlap of conditions is displayed in purple. Statistical threshold for t-statistics was set to p< 0.05FWE whole-brain corrected.

to SMT (mean=88.66%, SD= 6.90%) in a paired t-test (p< 0.005). No signicant correlation was found between task accuracies for IIT and SMT ( = 0.238, p = 0.341, aer outlier removal: =0.295, p = 0.267). However, a moderate positive correlation was observed for reaction times between mental rotation and instrument interpretation (=0.559, p=0.016).

Brain activation, related to both task conditions, was found in visual areas (occipital lobe, fusiform gyrus), superior parietal lobe, dorsolateral PFC, thalamus, putamen, somatosensory cortex, and motor cortex (Fig.2). Despite this substantial overlap, the contrast between the two conditions revealed task-specic dierences. Mental rotation elicited signicantly stronger activation in bilateral pre- and supplementary motor areas, and superior and inferior parietal lobe (Fig.3).

Instrument interpretation, on the contrary, induced signicant hyper activation in bilateral fusiform gyri and angular gyri, as well as precuneus and PCC. In the le hemisphere, more activation was found in Brocas and Wenickes area. In the right prefrontal cortex, DLPFC was signicantly activated in the IIT>SMT contrast.

The present study investigated the neurobiological correlates of spatial cognition in professional airline pilots while either interpreting information displayed on a realistic attitude indicator (instrument interpretation task, IIT) or performing a generic mental rotation task (Shepard-Metzler task, SMT).

Although a high level of safety has been achieved in aviation due to continuous developments in ight systems and automation, procedures, as well as pilot training, accidents and incidents still occur and human factors such as loss of SA, and specically, spatial SA, are reported as crucial in accident/incident reports5. While the origin of spatial disorientation is oen related to physiological processes outside the central nervous system (e.g., in the vestibular system)11, overcoming spatial disorientation can be inuenced by system and display design. Therefore, most spatial disorientation prevention training programs focus on these physiological processes. More knowledge about the neurological processes could lead to prevention programs also focusing on cognitive training.

A substantial overlap in brain activation related to instrument interpretation and Shepard-Metzler task was found. This implies that similar brain networks are required for abstract mental rotation paradigms and an airplane attitude judgement task that was carefully designed to emulate pilots real-life scenarios as closely as possible. The observed task activations for the SMT are consistent with previous studies where similar paradigms

SCIENTIFIC REPORTS

3

www.nature.com/scientificreports/

Figure 3. Statistical dierences of Shepard-Metzler task (cool) and instrument interpretation task (hot).

Statistical threshold for t-statistics was set to p< 0.05FWE cluster-level corrected. Increased activation during instrument interpretation was found in the fusiform gyri (rst row), Brocas and Wernickes area, angular gyri (third row), precuneus, DLPFC, and PCC (fourth row). The Shepard-Metzler task induced more activation in the inferior (fourth row) and superior parietal lobe and the pre-SMA (last row).

were applied12,13. The predominantly le-lateralized activation in the primary motor cortex can be attributed to the execution of the required button pressing. This interpretation is supported by evidence from a previous study that dierentiated these button pressing eorts from spatial cognition processes14. Instrument interpretation, when compared to SMT, induced stronger activation in the angular and fusiform gyri. The angular gyrus is central for context-dependent semantic integration of information originating from language-based but also non-language-based sources15. Neuronal activity within the fusiform gyrus has been implicated for the recognition of human faces, but also for animate and inanimate objects, in particular for making distinctions of those within a given category16,17. In accordance with this, a previous study has demonstrated that visual-spatial perspective taking is associated with activation in the (right) fusiform gyrus, when contrasted with a motor representation task, which elicited more activation in the superior parietal lobe18. Interestingly, IIT resulted in stronger activation of DLPFC, which might reect higher engagement of executive functions, including working memory. Notably the latter have been shown to play only a minor role in mental rotation task19, thus indicating again the discrepancy between task requirements. The stronger activation in Brocas and Wernickes area during IIT might be linked to verbalization processes that supported the pilots perceptual choices.

Most importantly, we found no signicant correlation between accuracies of both task types across pilots, which suggests that abstract mental rotation skills per se are not sufficient to predict task accuracy during instrument interpretation. Previous studies have shown that abstract mental competences might lead to better outcomes in early training phases of particular spatio-motor tasks. In experienced performers, however, these initial dierences were found to mitigate and mainly depend on the respective experience level9,10. Their ndings clearly suggest that spatio-motor expertise within a real-life scenario, such as surgery or car driving, only rely in part on the brain network necessary for abstract mental rotation skills. Dierentiating the role of perspective taking

SCIENTIFIC REPORTS

4

www.nature.com/scientificreports/

versus motor-based mental rotation is also of high relevance for display and instrument design. Maintaining and (re)establishing spatial awareness, i.e., awareness of the current and future aircra attitude, is a central requirement for pilots.

On the other hand, we observed a signicant correlation of reaction times between the two dierent tasks. Reaction times critically depend on individual performance indicators like vigilance and condence. Fast processing is, however, not the essential target in the tasks used in our study, and participants were instructed accordingly. In case of airline pilots, correct interpretation of the current state of the aircra is the basis for appropriate decision making. In general, we did not observe a signicant correlation between task accuracies and reaction times. With respect to the SMT, however, there was a signicant negative correlation between reaction times and task accuracies, which was not observed for the IIT. This provides additional support that SMT performance is not a reliable predictor for other spatial cognition tasks, such as the IIT and, thus, not a reliable stratication tool in pilot assessment.

Typically, spatial awareness is enabled by the multimodal integration of visual, somatosensory, and vestibular information in the medial superior temporal area20. However, the conclusions resulting from this functional processing are usually based on standard, earthbound situations and might be unreliable during ight operations, particular when additionally aected by reduced visibility (e.g., clouds, fog, smoke, low light) and spatial disorientation due to illusions within the human sensory system (e.g., somatogravic or oculogravic illusions)21. Therefore, pilots have to routinely supplement their sense of orientation by the use of articial instruments (i.e., attitude indicators). In current attitude indicators, two dierent designs can be distinguished: (A) the articial horizon is moving relative to a xed illustration of an aircra (sometimes referred to as inside-out, moving horizon or western-style attitude indicators), which was used in this study; (B) a xed horizon and an aircra symbol that is moving depending on the bank angle (outside-in or Russian-style attitude indicator). Both versions appear to have context- and training-dependent benets22 and there is no clear consensus in the aviation community about the superiority of one or the other. Given that both rely on dierent modes of perspective taking, a better characterization of the underlying neurobiological systems and their performance in situations with high workload and stress, can be highly informative for future instrument design. Additional experiments can be used to dierentiate more clearly the types of mental eort required for the comprehension of a particular display type (e.g., dierence between image retention and spatial transformation19) to allow for an evidence-based design process that minimizes the cognitive workload or potential ambiguities for the operators.

In summary, while we show that there is a substantial overlap of activation in neural networks across both task conditions, we found that there are also signicant dierences in activation patterns between instrument interpretation and non-aviation based mental rotation. In combination with the absent correlation between task accuracy for the two conditions, our results indicate that abstract mental rotation skills might not be a reliable predictor for actual real-life spatio-motor skills of pilots and likely other trained professionals. Regarding our experiment design, we show the applicability of aviation related displays as fMRI stimuli. In addition to inducing important dierences in brain activation, the novel IIT might be useful in future aviation studies due to the higher face validity compared with abstract tests, such as the SMT.

We investigated 18 male professional airline pilots (age: 37 7years, ight hours: 5,500 3,000h). Subjects performed a Shepard-Metzler mental rotation (SMT) and an instrument interpretation task (IIT) presented in a blocked design (block length = 20 s, 5 repetitions, SMT and IIT alternating) (Fig.1), while undergoing functional magnetic resonance imaging (fMRI) at the MR Center of Excellence at the Medical University of Vienna. All subjects were nancially reimbursed for their participation and provided informed written consent. The study protocol was approved by the institutional review board of the Medical University of Vienna. All methods were carried out in accordance with the approved study protocol, guidelines for good scientic practice, and the Declaration of Helsinki (1964) including current revisions.

Subjects were scanned in a 3 Tesla TIM Trio MR scanner (Siemens Medical, Erlangen, Germany) using the manufacturers 32-channel head coil. 270 whole-brain volumes were obtained at a repetition time employing a multiband echo planar imaging (EPI) sequence (TR/TE=1800/33ms, resolution =1.2 1.21.5mm3, matrix size: 12812854slices).

During the Shepard Metzler mental rotation task (SMT), subjects had to judge whether two three-dimensional objects, which were arbitrarily rotated in space, are identical or dierent in structure adapted from1. For the instrument interpretation task (IIT), a generic attitude indicator and a three-dimensional rendering of a simplied airplane as seen from posterior, were shown. Subjects had to indicate by button press if the pitch and roll displayed on the attitude indicator applies to the airplane (Fig.1). The fMRI task was implemented in Python (2.7.6) using pyglet (1.1.4, http://www.pyglet.org/) and PyWavefront (0.1.1, https://github.com/greenmoss/ PyWavefront) modules. To create the three-dimensional cubes and plane model, Blender (2.68, Stichting Blender Foundation, Amsterdam, Netherlands) 3D animation suite soware was used. SMT stimuli were freely rotated along all axes. IIT stimuli were rotated along the principal and secondary axes of the airplane resulting in roll and pitch angles between 2 and 45. In 20% of all trials, the two stimuli did not match: SMT items were mirrored along the y/z-plane and in the IIT the attitude indicator were le-right ipped, thus resulting in a wrong roll angle (e.g., indicating a 15 roll to the right instead of 15 to the le). There was no time limit for the individual task items, except the overall block length of 20s.

Data pre-processing and analyses were performed in SPM12b and comprised slice-timing correction23, realignment, unwarping, normalization to standard MNI template, and smoothing with a Gaussian kernel (6mm FWHM).

SCIENTIFIC REPORTS

5

www.nature.com/scientificreports/

Single-subject GLM analysis comprised two regressors (i.e., mental rotation and instrument interpretation), for which a boxcar function that encoded onset and duration of the respective condition was convolved with SPMs canonical HRF. The resulting contrast maps were used for subsequent group analysis to localize brain activation associated with the respective task conditions (p< 0.05 whole-brain FWE-corrected) and their potential dierences (p< 0.05 cluster-level FWE-corrected, cluster-forming threshold p<0.001 uncorrected).

1. Shepard, R. N. & Metzler, J. Mental rotation of three-dimensional objects. Science 171, 701703 (1971).2. Dror, I. E., Kosslyn, S. M. & Waag, W. L. Visual-Spatial Abilities of Pilots. Journal of Applied Psychology 78, 763773, doi: 10.1037/0021-9010.78.5.763 (1993).3. Endsley, M. R. Toward a Theory of Situation Awareness in Dynamic-Systems. Human Factors 37, 3264, doi: 10.1518/001872095779049543 (1995).

4. Endsley, M. R. Situation awareness: operationally necessary and scientically grounded. Cognition Technology & Work 17, 163167, doi: 10.1007/s10111-015-0323-5 (2015).

5. Newman, D. G. An overview of spatial disorientation as a factor in aviation accidents and incidents, https://atsb.com.au/ media/29971/b20070063.pdf (2007).

6. Hrmann, H. J. In Aviation Psychology: A Science and a Profession 5562 (1998).7. Kozhevnikov, M., Motes, M. A., Rasch, B. & Blajenkova, O. Perspective-taking vs. mental rotation transformations and how they predict spatial navigation performance. Applied Cognitive Psychology 20, 397417, doi: 10.1002/acp.1192 (2006).

8. Verde, P. et al. Gender Eects on Mental Rotation in Pilots vs. Nonpilots. Aviation, Space, and Environmental Medicine 84, 726729, doi: 10.3357/asem.3466.2013 (2013).

9. Wolf, C. C. et al. Sex dierences in parking are aected by biological and social factors. Psychological research 74, 429435, doi: 10.1007/s00426-009-0267-6 (2010).10. Wanzel, K. R., Hamstra, S. J., Anastakis, D. J., Matsumoto, E. D. & Cusimano, M. D. Eect of visual-spatial ability on learning of spatially-complex surgical skills. The Lancet 359, 230231, doi: 10.1016/s0140-6736(02)07441-x (2002).

11. Gibb, R., Ercoline, B. & Schar, L. Spatial disorientation: decades of pilot fatalities. Aviat Space Environ Med 82, 717724 (2011).12. Zacks, J. M. Neuroimaging studies of mental rotation: a meta-analysis and review. J Cogn Neurosci 20, 119, doi: 10.1162/ jocn.2008.20013 (2008).

13. Lamm, C., Windischberger, C., Leodolter, U., Moser, E. & Bauer, H. Evidence for premotor cortex activity during dynamic visuospatial imagery from single-trial functional magnetic resonance imaging and event-related slow cortical potentials. NeuroImage 14, 268283, doi: 10.1006/nimg.2001.0850 (2001).

14. Windischberger, C., Lamm, C., Bauer, H. & Moser, E. Human motor cortex activity during mental rotation. NeuroImage 20,

225232, doi: 10.1016/s1053-8119(03)00235-0 (2003).

15. Lau, E. F., Phillips, C. & Poeppel, D. A cortical network for semantics: (de)constructing the N400. Nature reviews. Neuroscience 9, 920933, doi: 10.1038/nrn2532 (2008).

16. Gauthier, I., Skudlarski, P., Gore, J. C. & Anderson, A. W. Expertise for cars and birds recruits brain areas involved in face recognition. Nat Neurosci 3, 191197, doi: 10.1038/72140 (2000).

17. McGugin, R. W., Gatenby, J. C., Gore, J. C. & Gauthier, I. High-resolution imaging of expertise reveals reliable object selectivity in the fusiform face area related to perceptual performance. Proc Natl Acad Sci USA 109, 1706317068, doi: 10.1073/pnas.1116333109 (2012).

18. Creem-Regehr, S. H., Neil, J. A. & Yeh, H. J. Neural correlates of two imagined egocentric transformations. NeuroImage 35, 916927, doi: 10.1016/j.neuroimage.2006.11.057 (2007).

19. Riecansky, I. et al. Visual image retention does not contribute to modulation of event-related potentials by mental rotation. Brain Cogn 83, 163170, doi: 10.1016/j.bandc.2013.07.011 (2013).

20. Angelaki, D. E., Klier, E. M. & Snyder, L. H. A vestibular sensation: probabilistic approaches to spatial perception. Neuron 64, 448461, doi: 10.1016/j.neuron.2009.11.010 (2009).

21. Stott, J. R. Orientation and disorientation in aviation. Extrem Physiol Med 2, 2, doi: 10.1186/2046-7648-2-2 (2013).22. Lee, B. G. & Myung, R. Attitude Indicator Design and Reference Frame Eects on Unusual Attitude Recoveries. International Journal of Aviation Psychology 23, 6390, doi: 10.1080/10508414.2013.746536 (2013).

23. Sladky, R. et al. Slice-timing effects and their correction in functional MRI. NeuroImage 58, 588594, doi: 10.1016/j. neuroimage.2011.06.078 (2011).

This research is funded as part of the FP7 2012 Aeronautics and Air Transport programme under EC contract ACP2-GA-2012-314765-Man4Gen. The views and opinions expressed in this publication are those of the authors and do not necessarily represent the position and opinions of the Man4Gen consortium or any of the individual partner organizations.

R.S., I.S., E.B., J.F., C.L. and C.W. designed and implemented the experiment. J.-P.B. and D.N. provided the aviation stimulus material. R.S. and I.S. performed the fMRI measurements. R.S., M.T. and A.H. conducted data processing and analyses. R.S. created the gures. R.S., I.S., C.L. and C.W. wrote the initial manuscript. All authors revised and reviewed the manuscript.

Competing nancial interests: The authors declare no competing nancial interests.

How to cite this article: Sladky, R. et al. Neurobiological dierences in mental rotation and instrument interpretation in airline pilots. Sci. Rep. 6, 28104; doi: 10.1038/srep28104 (2016).

This work is licensed under a Creative Commons Attribution 4.0 International License. The images or other third party material in this article are included in the articles Creative Commons license, unless indicated otherwise in the credit line; if the material is not included under the Creative Commons license, users will need to obtain permission from the license holder to reproduce the material. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

SCIENTIFIC REPORTS

6