Disease-Specific Contribution of Pulvinar Dysfunction to Impaired Emotion Recognition in Schizophrenia

Abstract

One important aspect for managing social interactions is the ability to perceive and respond to facial expressions rapidly and accurately. This ability is highly dependent upon intact processing within both cortical and subcortical components of the early visual pathways. Social cognitive deficits, including face emotion recognition (FER) deficits, are characteristic of several neuropsychiatric disorders including schizophrenia (Sz) and autism spectrum disorders (ASD). Here, we investigated potential visual sensory contributions to FER deficits in Sz (n=28, 8/20 female/male; age 21-54 years) and adult ASD (n=20, 4/16 female/male; age 19-43 years) participants compared to neurotypical (n=30, 8/22 female/male; age 19-54 years) controls using task-based fMRI during an implicit static/dynamic FER task. Compared to neurotypical controls, both Sz (d=1.97) and ASD (d=1.13) participants had significantly lower FER scores which interrelated with diminished activation of the superior temporal sulcus (STS). In Sz, STS deficits were predicted by reduced activation of early visual regions (d=.85, p=.002) and of the pulvinar nucleus of the thalamus (d=.44, p=.042), along with impaired cortico-pulvinar interaction. By contrast, ASD participants showed patterns of increased early visual cortical (d=1.03, p=.001) and pulvinar (d=.71, p=.015) activation. Large effect-size structural and histological abnormalities of pulvinar have previously been documented in Sz. Moreover, we have recently demonstrated impaired pulvinar activation to simple visual stimuli in Sz. Here, we provide the first demonstration of a disease-specific contribution of impaired pulvinar activation to social cognitive impairment in Sz.

Introduction

Social cognitive deficits are a core feature of schizophrenia (Sz) (Fernandes et al., 2018) and autism spectrum disorders (ASD) and contribute to impaired functional outcome (Mancuso et al., 2011; Bishop-Fitzpatrick et al., 2017). One important aspect of social functioning is the ability to rapidly and accurately perceive facial expressions. Impaired face-emotion recognition (FER) has been extensively reported in Sz (Edwards et al., 2002; Kohler et al., 2010) and ASD (Harms et al., 2010; Uljarevic and Hamilton, 2013; Tobe et al., 2016) however the underlying neuronal substrates of these deficits are not fully understood and, indeed, may arise from differential underlying neural pathologies (Foss-Feig et al., 2017). Over recent years, the contribution of sensory-processing deficits to cognitive impairments has been increasingly appreciated (Javitt and Freedman, 2015; Koshiyama et al., 2021), including the potential role of dysfunction within subcortical components of the afferent visual streams (Koshiyama et al., 2018; Martinez et al., 2019). Here, we utilize functional magnetic resonance imaging (fMRI) to evaluate the contributions of impaired early sensory processing to FER impairments in Sz. Data were collected as well from both neurotypical and ASD comparison groups to assess the specificity and magnitude of observed activation deficits in Sz.

During normative brain function, FER is supported by activation of specific components of the “social brain”(Adolphs, 2009), which includes structures along both the dorsal and ventral visual-cortical pathways (Allison et al., 2000; Haxby et al., 2000; Pitcher et al., 2011). These pathways receive retinal information from the lateral geniculate nucleus (LGN), which projects to primary visual cortex (V1). The dorsal pathway receives its primary input from the magnocellular geniculostriate pathway and is specialized for rapid detection of low spatial-frequency and motion information. Key dorsal structures include motion-sensitive mid-temporal regions (MT, MST). In Sz, differential deficits in magnocellular processing have been reported and related to potentially impaired patterns of sensory gain and functions of the N-methyl-D-aspartate-type glutamate receptors (NMDAR) (reviewed in Javitt and Freedman, 2015). Moreover, impairments in magnocellular function correlate with behavioral measures of impaired FER in Sz, supporting the involvement of this pathway in social cognition (Martinez et al., 2018, 2019; Marosi et al., 2019). By contrast, the ventral visual stream receives predominant input from the subcortical parvocellular system and is specialized for slower but higher-resolution processing of stimulus details. Key targets of the ventral stream include visual area V4 and the fusiform face complex (FFC).

Along with the ventral and dorsal pathways, the presence of an anatomically and functionally distinct third pathway specialized for social perception and comprising the superior temporal sulcus (STS) region, has recently been proposed (Pitcher and Ungerleider, 2021). The STS region has been reliably associated with processing biological motion signals (Kilts et al., 2003; Sato et al., 2004; Deen et al., 2015) including dynamic social cues such as the changeable aspects of facial features (eyes, lips). Impaired STS activation has been documented in Sz but the basis for the deficit remains unknown (Kim et al., 2011; Mier et al., 2014, 2017; Kronbichler et al., 2017; Matsumoto et al., 2018).

In addition to the cortical system, humans retain an evolutionarily old retinotectal system that mediates non-conscious affective processing via amygdala, superior colliculus and the pulvinar nucleus of the thalamus (PulN) (reviewed in Tamietto and Morrone, 2016). In addition to mediating retinogeniculate input into visual cortex, PulN also mediates cortico-cortical interactions between successive brain regions within the dorsal and ventral stream pathways (e.g., Bridge et al., 2016), and is the site of greatest NMDAR density within primate thalamus (Ibrahim et al., 2000).

PulN is anatomically divided into discrete, functionally differentiated subdivisions (e.g., Bourgeois et al., 2020; Guedj and Vuilleumier, 2020). For example, the “visual pulvinar,” consisting of its inferior (PI) and lateral (PL) subdivisions, has dense connections with early visual sensory regions (Guedj and Vuilleumier, 2020) and likely plays a modulatory role in visual information processing (de Souza et al., 2020). In addition, projections from PI specifically innervate motion sensitive regions surrounding area MT, especially MST (Kaas and Baldwin, 2019), and also serve as drivers to secondary areas of visual cortex (e.g., V2) and as modulators to V1 (de Souza et al., 2020). On the other hand, medial pulvinar (PM) is considered multimodal and is primarily coupled with prefrontal and temporal regions including STS (Homman-Ludiye and Bourne, 2019) and may play a unique role in processing emotional information (reviewed in Arend et al., 2015).

Here, we evaluate whole-brain fMRI activation patterns during FER in Sz, relative to both neurotypical individuals and ASD. Inclusion of ASD participants is based on a previous study involving simple visual stimuli in which we observed a divergent pattern of disturbance within early visual areas and PulN relative to Sz patients, despite similar magnitude of FER impairment (Martinez et al., 2019). We hypothesized that, in Sz, deficits within FER-related higher tier visual regions (e.g., STS) would be driven significantly by impaired activation of both early visual regions (e.g., V1, MST) and PulN as well as by impaired cortico-pulvinar interactions. Moreover, we hypothesized that deficit patterns would be differential across Sz and ASD participants despite similar levels of behavioral impairment, suggesting disorder-specific pathophysiological mechanisms underlying social cognitive impairments in neuropsychiatric populations.

Materials and Methods

Participants

Seventy-eight participants took part, including 28 participants (age range 21–54 years) diagnosed with schizophrenia (Sz) using the Structured Clinical Interview for DSM-IV (First et al., 1994), 20 adults with autism spectrum disorder (ASD) (age range 19–43 years), confirmed by the Autism Diagnostic Observation Schedule, Second Edition, and 30 neurotypical controls (age range 19–54 years) (Table 1). All Sz participants were on a stable dose of antipsychotic medication. All participants had at least 20/22 corrected visual acuity on a Logarithmic Visual Acuity Chart. On average, Sz participants were older [F(1, 56) = 7.24, p = 0.009] and had lower IQ scores [F(1, 56) = 6.54, p = 0.013] than controls. All ASD participants and a subset of 19 Sz and 17 controls participated in our previous EEG/fMRI study of visual sensory dysfunction as reported in Martinez et al. (2019), which did not include data from the present paradigm. Participants were recruited from the central research database and volunteer recruitment pool at the Nathan Kline Institute for Psychiatric Research (NKI). The investigation was approved by the Nathan Kline Institute (NKI) institutional review board. Informed consent was obtained from all participants.

TABLE 1

Table 1. Participant characteristics.

Paradigm

Unique video clips of five actors (three male) dynamically expressing each of four emotions (happy, sad, angry, fearful) were selected from the University of Cambridge Mind Reading Emotions Library (adult level 6) (Golan et al., 2006). Five additional actors (two male) from the NKI community acted a neutral expression consisting of non-emotionally salient head/eye movements (left/right, up/down). Neutral videos were matched in size, resolution and luminance to emotion videos. For each video, representative single frames were extracted and used as corresponding static stimuli. Both dynamic and static stimuli were presented for 2 s each followed by a 400 ms interstimulus interval (ISI). In each of two ∼7.5-min fMRI scans, dynamic and static stimuli of a single emotion or neutral were delivered in 12-s blocks (5 stimuli per block), interleaved with 10-s of fixation-only. A total of ten blocks of static and 10 blocks of dynamic faces were presented in random order per scan (Figure 1A). Across both scans a total of 200 stimuli were delivered, 20 of each of four emotions plus neutral, either dynamic or static. To ascertain that participants were attending the stimuli, participants responded by button press to a single predesignated actor chosen randomly for each participant, irrespective of emotion or motion. The target actor appeared in ∼10% of all stimuli.

FIGURE 1

Figure 1. (A) Schematic of fMRI paradigm. A total of 20 blocks lasting 12 s each were delivered in random order in each of two fMRI scans. Each block consisted of faces expressing a single emotion (happy, sad, angry, fear) or neutral expression either dynamically or statically. (B) FER accuracy determined in the behavioral paradigm administered outside the scanner, for dynamic (filled bars) and static (open) faces in control (CTL; blue), schizophrenia (SZ; orange), and autism (ASD; green). Relative to the CTL group, FER accuracy was significantly lower in the SZ and ASD groups for both dynamic (left bars) and static (right) faces. In SZ patients, FER accuracy for dynamic faces was especially reduced, relative to control participants. (C) Mean FER accuracy as a function of face-emotion and (below) sample stimuli used for happy, sad, angry and fear emotions. FER accuracy did not differ overall as a function of face-emotion and group membership. (D) Groupwise scores on the Penn Emotion Recognition (ER-40) test. As expected, both SZ and ASD participants had significantly lower ER-40 scores compared to the CTL group, which, (E) were correlated with accuracy on the FER task. In all cases, significance of group differences is denoted by asterisks, relative to CTL (*p < 0.05; **p < 0.01;***p < 0.005).

Behavior Measures

A forced-choice behavioral task was administered following the fMRI scan using the same static and dynamic emotional face stimuli (80 stimuli total, 20 of each type; neutral faces were not included). As in the fMRI scan, each static/dynamic stimulus was presented for 2 s. After each presentation, subjects were prompted to press one of five buttons to indicate if the actor’s expression was (1) happy, (2) sad, (3) angry, (4) fearful, or (5) none of the above. Accuracy, as opposed to response time was emphasized. The trial ended when subjects responded. To compare findings from our FER paradigm with those from a validated and reliable measure of FER, the Penn Emotion Recognition test (ER-40) (Taylor and MacDonald, 2012) was also administered to all participants and its results compared to those of the present FER paradigm. The ER-40 uses forty color photographs of faces expressing four basic emotions—happiness, sadness, anger, or fear—plus neutral—with eight photographs for each category, presented in random order. Participants were instructed to choose the correct emotion from among the five listed choices (forced choice) by clicking a computer mouse as quickly as possible without sacrificing accuracy. Each image was displayed until a choice was made.

Functional Imaging

Imaging took place on a Siemens 3T TiM Trio scanner. Two-hundred-twenty T2*-weighted echo-planar images (EPIs) (TR = 2,000 ms; TE = 30 ms; FA = 90°; FOV = 240 mm; slice thickness = 2.8 mm) were acquired on each of 36 contiguous slices in the axial plane. At least one high-resolution structural image of the entire brain was acquired from each participant using an MPRAGE sequence (TR = 2500 ms; TE = 3.5 ms; FOV = 256 mm, slice thickness = 1.0 mm, 192 slices).

Individual cortical surfaces were rendered from the high-resolution anatomical images using Freesurfer and registered to the std 0.141 fsaverage mesh (Fischl et al., 1999) with SUMA.1 The pulvinar and amygdala were derived individually using a Bayesian atlas-based automated segmentation methods (Saygin et al., 2017; Iglesias et al., 2018; Bocchetta et al., 2020) incorporated in Freesurfer. Functional data were preprocessed and analyzed using the Analysis of Functional NeuroImages (AFNI) software (Cox, 1996; Saad and Reynolds, 2012). Preprocessing consisted of concatenating data from two runs, removal of signal deviation >2.5 SDs from the mean (AFNI’s 3dDespike), temporal alignment, identification of motion outliers per run and scaling of blood-oxygen-level-dependent (BOLD) values to mean percent signal change (Taylor et al., 2018). For surface-based analyses, the data was spatially smoothed with a 6 mm full width at half maximum Gaussian kernel. Single-participant statistical analyses were conducted within the framework of the general linear model (GLM). The GLM model included regressors for each stimulus type (emotional dynamic, emotional static, neutral dynamic, neutral static) as well as regressors for the six motion parameters (three rotations, three translations) and their first derivatives, per run. Time points with large head motion between successive time points were censored. Surface-based analyses were carried out on the gray-matter ordinates of each individual cortical surface aligned to the Freesurfer 141-standard mesh. Cortical data was sampled to the Human Connectome Project multimodal cortical parcellation (HCP-MMP1.0) (Glasser et al., 2016), resampled to fsaverage, which delineates 180 brain parcels per hemisphere based on functional and structural properties. To assess activation of pulvinar and amygdala, analyses were conducted in the individual native-space volumes. Primary analyses involved the entire pulvinar. In secondary analyses, beta parameters were extracted from pulvinar subdivisions and tested separately.

To avoid issues related to circularity in data analysis (Kriegeskorte et al., 2009), activated parcels were first identified by an unpaired t-test of mean activation (vs. 0), collapsed over all stimuli, across all seventy-eight participants thresholded at an (uncorrected) p-value of 0.001 (Figure 2A). This analysis defined a mask consisting of 35 bilateral parcels (70 parcels total) (Figure 2B) in the HCP MMP1.0 parcellation atlas which was used in subsequent analyses of functional data. Table 2 lists each parcel.

FIGURE 2

Figure 2. (A) Whole-brain beta parameter maps of activation elicited by all stimuli and across all participants, superimposed on the template brain with borders of HCP parcels demarcated. (B) Thirty-five parcels with significant activation collapsed across subjects and face stimuli.

TABLE 2

Table 2. Mean beta parameter values in each of the 35 HCP-MMP1.0 (Glasser et al., 2016) parcels shown in Figure 2B for the control (CTL), schizophrenia (SZ), and autism (ASD) groups.

General Statistics

Mean beta parameter estimates were extracted from individual parcels and entered into an omnibus repeated measures analysis of variance (ANOVA) collapsing over all parcels identified as having significant across-group activation (n = 35 parcels per hemisphere, Table 2). Diagnostic group (control, Sz, ASD) was included as a between-subject factor. Face-motion type (dynamic, static) and face-emotion type (emotional, neutral) were included as within-subject factors. To minimize concerns regarding multiple comparisons, follow-up tests on individual parcels were conducted only if the initial group X parcel interaction was significant. Pre-planned subcortical regions (PulN, amygdala) were evaluated in separate ANOVAs with factors group, face-motion and face-emotion.

Effect sizes of between-group differences were calculated using Cohen’s d (mean/std dev).

The interrelationship between behavioral measures of FER and cortical/subcortical activation patterns was assessed by analysis of covariance (ANCOVA) with group membership as the categorical factor. Follow up ANCOVAs were conducted in a sequential fashion, for each significant covariate effect obtained in the prior analysis. In each case, only a single ANCOVA was performed for the region being tested as the dependent variable with all remaining covariates. The covariate x group interaction was used to evaluate group differences in the relationship between covariates.

All statistics were two-tailed with preset α level for significance of p < 0.05.

Mediation Analyses

Based on our prior findings (Martinez et al., 2019), we used exploratory linear mediation analyses to explore the role of PulN subdivisions and cortical activation patterns in Sz patients. Analyses were conducted within SPSS262 using the PROCESS macro (Hayes, 2013). A three-variable path model (model 4) was used to examine the predictor-outcome relationship between interrelated regions with impaired activation in Sz (relative to controls) and the potential mediating role of each PulN subdivision (PL, PI, PM). As per standard conventions, the link between the predictor and mediator variable is referred to as path a, and that between the mediator and the outcome (controlling for the predictor), is path b. The overall predictor-outcome relationship is effect c, and the direct effect, after controlling for the mediator is, c′. The indirect (mediation) effect is the product of a*b and tests the significance of c – c′. Statistical significance of indirect pathways, reflecting the impact of mediation, was evaluated using a non−parametric bootstrap approach with 10,000 replication samples to obtain a 95% confidence interval (CI) (Preacher and Hayes, 2008). The mediation effects were considered statistically significant if the bootstrapped 95% CI did not include zero.

Results

Behavior

Overall, FER accuracy was significantly lower in both Sz [F(1, 56) = 15.02, p < 0.001] and ASD participants [F(1, 48) = 7.67, p = 0.009] compared to controls but did not differ between Sz and ASD [F(1, 46) = 0.03, p = 0.863] participants (Figure 1B).

Across groups, FER accuracy was equivalent for dynamic and static faces [F(1, 75) = 0.314, p = 0.576], however, the group x face-motion interaction was significant [F(2, 75) = 8.26, p < 0.001], reflecting relatively greater deficits for processing dynamic faces in Sz patients compared to controls [F(1, 56) = 11.70, p = 0.001; d = 1.4 dynamic faces; d = 0.82 static faces]. Consequently, mean FER accuracy for dynamic vs. static faces was significantly lower in the Sz group [t(54) = −2.65, p = 0.010] but did not differ within either the ASD [t(38) = 1.01, p = 0.317] nor control [t(58) = 0.55, p = 0.584].

FER accuracy did not differ overall as a function of specific face-emotion type [F(6, 146) = 0.84, p = 0.541] (Figure 1C).

As expected, ER-40 scores were also lower in both Sz [F(1, 56) = 36.58, p < 0.001] and ASD [F(1, 48) = 24.57, p < 0.001] subjects compared to control participants (Figure 1D). Further, across subjects, ER-40 scores significantly predicted mean accuracy on the FER task [F(1, 74) = 8.22, p = 0.005; R² = 0.407] (Figure 1E).

Lastly, behavioral performance on the identity recognition task during fMRI was marginally worse in Sz [F(1, 56) = 3.76, p = 0.058] and significantly worse in ASD [F(1, 48) = 4.52, p = 0.039] compared to controls.

Between-Group Functional Magnetic Resonance Imaging Activation Differences

Cortical Surface

An initial omnibus analysis was carried out across all parcels in the mask (listed in Table 2) in order to test the null hypothesis that there were no significant activation differences across groups. The null hypothesis was falsified by the finding of a significant group x parcel interaction [F(68, 80) = 1.78, p = 0.007]. By contrast, the main effect of group membership was non-significant [F(2, 73) = 0.106, p = 0.899]. These findings were interpreted as indicating that activation of some, but not all, parcels differed significantly in activation across groups (Supplementary Table 1).

Follow-up analyses were therefore conducted to determine which parcels most contributed to the significant interaction effect observed in the omnibus test. The goal of these analyses was to identify regions that were most likely to contribute to between-group differences in behavioral task-performance. Therefore, these analyses were not considered to increase family-wise error rates.

In these protected follow-up analyses, significant main effects of group membership were obtained in seven parcels (V1,V2, MST, FFC, TPOJ1, STSdp, FEF) encompassing early visual areas [V1: F(2, 75) = 4.38, p = 0.016; V2: F(2, 75) = 4.49, p = 0.014]; motion-sensitive, medial superior temporal cortex [MST: F(2, 75) = 3.95, p = 0.023], fusiform face complex [FFC: F(2, 75) = 3.62, p = 0.032]; frontal eye fields [FEF: F(2, 75) = 3.63, p = 0.031] and the posterior superior-temporal cortex including the temporo-parietal junction [TPOJ1: F(2, 75) = 6.88, p = 0.002] and the dorsal posterior bank of the superior temporal sulcus [STSdp: F(2, 75) = 3.16, p = 0.048] (Table 2, highlighted parcels; Figure 3A).

FIGURE 3

Figure 3. (A) Localization of key parcels with overall group differences: early visual areas V1 and V2; medial superior temporal cortex (MST); fusiform face complex (FFC); frontal eye fields (FEF); temporo-parietal junction (TPOJ1); and dorsal posterior superior temporal sulcus (STSdp). (B) Mean activation (beta parameter) extracted from each key parcel for the control (CTL; blue bars), schizophrenia (SZ; orange bars), and autism (ASD; green bars) groups. In parcels with a face-motion effect, activation to dynamic (solid bars) and static (open bars) faces are plotted separately. Asterisks indicate a significant difference in SZ or ASD groups relative to CTL. (C) Activation in the STS parcels (TPOJ1, STSdp) showing differential activation patterns for dynamic/static, emotional (Emo.)/neutral (Neut.) faces in STSdp.

When compared independently to the control group, patients with Sz had reduced mean activation in early visual [V1: F(1, 56) = 10.41, p = 0.002, d = 0.85] and motion-sensitive [MST: F(1, 56) = 4.10, p = 0.022, d = 0.76] parcels. ASD participants, in contrast, showed significantly higher activation of early visual regions [V2: F(1, 48) = 12.52, p = 0.001, d = 1.03] as well as reduced activity in prefrontal cortex [FEF: F(1, 48) = 4.24, p = 0.045, d = 0.59] relative to control subjects (Figure 3B).

In parallel with these divergent activation patterns, convergent deficits were observed in the pSTS parcels with reduced mean activation in both Sz [STSdp: F(1, 56) = 5.81, p = 0.019, d = 0.97; TPOJ1: F(1, 56) = 11.83, p = 0.001, d = 0.93] and ASD [STSdp: F(1, 48) = 4.48, p = 0.040, d = 0.86; TPOJ1: F(1, 48) = 9.43, p = 0.004, d = 0.71] participants, compared to control subjects. Across subjects, activation of TPOJ1 was driven strongly by dynamic, relative to static, faces [F(1, 75) = 57.45, p < 0.001] (Figure 3C) with no main effect of emotion. In contrast, activation of STSdp was larger for both emotional (vs. neutral) [F(1, 75) = 9.19, p = 0.003] and dynamic (vs. static) [F(1, 75) = 8.73, p = 0.004] faces. Further, deficits in both clinical groups were differentially greatest in response to dynamic emotional faces [F(2, 75) = 5.28, p = 0.025].

Amygdala

Compared to control subjects, overall activation of the amygdala (collapsed across face-motion and face-emotion) was significantly higher in Sz patients [F(1, 56) = 7.58, p = 0.008, d = 0.83]. Further, the interaction between group-membership and face-emotion was significant [F(1, 56) = 4.93, p = 0.030] reflecting greater group differences for emotional, relative to neutral, faces [F(1, 56) = 4.93, p = 0.030] (Figure 4A).

FIGURE 4

Figure 4. (A) Localization and activation of the amygdala in each participant group, superimposed on an MNI template volume. (B) Localization of the medial (pink; PM), lateral (cyan; PL), and inferior (green, PI) subdivisions of pulvinar. For each group, bar plots (top) are of beta values for emotional and neutral faces extracted from the whole pulvinar and (bottom) of mean betas from each subdivision. *p < 0.05; **p < 0.01; ***p < 0.001.

Compared to controls, activation of the amygdala was equivalent, overall, in ASD participants [F(1, 48) = 1.39, p = 0.244].

Pulvinar

Relative to controls, subcortical activation within PulN was reduced overall in Sz [F(1, 56) = 4.35, p = 0.042, d = 0.44]. This reduction was relatively greater in response to emotional faces as indexed by the significant interaction between face-emotion and group membership [F(1, 56) = 4.96, p = 0.030, d = 0.74] (Figure 4B, top).

In contrast, mean activation of the PulN did not differ, overall, between ASD and control participants [F(1, 48) = 0.260, p = 0.614].

The PulN was segmented into its lateral (PL), inferior (PI) and medial (PM) subdivisions. Across subdivisions, mean activation differed overall as a function of group membership [F(2, 75) = 4.91, p = 0.009]. In PL [F(1, 56) = 9.97, p = 0.003, d = 0.82] and PM [F(1, 56) = 10.35, p = 0.002, d = 0.84], activation was significantly reduced in Sz relative to controls. Neither PL [F(1, 48) = 0.068, p = 0.795] nor PM [F(1, 48) = 0.127, p = 0.723] activation distinguished between ASD and control groups (Figure 4B, bottom). A different pattern was observed in PI where activation was equivalent in Sz [F(1, 56) = 0.43, p = 0.513] compared to controls, but significantly enhanced in ASD [F(1, 48) = 6.42, p = 0.015, d = 0.71], reflecting a disorder-specific, double-dissociation of subnucleus activation patterns in Sz vs. ASD.

Functional Magnetic Resonance Imaging Interrelationships

The interrelationship between FER performance and cortical/subcortical activation patterns was assessed by ANCOVA. The results are summarized schematically in Figure 5A (see also Supplementary Table 2).

FIGURE 5

Figure 5. (A) Interrelationship between activated cortical regions, pulvinar (PulN) activation and face-emotion recognition (FER) accuracy. Black lines denote a significant relationship across all groups. Orange lines denote significant relationship only in SZ group. Dashed green line denotes a significant (negative) correlation in ASD group only. (B) Correlation between FER accuracy and mean activation (beta) of the STSdp parcel. In all cases, black regression lines signify the correlation was significant across groups. (C) Correlation between STSdp-V1 and STSdp-TPOJ1 activation. (D) Correlation between STSdp-V2 activation. In ASD participants, enhanced V2 was associated with reduced STSdp activation. The opposite relationship was obtained in SZ and CTL groups. (E) Scatterplots showing a significant and positive correlation between PulN activation and TPOJ1 (left), STSdp (center) and MST (right) in the SZ group alone.

An initial omnibus analysis tested FER simultaneously against mean activation in all nine fMRI regions identified in the between-group fMRI analyses (seven cortical parcels, amygdala, PulN) (Supplementary Table 2A). This model tested the null hypothesis that no regions significantly predicted FER performance beyond the effect of group membership. The model incorporating these covariates (Adj. R² = 0.49) was statistically superior to a model incorporating group membership alone [Adj. R² = 0.35; F(9, 68) = 3.42, p = 0.0002] (Supplementary Table 2B) indicating that incorporation of these covariates significantly improved model fit.

The statistical contribution of the independent covariates was therefore considered in order to evaluate which regions contributed most to the overall model improvement. As expected, activation of the STSdp parcel most significantly predicted FER scores across participants [F(1, 66) = 16.61, p < 0.001]. Moreover, a model incorporating only STSdp as a covariate showed a model fit (Adj. R² = 0.51; Supplementary Table 2C) similar to that of the more complex model incorporating all covariates. In this simpler model, the relationship between STSdp and FER was highly significant [F(1, 74) = 26.24, p < 0.001]. In all groups, greater activation of STSdp correlated with improved behavioral performance (covaried by age and IQ) (Sz: r_p = 0.46, p = 0.018; ASD: r_p = 0.53, p = 0.020; control: r_p = 0.41, p = 0.029) (Figure 5B). By contrast, no significant correlation was observed between FER and the other covariates in the analysis.

The relationship between the nine fMRI covariates was assessed in follow-up ANCOVAs run in stepwise fashion and including interactions with group membership in the model. As these were not independent tests of the overall null hypothesis, they were not considered to increase family-wise error regarding potential predictors of FER impairments across groups. Rather, the goal was to determine stepwise contributions to impaired STSdp activation, which was shown in the omnibus test to significantly predict FER across groups.

Mean activation of STSdp was predicted across groups by both V1 [F(1, 51) = 6.19, p = 0.016] and TPOJ1 [F(1, 51) = 6.77, p = 0.012] (Figure 5C). In turn, V1 was predicted by PulN activity [F(1, 54) = 8.80, p = 0.004], and TPOJ1 was predicted by MST [F(1, 57) = 18.29, p < 0.001], which also predicted FEF activation [F(1, 61) = 6.63, p = 0.012]. There were no further significant interrelationships across groups.

An interaction involving group membership, indicative of different slopes, was obtained between STSdp and V2 [F(2, 51) = 5.95, p = 0.005] such that in ASD participants enhanced activation of V2 was associated with diminished STSdp activity (r_p = −0.71, p = 0.001) whereas in controls (r_p = 0.42, p = 0.023) and Sz (r_p = 0.44, p = 0.020) there was an inverse relationship (Figure 5D).

Lastly, group-specific interactions involving subcortical (PulN) activity were also obtained. Specifically, in Sz patients only, activation of STSdp [F(2, 51) = 4.51, p = 0.016], TPOJ1 [F(2, 54) = 4.48, p = 0.016], and MST [F(2, 63) = 3.01, p = 0.047] were all significantly and positively associated with PulN activation (Figure 5E).

Mediation Analyses

A more detailed analysis of subcortical activation patterns in Sz was carried out with exploratory mediation analyses involving specific PulN subdivisions and interrelated cortical regions. Based on known anatomical interrelationships within the early visual system (Bridge et al., 2016) as well as the regression analyses described above, three specific predictor-outcome paths were evaluated (V1-STSdp, MST-TPJO1 and TPOJ1-STSdp) with PL, PM, or PI as potential mediators. The results are detailed in Table 3.

TABLE 3

Table 3. Results of mediation analyses testing whether a proposed causal effect of X (predictor) on Y (outcome) may be transmitted through a mediating (M) variable.

Consistent with known anatomical projections of the lateral subdivision, activation of V1 significantly predicted mean PL activity (p = 0.001), which in turn predicted STSdp activation (p = 0.005) (Figure 6A). After controlling for PL activity, V1 was no longer associated with STSdp (p = 0.694), however, using a bootstrapping approach, the (unstandardized) coefficient for the indirect pathway from V1 to STSdp (V1 → PL → STSdp) was significant (CI: [0.08, 0.84]), consistent with full mediation.

FIGURE 6

Figure 6. (A) (Top) Correlations between PL and V1 activation (left Y-axis, solid circles) and between PL and STSdp activation (right Y-axis, open circles) in SZ patients. (Bottom) Schematic of the proposed V1 → PL → STSdp path. Beta coefficients are given for the direct (a: V1-PL; b: PL-STSdp) and indirect (a*b) paths. (B) Correlations and TPOJ1 → PM → STSdp mediated path. (C) Correlations and PI → MST → TPOJ1 path.

In addition, a significant mediating effect of PM was observed between TPOJ1 and STSdp (TPOJ1 → PM → STSdp; CI: [0.002, 1.13]) (Figure 6B) and of MST between PI and TPOJ1 (PI → MST → TPOJ1; CI: [0.13, 0.60]) (Figure 6C). In both cases, after controlling for the proposed mediator the direct paths between were not significant, suggesting full mediation.

Clinical Correlations

No significant correlations were observed between behavioral FER performance or cortical/subcortical activation patterns and medication dose (CPZ equivalents) in Sz patients (p > 0.15 for all). Functional activation strengths did not correlate with measures of general cognitive ability (PSI and IQ) in any group (p > 0.11 for all), however, in Sz (r = 0.378, p = 0.049) and control (r = 0.466, p = 0.044) participants, perceptual organization skill (POI) correlated with performance on the FER task as well as with STSdp activation (control: r = 0.539, p = 0.017; Sz: r = 0.399, p = 0.035).

Discussion

Deficits in social cognition contribute disproportionately to impaired functional outcome across a range of neurocognitive disorders, including Sz and ASD. FER is an important component of these deficits and, in the visual system, depends upon coordinated function of both subcortical and cortical regions for processing of static and dynamic facial features. Here, we investigated cortical and subcortical correlates of FER impairments in adults with Sz and ASD using a dynamic/static FER task that engages motion-sensitive areas as well as traditional face-processing regions. In addition, we investigated the subcortical pathway to cortex involving PulN.

The primary findings of the study relate to the relative involvement of cortico-cortical vs. thalamo-cortical transmission paths underlying impaired FER in Sz. Traditionally, it was assumed that cortical regions showing intercorrelated activity mediate their joint activations primarily through direct cortico-cortical connections (Felleman and Van Essen, 1991; Scannell and Young, 1993). More recent models by contrast propose that connections are mediated primarily by successive loops between cortex and thalamus, with higher-tier thalamic regions such as PulN and dorsomedial nuclei generally interacting with posterior and anterior association regions, respectively (Sherman and Guillery, 1996; Llinas et al., 1998). Within PulN, discrete subnuclei interact with specific visual cortical regions (Bridge et al., 2016). This theory converges with anatomical studies showing reduced PulN volume and cell number in schizophrenia (Byne et al., 2002, 2007; Dorph-Petersen and Lewis, 2017), along with our recent observations of impaired PulN activation to simple visual stimuli in Sz (Martinez et al., 2018, 2019).

In the present study, activation deficits in Sz were observed within the HCP-MMP1.0 (Glasser et al., 2016) parcels comprising lower-tier visual regions including early visual and motion-sensitive cortex, along with higher-tier (multisensory) regions associated with FER.

Within STS two discrete parcels were activated by the task– STSdp and TPOJ1. Activation of the STSdp, in particular, showed uniquely greater activation to dynamic emotional faces which correlated with behavioral measures of FER, in accord with the prominent role of STS in face-emotion assessment (Haist and Anzures, 2017). In Sz, STSdp deficits intercorrelated with impairments in activation of both early visual cortical regions and PulN.

Role of Pulvinar Subdivisions

PulN is divided into discrete anatomical subdivisions which mirror the dorsal/ventral stream distinction of visual cortex (Kaas and Baldwin, 2019) such that the more lateral regions (PL) project predominantly to primary visual cortex and ventral visual stream, whereas a subset of nuclei in the inferior subdivision (PI) project mainly to dorsal stream regions including motion-sensitive cortex (e.g., MST) (Arcaro et al., 2015; Tamietto and Morrone, 2016; Kaas and Baldwin, 2019). The medial subdivision (PM) is primarily connected with multimodal sensory association areas as well as prefrontal and cingulate cortices and has been tied to emotion processing (Homman-Ludiye and Bourne, 2019).

In the present study, in addition to intercorrelated cortical activation deficits, we observed correlations between cortical regions and PulN subnuclei. We therefore conducted a series of mediation analyses to evaluate underlying pathways.

An initial analysis evaluated the potential pathways underlying intercorrelated deficits between V1 and STS in the schizophrenia group. A significant indirect pathway from V1→PL→STSdp was observed, in support of indirect mediation by PL. Consistent with mediation by PM, the intercorrelation between the STS parcels (TPOJ1 and STSdp) was not significant once an indirect path via PM was modeled.

In contrast, PI appeared to mediate its effects via MST. Of note, unlike PL and PM, activation of PI was relatively intact in patients. Given that PI receives much of its driving inputs from neurons in the superior colliculus (Kaas and Baldwin, 2019), this finding is suggestive of unimpaired input via the retinotectal system. By contrast, the observed deficits in V1 are consistent with impaired input via the geniculostriate visual pathway (Martinez et al., 2008, 2019).

Although anatomical abnormalities in PulN are well-documented in schizophrenia (Dorph-Petersen and Lewis, 2017; Huang et al., 2020), the functional consequences of these abnormalities have, to date, remained relatively unexplored. Here, we provide evidence that impairments in visual PulN function significantly undermines visual processing required for effective face processing. In specific, deficits in PL function may mediate effects of impaired V1 activation, which, in turn likely reflects impaired magnocellular input to cortex. In addition, impaired PM activation mediated impaired input from more posterior (TPOJ1) to mid-STS (STSdp) regions, suggesting that accumulating deficits across successive cortico-pulvinar loops may lead to the large effect-sized deficits in FER-related reduced STS activation in Sz.

Comparison to Autism Spectrum Disorders

While deficits in social cognition are a prominent component of Sz, they are not unique to the disorder. In particular, we (Martinez et al., 2019) and others (Couture et al., 2010; Sasson et al., 2016) have reported FER deficits in adult ASD subjects that are as severe as those observed in Sz, despite the much higher level of overall function. Consistent with these prior results, adult ASD participants in the present study showed FER and STS activation deficits that were similar to those of Sz, supporting a role for STS as a common mediator of FER dysfunction across disorders.

Despite the similar STS impairments, ASD participants showed markedly different patterns of disturbance within early visual regions. In contrast to Sz patients, markedly increased responses and an opposite slope of the relationship between V2 and STSdp activation were observed in the ASD group. Activation within other task-activated visual regions, including V1 and MST, was unaffected in ASD, echoing our recent study in which response amplitudes were also normal within V1, but increased in early visual and dorsal visual regions (Martinez et al., 2019). Similar visual hypo/hyper activation patterns in Sz vs. ASD have been observed in both fMRI (reviewed in Samson et al., 2012; Taylor et al., 2012) and electrophysiological (Martinez et al., 2018, 2019; Shah et al., 2018; Kovarski et al., 2019) studies, supporting the concept that dysregulation of the early visual system may undermine later stages of visual processing.

Patterns of amygdalar and subcortical activation also distinguished between ASD and Sz participants. In the amygdala, activation was elevated overall in Sz (but not ASD), as reported previously (reviewed in Dugre et al., 2019), possibly reflecting abnormal salience attribution to neutral stimuli (Holt et al., 2006; Gur et al., 2007; Premkumar et al., 2008), heightened anxiety (Du and Grace, 2016) and/or paranoid ideation (Pinkham et al., 2015).

In contrast, whereas PulN activity was markedly reduced in Sz, activation of the inferior PulN subdivision was significantly elevated in ASD participants, in line with findings from our previous studies (Martinez et al., 2018, 2019) and those of others (Zurcher et al., 2013; Hadjikhani et al., 2017; Martinez et al., 2019). Although the source of the increased activation in ASD is not known, a parsimonious explanation would be hyperactivity of the subcortical retino-collicular pathway, which provides preferential input to PI (Kaas and Baldwin, 2019) and which, in turn, acts like a driver to V2 (de Souza et al., 2020). In humans, this system typically weakens with age as the retinogeniculate system increases in functionality (reviewed in Bourne and Morrone, 2017). Abnormal persistence of this system into adulthood could thus underlie the activation disturbance pattern observed in ASD.

Regardless of underlying pathophysiology, these findings support the concept that dysregulation of the early visual system, whether in the direction of increased or decreased activation, may undermine later stages of visual processing and further highlight the importance of sensory processing abnormalities to the pathophysiology of social cognitive impairment across neuropsychiatric disorders.

Limitations

Despite our differential findings, some limitations must be considered. First, Sz participants were receiving antipsychotic medication which may have impacted measures on brain activity. We did not observe any correlations with medication dose, however, this issue could be best addressed in future studies involving medication-naïve patients. Moreover, we note that the overall sample size remains small, and the findings need to be replicated in an independent sample. Additionally, we did not track fixation locations either in the scanner or during behavior. Thus, we do not know if activation failures relate to inability to process information, or simply from differential facial scanning approaches. Lastly, Sz participants had lower IQ and were older than controls or ASD participants, although correcting for IQ and age did not diminish the findings.

Conclusion

In summary, higher cortical (e.g., STS) contributions to impaired FER have been extensively documented in Sz and ASD (Fernandes et al., 2018), but early visual and subcortical contributions have been evaluated to only a limited degree. Here, we demonstrate significant but opposite abnormalities of circuits centered on PulN in Sz vs. ASD that correlate with impaired STS activation, which in turn correlated across groups with impaired FER. These findings highlight the importance of close integration between subcortical and cortical visual processing pathways and the potential breakdown of this tight coordination in Sz and ASD. Further, the findings reaffirm that similar behavioral deficits (e.g., impairment in social cognition) do not necessarily imply convergent pathophysiological mechanisms, and that physiological measures may be useful for guiding etiological and interventional studies in neuropsychiatry.

Data Availability Statement

The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author.

Ethics Statement

The studies involving human participants were reviewed and approved by the Nathan Kline Institute for Psychiatric Research, Institutional Review Board. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

AM wrote the first draft of the manuscript. PG and DCJ contributed to the conception and designed of the study. RT and GS were involved in subject recruitment and characterization. ED, PS, PL, and GP reviewed and edited drafts of the manuscript. DB analyzed parts of the data. DM conducted the mediation analyses. All authors contributed to the article and approved the submitted version.

Funding

This work was supported by the NIMH Grant MH49334 (DCJ).

Conflict of Interest

DCJ holds equity in Glytech, AASI, and NeuroRx; is part of the scientific advisory board for NRx pharma; and received consultant payments from Concert, Lundbeck, Phytec, Autifony, SK Life Sciences, Biogen, Cadence, Boehringer-Ingelheim, and Pfizer; DCJ has also received research support from Cerevance unrelated to this project.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

We thank the Clinical Research and Evaluation Facility at NKI and all research participants for their contributions.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fnbeh.2021.787383/full#supplementary-material

Footnotes

1. ^ https://afni.nimh.nih.gov

2. ^ https://www.ibm.com/products/spss-statistics

References

Adolphs, R. (2009). The social brain: neural basis of social knowledge. Annu. Rev. Psychol. 60, 693–716. doi: 10.1146/annurev.psych.60.110707.163514

Allison, T., Puce, A., and McCarthy, G. (2000). Social perception from visual cues: role of the STS region. Trends Cogn. Sci. 4, 267–278.

Arcaro, M. J., Pinsk, M. A., and Kastner, S. (2015). The anatomical and functional organization of the human visual pulvinar. J. Neurosci. 35, 9848–9871. doi: 10.1523/JNEUROSCI.1575-14.2015

Arend, I., Henik, A., and Okon-Singer, H. (2015). Dissociating emotion and attention functions in the pulvinar nucleus of the thalamus. Neuropsychology 29, 191–196. doi: 10.1037/neu0000139

Bishop-Fitzpatrick, L., Mazefsky, C. A., Eack, S. M., and Minshew, N. J. (2017). Correlates of social functioning in autism spectrum disorder: the role of social cognition. Res. Autism Spectr. Disord. 35, 25–34. doi: 10.1016/j.rasd.2016.11.013

Bocchetta, M., Iglesias, J. E., Neason, M., Cash, D. M., Warren, J. D., and Rohrer, J. D. (2020). Thalamic nuclei in frontotemporal dementia: mediodorsal nucleus involvement is universal but pulvinar atrophy is unique to C9orf72. Hum. Brain Mapp. 41, 1006–1016. doi: 10.1002/hbm.24856

Bourgeois, A., Guedj, C., Carrera, E., and Vuilleumier, P. (2020). Pulvino-cortical interaction: an integrative role in the control of attention. Neurosci. Biobehav. Rev. 111, 104–113. doi: 10.1016/j.neubiorev.2020.01.005

Bourne, J. A., and Morrone, M. C. (2017). Plasticity of visual pathways and function in the developing brain: is the pulvinar a crucial player? Front Syst. Neurosci. 11:3. doi: 10.3389/fnsys.2017.00003

Bridge, H., Leopold, D. A., and Bourne, J. A. (2016). Adaptive pulvinar circuitry supports visual cognition. Trends Cogn. Sci. 20, 146–157. doi: 10.1016/j.tics.2015.10.003

Byne, W., Buchsbaum, M. S., Mattiace, L. A., Hazlett, E. A., Kemether, E., Elhakem, S. L., et al. (2002). Postmortem assessment of thalamic nuclear volumes in subjects with schizophrenia. Am. J. Psychiatry 159, 59–65. doi: 10.1176/appi.ajp.159.1.59

Byne, W., Fernandes, J., Haroutunian, V., Huacon, D., Kidkardnee, S., Kim, J., et al. (2007). Reduction of right medial pulvinar volume and neuron number in schizophrenia. Schizophr. Res. 90, 71–75. doi: 10.1016/j.schres.2006.10.006

Couture, S. M., Penn, D. L., Losh, M., Adolphs, R., Hurley, R., and Piven, J. (2010). Comparison of social cognitive functioning in schizophrenia and high functioning autism: more convergence than divergence. Psychol. Med. 40, 569–579. doi: 10.1017/S003329170999078X

Cox, R. W. (1996). AFNI - Software for analysis and visualization of functional magnetic resonance neuroimages. Comput. Biomed. Res. 29, 162–173. doi: 10.1006/cbmr.1996.0014

de Souza, B. O. F., Cortes, N., and Casanova, C. (2020). Pulvinar modulates contrast responses in the visual cortex as a function of cortical hierarchy. Cereb. Cortex 30, 1068–1086. doi: 10.1093/cercor/bhz149

Deen, B., Koldewyn, K., Kanwisher, N., and Saxe, R. (2015). Functional organization of social perception and cognition in the superior temporal sulcus. Cereb. Cortex 25, 4596–4609. doi: 10.1093/cercor/bhv111

Dorph-Petersen, K. A., and Lewis, D. A. (2017). Postmortem structural studies of the thalamus in schizophrenia. Schizophr. Res. 180, 28–35. doi: 10.1016/j.schres.2016.08.007

Du, Y., and Grace, A. A. (2016). Amygdala hyperactivity in MAM model of schizophrenia is normalized by peripubertal diazepam administration. Neuropsychopharmacology 41, 2455–2462. doi: 10.1038/npp.2016.42

Dugre, J. R., Bitar, N., Dumais, A., and Potvin, S. (2019). Limbic hyperactivity in response to emotionally neutral stimuli in schizophrenia: a neuroimaging meta-analysis of the hypervigilant mind. Am. J. Psychiatry 176, 1021–1029. doi: 10.1176/appi.ajp.2019.19030247

Edwards, J., Jackson, H. J., and Pattison, P. E. (2002). Emotion recognition via facial expression and affective prosody in schizophrenia: a methodological review. Clin. Psychol. Rev. 22, 789–832. doi: 10.1016/s0272-7358(02)00130-7

Felleman, D. J., and Van Essen, D. C. (1991). Distributed hierarchical processing in the primate cerebral cortex. Cereb Cortex 1, 1–47. doi: 10.1093/cercor/1.1.1-a

Fernandes, J. M., Cajao, R., Lopes, R., Jeronimo, R., and Barahona-Correa, J. B. (2018). Social cognition in schizophrenia and autism spectrum disorders: a systematic review and meta-analysis of direct comparisons. Front Psychiatry 9:504. doi: 10.3389/fpsyt.2018.00504

First, M. B., Spitzer, R. L., Gibbon, M., Williams, J. B. W., and Benjamin, L. (1994). Structured Clinical Interview for DSM-IV Axis II Personality Disorders (Version 2.0). New York: New York State Psychiatric Institute.

Fischl, B., Sereno, M. I., Tootell, R. B., and Dale, A. M. (1999). High-resolution intersubject averaging and a coordinate system for the cortical surface. Hum. Brain Mapp. 8, 272–284. doi: 10.1002/(sici)1097-0193(1999)8:4<272::aid-hbm10>3.0.co;2-4

Foss-Feig, J. H., Adkinson, B. D., Ji, J. L., Yang, G., Srihari, V. H., McPartland, J. C., et al. (2017). Searching for cross-diagnostic convergence: neural mechanisms governing excitation and inhibition balance in schizophrenia and autism spectrum disorders. Biol. Psychiatry 81, 848–861.

Glasser, M. F., Coalson, T. S., Robinson, E. C., Hacker, C. D., Harwell, J., Yacoub, E., et al. (2016). A multi-modal parcellation of human cerebral cortex. Nature 536, 171–178. doi: 10.1038/nature18933

Golan, O., Baron-Cohen, S., and Hill, J. (2006). The Cambridge Mindreading (CAM) Face-Voice Battery: testing complex emotion recognition in adults with and without Asperger syndrome. J. Autism Dev. Disord. 36, 169–183. doi: 10.1007/s10803-005-0057-y

Guedj, C., and Vuilleumier, P. (2020). Functional connectivity fingerprints of the human pulvinar: decoding its role in cognition. Neuroimage 221:117162. doi: 10.1016/j.neuroimage.2020.117162

Gur, R. E., Loughead, J., Kohler, C. G., Elliott, M. A., Lesko, K., Ruparel, K., et al. (2007). Limbic activation associated with misidentification of fearful faces and flat affect in schizophrenia. Arch. Gen. Psychiatry 64, 1356–1366. doi: 10.1001/archpsyc.64.12.1356

Hadjikhani, N., Asberg Johnels, J., Zurcher, N. R., Lassalle, A., Guillon, Q., Hippolyte, L., et al. (2017). Look me in the eyes: constraining gaze in the eye-region provokes abnormally high subcortical activation in autism. Sci. Rep. 7:3163. doi: 10.1038/s41598-017-03378-5

Haist, F., and Anzures, G. (2017). Functional development of the brain’s face-processing system. Wiley Interdiscip. Rev. Cogn. Sci. 8:10.1002/wcs.1423. doi: 10.1002/wcs.1423

Harms, M. B., Martin, A., and Wallace, G. L. (2010). Facial emotion recognition in autism spectrum disorders: a review of behavioral and neuroimaging studies. Neuropsychol. Rev. 20, 290–322. doi: 10.1007/s11065-010-9138-6

Haxby, J. V., Hoffman, E. A., and Gobbini, M. I. (2000). The distributed human neural system for face perception. Trends Cogn. Sci. 4, 223–233. doi: 10.1016/s1364-6613(00)01482-0

Hayes, A. F. (2013). Introduction To Mediation, Moderation, And Conditional Process Analysis : A Regression-Based Approach. New York: The Guilford Press.

Holt, D. J., Kunkel, L., Weiss, A. P., Goff, D. C., Wright, C. I., Shin, L. M., et al. (2006). Increased medial temporal lobe activation during the passive viewing of emotional and neutral facial expressions in schizophrenia. Schizophr. Res. 82, 153–162. doi: 10.1016/j.schres.2005.09.021

Homman-Ludiye, J., and Bourne, J. A. (2019). The medial pulvinar: function, origin and association with neurodevelopmental disorders. J. Anat. 235, 507–520. doi: 10.1111/joa.12932

Huang, A. S., Rogers, B. P., Sheffield, J. M., Jalbrzikowski, M. E., Anticevic, A., Blackford, J. U., et al. (2020). Thalamic nuclei volumes in psychotic disorders and in youths with psychosis spectrum symptoms. Am. J. Psychiatry 177, 1159–1167. doi: 10.1176/appi.ajp.2020.19101099

Ibrahim, H. M., Healy, D. J., Hogg, A. J. Jr., and Meador-Woodruff, J. H. (2000). Nucleus-specific expression of ionotropic glutamate receptor subunit mRNAs and binding sites in primate thalamus. Brain Res. Mol. Brain Res. 79, 1–17. doi: 10.1016/s0169-328x(00)00072-3

Iglesias, J. E., Insausti, R., Lerma-Usabiaga, G., Bocchetta, M., Van Leemput, K., Greve, D. N., et al. (2018). A probabilistic atlas of the human thalamic nuclei combining ex vivo MRI and histology. Neuroimage 183, 314–326. doi: 10.1016/j.neuroimage.2018.08.012

Javitt, D. C., and Freedman, R. (2015). Sensory processing dysfunction in the personal experience and neuronal machinery of schizophrenia. Am. J. Psychiatry 172, 17–31. doi: 10.1176/appi.ajp.2014.13121691

Kaas, J. H., and Baldwin, M. K. L. (2019). The evolution of the pulvinar complex in primates and its role in the dorsal and ventral streams of cortical processing. Vision (Basel) 4:3. doi: 10.3390/vision4010003

Kilts, C. D., Egan, G., Gideon, D. A., Ely, T. D., and Hoffman, J. M. (2003). Dissociable neural pathways are involved in the recognition of emotion in static and dynamic facial expressions. NeuroImage 18, 156–168. doi: 10.1006/nimg.2002.1323

Kim, J., Park, S., and Blake, R. (2011). Perception of biological motion in schizophrenia and healthy individuals: a behavioral and FMRI study. PLoS One 6:e19971. doi: 10.1371/journal.pone.0019971

Kohler, C. G., Walker, J. B., Martin, E. A., Healey, K. M., and Moberg, P. J. (2010). Facial emotion perception in schizophrenia: a meta-analytic review. Schizophr. Bull. 36, 1009–1019. doi: 10.1093/schbul/sbn192

Koshiyama, D., Fukunaga, M., Okada, N., Yamashita, F., Yamamori, H., Yasuda, Y., et al. (2018). Role of subcortical structures on cognitive and social function in schizophrenia. Sci. Rep. 8:1183. doi: 10.1038/s41598-017-18950-2

Koshiyama, D., Thomas, M. L., Miyakoshi, M., Joshi, Y. B., Molina, J. L., Tanaka-Koshiyama, K., et al. (2021). Hierarchical pathways from sensory processing to cognitive, clinical, and functional impairments in schizophrenia. Schizophr. Bull. 47, 373–385. doi: 10.1093/schbul/sbaa116

Kovarski, K., Mennella, R., Wong, S. M., Dunkley, B. T., Taylor, M. J., and Batty, M. (2019). Enhanced early visual responses during implicit emotional faces processing in autism spectrum disorder. J. Autism Dev. Disord. 49, 871–886. doi: 10.1007/s10803-018-3787-3

Kriegeskorte, N., Simmons, W. K., Bellgowan, P. S., and Baker, C. I. (2009). Circular analysis in systems neuroscience: the dangers of double dipping. Nat. Neurosci. 12, 535–540. doi: 10.1038/nn.2303

Kronbichler, L., Tschernegg, M., Martin, A. I., Schurz, M., and Kronbichler, M. (2017). Abnormal brain activation during theory of mind tasks in schizophrenia: a meta-analysis. Schizophr. Bull. 43, 1240–1250. doi: 10.1093/schbul/sbx073

Llinas, R., Ribary, U., Contreras, D., and Pedroarena, C. (1998). The neuronal basis for consciousness. Philos. Trans. R. Soc. Lond. B Biol. Sci. 353, 1841–1849.

Mancuso, F., Horan, W. P., Kern, R. S., and Green, M. F. (2011). Social cognition in psychosis: multidimensional structure, clinical correlates, and relationship with functional outcome. Schizophr. Res. 125, 143–151. doi: 10.1016/j.schres.2010.11.007

Marosi, C., Fodor, Z., and Csukly, G. (2019). From basic perception deficits to facial affect recognition impairments in schizophrenia. Sci. Rep. 9:8958. doi: 10.1038/s41598-019-45231-x

Martinez, A., Gaspar, P. A., Hillyard, S. A., Andersen, S. K., Lopez-Calderon, J., Corcoran, C. M., et al. (2018). Impaired motion processing in schizophrenia and the attenuated psychosis syndrome: etiological and clinical implications. Am. J. Psychiatry 175, 1243–1254. doi: 10.1176/appi.ajp.2018.18010072

Martinez, A., Hillyard, S. A., Dias, E. C., Hagler, D. J. Jr., Butler, P. D., Guilfoyle, D. N., et al. (2008). Magnocellular pathway impairment in schizophrenia: evidence from functional magnetic resonance imaging. J. Neurosci. 28, 7492–7500. doi: 10.1523/JNEUROSCI.1852-08.2008

Martinez, A., Tobe, R., Dias, E. C., Ardekani, B. A., Veenstra-VanderWeele, J., Patel, G., et al. (2019). Differential patterns of visual sensory alteration underlying face emotion recognition impairment and motion perception deficits in schizophrenia and autism spectrum disorder. Biol. Psychiatry 86, 557–567. doi: 10.1016/j.biopsych.2019.05.016

Matsumoto, Y., Takahashi, H., Miyata, J., Sugihara, G., Murai, T., and Takahashi, H. (2018). Neural basis of altered earlier attention and higher order biological motion processing in schizophrenia. Soc. Neurosci. 13, 594–601. doi: 10.1080/17470919.2017.1366363

Mier, D., Eisenacher, S., Rausch, F., Englisch, S., Gerchen, M. F., Zamoscik, V., et al. (2017). Aberrant activity and connectivity of the posterior superior temporal sulcus during social cognition in schizophrenia. Eur. Arch. Psychiatry Clin. Neurosci. 267, 597–610. doi: 10.1007/s00406-016-0737-y

Mier, D., Lis, S., Zygrodnik, K., Sauer, C., Ulferts, J., Gallhofer, B., et al. (2014). Evidence for altered amygdala activation in schizophrenia in an adaptive emotion recognition task. Psychiatry Res. 221, 195–203.

Pinkham, A. E., Liu, P., Lu, H., Kriegsman, M., Simpson, C., and Tamminga, C. (2015). Amygdala hyperactivity at rest in paranoid individuals with schizophrenia. Am. J. Psychiatry 172, 784–792. doi: 10.1176/appi.ajp.2014.14081000

Pitcher, D., Dilks, D. D., Saxe, R. R., Triantafyllou, C., and Kanwisher, N. (2011). Differential selectivity for dynamic versus static information in face-selective cortical regions. Neuroimage 56, 2356–2363.

Pitcher, D., and Ungerleider, L. G. (2021). Evidence for a third visual pathway specialized for social perception. Trends Cogn. Sci. 25, 100–110. doi: 10.1016/j.tics.2020.11.006

Preacher, K. J., and Hayes, A. F. (2008). Asymptotic and resampling strategies for assessing and comparing indirect effects in multiple mediator models. Behav. Res. Methods 40, 879–891.

Premkumar, P., Cooke, M. A., Fannon, D., Peters, E., Michel, T. M., Aasen, I., et al. (2008). Misattribution bias of threat-related facial expressions is related to a longer duration of illness and poor executive function in schizophrenia and schizoaffective disorder. Eur. Psychiatry 23, 14–19. doi: 10.1016/j.eurpsy.2007.10.004

Saad, Z. S., and Reynolds, R. C. (2012). Suma. Neuroimage 62, 768–773.

Samson, F., Mottron, L., Soulieres, I., and Zeffiro, T. A. (2012). Enhanced visual functioning in autism: an ALE meta-analysis. Hum. Brain Mapp. 33, 1553–1581. doi: 10.1002/hbm.21307

Sasson, N. J., Pinkham, A. E., Weittenhiller, L. P., Faso, D. J., and Simpson, C. (2016). Context effects on facial affect recognition in schizophrenia and autism: behavioral and eye-tracking evidence. Schizophr. Bull. 42, 675–683. doi: 10.1093/schbul/sbv176

Sato, W., Kochiyama, T., Yoshikawa, S., Naito, E., and Matsumura, M. (2004). Enhanced neural activity in response to dynamic facial expressions of emotion: an fMRI study. Brain Res. Cogn. Brain Res. 20, 81–91. doi: 10.1016/j.cogbrainres.2004.01.008

Saygin, Z. M., Kliemann, D., Iglesias, J. E., van der Kouwe, A. J. W., Boyd, E., Reuter, M., et al. (2017). High-resolution magnetic resonance imaging reveals nuclei of the human amygdala: manual segmentation to automatic atlas. Neuroimage 155, 370–382. doi: 10.1016/j.neuroimage.2017.04.046

Scannell, J. W., and Young, M. P. (1993). The connectional organization of neural systems in the cat cerebral cortex. Curr. Biol. 3, 191–200. doi: 10.1016/0960-9822(93)90331-h

Shah, D., Knott, V., Baddeley, A., Bowers, H., Wright, N., Labelle, A., et al. (2018). Impairments of emotional face processing in schizophrenia patients: evidence from P100, N170 and P300 ERP components in a sample of auditory hallucinators. Int. J. Psychophysiol. 134, 120–134. doi: 10.1016/j.ijpsycho.2018.10.001

Sherman, S. M., and Guillery, R. W. (1996). Functional organization of thalamocortical relays. J. Neurophysiol. 76, 1367–1395. doi: 10.1152/jn.1996.76.3.1367

Tamietto, M., and Morrone, M. C. (2016). Visual plasticity: blindsight bridges anatomy and function in the visual system. Curr. Biol. 26, R70–R73. doi: 10.1016/j.cub.2015.11.026

Taylor, P. A., Chen, G., Glen, D. R., Rajendra, J. K., Reynolds, R. C., and Cox, R. W. (2018). FMRI processing with AFNI: some comments and corrections on “Exploring the Impact of Analysis Software on Task fMRI Results”. bioRXiv [Preprint]. doi: 10.1101/308643

Taylor, S. F., Kang, J., Brege, I. S., Tso, I. F., Hosanagar, A., and Johnson, T. D. (2012). Meta-analysis of functional neuroimaging studies of emotion perception and experience in schizophrenia. Biol. Psychiatry 71, 136–145. doi: 10.1016/j.biopsych.2011.09.007

Taylor, S. F., and MacDonald, A. W. III. (2012). Brain mapping biomarkers of socio-emotional processing in schizophrenia. Schizophr. Bull. 38, 73–80. doi: 10.1093/schbul/sbr105

Tobe, R. H., Corcoran, C. M., Breland, M., MacKay-Brandt, A., Klim, C., Colcombe, S. J., et al. (2016). Differential profiles in auditory social cognition deficits between adults with autism and schizophrenia spectrum disorders: a preliminary analysis. J. Psychiatr. Res. 79, 21–27. doi: 10.1016/j.jpsychires.2016.04.005

Uljarevic, M., and Hamilton, A. (2013). Recognition of emotions in autism: a formal meta-analysis. J. Autism Dev. Disord. 43, 1517–1526. doi: 10.1007/s10803-012-1695-5

Zurcher, N. R., Donnelly, N., Rogier, O., Russo, B., Hippolyte, L., Hadwin, J., et al. (2013). It’s all in the eyes: subcortical and cortical activation during grotesqueness perception in autism. PLoS One 8:e54313. doi: 10.1371/journal.pone.0054313