Introduction
Sharing the distress of others is central to empathy. fMRI studies show that a number of brain regions involved in the direct experience of pain also increase their activity while participants perceive the pain of others, including the cingulate cortex, the insula, and the somatosensory cortices (Jauniaux et al., 2019; Keysers et al., 2010; Lamm et al., 2011; Timmers et al., 2018). Across humans, primates, and rodents, lesions in these regions impair the perception and the sharing of others’ emotions (Paradiso et al., 2021), providing evidence for their causal contribution to the perception or sharing of the emotions of others. A number of recent studies have used multivoxel pattern analysis to explore how these regions encode the pain of others using fMRI signals, with particular attention to the insula. Krishnan et al., 2016 showed participants’ images of hands or feet in painful or innocuous situations and found a pattern across voxels in the insula that could predict how much pain people reported they would feel in the depicted situations. Corradi-Dell’Acqua et al., 2016 also reported that the pattern of insula activity could discriminate between trials in which a cue signaled that someone else was receiving a shock from nonshock trials. Finally, Zhou et al., 2020 reanalyzed a dataset in which participants viewed photographs of hands in painful or nonpainful situations, or of painful and neutral facial expressions. They found that, in the insula, similar but dissociable patterns supported painfulness decoding for hands and faces: similar in that a pattern trained to discriminate painfulness from faces could do so from hands and vice versa, with the rostral insula contributing to both patterns; but dissociable in that many voxels contributed only to decoding of either faces or hands.
Directly recording electrical signals from these regions in humans would complement these more indirect fMRI measurements and sharpen our understanding of how these regions represent the intensity of other people’s pain for at least two reasons. First, fMRI records a mixed signal that includes synaptic input and local neural processing. Localizing blood-oxygen-level-dependent (BOLD) activity that encodes a particular stimulus property thus cannot ensure that neurons in that region actually have spiking activity that encodes that property (Boynton, 2011). For instance, BOLD signals in V1 fluctuate based on whether a stimulus is perceived or not in binocular rivalry (Boynton, 2011; Maier et al., 2008). In contrast, simultaneous electrical recordings in V1 show that broadband gamma activity, which is tightly coupled to spiking (Bartoli et al., 2019; Buzsáki et al., 2012; Miller et al., 2014), responds to a stimulus equally well whether it is perceived or suppressed. Only the slower components, <20 Hz, that are known to carry feedback synaptic input fluctuate with perception (Maier et al., 2008). Being able to record electrical activity, particularly in the broadband gamma range, would thus be critical to localize where in this circuitry neuronal spiking indeed represents the pain of others. Second, fMRI’s low temporal resolution makes it difficult to characterize the time course of responses.
For the anterior cingulate, we have intracranial recordings: Hutchison et al., 1999 documented a single neuron in epileptic patients that responded to the sight of a finger being pin-pricked with increased firing rate, and a recent rodent study revealed that cingulate neurons responding to pain experience have responses that increase with the intensity of the pain experienced by another rat (Carrillo et al., 2019). In contrast, although the insula is central in the neuroimaging literature on empathy, and shows increases of BOLD signal for watching painful compared to nonpainful social stimuli (Jabbi et al., 2007; Jauniaux et al., 2019; Lamm et al., 2011; Meffert et al., 2013; Singer et al., 2004; Timmers et al., 2018; Wicker et al., 2003), and shows patterns that encode painfulness (Corradi-Dell’Acqua et al., 2016; Krishnan et al., 2016; Zhou et al., 2020), we still lack insular intracranial recordings while individuals witness the pain of others. Intracranial electroencephalography (iEEG) has been recorded in the insula during the self-experience of pain (Bastuji et al., 2018; Bastuji et al., 2016; Liberati et al., 2020), and the insula and adjacent SII are the only cortical regions where iEEG electrode stimulation can induce painful sensations (Jobst et al., 2019; Mazzola et al., 2012), but to our knowledge there are no published studies recording from insular electrodes while patients witness the pain of others. The degree to which neuronal activity local to the insula, as opposed to feedback synaptic input from other regions such as the cingulate, encodes the intensity of other people’s pain therefore remains unclear and the time course of such neural activity remains undercharacterized.
To fill this gap and characterize the electrophysiological responses of the insula to the pain of others, we collected depth electrode recordings from seven epileptic patients during presurgical exploration, while they rated the different intensities of pain they perceived in another person in a video (Figure 1a and b). All these patients had macroelectrodes in their insulae that yielded local field potentials (LFPs) capable of measuring broadband gamma activity (circles in Figure 1c). Three patients, additionally, had microelectrodes at the tip of some macroelectrodes to record from isolated insular neurons (pluses in Figure 1c). Our stimuli also included two ways in which pain is perceived in others (Figure 1a). Half the stimuli (Faces) showed a female receiving electroshocks on the hand and expressing pain through facial expressions (furrowing eyebrows and tightening eyes). The other half (Hands) showed the protagonist’s hand slapped by a leather belt, and pain intensity had to be deduced from the movements of the belt and the hand. In both cases, movies, rather than static images, were chosen to provide richer and more ecological stimuli and provide information about the temporal dynamics with which such movies are represented in a field still dominated by the presentation of static images (Adolphs et al., 2003; Zinchenko et al., 2018). We used these two classes of stimuli because both tap into the visual perception of other people’s pain, and we start to understand that they do so through partially overlapping and partially dissociable routes (Jauniaux et al., 2019; Keysers et al., 2010; Timmers et al., 2018). For instance, the Hand stimuli depend on the hand region of the somatosensory cortex (Gallo et al., 2018), while facial expressions depend on both the ventral somatosensory cortex and the insula (Adolphs et al., 2000; Dal Monte et al., 2013; Mattavelli et al., 2019), whereby the insula appears to show partially overlapping and partially discriminable patterns encoding painfulness for these two types of stimuli (Zhou et al., 2020).
Figure 1.
Experimental design, recording site locations, and behavioral pain ratings.
(a) Frames extracted from a Hand and a Face movie. For the Face, the first second of each movie showed a neutral facial expression, the second, the facial reaction to the shock. For the Hand, the movie started with the belt resting on the hand. The first second showed the belt lifting and coming down again, to hit the hand at the 1 s mark exactly. The hand then reacted to the force of the belt. Both the slap and the shock delivery happened in the middle of the movies, splitting them into a 1 s neutral and a 1 s pain period. (b) Single-trial structure diagram. After the presentation of each video, patients expressed their choice at their pace using the keyboard keys f, g, j, and k for pain intensities 1–2, 3–4, 5–6, and 7–8, respectively. ITI started with participant’s response. (c) Position (i.e., the midpoint between two adjacent electrodes) of the 85 bipolar macroelectrode recording sites shown as dots and of the microelectrode locations shown as pluses, color-coded by patient. Data from the two hemispheres and all lateromedial coordinates are projected here onto a single sagittal slice of the insula taken at X = 38 from the brain of one of the patients. For a list of all MNI coordinates, see Figure 1—source data 1. (d) Graphical illustration of how a bipolar recording for one patient and one insular electrode was computed. In green, the CT, and in gray, the T1 scan from patient C. The annular structures along the electrode shaft in the CT correspond to individual macroelectrode contacts (green 1, 2, 3 …). Recordings from adjacent pairs of contacts along the electrode were subtracted to calculate bipolar recordings (white 1–2, 2–3 …). (e) From left to right, Spearman’s correlation coefficient
Our aim is to localize insular recording sites that do or do not encode the perceived intensity of others’ experiences from Face and/or Hand stimuli and to exploit the temporal resolution of iEEG to explore the timing of intensity coding and how it relates to the timing of intensity-relevant stimulus features. To provide evidence that a specific recording site does not encode intensity for a stimulus type, we supplement frequentist statistics with Bayesian statistics, which generate a Bayes factor (BF10; Keysers et al., 2020). The Bayes factor indicates the probability of the data if there is intensity encoding (H1) divided by that if there isn’t (H0). A BF10 < ⅓, which indicates the data is at least three times more likely under H0 than H1, is considered evidence for the absence of intensity coding. Importantly, in Bayesian statistics, whenever possible, it is advised to commit to a directional hypothesis (H+) that endows the BF+0 with higher sensitivity to detect evidence for both H+ and H0 (Keysers et al., 2020; Wagenmakers et al., 2016). Given that fMRI experiments overwhelmingly report voxels with
Results
The pain intensity ratings of patients were within the normal range
To assess whether the behavior of the patients was representative of the general population, we compared patients’ (three males, four females, 34.3 years ± 9 SD) ratings with those of 93 healthy volunteers (54 females, 32.7 years ± 9 SD, Table 4), who took part in an online version of the video pain rating task. Table 1 shows the distribution of ratings separately for patients and on average for controls. We calculated three metrics of similarity between the ratings of the patients and the control group: the Spearman’s rank-order correlation, the slope, and the intercept of a simple linear regression between each patient’s ratings and the average ratings of the control sample (Figure 1e). The patients revealed correlation coefficients, slopes, and intercepts (green and purple bars) within the 2.5 and 97.5 percentiles of the corresponding control sample distributions (gray bars), except for one correlation coefficient for Faces, where a patient rated the Faces with unusually high concordance with the average of the control groups. This verified that these seven patients were not impaired in their ability to rate intensity from our videos. In both the patient and the control sample, we observed that correlation coefficients for Faces were significantly greater than for Hands (patient:
Table 1.
Pain ratings in patients and controls.
Left: the percentage of trials (out of the 60 Hand and 60 Face trials for patients, or 30 and 30 for controls) per rating per participant for the Hand and Face conditions. For the age- and gender-matched control group, only the average across the 93 controls is shown. Middle: mean (M) rating for the Hand or Face. Our patients reported slightly higher pain intensity ratings for our Hand than Face stimuli (
| Rating | M (rating) | SD (rating) | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Hand | Face | Hand | Face | Hand | Face | ||||||||
| 1–2 | 3–4 | 5–6 | 7–8 | 1–2 | 3–4 | 5–6 | 7–8 | ||||||
| Patient | A | 1.67 | 38.33 | 43.33 | 16.67 | 40.00 | 28.33 | 25.00 | 6.67 | 2.75 | 1.98 | 0.75 | 0.97 |
| B | 0.00 | 16.67 | 50.00 | 33.33 | 36.67 | 35.00 | 13.33 | 15.00 | 3.17 | 2.07 | 0.69 | 1.06 | |
| C | 0.00 | 0.00 | 43.33 | 56.67 | 56.67 | 43.33 | 0.00 | 0.00 | 3.57 | 1.43 | 0.50 | 0.50 | |
| D | 1.67 | 38.33 | 38.33 | 21.67 | 35.00 | 41.67 | 23.33 | 0.00 | 2.80 | 1.88 | 0.80 | 0.76 | |
| E | 28.33 | 28.33 | 31.67 | 11.67 | 35.00 | 20.00 | 28.33 | 16.67 | 2.27 | 2.27 | 1.01 | 1.12 | |
| F | 25.00 | 40.00 | 25.00 | 10.00 | 38.33 | 31.67 | 25.00 | 5.00 | 2.20 | 1.97 | 0.94 | 0.92 | |
| G | 38.33 | 23.33 | 26.67 | 11.67 | 36.67 | 28.33 | 25.00 | 10.00 | 2.12 | 2.08 | 1.06 | 1.01 | |
| Mean ± SEM | 13.57 | 26.43 | 36.90 | 23.10 | 39.76 | 32.62 | 20.00 | 7.62 | 2.70 | 1.95 | 0.82 | 0.91 | |
| Control | Mean ± SEM | 33.01 | 38.89 | 21.25 | 6.85 | 44.55 | 32.76 | 16.99 | 5.70 | 2.02 | 1.84 | 0.74 | 0.81 |
LFP activity in the insula correlates with the perceived intensity of the pain of others
Table 2.
Post-hoc comparisons of Figure 2f.
To follow up on the repeated-measures ANOVA (rmANOVA) on the 1 s broadband power (BBP) with factors period (neutral, pain) × rating (1–2, 3–4, 5–6, 7–8), the table reports, for each contrast of interest indicated over the first two left columns: the average (SEM) of % power change, the
| Period | Pain rating | % power change |
|
| p2 | BF10 |
|---|---|---|---|---|---|---|
| Neutral vs. pain period | 1–2 | –1.34 (0.35) vs. –1.16 (0.56) | 1903 | 0.742 | 0.13 | |
| 3–4 | –1.43 (0.32) vs. –0.92 (0.55) | 1801 | 0.909 | 0.15 | ||
| 5–6 | –1.48 (0.35) vs. 0.36 (0.67) | 3.42 | 0.001 | 24.31 | ||
| 7–8 | 0.84 (0.67) vs. 6.07 (1.05) | 7.29 | 2 × 10–10 | 6 × 107 | ||
| Pain period | 1–2 vs. 3–4 | –1.16 (0.56) vs. –0.92 (0.55) | 1966 | 0.545 | 0.12 | |
| 3–4 vs. 5–6 | –0.92 (0.55) vs. 0.36 (0.67) | 1065 | 8 × 10–4 | 15.21 | ||
| 5–6 vs. 7–8 | 0.36 (0.67) vs. 6.07 (1.05) | 309 | 3 × 10–11 | 950,944 |
Table 3.
Post-hoc comparisons of Figure 2h.
To follow up on the two significant stimulus (Hand, Face) × rating repeated-measures ANOVAs (rmANOVAs), one for the early, one for the late period, the table reports, for each contrast of interest indicated over the first three left columns: the average (SEM) of % power change, the
| Period | Stimulus | Pain rating | % power change |
|
| p2 | BF10 |
|---|---|---|---|---|---|---|---|
| Early period | Hand | 1–2 vs. 3–4 | 2.11 (1.59) vs. –3.16 (0.94) | 1014 | 3 × 10–6 | 847.14 | |
| 3–4 vs. 5–6 | –3.16 (0.94) vs. 2.44 (1.03) | 5.97 | 3 × 10–7 | 51,110 | |||
| 5–6 vs. 7–8 | 2.44 (1.03) vs. 8.32 (1.61) | 188 | 2 × 10–5 | 764.63 | |||
| Face | 1–2 vs. 3–4 | –1.81 (0.6) vs. –1.75 (0.6) | 0.1 | 0.92 | 0.16 | ||
| 3–4 vs. 5–6 | –1.75 (0.6) vs. –1.54 (0.95) | 0.25 | 0.803 | 0.16 | |||
| 5–6 vs. 7–8 | –1.54 (0.95) vs. –1.83 (1.35) | 0.23 | 0.817 | 0.16 | |||
| Late period | Hand and Face | 1–2 vs. 3–4 | –0.76 (1.01) vs. –0.73 (0.84) | 597 | 0.931 | 0.16 | |
| 3–4 vs. 5–6 | –0.73 (0.84) vs. 1.96 (1.13) | 3.46 | 0.001 | 25.15 | |||
| 5–6 vs. 7–8 | 1.96 (1.13) vs. 4.75 (1.44) | 2.9 | 0.006 | 6.29 |
Correlating power with reported pain intensity, irrespectively of whether Hand or Face videos were shown, revealed a cluster of positive correlations ranging from 20 to 190 Hz and 1.12–1.62 s (p1<0.001; p1=one-tailed p-value), another cluster of positive correlations at very low frequencies (1–6 Hz, 0.02–2.06 s; p1<0.001), and a small cluster of negative correlations (13–17 Hz, 1.30–1.83 s; p1=0.004, not further discussed; Figure 2a, Figure 2—figure supplement 1a). Intensity coding was apparent in all traditional frequency ranges, except alpha (Figure 2b), and, as expected, was significant in the pain period. With no obvious differences among frequency bands above alpha, we henceforth used the frequency band 20–190 Hz for all analyses and refer to it as broadband power (BBP). We concentrate on BBP rather than oscillatory signals in lower frequencies because BBP is more closely linked to neural spiking (Bartoli et al., 2019; Buzsáki et al., 2012; Miller et al., 2014), cannot be explored in noninvasive EEG recordings, and is the frequency range that can supplement the information available for the substantial fMRI literature (Boynton, 2011; Maier et al., 2008). The temporal profile of the BBP–rating association revealed two periods with significant positive correlations: 1.14–1.54 s and 1.74–1.96 s (Figure 2b). Averaging BBP over the entire pain period revealed that, out of 85 macrocontacts within the insula, 27 (32%) showed a significant positive correlation (assessed as p1<0.05, Figure 2c) between perceived intensity and BBP (n = 120 trials, all
Figure 2.
Intensity coding in the insula local field potential (LFP) activity for Hands and Faces together.
(a) For each frequency and time relative to stimulus onset, the average
Figure 2—figure supplement 1.
Time–frequency decomposition as a function of intensity rating.
(a) Mean percent power changes relative to the baseline period (1 s before the onset of videos) over all 85 insular bipolar recordings as a function of time and frequency for all trials rated 1–2 (first column), 3–4 (second column), 5–6 (third column), and 7–8 (fourth column) combining Hand and Face stimuli. (b) Same as (a), but separately for the Hand (top row) and Face (bottom row) trials.
Figure 2—figure supplement 2.
Glass brain representation of all macrocontacts available in the seven patients.
Patients are color-coded as in Figure 1c and Figure 1—source data 1. The analysis depicted in Figure 2c was performed by randomly sampling 85 electrodes 100,000 times from these macrocontacts.
Figure 2—figure supplement 3.
Overview of the anatomical distribution of all macrocontacts available in the seven patients.
For each electrode, the region it belongs to was determined based on the Faillenot et al., 2017 atlas by choosing the region with the highest probability of occurrence within 3 mm from the center of the selected electrode. The pie chart simply illustrates the total number of contacts falling within a particular lobe. The complete list can be found at here.
Figure 2—figure supplement 4.
Broadband power (BBP) time course as a function of rating and stimulus.
Mean ± SEM time course of percent power change from baseline in BBP (20–190 Hz) over all insular bipolar recordings plotted separately for trials rated 1–2, 3–4, 5–6, and 7–8, and for the Hand and Face conditions.
Intensity coding arises earlier for Hands than Faces
To investigate how intensity coding depends on the stimulus, we focused on the BBP range of interest (20–190 Hz), identified independently of stimulus type (Figure 2a), and found significant intensity coding for the Hand from 1.01 to 1.44 s (hereafter called early period) and for the Face from 1.75 to 1.86 s and from 1.91 to 1.98 s (jointly called late period, Figure 2g). The insula thus reflects, in broadband activity, the perceived intensity with differential time courses for the Hand and Face videos in this study.
To explore the shape of the BBP–rating relation, we averaged BBP over time for the early and the late periods for each pain rating separately (Figure 2h). For the early period, a stimulus (Hand, Face) × rating rmANOVA revealed a significant interaction (Greenhouse×Geisser-corrected
The timing of shape information matches that of Face intensity coding
Having observed differences in the temporal profiles of intensity coding for Hands and Faces, we next assessed whether these differences could arise from the timing of different intensity cues depicted in the two video types. We first subjected our stimuli to more detailed, time-resolved analyses to describe the temporal evolution of the motion information in the Hand videos, and the motion and the shape information in the Face videos. Motion information for both Hands and Faces was quantified based on pixel-based intensity changes across consecutive frame pairs. Shape information for Faces was estimated using an automated face analysis software to extract the two most reliable shape features of painful facial expressions: how lowered the eyebrows and how tightened the eyelids are (facial action units [AUs] 4 and 7, respectively, Figure 3a, Kunz et al., 2019). Appendix 1 indicates that for Faces static shape information was sufficient to explain video ratings, while for Hands, the shape information was not.
Figure 3.
Temporal dynamics of pain rating and intensity coding in the insula broadband activity.
(a–c) Motion and shape signals as a function of time and perceived intensity for the Face and Hand videos rated as 1–2, 3–4, 5–6, and 7–8 separately. Each colored curve represents the mean ± SEM for each rating. Purple and green bars indicate the periods with significant broadband power (BBP)–rating correlations for Faces and Hands, respectively (as in Figure 2g). Black lines represent the partial least-square regression (PLSR) beta coefficients predicting perceived intensity ratings using motion (for Hand and Faces) or shape information (for Faces). The white transparent circles over the inlet figure in (a) shows the action units (AUs) 4 and 7 that were used to estimate the intensity of the shape information in Face videos. (d–f) Accuracy with which the motion or shape signal across all frames can be used to predict the intensity rating of the movie. The histogram shows the actual predictive accuracy averaged over cross-validation folds (green and purple) relative to the null distribution of shuffled ratings (gray), with the median and top 5% of the null distribution shown as full and dashed line. In all cases, the actual accuracy was higher than all 10,000 shuffling values, as indicated by p<10–4. (g–i) Mean ± SEM time courses of the correlations between BBP and pain ratings (green and purple, as in Figure 2g) superimposed with black lines from (a–c) for visualization of the temporal similarity between the two curves. (j–l) Mean ± SEM lagged correlation (left) and partial correlation coefficients (middle and right) between the temporal profile of BBP–rating correlations and that of the PLSR beta coefficients for the corresponding stimulus information. For partial correlation analyses, middle panel shows
Leveraging the high temporal resolution of our iEEG recordings, we next asked whether the motion or the shape information better matches the timing of our intensity coding for Faces in the insula. Figure 3a shows shape information increases toward the end of the movies with rating intensity. Comparing the timing of intensity coding for the Face in the insula BBP (purple bar in Figures 2g and 3a) with the timing of the shape information for Faces (separation between the curves in Figure 3a) shows a nice correspondence, with both BBP and shape information being highest toward the end of the movie. Furthermore, a partial least-squares regression (PLSR) analysis indicated that the time course of shape information could predict the rating of our patients with very high accuracy (Figure 3d). Regarding kinematics, we calculated the changes in pixel values across consecutive frames to track the timing of motion (Figure 3b), and this information could also predict the rating of our patients with high accuracy for Faces (Figure 3e). Comparing the timing of intensity coding in the insula for Faces (purple bar in Figure 2g) with the timing of motion information (separation between the curves in Figure 3b) shows that intensity coding maximizes when motion information has already declined significantly.
We complemented these observations with a quantitative approach that estimates how the neural intensity coding lags behind the shape or motion information (Figure 3g, h, j and k). If motion were, for instance, the driver of neural response to Face stimuli, we would expect that when motion information increases, neural responses should start increasing within ~200 ms, given typical latencies in the insula for facial expressions (Chen et al., 2009; Cornwell et al., 2008; Krolak-Salmon et al., 2003; Meeren et al., 2013) or other painful or nonpainful sensory stimulations (Bastuji et al., 2018; Bastuji et al., 2018; Kobayakawa et al., 1996; Liberati et al., 2020; Liberati et al., 2016; Taniguchi et al., 2022). Thus, we conducted correlation analyses to test how much the temporal profiles of shape or motion information are associated with the temporal profiles of intensity coding at various lags. Note that to directly contrast the predictive power of the shape and motion information, we used partial correlations. For Faces, partial correlations were positive for shape information in time windows centered at 40–320 ms and for motion at 560–1000 ms lags (Figure 3j and k). As discussed in more detail in the section ‘Discussion,’ given typical insular response onset latencies, intensity coding for Faces in the insula is more likely to be primarily triggered by shape information.
Rating-related motion information could drive Hand intensity coding
Motion energy is also a reliable predictor of pain intensity ratings for Hands (Figure 3f). In our Hand videos, motion occurs at two time points: early, when the belt is lifted up, and then again, around the moment when the belt hits the hand (Figure 3c). Pain, however, occurs only at the second point in time. This allows us to explore whether the insula is coding movement in general, or movement that is associated with pain more specifically. We thus divided the 2 s of the Hand movies into six segments and asked, for each segment, how well BBP relates to motion energy in the same segment (Figure 3m and n). Over the 85 channels, we had evidence of absence for a relationship during the neutral period that contained the period during which the belt was seen to move upward (all BF10 < 1/3), and evidence for a relationship during the first 666 ms of the pain period when the belt is seen to slap the hand (both punc<0.003 or pFDR<0.018 corrected for six bins, both BF10 > 8.8). Indeed an rmANOVA comparing the correlation values across the six bins confirms that the relationship between motion energy and BBP changes as a function of time (
We also conducted the same lagged correlation analysis for Hands, as described for Faces above: that is, calculating the correlation coefficients between the temporal profile of the motion information and the temporal profile of intensity coding for Hands at various lags (Figure 3i and l). This analysis showed that intensity coding in the insula is best associated with motion information in Hands in time windows with center points that lag behind by 0–80ms (Figure 3l).
The insula contains a surprising number of intensity coding locations with stimulus preference
We next focused on how individual recording sites in the insula reflected perceived intensity. In the early period, for Hands, 21/85 (25%) showed significant intensity coding (rating–BBP correlations, n = 60 trials, all
Figure 4.
The relationship between Hand and Face intensity coding in the insula broadband activity.
(a) Topographical maps of broadband power (BBP)–rating correlation coefficients for Hands and Faces in the early and late periods. Each circle is one of the recording sites (as in Figure 1c), with filled circles indicating locations with significant correlation coefficients (p1<0.05). (b) Classification of recording locations based on their Hand (early period) and Face (late period) intensity coding. Bipolar recordings in the gray zone (n = 5) significantly co-represent intensity for Hands and Faces (both p1<0.05, i.e., beyond dashed line). Recordings in the purple (n = 6) and green (n = 10) zone represent intensity coding preference for Faces or Hands, respectively (i.e., p1<0.05 for Hands and BF+0 < ⅓ for Faces, and vice versa). (c) Location of all 85 bipolar recordings, color-coded by stimulus preference as described in (b). Note that locations Hand and Face without further specification are those with
If the insula simply represents salience, one might expect a tight association between intensity coding for Hands and Faces, and an above-chance number of locations showing dual-intensity coding for both Faces and Hands. In contrast, if the insula also represents more specific information, we would expect above-chance numbers of locations with intensity coding for Faces, but not Hands and vice versa. Statistically, we infer the presence of intensity coding based on
In addition, we used a multivariate partial least-square regression (PLSR) approach to assess how well the pattern of BBP across the insula can predict participants’ pain ratings. BBP across the 85 sites in the early period for Hands can be used to predict the patients’ average rating of the stimulus with reasonably high accuracy (n = 10 trials since 1/3 of the 30 unique videos were used for testing decoding performance for each randomization,
Intensity coding for Hands increases anteriorly as in a similar fMRI experiment
To examine the spatial distribution of intensity coding, we examined the relationship between MNI coordinates of the bipolar recordings and intensity coding (i.e.,
Figure 5.
The relationship between the insula broadband and blood-oxygen-level-dependent (BOLD) activity during pain intensity ratings.
(a) Correlations (
To better understand the origin of the anterior gradient for intensity coding for Hands, we performed a regression analysis between intensity coding of the 85 insular recording locations (for Hands and Faces separately) and resting state connectivity seeded at corresponding MNI locations in Neurosynth (Yarkoni et al., 2011). Insular locations with higher Hand intensity coding had higher resting state connectivity with the left anterior insula and ventral prefrontal cortex (including BA44/45, OP8/9, Fp1), with the right frontal orbital cortex; with the bilateral cingulate (including BA24/33); and the right cerebellum (Crus I and lobules VI, VII, and VIII, Figure 5b, Figure 5—source data 1). In line with the lack of spatial gradients for Faces in the insula of our patients, examining which voxels had higher resting state connectivity with insular locations with higher Face intensity coding did not yield any significant voxels (all punc>0.001).
Finally, we leveraged existing unpublished fMRI data from our lab to test whether the spatial gradient observed in the iEEG data resembles the spatial distribution of BOLD signals in the insula correlating with intensity ratings. Twenty-three independent healthy volunteers participated in the fMRI-adapted version of the rating task. As for the iEEG experiment, participants were asked to rate Hand and Face 2 s videos on painfulness. The stimuli depicted the same actor as in the iEEG experiment and were made following the same procedure. To test whether the pattern of BOLD activity in the insula can be used to predict the ratings of the participants, we defined eight separate regressors in each participant, capturing all trials that participants rated with intensity 0–2, 3–4, 5–6, or 7–8 separately for Hand and Face trials. We then performed a PLSR with either two or three components using all voxels in the insula to predict the intensity rating. A leave-one-out cross-validation was used, and the correlation between the predicted and actual rating for each left-out participant was compared against zero. This confirmed that BOLD activity in the insula can be used to predict the perceived intensity for Hands (one-sample
The insula contains neurons with intensity coding for Hands and/or Faces
The insula thus displays intensity coding in a broad frequency range, including locations with Hand-but-not-Face or Face-but-not-Hand intensity coding, as well as locations showing intensity coding for both stimulus types. To explore this representation at the level of single neurons, we analyzed the microelectrode data from the three patients (patients C, D, and E) that had microwires in the ventral anterior insula (pluses in Figure 1c). Spike sorting resulted in a total of 28 candidate neurons. From these, 13 showed more spikes during the pain period than the pre-stimulus baseline. Among those, eight show intensity coding for Faces and/or Hands (Figure 6), with significant Kendall’s tau correlations between perceived intensity (1–2, 3–4, 5–6, 7–8) and spike count during the pain period (1–2 s post-stimulus onset) for at least one stimulus type: 4/8 for Faces and 5/8 for Hands (binomial test, Face: p1=0.003, BF+0 = 27; Hands: p1=3 × 10–4, BF+0 = 282). Considering the p1-value for the intensity coding, two cells (a, b) showed intensity coding for both Hands and Faces, three (c–e) only for Hands, and three (f–h) only for Faces. If we additionally consider the BF+0 values below ⅓ as evidence for the absence of coding in the other stimulus type, we find three Hand-but-not-Face coding cells (c–e) and two Face-but-not-Hand coding cells (g, h). Importantly, within patient D, we observe the coexistence of Hand-but-not-Face (c, d) and Face-but-not-Hand intensity coding (g).
Figure 6.
Intensity coding in the insula single units and the corresponding broadband activity.
(a–h) Left (Face) and right (Hand) columns display, for each single unit, the rastergrams and peri-stimulus time histograms (PSTH) for eight cells that showed intensity coding for at least one stimulus type. For the PSTH, each curve represents the mean ± SEM of the firing rate in each bin for trials with the corresponding rating. Not all patients gave all possible ratings in each condition. For the rastergram, trials are sorted in order of rating, with the highest ratings upmost. The color bar next to the rastergram indicates the rows corresponding to each rating. p1 and BF+0 values result from a one-tailed test of the Kendall’s tau between rating and spike count in the pain period (marked by the dashed lines). *Significant intensity coding (p1<0.05), t: trend (p1<0.1), X: evidence of absence for a positive intensity coding (BF+0 < 1/3). The x-axis (time) is relative to the movie onset. The color bar on the leftmost side indicates from which patient the data is taken. Middle columns show the broadband power (BBP) averaged over the pain period for the microelectrode (from which the corresponding single unit was extracted) as a function of rating where boxplots show variance across trials. The box and whiskers represent the quartiles across trials, and the p1 and BF+0, the Kendall’s tau test of the association of rating and BBP. Note that cells c and d were taken from the same microwire, and therefore have only one BBP graph.
To explore how spiking relates to BBP, we analyzed the BBP from the 10 microelectrodes that yielded the 13 cells showing stimulus triggered responses. Using Kendall’s tau correlations between BBP (20–190 Hz) and patients’ intensity ratings (1–2, 3–4, 5–6, 7–8) and comparing these results with the coding of the cells on the same wire reveals a relationship between the two. For Hands, 2/3 microelectrodes that yielded cells with intensity coding also showed significant association between ratings and BBP (Figure 6a, c and d). Indeed, intensity coding (i.e., correlation between intensity rating and spiking/BBP) was significantly correlated across the 10 microwires (
Discussion
Here we characterize how the insula’s iEEG activity encodes the intensity of other people’s emotions using pain as an important category. LFPs indicate that neural activity in the insula within a broad range of frequencies, including the conventional theta, beta, and gamma frequency bands, scales with the perceived intensity of pain expressed by others. Interestingly, the insula only appeared to be recruited once the perceived pain level was at least moderate: activity was increased for moderate (5–6) compared to mild (3–4) and for severe (7–8) compared to moderate. However, activity for mild pain (3–4) was not significantly increased compared to minimal pain (1–2), or baseline activity. This echoes a recent finding that BBP activity in the insula is selectively increased only once thermal stimulation is consistently painful (Liberati et al., 2020). Furthermore, we isolate a small number of insular neurons increasing their firing with increases in the intensity of pain experienced by others.
As mentioned in the section ‘Introduction,’ BOLD signals can dissociate from neural spiking (Boynton, 2011; Maier et al., 2008). Just as V1 BOLD signals fluctuate with perception while spiking in V1 does not (Maier et al., 2008), the observation that BOLD signals in the insula fluctuate with perceived pain intensity alone cannot guarantee that neuronal spiking in the insula does. The insula’s BOLD signal could instead fluctuate with perceived intensity simply as a result of feedback synaptic input from other brain regions that encode perceived intensity (e.g., area 24 in the cingulate gyrus; Carrillo et al., 2019). The foremost impact of our broadband gamma and spiking data is thus to provide what is arguably the first evidence that the intensity of other people’s pain is indeed locally encoded in the activity of neurons in the insula.
The human insula has been in the focus of pain neuroscience as part of the pain matrix recruited by first-hand experience of pain (Ingvar, 1999). In this tradition, neuroimaging evidence for activation of the insula while witnessing pain experienced by others has led many to suggest it may represent a neural basis for empathy for pain (Bernhardt and Singer, 2012; Jauniaux et al., 2019; Lamm et al., 2011; Timmers et al., 2018). However, the insula is also recruited by a variety of tasks beyond nociception and pain empathy, including other affective states, sensorimotor functions, and decision-making under uncertainty (Craig, 2002; Craig, 2009; Uddin, 2015; Uddin et al., 2017). With so many tasks recruiting the insula, the idea has become popular that it may play a key role in attaching salience to behaviorally important stimuli (Legrain et al., 2011; Uddin, 2015; Uddin et al., 2017; Zaki et al., 2016). In this study, we do not intend to (and cannot) address the selectivity of the insula for the pain of others over other emotions, and we do not claim the neural responses we report are pain-specific. Instead, we characterize how the insula’s iEEG activity encodes the intensity of other people’s emotions, using pain as an important category, and use the agnostic terminology of ‘intensity coding’ rather than ‘pain intensity coding’ throughout the article to refer to recording sites. That the insula’s broadband activity correlated with motion energy in our Hand stimuli only while the motion was associated with pain (during the slapping), but not while motion was innocuous (during the initial belt lifting), shows that the effects we measured cannot be reduced to visual motion detection. That perceived intensity was mediating the association between motion and broadband activity further speaks against an interpretation of our results as reflecting unspecific motion processing. Future experiments using a wider gamut of control stimuli that are matched in motion but differ in emotions will be critical to address the specificity of the responses we describe, and hence, what state they could reliably signal (Zaki et al., 2016). For the Hand stimuli, this could include seeing hands experience a range of different salient affective experiences, such as a hand being hurt, being caressed, and being rejected in addition to a neutral handshake (Meffert et al., 2013), or the actor in our Hand movies could have been asked to reposition their hand after each slap. For the face, this could include disgusted, angry, fearful, and happy facial expressions matched for motion. In general, using multiple actors and painful hand interaction could help characterize how intensity coding in the insula is influenced by details of the stimuli that convey it, including the identity and gender of the actor, and how well our results generalize to other stimulus sets (Yarkoni, 2020).
An important and somewhat related question has been whether pain or salience cues are represented in a modality-specific or modality-general manner in the insula. fMRI studies have shown that the anterior insula is coactivated with different brain regions depending on whether the pain of others is conveyed via indirect cues or via the actual body part, such as the hand, that directly receives the noxious stimulation (Gallo et al., 2018; Jauniaux et al., 2019; Keysers et al., 2010; Lamm et al., 2011; Timmers et al., 2018). Here, we provide electrophysiological measures of neural activity that speak to that issue. We focus on the broadband gamma signal known to have comparatively high spatial specificity and be closest to neural spiking (Bartoli et al., 2019; Buzsáki et al., 2012; Miller et al., 2014), and find a mixed organization, consisting of modality-specific and -general locations in a partially intermixed layout. That is, we found locations with broadband activity and spiking associated with perceived intensity for the Hand, but not the Face; others associated with the Face, but not the Hand; and others still associated with perceived intensity for both. This echoes findings in recent fMRI studies that show that at the level of BOLD response within the insula some voxels encode intensity for both Hand and Face stimuli, while some only significantly contribute to intensity coding for one stimulus type (Zhou et al., 2020).
Leveraging our high temporal and spatial resolution, we found that locations showing intensity coding for the Hand stimuli have activity timing that echoes that of pain-related motion cues with very short lags (<80 ms). These optimal lags seem at odds with the much longer response onset latencies reported in the literature (Bastuji et al., 2018; Bastuji et al., 2016; Chen et al., 2009; Cornwell et al., 2008; Krolak-Salmon et al., 2003; Liberati et al., 2020; Liberati et al., 2016), but differ from response onset latencies in two ways. First, our BBP is calculated within time windows of 8× the inverse of the frequency. Hence, for the lowest broadband frequency of 20 Hz, power at time
An important consideration is whether preference for Face or Hand stimuli in specific locations could simply originate from some participants finding our facial stimuli more salient, and others the Hand stimuli, particularly given that our patients rated the Hand stimuli with slightly higher pain intensity than our Face stimuli. That we find Face and Hand preference side-by-side in simultaneously recorded locations and neurons in single patients suggests that this cannot suffice to explain our data. This is because if a patient were to find Hand stimuli more salient than Face stimuli, and the insula simply codes saliency, we would expect to find Hand-but-not-Face intensity coding in that patient’s insula, but we would not expect to find side-by-side locations with Hand-but-not-Face and Face-but-not-Hand coding. This also makes it unlikely that broadband activity simply reflects categorization uncertainty (Grinband et al., 2006). Indeed, it would be uneconomical for all insular neurons to redundantly encode the same exact signal of salience or categorization uncertainty. Our findings instead suggest that different locations might encode behaviorally relevant (and hence salient) information about the intensity of the pain of others with some degree of specialization for a particular type of information (Hand vs. Face). The insula intensity coding we measure could have a dual function: help perceive the intensity of other people’s emotions and tag intense emotions as salient, thereby reconciling the notion that it contributes to empathy and saliency. One peculiar observation in our data is that BBP for the Hand stimuli did not increase monotonically as a function of reported pain intensity, but in a J shape, with higher power for the lowest than second lowest rating. Some have argued that the insula may be part of an active inference circuitry in the brain that learns to predict emotionally relevant states and stimuli (Seth and Friston, 2016). In such a perspective, a J-shaped response curve could reflect a combination of underlying neurons representing the intensity of other people’s emotions (with a monotonic increase in activity) and others representing the prediction error (which for randomly presented intensities would have a U shape centered on the average intensity). Future experiments could contrast responses to a given pain intensity in blocks of high and blocks of low average presented intensity to disentangle the effect of observed and expected intensity on neural responses in the insula and shed further light on this predictive component.
On the other hand, we also found evidence that some locations and cells represent intensity coding for both the Hand and the Face stimuli. In addition, if we train a PLSR to predict perceived intensity from the activity pattern across all recorded locations, we find that training the decoder on Hand activity pattern and testing it on Face activity patterns (or vice versa) leads to above-chance decoding. This confirms that the insular representation of intensity can support stimulus-independent decoding – despite our PLSR not being biased to focus on signals that do generalize well across stimulus types. This provides an electrophysiological basis for recent fMRI studies that show that stimuli depicting situations in which others’ experience is painful or not can be discriminated using the same pattern across Hand and Face stimuli (Zhou et al., 2020).
In addition to the broadband results we report in detail, we find that theta power increases with perceived intensity. Given a growing animal literature establishing that interareal theta synchronization promotes learning about threats (Likhtik et al., 2014; Likhtik and Gordon, 2014; Taub et al., 2018; Tovote et al., 2015), examining the coherence in the theta range across iEEG electrodes in different brain regions during pain observation may in the future shed light on how humans learn about safety through others.
Spatially, we found that Hand intensity coding was enriched in the anterior dorsal insula, where we also found the largest proportion of locations encoding both Hand and Face intensity. This anterior bias was also observed in our BOLD signal for similar Hand stimuli. A recent meta-analysis identified that the most consistent BOLD activations when observing limbs in painful situations within the insula occur bilaterally around MNI coordinates y = 13 and z = 10 (Jauniaux et al., 2019), which closely matches where we find the highest density of Hand intensity coding (Figure 4a). Interestingly, locations with higher Hand intensity coding have increased connectivity at rest with extrainsular regions involved in processing two relevant stimulus dimensions. Connectivity was higher with cerebellar lobules VI, VII, and VIII, and with the inferior frontal gyrus, all of which are recruited by Abdelgabar et al., 2019; Caspers et al., 2010; Gazzola and Keysers, 2009 and necessary for (Abdelgabar et al., 2019; Keysers et al., 2018; Pobric and Hamilton, 2006) perceiving the very kinematics of hand actions we find to be good predictors of BBP activity in this study. Connectivity is also higher with the mid- and anterior cingulate cortex, which is associated with pain witnessing in humans (Jauniaux et al., 2019; Lamm et al., 2011; Timmers et al., 2018) and contains neurons with pain intensity coding in rats witnessing the pain of other rats (Carrillo et al., 2019).
With respect to Faces, somewhat surprisingly, we did not find a clear spatial clustering of intensity coding in our electrophysiological data. Overall, our ability to decode perceived intensity from our Face stimuli was also lower than that from the Hand stimuli. In that context, it is important to reflect on the fact that, despite our efforts to match the perceived intensity based on previous data (Gallo et al., 2018), patients (and to a lesser extent an age- and gender-matched control group) perceived the Hand stimuli as more intense than the Face stimuli. Given that responses were strongest for the highest rating, and that this rating was given more rarely for Faces than Hands, this difference could have contributed to making our Face results less consistent. At the same time, the variance in rating, a critical factor for determining the efficiency with which a regression can detect the presence of a rating–BBP relationship, did not differ across Hand and Face. Together, this difference in perceived intensity cautions us against overinterpreting the lack of detectability of a topographic organization of responses to Faces. Indeed, in our BOLD data, where decoding performance was similar for Hand and Face stimuli, Face intensity coding also had a clear spatial organization, being stronger more anteriorly, and meta-analyses show the left anterior insula to be reliably recruited by the observation of painful facial expressions (Jauniaux et al., 2019). However, that we found fewer locations and less reliable spatial organization for Face than Hand intensity coding dovetails with recent meta-analyses of the fMRI literature showing that the insula is more reliably recruited by the sight of limbs than by painful facial expressions (Jauniaux et al., 2019; Timmers et al., 2018). Indeed, that we find a macroscopic organization of intensity coding for the Hand, but not the Face, is echoed at the mesoscale: microwires with cells with Hand intensity coding also tend to show Hand intensity coding in the BBP signal that is thought to pool the spiking of many neighboring neurons, but the same is not true for the Face. In terms of lateralization, we find that our data is more likely if one assumes that both hemispheres have similar intensity coding than if one hemisphere were dominant. This echoes the fact that during noxious stimulation of the right hand both insulae show significant iEEG responses (although slightly stronger in the left insula; Liberati et al., 2020), and that fMRI fails to find robust lateralization of responses to empathy for pain (Jauniaux et al., 2019; Timmers et al., 2018).
Materials and methods
iEEG experiment
Participants
Patients
Depth electrode recordings were collected from nine epileptic volunteers admitted at the Amsterdam UMC to localize seizure origin. Patients received initial study information from the neurosurgeon and provided informed consent to the experimenter before the surgery occurred. Our single-session experiment started on average 4 days after surgery (SD = 1.89 days). Preliminary analyses indicated that the pain rating performance of two patients (Xs in Figure 1e) for Face videos was significantly poorer compared to an age- and gender-matched healthy control group. Hence, these two patients were excluded from all analyses, which yielded a final sample of seven patients (four females, 34.3 years ± 9 SD, Table 4). The study was approved by the medical ethical committee of the Vrije University Medical Center (protocol 2016/037), and each patient signed a written informed consent according to the Declaration of Helsinki.
Table 4.
Participants’ demographics and epileptic status.
Our seven patients were matched in age and gender to the online control group from which we obtained normative movie ratings. The last column indicates the postoperative status of our patients. Three patients had other brain regions than insula surgically removed, and afterward had no more attacks (marked with 1), suggesting that the foci were clearly outside the insula. One patient had a region other than the insula surgically removed because the monitoring had suggested that the foci was outside of the insula; however, the patient continued to have post-surgical attacks (marked with 2). For three patients, no surgery was performed because there was no clear link between electrode locations and epileptic attacks (marked with 3).
| Group | Sample size (n) | Age (mean ± SD years) | Age-matched | Gender (M/F) | Gender-matched | Epileptic location score |
|---|---|---|---|---|---|---|
| Patient | 7 | 34.3 ± 9 | Mann–Whitney | 3/4 | Multinomial test, p=0.7, BF01 = 7 .3 | A = 1, B = 1, C = 3, D = 3, E = 1, F = 3, G = 2 |
| Control | 93 | 33.7 ± 9 | 38/55 | N/A |
Clinical investigation revealed that for all our patients the epileptic incidents did not appear to originate around the electrode contacts in the insula that we analyzed here. In addition, for four of them, recordings pointed to origins of the epilepsy to be clearly outside the insula, leading to the surgical removal of extrainsular regions (Table 4). Finally, for the remaining three, no clear origin for the epilepsy could be localized, but there was no indication that the insula was involved in the initiation of the attacks.
Control participants in the online video rating task
To assess whether the behavior of the patients was representative of the general population, we compared patients’ ratings with those of 93 volunteers (54 females, 32.7 years ± 9 SD), who took part in an online version of the video pain rating task. The matching with the seven patients was done by only including age- and gender-matched Dutch participants, and was successful (Table 4). The study was approved by the local ethical committee of the University of Amsterdam (2021-EXT-13608), and each participant signed an online informed consent form to participate in the study.
Control participants in the online frame rating task
To determine whether participants could use shape information available in single frames to determine pain intensity in Hand and/or Face stimuli, 40 volunteers (23 females, 33.7 years ± 9 SD) from the same group who also performed the online video pain rating task participated in the online frame rating task, so they were already familiar with the videos and had a better understanding of where the single frames came from. This also allowed us to directly compare how they rate single frames with how they rated the movies from which the frames were taken. They were selected to approximate the age and gender distribution of the patient group.
Stimuli and procedure
Video rating task
The 2 s videos were generated as in Gallo et al., 2018 and showed a Caucasian female receiving either electrical shocks to the hand (reaction conveyed by the facial expression only; Face condition) or a slap with a belt to the hand (reaction conveyed by the hand only; Hand condition). Hence, the location of the noxious stimulation was maintained across conditions (dorsum of the left hand), but the cues through which participants could deduce the painfulness differed. All videos started with 1 s of baseline: neutral facial expression for Face and static hand for Hand stimuli. The reactions of the actor were genuine, but enhanced: truly noxious stimuli were applied during recording, but the actor was encouraged to enhance her expressivity and not to suppress her reactions. This instruction was given in order to compensate for the fact that the intensity of the shock delivered had to be in balance with the fact that many videos had to be recorded. A mild pain was therefore used, in agreement with the actress, and enhancement was necessary to fully convey expressions of stronger pain. Movies were cut, so that evidence of pain started at 1 s (Figure 1a). Before the experiment, participants were instructed to rate pain intensity (‘How much pain do you think the person felt?’) on a scale from 0 (no pain at all) to 10 (the worst imaginable pain). To reassure patients that no real harm was inflicted to the actor in the movie, they were informed that during video recording stimulations in the 9–10 range were never used. Participants had to rate pain intensity after each video at their own pace, using four keyboard keys (Figure 1b). Only the relevant keys were presented on the screen, intensities were not indicated. Patients watched each of the 60 videos (30 Hand, 30 Face) twice in fully randomized fashion with a random interval of 1.5 s ± 0.5. The videos were matched in terms of intensity and standard deviation based on a validation in Gallo et al., 2018.
Online video rating task
The stimuli and the task were the same as in the electrophysiology experiment, except each video was presented only once. Prolific Academic (https://www.prolific.co/) was used to recruit participants, and the experiment was implemented on Gorilla (Anwyl-Irvine et al., 2020; https://gorilla.sc/).
Online frame rating task
The task was similar to the pain rating experiment, except still frames instead of the full videos were presented for 2 s. For faces, frames at the 1.8 s of the Face videos were used (except for one video where the eyes were closed at 1.8 s, so the frame was taken at 1.68 s). This timepoint was selected because facial expressions were most pronounced toward the end of the movies, and more formal analyses confirmed that this corresponds to a time where shape information plateaued (Figure 3a). To use a comparable stimulus set for Hands, which portrayed maximal configuration information, we selected from each Hand video separately, the frame at which the hand was maximally depressed by the force of the belt slap (timepoint mean ± SD = 1.001 ± 0.013 s).
Data acquisition
Patients were implanted with Behnke–Fried depth electrodes (Ad-Tech Medical Instrument Corporation; Fried et al., 1999) targeted at the right or left, anterior or posterior insula. Electrodes were inserted via a guide tube under the guidance of an online stereotactic positioning system. They consisted of a silastic hollow tube with 9–12 platinum outer macrocontacts, 1.28 mm in diameter, 1.57 mm in length with the first two macrocontacts spaced 3 mm from each other and the rest spaced 5 mm from each other. This hollow tube had nine platinum microwires (eight recording and one reference contact) running through it, each 38 micron in diameter, protruding as a ‘pigtail’ formation out of the tip of the electrode. Macrocontact recordings were amplified using unity gain, DC amplifiers (Braintronics BrainBox 1166 system), low-pass filtered at 1500 Hz (–3 dB point, –12 db/octave), and sampled at 32,768 Hz. The digital signal was decimated to a final sample rate of 512 Hz or 1024 Hz and was pre-filtered with a three-section FIR equiripple filter (0.01 dB passband ripple) with the passband set to 1/3 of the sample frequency and the stopband set to 1/2 of the sample frequency. Signals from the microcontacts were amplified with respect to a skull-screw ground using a unity gain HS-9 head-stage amplifier (NeuraLynx). The signal was high-pass filtered at 1 Hz and low-pass filtered at 5 kHz and had a sampling rate of 32 kHz. There were a total of 85 macroelectrodes and 32 microwires across all patients in the insula that we recorded from.
Electrode localization
For each patient, the T1 structural MR image taken before the electrode implantation surgery and the CT scan taken after the electrode placement were co-registered (Figure 1d). Using SPM12 (https://www.fil.ion.ucl.ac.uk), the T1 image was segmented to determine the normalization parameters, and MR and CT images were then normalized to the MNI space using these parameters. CT scan and gray matter were overlaid with insula probability maps (Faillenot et al., 2017), and macrocontacts within the boundaries of the insula map were detected based on detailed visual examination using MRIcron (https://www.nitrc.org/projects/mricron). Not the individual subdivision maps, but the combination of all the subdivision maps as one general insula map was used for localization because the coverage of different subdivisions was highly uneven across patients. Since macrocontact recordings were analyzed in a bipolar layout, the coordinates of each bipolar recording were estimated as the midpoint of its macrocontacts (see Figure 1—source data 1 for MNI coordinates).
Data analysis
General statistical approach
Much of the analyses in this article assess intensity coding, which examines the relationship between brain activity (measured based on LFP, spiking, or BOLD activity) and rating. Because the rating of pain intensity was along discrete categories (1–2, 3–4, 5–6, 7–8), which might be linear but is certainly ordinal, we tend to use association measures that are appropriate for ordinal scales when we relate brain activity to the rating of a single participant. That includes Spearman’s
When using
Behavioral analyses
To explore whether patients were impaired in their ability to perform the task, our rationale was to consider the average rating of all control participants as the normative rating. We then compared the vector of 30 ratings (one per movie for 30 movies) of each member of the control group against the average of the other members of the control group to define a distribution of how far from the normative rating healthy volunteers tend to fall. For the patients, we compared their ratings against the average rating of the control group, and compared how similar patient ratings were to the normative average against the distribution of how similar left-out control participants are to the normative average. We calculated three metrics of similarity: the Spearman’s rank-order correlation, the slope, and the intercept of a simple linear regressions between the ratings of each of the patients and the average rating of all control samples.
Preprocessing of LFPs
To reduce artifacts and extract local signals, iEEG macrocontact recordings were digitally re-referenced in a bipolar layout (Figure 1d). This generated 85 bipolar recordings from 102 contacts in the insula, with patients having between 5 and 19 bipolar channels (Figure 1c, Figure 1—source data 1). Re-referencing attenuated 50 Hz noise sufficiently to omit digital filters that distort data. Continuous recordings were separated into trials of 4 s: 1 s premovie baseline, 2 s video, and 1 s postmovie. Trials were visually checked for ground failure and amplitude saturation (none was detected), downsampled to 400 Hz, and detrended.
Time–frequency decomposition of LFPs
A sliding window Hanning taper-based approach was used for each trial with the following parameters: frequencies from 1 to 200 Hz in steps of 1 Hz; timepoints from –1 s (relative to movie onset) to 3 s in steps of 0.0025 s; and for each frequency, a single-sliding Hanning taper window with the duration of eight cycles (maximum = 1 s; minimum = 0.1 s). Trials were expressed as % power change relative to baseline (–1 s to 0 s) separately for each frequency: y(t) = ((P(t) – P0)/P0), with P0 = average of baseline. Points with y(t) ± 10 standard deviations from the mean of the other trials were excluded to not reject entire trials, but only outlier time–frequency points in some trials (rejections were rare, mean rejected time–frequency points = 0.0032% ± 0.0035 SD).
Intensity coding in LFPs
In the LFP signal, we consider that a bipolar channel shows intensity coding if its trial-by-trial power variations correlate positively with the variation in the pain intensity reported by the patient. We always coded the 1–2, 3–4, 5–6, and 7–8 rating options as 1, 2, 3, and 4. For each bipolar recording, we then calculated the Spearman’s rank correlations (due to the ordinal nature of intensity ratings) between the patient’s behavioral intensity ratings and the neural activity power estimate over all trials. In time–frequency-resolved analyses, these correlation analyses were conducted separately using the power estimates at each time–frequency intersection separately. In frequency band analyses, the correlation was calculated using the average of the power estimates within a specific frequency band. For both analysis types, a one-sample
We performed a similar analysis to identify time–frequency bands with significant intensity coding when separating the Hand and Face trials. Note that including half the number of trials makes this analysis less powerful than the Hand and Face combined analysis, which is why we used the broadband frequency band (20–190 Hz) resulting from the combined analysis. To directly compare the time–frequency profiles of intensity coding for Hand vs. Face trials, we performed a similar analysis for cluster correction, except that for each time–frequency point separately, the Hand and Face correlation coefficient distributions across the 85 electrodes were directly compared with a paired-samples
Resampling LFPs from the entire brain
To test whether the BBP–rating association observed in the insular electrodes was enriched compared to what could have been observed in any 85 bipolar recordings anywhere in the brain, we made use of not just the insular but all the intracranial macroelectrodes that were implanted in the same patients (see Figure 2—figure supplements 2 and 3 for anatomical locations of all macrocontacts). The recordings from these electrodes were preprocessed exactly as described for the insular electrodes. The seven patients included in these analyses had between 91 and 149 (mean ± SD = 114 ± 20) bipolar recordings distributed throughout the two hemispheres and various regions of the four brain lobes. The BBP–rating Spearman’s correlation coefficients of each of these electrodes were entered into a resampling method, in which for each of the 100,000 iterations, from each patient, a random subset of these correlation coefficients were selected. The number that was selected for each patient was determined by the number of insular electrodes that patient had; that is, since patient A had five insular electrodes in the main analyses, five random electrodes were selected from the entire brain in these analyses. This was done to ensure that possible patient-specific biases in the analysis of insular electrodes were maintained in these analyses. This way, in each iteration, a total of 85 electrodes were selected randomly from the entire brain and tested with a one-sample
Extracting shape and motion information from videos
A recent systematic review has revealed that facial expressions of pain are most consistently characterized by lowering of the brow and tightening of the eyelid (Kunz et al., 2019), corresponding to facial AUs 4 and 7 (Ekman and Friesen, 1978; inlet in Figure 3a). More specifically, research has evidenced that people fall into four clusters that differ in how they express pain (Kunz and Lautenbacher, 2014), and our protagonist fell within cluster IV, who express pain by furrowing brows and tightening eyes, but not opening the mouth or wrinkling the nose. Accordingly, we quantify the painfulness expressed in the shape of our protagonist’s face based on facial AUs 4 and 7. To get a replicable and objective measurement of these AU, we used the FaceReader software (Noldus, the Netherlands), which uses a deep convolutional neural net to automatically extract the level of activation of the classic facial action units (Ekman and Friesen, 1978). FaceReader reliably obtained estimates for the facial AUs 4 and 7 from all but three frames from our 30 movies, and we thus quantified the pain-related shape information contained in each frame of our movies as the average of AUs 4 and 7. When applied to the frames used in our psychophysical experiment mentioned above, the average activation of AUs 4 and 7 correlated at
To quantify motion over time for each video, we use motion energy, an established and objective way to extract dynamics from any movie, using the average of the Euclidean distances between the RGB vectors of the corresponding pixels across every two consecutive frames.
PLSR decoding of intensity coding from shape and motion information
To identify when motion or shape information may contribute to predicting the overall intensity rating R of the movie i, we used PLSR analyses using the ‘plsregress’ function in MATLAB, with a single component. For the Hand, where no shape information was available, the predictor for each movie i was the motion M at frame t, and the plsregress thus identified the weights (B), such that R(i) = M(i,t)B(t) + B0(i). For the Face, where both motion and shape information S was available, as the average of AUs 4 and 7 in each frame, we concatenated M(i,t) and S(i,t) into a single predictor X to identify weights such that R(i) = XB + B0(i). We used PLSR here in particular because both M and S have high temporal autocorrelations and are mutually correlated, and PLSR is well suited for such cases.
Second, we can use the PLSR method to see how accurately the motion and/or shape profile across the entire movie can be used to predict the rating of the patients using a cross-validation. The predictive accuracy for both the motion and shape time course was calculated in 1000 iterations. In each iteration, the videos were randomly divided into three equal-sized samples. For each of these three samples separately, the remaining two samples were used to calculate PLSR beta coefficients, which were then used to predict the ratings of our patients. The Pearson’s correlation coefficient between the actual ratings and the predicted ratings was taken as measures of predictive accuracy and averaged across all iterations. A similar procedure was also applied to calculate the null distribution of such correlation coefficients, in which the 1000-iteration step described above was repeated for 10,000 times, each time with a different randomly shuffled version of the observed ratings. The probability of the observed decoding accuracies was then estimated by ranking the accuracy based on the actual ratings within the distribution of shuffled ratings. Figure 3d–f shows that motion and shape information each allows one to predict movie ratings with high accuracy.
Probability of Face-but-not-Hand, Hand-but-not-Face, and dual-intensity coding LFPs
Correlations between BBP and rating were thresholded as significant or as providing evidence of absence as follows (Figure 4b). At n = 60 trials, values above
The same analysis was applied to an exemplar participant (Figure 4d). Using the same logic in that patient, we find that the number of Hand-but-not-Face coding locations (p1=2 × 10–4, BF+0 = 701) and the number of Face-but-not-Hand coding locations (p1=0.007, BF+0 = 37) are surprising, but the number of dual-coding locations (1/15) is not surprising among the three Face (p1=0.143, BF+0 = 1.9) or seven Hand (p1=0.3, BF+0 = 0.48) coding locations.
PLSR decoding of intensity from the LFP insula activity pattern
To explore how well the pattern of activity across all 85 bipolar recordings reflects the perceived intensity reported by our patients, we applied a PLSR regression approach similar to that used to infer how well shape or motion predicts ratings, except that instead of using motion over 50 frames, we used BBP over 85 electrodes. Specifically, BBP across the 85 sites, averaged over the early period for Hand videos and in the late period for Face videos, were separately used as predictors in two separate PLSR analyses predicting the participants’ average pain ratings. The decoding accuracies were each calculated in 1000 iterations. In each iteration, the videos were randomly divided into three equal-sized samples. For each one of these three samples, the remaining two samples were used as a training set to calculate PLSR beta coefficients, which were then used to predict the ratings in the remaining test sample. The Pearson’s correlation coefficient between the predicted ratings and the actual ratings of the patients was then taken as a measure of decoding accuracy and averaged across all iterations. We used Pearson here because the data was normally distributed, and we compared two ratings. A similar procedure was also applied to calculate the null distribution of such correlation coefficients, in which the 1000-iteration step described above was repeated for 10,000 times, each time with a different randomly shuffled version of the original ratings. The probability of the observed decoding accuracies was estimated under the corresponding null distributions as the rank of the actual average accuracy against the shuffled accuracies. We first performed this analysis within each stimulus type; that is, we trained and tested on Hand stimuli or we trained and tested on Face stimuli. Then, to explore whether the pattern of activity could generalize across stimulus type, we also performed cross-decoding analyses where we trained on one stimulus type (e.g., we determined the PLSR weights using ⅔ of Hand stimuli) and then tested them on the other (e.g., predicted 1/3 of the Face stimuli). We first performed this analysis using a single PLSR component and found a trend (Hand:
Resting state connectivity analysis
To interrogate what connectivity profile is characteristic for electrode pairs with high-intensity coding, we used Neurosynth (Yarkoni et al., 2011) to extract a whole-brain resting state connectivity map for the MNI location of each of the 85 contact pairs in the insula (Figure 5—source data 1). Using SPM12 (https://www.fil.ion.ucl.ac.uk/spm/software/spm12/), we performed a regression analysis (general linear model) that included the 85 voxel-wise resting state connectivity maps and two predictors: the correlation between power and rating for the Hand in the early window, and for the Face in the late window. Results were thresholded at p<0.001, corrected for multiple comparisons using family-wise error correction at the cluster level. Results were then illustrated on an inflated cortical template provided in SPM12, and significant voxels were attributed to specific brain regions using the Anatomy toolbox 3.0.
Spike sorting and selection of responsive single units
Three patients had microwires (Behnke–Fried electrodes, Ad-Tech Medical; Fried et al., 1999) in the insula protruding from the electrode tip (plusses in Figure 1c). Spikes were detected and sorted using Wave_Clus2 (Quiroga et al., 2004). In short, raw data was filtered between 300 and 3000 Hz. As per default settings, spike waveforms were extracted from 0.625 ms before to 1.375 ms after the signal exceeded a 5*noise threshold, where noise was the unbiased estimate of the median absolute deviation. Wave_Clus2 sorted and clustered the waveforms automatically and were manually checked by author RB. Clusters were excluded in which >2% of spikes were observed with an interspike interval < 2 ms or with firing rate < 1 Hz. To identify cells that responded to our stimuli, we used a Wilcoxon signed-rank test comparing spike counts during baseline (–1 s to 0 s) against that during the pain period (1–2 s) for Hand and Face trials together. Only cells that showed a response to the stimuli (p1<0.05), irrespective of pain intensity, were considered for further analysis.
Similar to LFP analyses, a cell was said to show intensity coding if spike counts rank correlated positively with reported intensity. Because JASP includes Bayesian statistics using Kendall’s tau but not Spearman’s
Broadband power analysis in microelectrodes
To explore whether intensity coding in cells and the BBP (20–190 Hz; Figure 2a) from the same microwire were related, for the 10 microwires that yielded responsive neurons (whether these neurons showed intensity coding or not) we quantified the association between BBP averaged over the pain period (1–2 s) and intensity ratings (1–2, 3–4, 5–6, 7–8) using rank correlation coefficients separately for Face and Hand videos (again using Kendall’s tau to provide BF+0 estimates). All eight microwires protruding from the same electrode were first re-referenced to the microwire with the least spiking and lowest artifacts, yielding seven microwire recordings for each of the four electrode tips with wires in the insula. Data were filtered to remove 50 Hz noise and harmonics at 100 and 150 Hz. Subsequently, they were separated into trials of 4 s (–1 s to 3 s relative to video onset), downsampled to 400 Hz, and visually checked for artifacts. The time–frequency decomposition of power followed the same procedure as for the macrocontact recordings. Finally, intensity coding at the level of spikes (i.e.,
fMRI experiment
Participants
Twenty-five healthy volunteers participated in the study. The full dataset of two participants was excluded from the analyses because head motions were above 3 mm. Analyses were performed on the remaining 23 participants (13 females; mean age = 28.76 years old ± 6.16 SD). The study was approved by the local ethics committee of the University of Amsterdam (project number: 2017-EXT-8542).
Stimuli and procedure
The video pain rating task was performed as described in the section
Data acquisition
MRI images were acquired with a 3-Tesla Philips Ingenia CX system using a 32-channel head coil. One T1-weighted structural image (matrix = 240 × 222; 170 slices; voxel size = 1 × 1 × 1 mm) was collected per participant together with echoplanar imaging (EPI) volumes (matrix M × P: 80 × 78; 32 transversal slices acquired in ascending order; TR = 1.7 s; TE = 27.6 ms; flip angle: 72.90°; voxel size = 3 × 3 × 3 mm, including a 0.349 mm slice gap).
Data analysis
Preprocessing
MRI data were processed in SPM12. EPI images were slice-time-corrected to the middle slice and realigned to the mean EPI. High-quality T1 images were co-registered to the mean EPI image and segmented. The normalization parameters computed during the segmentation were used to normalize the gray matter segment (1 × 1 × 1 mm) and the EPI images (2 × 2 × 2 mm) to the MNI templates. Finally, EPIs images were smoothed with a 6 mm kernel.
Voxel-wise analysis
At the first level, we defined separate regressors for the Hand videos, Face videos, rating scale, and button presses. Analyses focused on the 2 s that the videos were presented. The trial-by-trial ratings given by the participants were used as a parametric modulator, one modulator for the Face and one for the Hand trials, on the respective video regressor. The rating-scale regressor started from the moment the scale appeared and ended with participants’ confirmation button press. The button-press regressor, finally, had zero duration events aligned to the moment of each button press. Six additional regressors of no interest were included to model head movements. To quantify the degree to which each voxel in the insula had BOLD activity associated with trial-by-trial ratings, we then brought the parameter estimate for the parametric modulator obtained for Hand and Face trials separately to a second-level
Multivariate regression
To investigate whether the pattern of BOLD activity across all voxels in the insula encodes intensity, we additionally performed a multivariate regression analysis akin to the PLSR for the BBP described in above. For each participant, we performed a general linear model that estimated a separate parameter estimate for the video epoch of trials in which participants gave a rating of 0–2, 3–4, 5–6, or 7–8, respectively, separately for Hand and Face trials. In MATLAB, we then loaded for each subject the parameter estimate images for each level of rating and only included voxels that fell within our insula mask (Faillenot et al., 2017). We then trained a weighted PLSR using the MATLAB function plsregress and the data from all but one participant to predict rating based on a linear combination of the parameter estimates in each voxel and used this linear combination to predict the rating of the left-out participant, repeating the procedure for each participant. We weighted the regression by replicating each parameter estimate image in the training and testing set by the number of trials that went into it. We quantified how accurately the regression predicted the rating of the left-out participants using Kendall’s tau, then tested whether the performance was above chance by comparing the 23 prediction accuracies (Kendall’s tau) against zero in a one-tailed test. Based on the analysis on the BBP, we performed this analysis with procedure using two or three components.
You have requested "on-the-fly" machine translation of selected content from our databases. This functionality is provided solely for your convenience and is in no way intended to replace human translation. Show full disclaimer
Neither ProQuest nor its licensors make any representations or warranties with respect to the translations. The translations are automatically generated "AS IS" and "AS AVAILABLE" and are not retained in our systems. PROQUEST AND ITS LICENSORS SPECIFICALLY DISCLAIM ANY AND ALL EXPRESS OR IMPLIED WARRANTIES, INCLUDING WITHOUT LIMITATION, ANY WARRANTIES FOR AVAILABILITY, ACCURACY, TIMELINESS, COMPLETENESS, NON-INFRINGMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Your use of the translations is subject to all use restrictions contained in your Electronic Products License Agreement and by using the translation functionality you agree to forgo any and all claims against ProQuest or its licensors for your use of the translation functionality and any output derived there from. Hide full disclaimer
© 2022, Soyman, Bruls, Ioumpa et al. This work is published under https://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Abstract
Based on neuroimaging data, the insula is considered important for people to empathize with the pain of others. Here, we present intracranial electroencephalographic (iEEG) recordings and single-cell recordings from the human insula while seven epilepsy patients rated the intensity of a woman’s painful experiences seen in short movie clips. Pain had to be deduced from seeing facial expressions or a hand being slapped by a belt. We found activity in the broadband 20–190 Hz range correlated with the trial-by-trial perceived intensity in the insula for both types of stimuli. Within the insula, some locations had activity correlating with perceived intensity for our facial expressions but not for our hand stimuli, others only for our hand but not our face stimuli, and others for both. The timing of responses to the sight of the hand being hit is best explained by kinematic information; that for our facial expressions, by shape information. Comparing the broadband activity in the iEEG signal with spiking activity from a small number of neurons and an fMRI experiment with similar stimuli revealed a consistent spatial organization, with stronger associations with intensity more anteriorly, while viewing the hand being slapped.
You have requested "on-the-fly" machine translation of selected content from our databases. This functionality is provided solely for your convenience and is in no way intended to replace human translation. Show full disclaimer
Neither ProQuest nor its licensors make any representations or warranties with respect to the translations. The translations are automatically generated "AS IS" and "AS AVAILABLE" and are not retained in our systems. PROQUEST AND ITS LICENSORS SPECIFICALLY DISCLAIM ANY AND ALL EXPRESS OR IMPLIED WARRANTIES, INCLUDING WITHOUT LIMITATION, ANY WARRANTIES FOR AVAILABILITY, ACCURACY, TIMELINESS, COMPLETENESS, NON-INFRINGMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Your use of the translations is subject to all use restrictions contained in your Electronic Products License Agreement and by using the translation functionality you agree to forgo any and all claims against ProQuest or its licensors for your use of the translation functionality and any output derived there from. Hide full disclaimer