1. Introduction
The research on the dynamics of human brain activity has largely taken one of the three main approaches. The first approach comprises the classic cognitive neuroscience paradigm, i.e., the measurement of intra-personal (within-person) brain activity, with a focus on the functional specialization of the individual brain as well as its activity in creating representations of the inner and outer world [1]. The second approach emerged from the field of social neuroscience as a ‘multi-level analysis of social psychological phenomena’ that investigates intra-personal brain dynamics during inter-personal interactions [2]. A new, third approach has recently been emerging whereby brain activity from two or more people can be measured and quantified simultaneously during a particular motor or cognitive task [3,4]. Studying the so-called ‘social brain,’ which allows us to interact with other people [5], has been one of the fastest growing and most challenging issues in neuroscience [6]. However, since the introduction of the simultaneous recording of the cerebral activity from different subjects using various neuroimaging methods such as electroencephalography (EEG) [7], functional magnetic resonance imaging (fMRI) [8], functional near-infrared spectroscopy (fNIRS) [9] and magnetoencephalography (MEG) [10], it has been possible to assess the neurophysiological basis of social behavior by analyzing the synchronization between multiple brains [11,12]. Such approach, whereby brain activity from two or more people can be measured and quantified simultaneously, has been called ‘two-person neuroscience’ [13], ‘inter-personal brain-to-brain coupling’ [4], ’dynamical coupling’ [14], ’entrainment’ [15], or collectively ‘hyperscanning’ [16].
Figure 1 shows a timeline of major events in hyperscanning research. Montague et al. [8] first coined the term ‘hyperscanning’ in a proposal which described both the hardware and software necessary to make simultaneous fMRI recordings of two people playing a game. King-Casas et al. [17] were the first to used fMRI to measure brain activities and compare the activations in subject pairs. Ahn et al. [18] conducted the first simultaneous EEG/MEG hyperscanning study of inter-brain phase synchronization during social interaction. Recently, Cha and Lee [19] developed a novel approach to quantifying inter-brain synchronization via bispectral analysis.
1.1. Research Motivation
Despite its relatively short history, the field of hyperscanning has made significant progress in the past years on (1) applications, (2) neuroimaging methods, (3) experiment paradigms, and (4) inter-brain synchrony methodologies. First, plenty of successful applications have been demonstrated in many areas, such as common vision [20], temporal motor synchronization [9,21,22], music production [23,24,25], speech [26,27], shared attention [28], decision-making [3,29,30], classroom learning [31], and cooperative piloting [6]. Second, through technological advancements in neuroimaging methods, NIRS hyperscanning could be conducted using one NIRS scanner to measure simultaneous movements in two subjects [9]. Zhao et al. [32] developed a processing pipeline which incorporated independent component analysis (ICA) as a precursor to calculating hyperconnectivity with fNIRS in source space; previous fNIRS hyperscanning studies had done calculations in sensor space without ICA. Third, recent hyperscanning studies have focused on increasing ecological validity through face-to-face interaction and naturalistic environments [33,34,35,36]. In this past year, numerous discoveries have been made including EEG hyperscanning while monitoring in real-time [19] and using fNIRS hyperscanning to simultaneously scan musicians in an ecological setting [37]. Finally, various methods that can quantify inter-brain synchrony have been developed in fNIRS [37], fMRI [38]), MEG [18], and EEG [6] hyperscanning studies. For example, Granger causality was first used in fMRI hyperscanning in 2010 by Schippers et al. [39]. This effective connectivity measure was first adopted in fNIRS hyperscanning two years later by Holper et al. [40]. Tognoli et al. [41] used Phase Locking Value (PLV) as a functional connectivity measure, in an EEG hyperscanning experiment. In 2008, Yun and colleagues [42] conducted the first functional connectivity-based EEG hyperscanning study with the link patterns of intra- and inter-brain.
However, it is also true that contemporary hyperscanning research identifies many gaps in the literature that warrant further investigation. A rapidly-increasing number of published hyperscanning studies occasions various problems, including (i) how to highlight methodological concerns in research studies [43] that can be used to improve future work in the hyperscanning area and (ii) how to identify questions for which the available evidence provides clear answers and thus for which further research is not necessary [44]. Systematic reviews address both of these issues. A systematic review, a “review of the evidence on a clearly formulated question that uses systematic and explicit methods to identify,” selects and critically appraises relevant primary research, and extracts and analyzes the data from the studies that are included in the review [45]. However, the literature on hyperscanning has mostly been narratively reviewed, which simply records and assesses the state of knowledge on a particular topic in hyperscanning.
For example, we identified 24 review papers on hyperscanning research published since 2002 (see Table 1). We found these review papers shared a couple of significant limitations in their methods and approaches, which indicates a need for a more systematic review study on hyperscanning. A number of features distinguish our review study from most other review papers. First, in this paper we reviewed state-of-the-art hyperscanning research by neuroimaging methods, research paradigms and application types, as well as various methods or indices for quantifying inter-brain synchrony, which other review papers have failed to address specifically. Out of the 24 review papers, two review papers [46,47] gave a general timeline of hyperscanning research but failed to dive into the research paradigms and indices in hyperscanning. Six papers discussed one methodology [48,49,50,51,52,53] and four papers reviewed multiple modalities [12,54,55,56]. The applications discussed were joint action [57,58], social neuroscience [59,60], social interaction [13,61,62,63,64], and interpersonal coordination [65,66]. Finally, this review draws its conclusions from the empirical data collected from hyperscanning experimental studies. Our review is unique in that it examines neuroimaging methods e.g., [48,52,54,67], research paradigms, and application domains e.g., [57,68] for future hyperscanning research.
1.2. Review Objectives
The overarching objective of this study was to conduct a comprehensive review on hyperscanning research, with the goal of systematically identifying, critically appraising, and synthesizing all relevant studies on inter-brain neural synchrony during social interactions. An explicit systematic method, the preferred reporting items for systematic reviews and meta-analyses (PRISMA) was used to address three specific research questions (RQs) regarding the neuroimaging method (RQ1), the application domain (RQ2), the and experiment paradigm (RQ3), which articulate the current-state-of research being conducted with hyperscanning. PRISMA is known to minimize bias and thus provide reliable findings from which conclusions can be drawn [69]. To the best of our knowledge, this is the first systematic review study that used PRISMA to compile all relevant and cutting-edge hyperscanning research to address the current state-of-the-art hyperscanning research since the first publication in 2002.
RQ1. What is the frequency of the four major neuroimaging methods being used in hyperscanning studies? To answer RQ1, this paper calculated the frequency of research conducted by different neuro-imaging methods being used in the selected 126 hyperscanning studies. To effectively organize and draw data from these papers, we first sorted them by the four major neuroimaging methods: magnetoencephalography (MEG) and electroencephalography (EEG); near-infrared spectroscopy (NIRS) and functional near-infrared spectroscopy (fNIRS); magnetic resonance imaging (MRI) and functional magnetic resonance imaging (fMRI); and others. It is important to know which neuroimaging methods are most frequently used, because that information can show us which hyperscanning methodologies researchers should focus on improving first to further neural synchrony research.
RQ2. In what areas has past and current hyperscanning research been conducted within each neuroimaging method? To address RQ2, this paper further categorized the papers in each neuroimaging method by application domain (e.g., cognition, decision-making, motor synchronization, and education) to report the number of studies performed in each application.
RQ3. How have past and current hyperscanning research studies been experimentally designed within each neuroimaging method and application area? To address RQ3, this paper looked into the frequency of different experimental methods (setting, orientation, and group size) within each application. This information is able to tell us what current research is using for hyperscanning experiments and how that changes based on application. From here, we are able to discuss why certain applications prefer one neuroimaging method, setting, orientation, or group size over another, and whether research should be conducted using other experimental methods to advance research in that application.
2. Review Method
A systematical approach, PRISMA [69], was utilized in this review. Research articles were sought out from five major search engines: (a) IEEE Xplore, to provide an electrical/electronic engineering perspective; (b) PubMed, to provide a medical perspective, (c) Engineering Village, to provide an engineering perspective; (d) Web of Science, to provide a cross-disciplinary perspective; and (e) Scopus, to provide a broad-spectrum perspective [70,71].
2.1. Inclusion and Prescreening Criteria
Inclusion criteria were journal articles written in English from 2000 to 19 June 2019, since the first journal article related to hyperscanning research was published in 2002 [8]. Other publication forms (e.g., proceeding papers, unpublished working papers, master’s and doctoral dissertations, newspapers, and books, etc.) were not included. Since journal articles indicate a high level of research, journal articles can help both practitioners and academicians to obtain knowledge and spread their study findings.
Keywords used in search engines were (1) hyperscanning, (2) hyperscanning and inter-brain synchronization or dynamic coupling or brain network or social interaction, (3) hyperscanning or social neuroscience and network index or EEG or MEG or fNIRS or fMRI. Figure 2 shows the flow diagram of PRISMA with the combinations of keywords and the number of studies for each keyword combination from the online databases. After the keyword search, duplicates were removed and 986 articles remained. Those articles were screened again based on titles and abstracts, and 126 research studies remained.
2.2. Eligibility Criteria
Prescreened articles were checked for eligibility via full-text screening by following analyses of populations, interventions, comparisons, outcomes, and study design (PICOS) [69]:
Populations: Experiment studies conducted with multiple human subjects for any age, gender, or clinical conditions that meet the inclusion criteria were included. Studies where subjects interacted with each other but their brains were not measured simultaneously were excluded.
Interventions: For RQ1, we screened for studies that used one of the four neuroimaging methods in one of the four application domains. For RQ2, we screened for studies that demonstrated use of a methodology for measuring the inter-brain synchrony between the human subjects.
Comparators: We screened for studies with a design that included a control group or that measured multiple brains so as to provide an inter-brain synchrony comparison.
Outcomes: We screened for studies that accounted for intra-personal brain dynamics during inter-personal interactions using hyperscanning methods.
Study designs: We screened for studies that used one of the three experimental paradigms.
3. Results and Discussion
Figure 3 shows the status of hyperscanning based on the 126 selected hyperscanning papers (number of papers, percentage). As shown in Figure 3a, the most used neuroimaging method with hyperscanning was MEG/EEG (60, 47%) and the least used neuroimaging method was hyper-transcranial Alternating Current Stimulation (ACS) (1, 1%). fNIRS/NIRS hyperscanning was the second most used neuroimaging method (44, 35%), followed by fMRI/MRI hyperscanning (21, 17%). Figure 3b shows the frequencies of the research applications across the 126 selected papers. Applications in cognition accounted for almost half the studies (60, 48%), while educational applications accounted for less than 5% of the studies (4, 3%). Applications in decision-making tasks were the second most common (33, 26%), shortly followed by applications in motor synchronization (29, 23%).
3.1. MEG/EEG-Based Hyperscanning Research
Table 2 shows a summary of 60 MEG/EEG-based hyperscanning papers. Of 60 MEG/EEG-based hyperscanning papers, the cognition category was the most common application (28, 47%), followed by decision-making (16, 27%), motor synchronization (14, 23%) and education (2, 3%). Out of the hyperscanning studies performed with MEG/EEG, 83% (50/60) of the studies were performed in a laboratory setting. As far as participant orientation and organization, 57% (34/60) of the studies were performed with participants in a face-to-face orientation, and 83% (50/60) were performed with participants in pairs.
3.1.1. Cognition
The cognition application makes up 47% (28/60) of the research done with the MEG/EEG neuro-imaging methodology. The majority of studies were done by giving participants a visual, auditory, or speech task. Computer games were used to investigate the cognition applications [57,72,73,74]. Another task was facial expression or visual search activity [19,75,76,77]. For example, Lachat [78] did the first study to associate joint attention with alpha and mu rhythm modulations and used an eye-gaze task. Some studies used listen/response activities [27,79,80,81]. Perez et al. [80] investigated interbrain synchronization patterns in pairs of participants interacting through speech by having participants alternate between the role of speaker/listener for semi-structured oral narratives without interpersonal visual contact. Some studies utilized in-person games or role-play [34,82,83,84]. Venturella et al. [82] looked at how hyperscanning can be used in leadership/management of a business. This study had participants role-play as a leader and an employee during a work performance evaluation. This role-play showed information about leadership and communicative styles. Six studies looked at the neural-synchrony between musicians during a performance. Three of the studies looked at musicians who played guitar [25,85,86], and three other studies looked at musicians who played the saxophone [23,87,88]. One unique study was the first to look at the effect of physical touch (handholding) on pain alleviation [89]. Using couples that were either married or strangers, the study found that hand-holding during pain increases the neural synchrony between pairs and improves the accuracy of observer’s empathic reaction based on how accurately the second participant rated his partner’s pain level.
Setting. A majority of studies (25/28, 89%) were conducted in the lab while 11% (3/28) of the studies were done in a natural setting [23,87,90]. For example, Babiloni et al. [87] demonstrated that the EEG methodology is suitable for high-quality EEG data in subjects playing in an ensemble outside of a lab. Recently, Fenwick et al. [90] was the first study to investigate neural synchrony during meditation. In this experiment, a 60-year old French philosopher and meditation teacher led a meditation practice (without verbal communication) for himself and two male pupils in a natural meditation setting.
Orientation. Of the 28 studies in the cognition application, 61% (17/28) were done with participants seated side-by-side, while 39% (11/28) were done with participants seated face-to-face. Studies on musicians were arranged side-by-side as they were recorded with EEG while playing their instrument [23,25,87,91]. Recently, Cha et al. [19] had pairs of male subjects look for the wrong pieces of a puzzle in order to quantify the neural synchrony between subjects during a collaborative process through EEG hyperscanning.
Group Size. Eighteen percent (5/28) of the studies looked at the neural responses in more than two participants compared to pairs [23,75,87,88,91]. For example, Greco et al. [88] was the first study on neural synchrony between musicians. This study evaluated the eventual occurrence of synchronous oscillatory brain activity across subjects through an EEG hyperconnectivity study done with a professional quartet. Muller et al. [91] explored the role of interbrain synchronization based on brain network indices (e.g., small-worldness, efficiency) when playing in music in an ensemble. Burgess et al. [75] aimed to compare the performance of different hyper-connectivity measures using simulated data and individually recorded EEG hyperscanning.
3.1.2. Decision-Making Task
The decision-making task application makes up 27% (16/60) of the research done with the MEG/EEG neuro-imaging methodology [3,6,16,29,36,42,92,93,94,95,96,97,98,99,100,101]. The goal of these studies was to see the neural-synchrony between participants as they either collaborated and made decisions (trust/cooperative) or made secret decisions (deception/competitive). These experiments most frequently used the prisoner’s dilemma game and/or a card game to test participants ability to decide under different conditions. Four studies looked at decision-making between pilots and co-pilots in a flight simulator [6,16,92,98]. One study used a version of the ultimatum game to study decision-making [42], while one other study used a cooperation/competitive game [99].
Setting. Most of the studies (75%, 12/16) were done in the lab, while the rest were done in a natural setting. For example, Astolfi et al. [95] used the Prisoner’s Dilemma Game, which has the players choose to either cooperate or defect, to understand the cerebral processes generating and generated by social cooperation or competition. Toppi et al. [6] used the pilot flight simulator to investigate brain-to-brain synchrony of cooperative behavior in ecological settings.
Orientation. Half of the studies (9/18) were done with participants facing each other. For example, Babiloni et al. [94] was the first study to use EEG hyperscanning and PDC for brain network. This study investigated the links between prefrontal areas of the different subjects while they played a card game. Two research papers in this application performed two different experiments with different participant orientations and a different number of participants. The first study [100] focused on understanding the cerebral processes generated by social cooperation or competition by doing the following two different experiments: Prisoner’s Dilemma Game (pairs seated side-by-side) and a Card game (more than two people seated face-to-face). The second study [3] expanded on the previous study. To account for the different experiments, we counted those two studies twice in the correct category.
Group Size. More than half of the studies (81%, 13/16) looked at the neural responses in pairs compared to more than two participants (19%, 3/16). For example, Babiloni et al. [96] focused the concurrent multiple-brains activity depending on cooperation or competition activities during a card game. Astolfi et al. [101] investigated simultaneous multiple-brains activities for cooperation between individuals in the card game domain.
3.1.3. Motor Synchronization
The motor synchronization application makes up 23% (14/60) of the research done with the MEG/EEG neuro-imaging methodology [41,58,102,103,104,105,106,107,108,109,110,111,112,113]. The goal of these studies was to look at how participants are able to coordinate their actions with each other. This was most commonly done by asking participants to perform a motor task such as rhythmically tapping their fingers [41,104,105], following someone’s lead (imitation) to tap the sensors at the same time [102,106,108,109,110], or other motor/selective attention tasks [68,107,111,112,113]. Motor synchronization studies were also done via a computer by asking participants to try to coordinate their response times to different matching/shapes games.
Setting. Filho et al. [68] was the only study which explored the shared and complementary mental models in a natural setting. It performed EEG mapping of two brains engaging in the real-world interactive motor task of juggling. The researchers increased difficulty by having participants juggle an increasing number of objects.
Orientation. Less than half the studies, 43% (6/14), were done with participants facing each other, while 57% (8/14) of the studies employed the side-by-side orientation [102,103,104,105,106,110,112,113]. The studies that were done with participants facing each other used motor synchronization tasks like finger tapping. For example, Naeem et al. [113] investigated sub-band modulations in the mu domain in pairs performing different social coordination tasks—variations of finger tapping and motion imitation. Zhou et al. [108] explored neural signatures of leaders and followers in terms of hand kinematics using dual-MEG.
Group Size. All 14 studies (100%) looked at the neural responses between pairs of participants, because they all focused on neuro-synchrony and mirror-neuron responses between pairs.
3.1.4. Education
These studies were conducted in a classroom setting with portable EEG equipment to measure the brain-to-brain neural synchrony and joint attention between a teacher and her students. The education application makes up 3% (2/60) of the research done with the MEG/EEG neuro-imaging methodology. The goal of these studies was to gain more knowledge in the learning-process—specifically in a classroom setting. These studies looked into the most effective teaching methods by measuring their neural-responsivity/attention and testing their material retention with quizzes. In the first study done in a classroom application by Dikker et al. [35]. EEG was used to record 12 students’ brains simultaneously under different teaching conditions (video vs. lecture) and post-class quizzes were given to measure engagement and material retention. This study showed an increase in students’ brain-to-brain synchrony when they are more engaged in class and a positive correlation between brain-to-brain synchrony and teacher likeability. Interestingly, this study also showed that brain-to-brain synchronization was highest in the preferred teaching style (videos), not lectures Bevilacqua et al. [31] extended Dikker et al. [35] research of brain-to-brain synchrony and learning outcomes in a real-world classroom. Specifically, this study evaluated the effect of student-teacher relationships on student-teacher brain-to-brain synchrony in a high school biology class (12th graders). This study found that brain synchrony and quiz scores on class material were higher for educational videos compared to lectures, which is possibly due to videos being more entertaining and therefore more engaging. The study on the biology class found no correlation between student-teacher synchronization and quiz scores, but a positive correlation between student-teacher closeness and quiz scores. However, this study failed to look at retention of information as a function of time and failed to record when, during the class, student-teacher synchrony was highest, and if that timing agrees with the topics covered by missed quiz questions. This information would help to accurately reflect the retention of information.
Setting. No study was done in the lab, instead both studies took place in a natural classroom setting [31,35]. During both studies students were able to look at each other and the teacher, thus we classified this as face-to-face orientation.
Orientation. Both studies also evaluated the neural synchrony between the students (group synchrony) and between the student(s) and teacher, thus 100% (2/2) of the studies looked at the neural responses in a classroom of students (more than two participants). This is an effective way to research a classroom setting, because it takes into account student-to-student interaction as a factor.
Group Size. One study was done over 11 class sessions [35], while the other study took place over six class sessions [31]. Both studies had a sample (class) size of 12 [31,35].
Summary of 60 magnetoencephalography/electroencephalography (EEG/MEG)-based hyperscanning studies.
| Application | Experiment Paradigm | Reference | |
|---|---|---|---|
| Cognition |
Setting | Lab (25) | [10,18,19,25,27,34,57,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,88,89,91] |
| Natural (3) | [23,87,90] | ||
| Orientation | Face-to-Face (11) | [10,27,34,78,79,82,84,85,86,89,90] | |
| Side-by-Side (17) | [18,19,23,25,57,72,73,74,75,76,77,80,81,83,87,88,91] | ||
| Group Size | n = 2 (23) | [10,18,19,25,27,34,57,72,73,74,76,77,78,79,80,81,82,83,84,85,86,89,90] | |
| n > 2 (5) | [23,75,87,88,91] | ||
| Decision- |
Setting | Lab (12) | [16,29,36,42,92,93,94,95,96,97,98,99] |
| Natural (4) | [3,6,100,101] | ||
| Orientation | Face-to-Face (9) | [3,36,42,93,94,96,99,100,101] | |
| Side-by-Side (9) | [3,6,16,29,92,95,97,98,100] | ||
| Group Size | n = 2 (13) | [3,6,16,29,36,42,93,95,96,97,98,99,101] | |
| n > 2 (5) | [3,92,94,100,101] | ||
| Motor |
Setting | Lab (13) | [41,102,103,104,105,106,107,108,109,110,111,112,113] |
| Natural (1) | [58] | ||
| Orientation | Face-to-Face (6) | [41,68,105,108,112,113] | |
| Side-by-Side (8) | [102,103,104,106,107,109,110,111] | ||
| Group Size | n = 2 (14) | [41,68,102,103,104,105,106,107,108,109,110,111,112,113] | |
| Education |
Setting | Natural (2) | [31,35] |
| Orientation | Face-to-Face (2) | [31,35] | |
| Group Size | n > 2 (2) | [31,35] | |
3.2. NIRS/fNIRS-Based Hyperscanning Research
NIRS and fNIRS hyperscanning are common methodologies for experimental studies, given their increased ecological validity compared to other neuro-imaging methods. While NIRS has a lower temporal resolution compared to EEG, it has a significantly higher spatial resolution [114]. Table 3 shows a summary of 44 NIRS/fNIRS-based hyperscanning papers. The most common application is cognition, representing seventeen of the 44 studies (38.6%). Studies with decision-making task applications comprised thirteen of the 44 studies (29.5%), closely followed by applications in motor synchronization, composed of twelve studies (27.3%). Applications in education were the least common, with only two studies (4.6%). Some examples of fNIRS/NIRS hyperscanning experimental tasks include computer games, motor tasks, synchronization games, musical activities, and facilitated discussion/brainstorming sessions.
3.2.1. Cognition
NIRS and fNIRS hyperscanning have been used for cognitive applications in numerous experimental tasks. These experimental tasks include computer games [114,115,116,117,118], verbal communication [26,119,120], music [37,116,121], gift exchange [122], photography [123], singing [124], the Prisoner’s Dilemma Game [125,126], and puzzles [127]. Using cooperative and competitive computer games, Reindl et al. [117] examined the neurobiological underpinnings of emotional bond formation and a child’s development of emotional regulation. Furthermore, given the increased prevalence of virtual communication, another study [26] investigated the neural difference between face-to-face communication and other forms of communication, such as through a computer. For example, the most recent fNIRS hyperscanning study for applications in cognition explored the effect of prosocial behavior through all-female pairs engaging in a gift exchange task [122]. The results showed that a greater interpersonal engagement between two individuals can lead to significant increase in the coordination of behavioral activities.
Setting. Out of the seventeen studies, two were conducted in a natural setting [37,120], while the majority (15) were conducted in a lab setting. For example, Reindl et al. [117] tested the coherence between parents and children pairs while they played a cooperation-or-competition game in the lab.
Orientation. Eight studies oriented participants side-by-side [37,114,115,116,117,121,122] while nine studies had participants face each other [26,116,118,119,120,123,124,126,127] to complete experimental tasks.
Group Size. Three studies had participant group sizes greater than two [119,120,127], but a majority (13) organized participants into pairs. Some studies only selected male participants [114,118,121], others selected only female participants [116,122], and the remaining studies included both male and female participants.
3.2.2. Decision-Making Task
NIRS and fNIRS hyperscanning has been used in a variety of decision-making task applications, particularly forms of cooperative and competitive games. Common experimental tasks include computer games [9,128,129,130,131], card games [33,132], an economic exchange game [30], creative thinking [133,134], and cooperative tasks [135,136]. A unique experimental task incorporated the game of Jenga [130].
Setting. Out of the thirteen studies, two were conducted in a natural setting [9,130], while the majority were conducted in a lab setting.
Orientation. Five studies oriented participants side-by-side [9,128,129,130,131,134], while eight studies had participants sit face-to-face [30,33,130,131,132,133,135,136].
Group Size. The majority (12) organized participants into pairs as in the cognition application e.g., [137], while two studies organized participants in groups greater than two [135,136,138].
Only one study selected all female participants [135], while the remaining incorporated both male and female subjects in the studies. A study with a large sample size (n = 222, 112 males) sought to understand how cooperation is influenced by male-male, female-female, and male-female pairings [131]. For example, the most recent fNIRS hyperscanning study which focused on decision-making tasks investigated how different feedback affects group creative performance to reveal the underlying interpersonal neural correlates [134]. The experiment consisted of a group brainstorming and feedback session with a large sample size (n = 118, 16 males).
3.2.3. Motor Synchronization
Applications in motor synchronization were relatively common and generally had participants complete synchronous physical motions as an experimental task in the lab e.g., [139]. These tasks included a cooperative button press task [140] where participants interact either side-by-side e.g., [141] or face-to-face e.g., [142], computer games [51,141,142,143], joint-tapping tasks [32,40,144], and synchronization tasks [145,146,147]. One study utilized a finger-tapping task to record between-brain hemodynamics [40]. A unique study examined cooperation-related inter-brain synchrony between mothers and their children [147].
Setting. All of the twelve studies in applications in motor synchronization were conducted in a lab setting such as in [32,41,139]. For example, Funane et al. [140] tested the correlation between fNIRS signals and time intervals of button pressing after hearing auditory cues. They were able to manipulate the auditory stimulations and sync with the fNIRS equipment in a lab setting.
Orientation. Applications in motor synchronization adopted various orientation methods e.g., [32,40,51,139,140,141,142,143,144,145,146,147]. For example, Six studies had participants oriented side-by-side [51,139,141,142,143,146], Two studies had participants oriented back-to-back [32,144], and four of the studies had participants interact face-to-face [40,140,145,147].
Group Size. One group organized participants into groups greater than two [139], but the majority (11) organized participants into pairs. One study selected only females for a coordination task [145], while the remaining studies incorporated both males and females.
3.2.4. Education
Applications in education were the least common among NIRS and fNIRS hyperscanning studies, with only two studies investigating education. First, Brockington et al. [148] utilized fNIRS hyperscanning to explore different paradigms between teachers and students in an academic context. The study conducted three small experiments, two in a classroom and one featuring a single student with eye-tracking glasses.
The results provide proofs of concept for the application of near-infrared spectroscopy in scenarios that more closely resemble authentic classroom routines and daily activities [148]. Recently, fNIRS hyperscanning has expanded its application in education by investigating how communication mode and knowledge state impacted teaching effectiveness [149]. The results found that higher task-related interpersonal neural synchronization (INS) in the left prefrontal cortex (PFC) was found in the FTF teaching condition with prior knowledge. The findings suggest that INS could be a possible neuromarker for dynamic evaluation of teacher-student interaction and teaching effectiveness. Both studies had experimental tasks where participants role-played in classroom scenarios.
Setting. The two studies were conducted in a classroom-like lab setting [148,149] and had subjects interact face-to-face.
Group Size. One study organized participants in a group larger than two [148], while the other study organized participants into pairs [149].
Summary of 44 fNIRS/NIRS-based hyperscanning research.
| Application | Experiment Paradigm | Reference | |
|---|---|---|---|
| Cognition |
Setting | Lab (15) | [26,114,116,117,118,119,121,122,123,124,125,126,127,137] |
| Natural (2) | [37,120] | ||
| Orientation | Face-to-Face (9) | [26,117,119,120,123,124,126,127,149] | |
| Side-by-Side (8) | [37,114,115,116,117,121,122,125] | ||
| Group Size | n = 2 (14) | [26,114,116,117,118,121,122,123,124,125,126,137] | |
| n > 2 (3) | [119,120,127] | ||
| Decision- |
Setting | Lab (11) | [30,33,128,129,130,131,132,133,134,135,136,138] |
| Natural (2) | [9,130] | ||
| Orientation | Face-to-Face (8) | [30,33,130,131,132,133,135,136] | |
| Side-by-Side (5) | [9,128,129,136,138] | ||
| Group Size | n = 2 (11) | [9,30,33,128,129,130,131,132,133,135,138] | |
| n > 2 (2) | [134,136] | ||
| Motor |
Setting | Lab (12) | [32,40,51,139,140,141,142,143,144,145,146,147] |
| Orientation | Face-to-Face (4) | [40,140,145,147] | |
| Side-by-Side (6) | [51,139,141,142,143,146] | ||
| Back-to-Back (2) | [32,144] | ||
| Group Size | n = 2 (11) | [32,40,51,140,141,142,143,144,145,146,147] | |
| n > 2 (1) | [139] | ||
| Education |
Setting | Lab (2) | [148,149] |
| Orientation | Face-to-Face (2) | [148,149] | |
| Group Size | n = 2 (1) | [149] | |
| n > 2 (1) | [148] | ||
3.3. MRI/fMRI-Based Hyperscanning Research
MRI/fMRI neuro-imaging hyperscanning studies require each participant to lie completely still in the scanner, yet in order to study social interaction, participants need to interact with each other [55]. Participant interaction in an MRI scanner is performed through a computer interface, which poses questions of real-world validity. Table 4 shows a summary of 21 MRI/fMRI-based hyperscanning papers. Out of the 126 research papers selected for this review study, 21 papers used MRI/fMRI hyperscanning. Figure 3 shows that the cognition category was the most common application addressed with MRI/fMRI (15, 71.4%), followed by decision-making (4, 19%) and motor Synchronization (2, 10%).
Of the hyperscanning studies which used MRI/fMRI, 85.7% (18/21) of the studies were performed in a laboratory setting. As for participant orientation and organization, 19% (4/21) of the studies placed participants in a face-to-face orientation and 85.7% (18/21) put the participants in pairs.
3.3.1. Cognition
The cognition application is the most common area for research when using MRI/fMRI hyperscanning (15, 71.4%). These studies analyzed neural synchrony between participants who engaged in various activities, with communication, eye contact, and joint attention being common activities of interest. Six studies looked at neural synchrony during communication and speech [38,150,151,152,153,154]. Three studies evaluated neural synchrony in pairs by recording eye-contact [155,156,157]. One unique study in particular used online video chat between two participants to analyze the neural activity associated with real-time eye contact (in person) versus non-real time eye-contact (video chat). Three studies used computer games to analyze social interaction [28,158,159]. Schippers et al. [39] used charades to map information flow between two brains during gestural (non-verbal) communication. They measured the neural activity between the participant guessing and the participant acting out the word to understand the role of the putative mirror neuron system when not taking part in verbal interactions. Two studies used a movie to measure neural synchrony between subjects while observing social reactions in a movie [160,161].
Setting and Orientation. The most common experimental methods included pairs of participants positioned side-by-side in a lab setting. Twelve of the fifteen experiments done by MRI/fMRI in the cognition category were performed in a lab (80%), while the other three studies were done in a natural setting. For example, Jääskeläinen et al. [160] used fMRI to reveal between-subject correlation of fMRI activity in the right hemisphere and frontal cortical areas while watching a movie. During this study, participants watched the first 72 min on an LCD screen with surround sound in a dimly lit room (to simulate a movie theater). Participants were instructed to act as though they were in a movie theater.
The last 36 min of the movie section was shown to participants during fMRI scanning. In reference to participant orientation, 80% of the fifteen experiments were done with participants seated side-by-side. For example, Tanabe et al. [156] compared brain activity during eye contact and joint attention between autistic spectrum disorder participants and normal participants. This study was the first to demonstrate neural correlation between individuals with ASD and normal individuals during direct/real-time interaction. The simple idea behind their eye contact/gaze shifting task used the image of balls on a screen to represent where the other participant was looking and had the other participant shift their gaze to where the ball on the screen was. Recently, Koike et al. [157] evaluated the neural activity associated with eye-contact during a mutual interaction. This study used a video capturing and displaying set-up so the participants could look at each other’s faces. This study found that mutual interaction during eye contact is mediated by the cerebellum and the limbic mirror system.
Group Size. Looking at the group size in experiments done in the cognition application with MRI/fMRI, 80% of the fifteen experiments were done with pairs of participants, while the other studies were done with more than two participants in one trial. For example, Wilson et al. [150] used auditory/audiovisual narratives to measure neural synchrony between 24 subjects during the narratives.
3.3.2. Decision-Making Task
MRI and fMRI hyperscanning had four studies focusing on applications in decision-making tasks e.g., [8,17,162,163]. The first hyperscanning study done was by Montague in 2002. This study used a competitive two-player computer game to study decision-making during social interactions, and paved the way for future hyperscanning research [8]. King-Casas et al. [17] had participants play an economic trust game to understand social interaction through reciprocity, while another study conducted two experiments using the economic trust game to investigate social credit during a two-person social exchange [163]. The fourth study used an ultimatum game to explore brain processes associated with bidirectional reciprocity [162].
Setting, Orientation, and Group Sizes. All four experiments were conducted in a lab setting, with pairs of participants oriented side-by-side. King-Casas et al. [17] used an economic (trust) game to explore the effect of reciprocity on another person (their partner). This study reports that that reciprocity expressed by one player strongly predicts future trust expressed by their partner—a behavioral finding mirrored by neural responses in the dorsal striatum. Recently, Shaw et al. [162] explored brain processes associated with the bidirectional reciprocity characterizing real-world, repeated dyadic interactions. This study revealed, for the first time, that brain signals implicated in social decision-making are modulated by these estimates of EU, and become correlated more strongly between interacting players who reciprocate one another.
3.3.3. Motor Synchronization
MRI and fMRI were the least commonly used methods for investigating motor synchronization. The first study investigated how attention is shared between pairs of subjects [164]. This study conducted three experiments which used an eye blink synchronization experimental task to measure shared attention between a pair of participants. Results from this experiment suggest that shared attention is represented and retained by pair-specific neural synchronization that cannot be reduced to the individual level. The second study [165] evaluated the neural substrates of cooperation using a grip-force synchronization task to evaluate the neural substrates of cooperation during a joint force-production task. Participants were positioned side by side in two separate fMRI machines to measure the neural synchrony while participants attempt to synchronize their grip strength force.
Setting, Orientation and Group Size. Both studies were conducted in a lab setting and organized groups of participants into pairs. Koike et al. [157] evaluated how information is shared and retained between two people by asking participants to interact face-to-face. This study featured an equal number of male and female subjects. On the other hand, Abe et al. [165] looked into the neural substrates of cooperation during a joint force-production task, and had participants engage side-by-side. This study only included female subjects.
Summary of 21 fMRI/MRI-based hyperscanning research.
| Application | Experiment Paradigm | Reference | |
|---|---|---|---|
| Cognition |
Setting | Lab (12) | [28,38,39,150,152,153,154,155,156,157,158,159] |
| Natural (3) | [151,160,161] | ||
| Orientation | Face-to-Face (3) | [154,156,157] | |
| Side-by-Side (12) | [28,38,39,150,151,152,153,155,158,159,160,161] | ||
| Group Size | n = 2 (12) | [28,38,39,150,151,152,153,155,158,159,160,161] | |
| n > 2 (3) | [150,160,161] | ||
| Decision- |
Setting | Lab (4) | [8,17,162,163] |
| Orientation | Side-by-Side (4) | [8,17,162,163] | |
| Group Size | n = 2 (4) | [8,17,162,163] | |
| Motor |
Setting | Lab (2) | [164,165] |
| Orientation | Face-to-Face (1) | [164] | |
| Side-by-Side (1) | [165] | ||
| Group Size | n = 2 (2) | [164,165] | |
4. Conclusions
We systematically identified, critically appraised, and synthesized 126 relevant studies on inter-brain neural synchrony during social interactions since the first publication in 2002, which fitted the pre-specified eligibility criteria. We used an explicit systematic method, the preferred reporting items for systematic reviews and meta-analyses (PRISMA) to address three specific questions regarding the neuroimaging method (RQ1), the application domain (RQ2), and the experiment paradigm (RQ3).
First, it is important to understand what neuroimaging methods have been frequently used, because that information can show us which hyperscanning methodologies researchers should focus on improving first to further neural synchrony research. To answer RQ1 (What is the frequency of the four major neuroimaging methods being used in hyperscanning studies?), we systematically categorized the selected 126 hyperscanning studies by the four major neuroimaging methods: EEG/MEG, NIRS/fNIRS, MRI/fMRI and hyper-tACS. We found MEG/EEG (47%) the most widely used neuroimaging method with hyperscanning, and the hyper-tACS method was rarely applied. Second, we further categorized the papers by the application domains of cognition, decision-making, motor synchronization, and education. Applications in cognition accounted for almost half the studies (48%), while educational applications accounted for less than 5% of the studies. Applications in decision-making tasks were the second most common (26%), shortly followed by applications in motor synchronization (23%). Finally, we found that hyperscanning research studies have been experimentally designed in various settings, orientations, and group sizes within each neuroimaging method and application area. Measuring brain activity simultaneously from more than two people interacting, or “hyperscanning”, has enhanced our understanding of the neurobiological aspects of human social interaction. However, hyperscanning literature has been narratively reviewed in that the reviews have only recorded and assessed the state of knowledge on a particular topic in hyperscanning. PRISMA is known to minimize bias and thus provide reliable findings from which conclusions can be drawn [71].
To the best of our knowledge, this is the first systematic review study that used PRISMA to compile all relevant and cutting-edge hyperscanning research to address the current state-of-the-art hyperscanning research since the first publication in 2002. Future challenges and directions from this review of hyperscanning studies include:
Various neuroimaging methods (e.g., hyper-tACS) are still required to assess the synchronization between multiple brains.
The inter-brain synchrony study that compares social interactions between normal individuals and groups with social cognitive impairment would be able to explore research findings on the corresponding brain regions to social cognitive impairment.
Future studies should pay more attention to more factors that may influence the anatomical and functional similarity of the two brains as well as the inter-brain synchrony.
Simple and easily interpretable experiment paradigms are still required and need to be extended to a wide range of fields (e.g., natural settings) if this is to be a useful design of social interactive experiments to conduct hyperscanning studies.
This systematic review, based on the findings of documented, transparent, and reproducible searches, should help build cumulative knowledge and guide future research regarding inter-brain neural synchrony during social interactions, that is, hyperscanning research.
Author Contributions
Conceptualization, C.S.N.; methodology, C.S.N., S.C. and J.H.; validation, C.S.N., S.C. and J.H.; formal analysis, C.S.N., S.C., J.H. and J.P.; investigation, C.S.N., J.H. and J.P.; data curation, C.S.N., J.H. and J.P.; writing—original draft preparation, C.S.N., S.C. and J.H.; writing—review and editing, C.S.N.; visualization, S.C. and J.H.; supervision, C.S.N.; project administration, C.S.N.; funding acquisition, C.S.N. All authors have read and agreed to the published version of the manuscript.
Funding
This research was partly supported by the National Science Foundation (NSF) under Grant NSF BCS-1551688. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the NSF.
Acknowledgments
We would like to thank all those who have helped in carrying out the research, including Katie Lawson and Erica Zack for her assistance in data collection and initial analysis.
Conflicts of Interest
The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.
Figures and Table
Enlarge this image.Figure 2. Modified preferred reporting items for systematic reviews and meta-analyses (PRISMA) flow diagram of hyperscanning paper review.
Enlarge this image.Figure 3. Current state of the art of hyperscanning research. The two numerical values in the parenthesis indicate that, out of the 126 selected hyperscanning papers, the number of hyperscanning studies that used (a) each neuroimaging method (e.g., 21 fMRI/MRI papers) and its percentage (e.g., 21/126 = 17%), and that were conducted in (b) each application domain (e.g., 29 motor synchronization) and its percentage (e.g., 29/126 = 23%).
A summary of 24 review papers on hyperscanning research published since 2002.
| Catagory | Studies | |
|---|---|---|
| Modality | fMRI family | [48] |
| fNIRS family | [52] | |
| EEG family | [49,50,51,54,67] | |
| Multimodality | [12,54,55,56] | |
| Application | Joint action | [57,68] |
| Social neuroscience | [59,60] | |
| Social interaction | [11,61,62,63,64] | |
| Interpersonalcoordination | [65,66] | |
| Other | General hyperscanning | [46,47] |
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Details
; Choo, Sanghyun 1
; Huang, Jiali 1
; Park, Jiyoung 2
1 Fitts Department of Industrial and Systems Engineering, North Carolina State University, Raleigh, NC 27695, USA;
2 Department of Clinical Research for Rehabilitation, National Rehabilitation Research Institute, Seoul 01022, Korea;