Combined real-time fMRI and real time fNIRS brain computer interface (BCI): Training of volitional wrist extension after stroke, a case series pilot study

Abstract

[...]EEG-based BCIs possess a lower spatial resolution in comparison to other imaging methods. [...]we turned our attention to the neurovascular information captured by functional magnetic resonance imaging (fMRI); the greater spatial precision of fMRI may lend itself to valuable neural feedback in a BCI system for motor learning after stroke. [...]recent research supports the need for many sessions of motor training after stroke [5–7]. [...]in order to extend the number of BCI sessions we included neural feedback training with functional near infrared spectroscopy (real time fNIRS (rt-fNIRS)) as a component of the intervention protocol, even though NIRS is not as spatially precise as fMRI. [...]fNIRS is based on the same signal as fMRI, oxy-and deoxy-hemoglobin, and fNIRS affords the advantages of a functional body position, no distracting noise, easy interactions between therapist and patient, and less overall cost for future consideration; thus, we incorporated the component of rt-fNIRS into our intervention, to be offered sequentially subsequent to rt-fMRI. [...]our primary objective was to conduct a pilot study in which we would develop a sequential rt-fMRI and rt-fNIRS neural feedback system and test it in a small case series of stroke survivors. [...]we framed the BCI intervention within a long-dose protocol containing treatment components targeting an array of impairments, and proven to have been efficacious upper limb motor function in past work [5–6].

About the Authors:

Avi K. Matarasso

Roles Data curation, Formal analysis, Investigation, Software, Validation, Visualization

* E-mail: [email protected] (JJD); [email protected] (AKM)

Current address: Department of Bioengineering, University of Washington, Seattle, Washington, United States of America

Affiliations Brain Rehabilitation Research Center, Malcom Randall VA Medical Center, Gainesville, Florida, United States of America, Department of Chemical Engineering, College of Engineering, University of Florida, Gainesville, Florida, United States of America

ORCID logo https://orcid.org/0000-0001-5876-1871

Jake D. Rieke

Roles Data curation, Formal analysis, Methodology

Affiliation: Brain Rehabilitation Research Center, Malcom Randall VA Medical Center, Gainesville, Florida, United States of America

Keith White

Roles Formal analysis, Investigation, Methodology, Validation, Visualization, Writing – review & editing

Affiliation: Brain Rehabilitation Research Center, Malcom Randall VA Medical Center, Gainesville, Florida, United States of America

M. Minhal Yusufali

Roles Data curation, Formal analysis

Affiliations Brain Rehabilitation Research Center, Malcom Randall VA Medical Center, Gainesville, Florida, United States of America, J. Pruitt Family Department of Biomedical Engineering, College of Engineering, University of Florida, Gainesville, Florida, United States of America

Janis J. Daly

Roles Conceptualization, Data curation, Formal analysis, Funding acquisition

* E-mail: [email protected] (JJD); [email protected] (AKM)

Affiliations Brain Rehabilitation Research Center, Malcom Randall VA Medical Center, Gainesville, Florida, United States of America, Department of Neurology, College of Medicine, University of Florida, Gainesville, Florida, United States of America

ORCID logo https://orcid.org/0000-0002-8985-7240

Introduction

Dyscoordination of the upper limb is a common impairment after stroke, persisting even after standard neurorehabilitation. Dyscoordination of the upper limb impairs performance of normal functional tasks which require coordinated wrist and forearm joint movements [1]. Wrist extension is critical to normal function of the hand in activities of daily living, workplace and leisure activities [2]. Therefore, we focused on wrist coordination training in the current work.

For moderately or severely impaired stroke survivors with chronic motor impairment, standard approaches as well as many emerging therapies have shown mixed or limited success [3, 4]. Recently, we and others have published studies that employed innovative peripherally-directed treatment (no neural feedback), and have shown promising results in terms of clinically and statistically significant improvement in functional task performance and arm/hand coordination [5–7]. The shortcoming of these protocols is that they require either intensive therapist time (Ward 2019b) or long-dose treatment [5, 6]. In contrast to this past work employing peripherally-directed exercises, we employed a neural feedback training paradigm. This is possible due to the emergence of rapid signal processing capabilities that can provide ‘real-time’ neural feedback to a stroke survivor, to characterize the participant’s brain signal during coordination practice and training. Brain computer interfaces (BCIs) provide feedback composed of brain signal features to a system user. There are multiple possible applications of BCIs, such as movement assistance for one who is quadriplegic. The use of BCI systems to enhance motor learning after stroke is an emerging field of study [8–10]. BCIs can be based on a number of different imaging methods, all of which are used to provide the stroke survivor with brain neural feedback to improve motor control through neuroplasticity [11, 12].

Several different modalities have been used to construct BCIs. BCIs can be based on electrophysiological signal, such as electroencephalography (EEG). EEG-based BCIs show some promise and mixed results in application to stroke survivors [13–19]. An advantage of EEG is its high temporal resolution. However, EEG-based BCIs possess a lower spatial resolution in comparison to other imaging methods. Therefore, we turned our attention to the neurovascular information captured by functional magnetic resonance imaging (fMRI); the greater spatial precision of fMRI may lend itself to valuable neural feedback in a BCI system for motor learning after stroke. Magnetic resonance imaging (MRI) signal has been the least studied imaging method upon which to base a BCI system for motor re-training after stroke [20]. Though there is currently no evidence that MRI-based BCI produced recovery of upper limb coordination, there is evidence that real time fMRI (rt-fMRI) neurofeedback enabled study participants to volitionally regulate brain signal in regions relevant to motor control as follows: primary motor cortex [21, 22]; pre-motor cortex [23, 24]; ventral premotor cortex [25]; and motor cortico-thalamic communication [26]. Therefore, with evidence of brain signal modification capability, we incorporated the component of real-time fMRI (rt-fMRI) neural training into our intervention protocol.

Though MRI is the most spatially precise imaging technology available, disadvantages include the following: expense of MRI (important for future clinical practice issues), the supine position imposed (supine is not a body position for most functional tasks), and the patient burdens of disturbingly loud noise, restricted head/body movements, and the small, confining space. These disadvantages restrict the practical number of MRI-based BCI sessions that would be realistic for clinical care of stroke survivors. At the same time, recent research supports the need for many sessions of motor training after stroke [5–7]. Therefore, in order to extend the number of BCI sessions we included neural feedback training with functional near infrared spectroscopy (real time fNIRS (rt-fNIRS)) as a component of the intervention protocol, even though NIRS is not as spatially precise as fMRI. Functional near infrared spectroscopy (fNIRS) has been recently studied [27], with mixed results. Still, fNIRS is based on the same signal as fMRI, oxy-and deoxy-hemoglobin, and fNIRS affords the advantages of a functional body position, no distracting noise, easy interactions between therapist and patient, and less overall cost for future consideration; thus, we incorporated the component of rt-fNIRS into our intervention, to be offered sequentially subsequent to rt-fMRI. Thus, our primary objective was to conduct a pilot study in which we would develop a sequential rt-fMRI and rt-fNIRS neural feedback system and test it in a small case series of stroke survivors.

The dearth of information on fMRI-based BCIs and the mixed results of fNIRS-based BCIs led us to an additional treatment consideration. Our recent studies and that of others support two important treatment protocol considerations, as follows: 1) target the array of impairments underlying dyscoordination after stroke (Daly 2019); and 2) offer intensive, long-dose neurorehabilitation [5–7, 28]. Therefore, we framed the BCI intervention within a long-dose protocol containing treatment components targeting an array of impairments, and proven to have been efficacious upper limb motor function in past work [5–6]. Specifically, we interleaved clinically-based motor training sessions with the rt-fMRI and the rt-fNIRS; these clinically-based motor training sessions included exercise and standard-practice functional electrical stimulation (FES), both supported by research findings [6, 29, 30]. Thus, the secondary objective was to incorporate the sequential rt-fMRI, rt-fNIRS neural feedback system into a motor learning protocol.

Taken together, the overall purpose of the current work was to conduct a preliminary case series pilot test of a BCI protocol based upon neural signals of sequentially applied rt-fMRI and rt-fNIRS, and framed within clinically-based motor learning components.

Methods

Overall study design and subjects

Design summary.

The design of this pilot study was a small case series of stroke survivors. In addition, in order to characterize the range of normal fMRI and fNIRS brain signal for the wrist extension task, we enrolled and acquired imaging data from ten healthy adults performing the wrist extension task during a single session. The study was conducted in a research laboratory setting and a research imaging facility. For each of the four stroke survivor subjects, participation spanned 3 months (data acquisition and intervention), followed by 3 additional months of no intervention and follow-up data acquisition. For the four stroke survivor cases, the procedures and schedule were as follows:

* Acquisition of baseline measures: motor, fMRI, and fNIRS;

* Three motor learning orientation sessions (once per day; three days; three hr per day);

* Three rt-fMRI neural training sessions (every other day, one hr per day);

* 10 rt-fNIRS neural training sessions (1hr each), alternated with 11 motor learning sessions (3hr per day, motor learning session). These sessions were every weekday for 21 week days;

* Acquisition of fMRI mid-treatment outcome measure;

* 33 additional motor learning sessions (two hr each, every weekday).

* Acquisition of post-treatment outcome measures: motor, fMRI, and fNIRS;

* Acquisition of 3-month follow-up outcome measures: motor, fMRI, and fNIRS.

Subjects.

Stroke survivors and ten healthy adults were recruited by flier advertisement and word of mouth. The study was conducted under the oversight and specific approval of the University of Florida and Malcom Randall VA Hospital ‘Institutional Review Board (IRB) for human subjects’ protection. Subjects provided written informed consent. Stroke survivors were >6mo post stroke and other criteria included: impaired wrist extension, with presence of at least a Trace grade muscle contraction, of the affected wrist extensors; medically stable, with no other prior neurological condition; ambulatory with or without an assistive device, and ability to follow two-step commands. Healthy adult subjects had no known neurological diagnoses and no impairment of wrist extension. Of the six potential stroke survivors who were screened (Fig 1), one did not meet inclusion criteria and one did not enroll due to practical issues (time, etc). For purposes of preliminary study (constrained by funding and time) of the new protocol, four chronic stroke survivors were enrolled and completed the study.

[Figure omitted. See PDF.]

Fig 1. Subject screening, enrollment, and study flow diagram.

https://doi.org/10.1371/journal.pone.0250431.g001

Outcome measures

Motor measures.

Because it is a functional level measure, the Arm Motor Assessment Tool (AMAT) was the primary measure, specifically the AMAT, functional domain (AMAT-F), an ordinal measure assessing the coordination displayed during performance of the functional tasks contained within the AMAT. A secondary measure was the AMAT, timed domain (AMAT-T), which is the summed-time taken to complete the functional tasks in the AMAT. The AMAT is composed of 13 functional tasks of everyday living, for example, ‘use a spoon to scoop up bean’, and ‘unscrew jar lid’. For the AMAT-F, the known minimal clinically important difference (MCID) is .44 points [31]. The MCID of .44 points for the AMAT is the value of change in score (from baseline, in this study) in response to a treatment that is required in order for the change score to be considered clinically important. The AMAT is reliable, valid, and a homogeneous measure of functional performance [32]. AMAT is valid across a broad range of impairment levels [33] and notably for the current work, the AMAT is strongly correlated with the impairment measure of joint coordination, the Fugl-Meyer (FM) [34]. The FM is the most widely used measure of coordination in stroke research on an international basis [35]. Thus, this FM impairment measure was included as a secondary measure, measuring single and multiple joint movement coordination [36]. For the FM, the minimally important difference is 4.24 points [37]. The MCID of 4.24 points for the FM is the value of change in score (from baseline, in this study) in response to a treatment that is required in order to be considered clinically important.

Active wrist extension is performed as an integral component of the majority of normally coordinated hand tasks. Therefore, goniometric measures were made for two active wrist extension tasks, as follows:

1. Wrist extension beginning from the wrist-neutral position, with the palm and forearm resting on a flat surface (normal range of motion (ROM) is 70°; [38].

2. Wrist extension beginning from the fully-flexed wrist position (normal ROM is 150°; [38].

For this small pilot, we generated descriptive statistics. For the FM and AMAT, we generated the baseline group mean and standard deviation, the post-treatment group mean and standard deviation, and the group change score and standard deviation from baseline to post treatment. For the AMAT-F and the FM, the resulting change score was compared with the known minimal clinically important difference. Additionally, data for each individual subject are reported in the S1 File. The S1 File of individual motor measures data shows that all data are intact, with the exception that S1 shows missing data at the 3-month follow-up. Motor function data were acquired as follows: pre-treatment; post-treatment, and at 3-month follow up (3moF/U; three months after the last treatment session).

fMRI acquisition and offline analysis.

For stroke survivors, brain function data during wrist extension were acquired at pre-treatment, mid-treatment (at the end of the neural feedback training); post-treatment (at the end of motor learning sessions); and at 3-month follow up (3moF/U; three months after the last treatment session). For each healthy adult, data were acquired during wrist extension at a single session.

Acquisition. For fMRI data acquisition, we used a 3T Phillips Achieva scanner acquiring structural MRI data and functional MRI data during wrist extension. At each MRI acquisition session, we acquired a T1-weighted, high resolution, structural scan using the following parameters: magnetization-prepared, rapid acquisition gradient-echo (MPRAGE) sequence [repetition time (TR) = 7.0 ms; echo time (TE) = 3.2 ms; flip angle (FA) = 8 degrees; acquisition matrix = 240 x 240, field of view (FOV) = 240 mm x 240 mm, slice thickness = 1 mm; and voxel size = 1 mm x 1 mm x 1 mm, 176 sagittal slices]. For functional images, we used the following parameters: a T2*-weighted echo planar sequence (conventional fMRI, TR = 3000 ms, or real-time fMRI, TR = 1500ms); TE = 30ms; SENSE factor = 2; FA (degrees) = 90 degrees; matrix = 80x78; FOV = 240 mm x 240 mm; slice thickness = 3 mm; voxel size = 3.0 mm x 3.08 mm x 3.0 mm (conventional fMRI, 43 axial slices and 116 volumes per run; real-time fMRI, 21 axial slices prescribed dorsally and 232 volumes per run).

Analysis. We conducted conventional analyses, summarized here (detail, [39]. We employed the Statistical Parametric Mapping software (SPM12; Wellcome Trust Centre for Neuroimaging, UCL, UK) and MATLAB R2017a (MathWorks Inc., Natick, MA) to analyze functional and structural MRI data. We co-registered the functional images to the T1-weighted structural image. We used a Gaussian kernel of 6 mm full width at half maximum (FWHM) for smoothing functional images.

To detect outliers of intensity and artifacts of head movement, we used the Artifact Detection Tool (ART; motion artifacts, >2mm translation or >0.01 radian rotation; whole brain intensity variation outliers, Z ≥ 3). Runs were excluded from further analysis if the number of valid scans after artifact detection did not exceed a total time of 5 minutes. In our case, a TR of 1.5 s for rt-fMRI would require at least 200/232 valid scans per run (TR of 3 s for fMRI required at least 100/116 valid scans per run) to be considered for further analysis. Images which contained outliers or artifacts were excluded from subsequent analysis.

To obtain volume of activation, we extracted regions of interest (ROIs) using FreeSurfer. We used FreeSurfer longitudinal processing pipeline to mitigate selection bias across sessions. Also, we visually inspected each extracted ROI to preclude any potential errors in the FreeSurfer longitudinal process. Participant-specific masks were created for a subset of the Brodmann Area (BA) atlas from the Martinos Center for Biomedical Imaging and the Institute of Neurosciences and Biophysics [40]. Regions of interest included primary motor (BA 4, subdivided into Hand Knob and the primary motor region remaining after the hand knob region was removed (Primary Motor–Hand Knob; [39, 41], sensory (BA 3), and pre-motor (BA 6). Variability of ROI size across data acquisition sessions was 1–7 voxels or < 10%, for a given ROI, for each subject. We used small volume family-wise error correction (FWEc) at p = .05, for thresholding activation of voxels in each ROI).

fNIRS acquisition and offline analysis.

Acquisition. For S1 and healthy adults, we used the ETG-400 NIRS system (Hitachi Medical Systems, Japan) to acquire fNIRS data at wavelengths of 695 nm and 830 nm during wrist extension. The NIRSport system (NIRx Medical Technologies, New York, NY, USA) was used to acquire fNIRS data at wavelengths 760 nm and 850 nm during wrist extension for S2, S3, S4. For both, the probe array included 8 sources (transmitters) and 8 receivers (detectors), donned over the ipsilesional hemisphere contralateral to the working arm. The Brainsight (Rogue Research, Montreal, QA, Canada) neuronavigation device ensured proper positioning to acquire signals from the hand-knob area of the primary motor cortext. The fNIRS sampling rate was 10 Hz (ETG-400 Hitachi) and 3.47 Hz (NIRSport system). HbO concentration.

Analysis. We used HOMER2 [42] to analyze data and calculate peak intensity. We followed conventional analysis steps including the following: raw voltage was converted to optical density; we detected and rejected motion artifact [43]; signal was band-pass filtered (cut-off frequencies, 0.01 and 0.5 Hz). Finally, we converted optical density to concentration (μmol/l).

We used deconvolution to recover a canonical, evoked response associated with a given movement within a movement series to calculate HbO concentration. We modeled the oxyhemodynamic response using a Gaussian temporal basis-set, within an epoch beginning 2 s before the cue to ‘move’ and ending 10 s after the cue. The temporal basis-set was composed of multiple Gaussian functions implemented with a standard deviation of 1 s, equivalent to full width half-max 2.355 s, each function offset from the next by a period of 1 s per Gaussian function.

For quality assurance purposes, the time-to-peak (TTP) derived from the deconvolved hemodynamic response was considered as a separate variable. The criterion to exclude a run from further analysis was deviation of TTP greater than two standard deviations from the mean TTP across all runs. Based on that criterion, the following number of movement series were excluded from further analysis: S01 and S02, none excluded; S3, two excluded; and S4, four excluded.

Interventions

Motor learning intervention.

The motor learning protocol was provided at three times throughout the overall intervention, as follows (and as listed in the previous design summary section):

1. 3 ML introductory sessions prior to the rt-fMRI neural feedback sessions;

2. 11 ML sessions interleaved between rt-fNIRS sessions; and

3. 33 ML sessions after completion of the rt-fNIRS sessions (five days/wk).

Motor learning sessions were each 3 hours, provided by an experienced neuro-physical therapist in a clinical research laboratory. Functional electrical stimulation for wrist/hand extensor muscles was provided as a coordination practice, movement-assist device (EMS+2™ Staodyn, Inc, Longmont, Colorado). Motor learning sessions included practice of single joint movement, multiple joint movements, task component, and full task practice [5, 6]. Our motor learning training goal was recovery of the movement components of complex functional tasks in order to recover whole task performance. Wrist extension coordination is essential to use of the hand in functional activities; therefore, the brain neural feedback sessions focused on the wrist dyscoordination present in the subjects. The motor learning sessions extended the brain neural feedback training of wrist extension, continuing wrist extension coordination training without the neural feedback. As the control improved from treatment at this neurophysiological level [44–46], we progressed treatment according to a motor task difficulty hierarchy. Detail regarding treatment content of the ML sessions was provided elsewhere [6]. The hierarchy included single joint movements, multiple joint movements, functional task components, and whole task practice. During training within each of the difficulty hierarchies, we followed particular motor learning principles: movement practice as close to normal as possible [47, 48]; high number of repetitions [49–52]; attention to the motor task [53]; and training specificity [54]. Subjects practiced grasp preparation and grasp, alone and within task components.

rt-fMRI intervention.

Data acquisition, processing, and neural feedback training methods. After the first three motor learning orientation sessions, real time fMRI commenced and was provided by a team of engineers/therapist in three separate sessions of 1 hour’s duration (Fig 2), in a research imaging laboratory.

[Figure omitted. See PDF.]

Fig 2. Real-time functional magnetic resonance imaging (fMRI) acquisition and neurofeedback system: Visual stimulation displayed by custom Python software running on a dedicated presentation computer (8-GB RAM, 2.70 GHz, Core i7 processor) running Windows 7; reconstructed image was sent via DRIN protocol (Phillips Medical Systems) to Turbo Brain Voyager software (TBV, version 3.2) running on a dedicated computer (8GB RAM, 3.6GHz, Xeon E5-1620) to identify active voxels within a defined region of interest (ROI) in real-time.

The feedback estimation was performed in TBV and saved as a bitmap (.PNG file) to disk in a shared network directory. These files were picked up by custom Python software for final display to the subject. The feedback stimulus consisted of a brain picture with a purple icon superimposed. The icon changed in size and brightness to reflect the level of activation in the ROI. This visual stimulus was the source of neurofeedback. (With permission, [39]).

https://doi.org/10.1371/journal.pone.0250431.g002

During these three rt-fMRI sessions, acquisition of MRI data is described above. We used Turbo-BrainVoyager software suite (v3.2; Brain Innovation B.V., Maastricht, Netherlands) for real time fMRI processing.

Testing movement series. For each rt-fMRI neural feedback training day, we conducted one movement series without providing feedback, termed the ‘functional localizer’ movement series. This was a mixed block/event-related design (each move cue and each rest cue was considered an event), was composed of 42 wrist extension movement attempts with interleaved rest periods.

We conducted the functional localizer to define a feedback region-of-interest (ROI), at the beginning of each training session. Our pre-processing steps included the following: motion correction, spatial smoothing (FWHM = 4mm x 4mm x 4mm), and linear drift correction. For statistical analysis we employed a real-time implementation of the general linear model (GLM), recursive least squares GLM. We defined significant voxels using a cluster-forming threshold of a t-score greater than or equal to 4.2, and a cluster-size threshold of 4 voxels. Statistical maps were visualized and we created a region-of-interest mask by manually drawing a rectangular volume around significant voxels. In this manner, we created a feedback ROI mask in the motor region, specifically containing the “hand knob” sensorimotor cortex of the involved hemisphere.

Identification of rt-fMRI signal threshold for neural training. We used the spatially averaged time course of significantly activated voxels within the selected feedback ROI mask in calculating the feedback during the neural training session. Our threshold for rt-fMRI feedback was consistent with the pre-processing data, customized for each participant and included (1) voxels that were significant as defined using a cluster-forming threshold with a t-score greater than or equal to 4.2, and a cluster-size threshold of 4 voxels; and (2) feedback calculation was based on the percent signal change from preceding rest to current feedback movement series. Rest (baseline) was defined based on the “rest” condition prior to an active movement condition. We developed custom Python software to provide the user/participant neural feedback during neural feedback training.

Neural training movement series. For the subsequent rt-fMRI training sessions (Fig 2), we maintained consistency with the content of the functional localizer movement series, and now with the addition of neural feedback training and additional practice wrist extensions. On a given day of rt-fMRI, we provided three training movement series, each containing 54 wrist extensions (total rt-fMRI wrist extensions per day = 162). With three days of training, there was a total of 486 wrist extension practices across the three rt-fMRI neural feedback training (greater detail, [39]).

rt-fMRI feedback presentation. We presented visual feedback to the participant using a computer monitor, with custom-designed Python script. Prior to the real time neural training, we provided verbal instructions and practice recognizing the move cues. During the practice period, we stated, “when you see the ‘move’ cue, please extend the wrist one time, simultaneously increasing the brain signal, hold the extended position for a ‘beat’, and then begin to rest”. The move command consisted of a cartoon of a hand and forearm with the wrist extending. Simultaneously and on the same screen, we provided an animation of brain activity within the outline of a cartoon brain containing a circle located in the hand-knob motor region, and which increased in size and brightness as the stroke survivor increased the brain signal. The color and diameter of the circle changed in response to the brain activity, ranging from light blue to purple. Our system updated the visual feedback every 1.5 s, matching our TR capture.

rt-fNIRS intervention.

Data acquisition and neural feedback training methods. After completion of the rt-fMRI neural training sessions, we initiated a series of 10 rt-fNIRS training sessions (Fig 3). fNIRS acquisition is detailed above, section 2.2.3. Processing of fNIRS data was performed using NIRStar (NIRx Medical Technologies, New York, NY, USA) and custom designed software to implement real-time filtering, feedback estimation, and visualization. Details provided below.

[Figure omitted. See PDF.]

Fig 3. Real-time functional near infrared spectroscopy (rt-fNIRS) neural feedback system.

The command to move was a visual cue including a cartoon of the brain, the words “Wrist extension—GO”, and a cartoon of the wrist moving from a neutral to an extended position in 1 s. Brain activation during the attempt to extend the wrist was captured by NIRx NIRSport. Output from the NIRSport was synchronized with NIRx NIRStar Version 15, which ran on a dedicated acquisition computer (24-GB RAM, 2.60 GHz, Core i7 processor) running Windows 8. The data were pre-processed and thresholded by custom Matlab code. Matlab output informed Python code how to update the visual participant feedback, shown as a purple color in the brain region. Matlab output also triggered functional electrical stimulation (FES) of wrist extensors. (With permission, [39]).

https://doi.org/10.1371/journal.pone.0250431.g003

Testing movement series. We conducted an assessment-only movement series at the beginning of each rt-fNIRS training day; this ‘functional localizer’ movement series was a mixed block/event-related design (an event was a move or rest cue), which contained 36 practice wrist extension attempts.

The time series streams of oxyhemoglobin and deoxyhemoglobin concentrations (μmol/L) were received sample-by-sample and consisted of multichannel data obtained in real time at a regular sampling rate of 3.47 Hz. The Lab Streaming Layer protocol allowed for communication between custom Matlab scripts and NIRx NIRStar software. NIRStar interfaced directly with the fNIRS hardware, acquired the data, converted raw light intensity to concentration data, and provided the functionality to stream data in real-time. NIRStar transmitted in (near) real-time fNIRS data using the Lab Streaming Layer (LSL) protocol. We generated Matlab scripts that served to receive the fNIRS data one sample at a time, pre-process the data, calculate feedback score, and update the visual display to the user. Matlab also handled the time-synchronization of the experimental design.

In real time, we preprocessed using a sliding window. We employed a low-pass Gaussian filter (FWHM = 1.5 s, 15th order) with a cutoff frequency of 0.22 Hz, followed by a DC blocking filter (R = 0.99) with a cutoff frequency of 0.012 Hz to mitigate low-frequency drifts. We then calculated feedback values which were then provided to the user during neural feedback training. Using standard score (z-score) calculation, we calculated feedback values by subtracting the preceding rest period average from the 1s moving average of the movement block and then dividing the difference by the standard deviation of the preceding rest period. Thus, positive (increasing) feedback was provided if the 1s moving average was greater than the average of the preceding rest period; this procedure assisted in compensating for potential spurious changes in the baseline brain activity that could occur during a given training session. We applied the 1s moving average to the feedback values in order to mitigate any potential rapid signal changes (potentially artifactual) from causing sudden spikes.

Identification of signal threshold for neural training (fNIRS). Structural MRI was used as input to the Brainsight Neuronavigation, to locate the hand knob region for NIRS. Functional MRI from the training sessions was analyzed and overlaid on the structural image to confirm activation in the hand-knob region during wrist movement. We conducted the fNIRS functional localizer movement block, with behavioral protocol described above.

Using Brainsight neuronavigation (Rogue Research, Montréal, QA, Canada) to specify the location of the desired optode location, we began by identifying the subset of optodes closest to the hand knob region. We calculated a goodness-of-fit measure using linear regression for each of those channels in the subset. Our work was based on these following assumptions: (1) The evoked fNIRS response conforms to the canonical hemodynamic response function (HRF) and (2) the responses would add linearly. In addition, we added third order polynomial regressors to account for drift. Matrix inversion used ordinary least squares. We ranked the subset channels based on the beta coefficient corresponding to the scale of the HRF divided by the standard deviation of residuals. We then inspected the top-ranked channels to select the feedback channel for the session.

In summary, criteria for selection of the feedback channel included:

1. (a) Hand knob region, determined using Neuronavigation

2. (b) An increase in oxyhemoglobin from rest to movement periods, discounting channels with considerable noise or high frequency changes indicating possible artefacts.

3. (c) Tie breaker: closest proximity to hand knob region.

We calculated feedback values from the selected fNIRS channel. We maintained this channel throughout the given session. During rt-fNIRS neural feedback training, the presentation of feedback to the subject was identical to that described above for rt-fMRI neural feedback training.

We generated a user-defined, signal-to-noise threshold, which we scaled, 0–10 facilitate threshold achievement at which positive feedback was provided. We began each rt-fNIRS neural training session with a threshold of 6 on the scale, at which positive feedback was provided. We varied this threshold between 6 or 7 during the session if performance indicated the need. For example, if 100% of trials achieved threshold at a value of 6, then the threshold was increased to a 7.

We assessed successful control of brain signal during rt-NIRS neural training with a measure of ‘sensitivity’ which was defined as the percentage of wrist movements during which the brain signal was above threshold.

where

TP = (true positive) brain signal above threshold during wrist movement

FN = (false negative) brain signal below threshold during wrist movement (threshold not achieved)

and TP + FN is the total number of possible wrist extension movements.

Neural training movement series. We constructed the subsequent rt-fNIRS training movement series with the same content as the functional localizer movement series, but with the neural feedback added for each wrist movement attempt. We provided four training movement series on each training day (total 144 wrist extensions 36 per movement series, 4 of the movement series).

Rt-fNIRS-triggered functional electrical stimulation. We used surface functional electrical stimulation (FES; EMS+2™ Staodyn, Inc, Longmont, Colorado) during rt-fNIRS. FES was triggered for wrist extension movement-assist when the fNIRS brain signal reached the custom-set threshold. The FES parameters were 300μs pulse width, 30Hz, amplitude set to comfort, and 3s duration of stimulus. We placed the electrodes (1.5 cm x 1.5 cm) over the muscle belly of the wrist extensor muscles, to provide movement-assist for the attempted wrist extension movements.

rt-fNIRS feedback presentation.

Feedback for the stroke survivor during rt-fNIRS was consistent with that provided during rt-fMRI, described above, with two exceptions. First, updates to the visual feedback occurred once per second rt-fNIRS. Second, the stroke survivor received FES-assisted movement that was triggered each time the brain signal was at or above the custom-derived threshold.

Results

Subjects

Stroke survivors characteristics are provided in Table 1. Characteristics for the 10 healthy adults were as follows: 6 males and 4 females, average age 43.8 ± 22.9 years, all right-handed except one male. There were no study-related adverse events.

[Figure omitted. See PDF.]

Table 1. Subject characteristics.

https://doi.org/10.1371/journal.pone.0250431.t001

Outcome measures results

Motor outcome measures.

Group descriptive statistics are provided in Table 2. The primary measure, mean improvement in AMAT-F, functional task performance, was just above the MCID of .44 for the AMAT-F (Table 2, row 1, column D), indicating clinically significant improvement. Three of four subjects had a clinically significant improvement in AMAT-F by post-treatment, and the remaining subject had a clinically significant improvement by follow-up (Individual data are in the S1 File (Section I. Outcome Measures, Section 1.1. Motor Results, Table 1a-1d in S1 File).

[Figure omitted. See PDF.]

Table 2. Group data for motor measures.

https://doi.org/10.1371/journal.pone.0250431.t002

Mean improvement in coordination (FM) was more than double the MCID of 4.25 points for the FM (Table 2, row 2, column D). The mean change in AMAT time was 418s. Individual Subject responses are in in the S1 File (Section I. Outcome Measures, Section 1.1. Motor Results, Table 1a-1d in S1 File).

fMRI outcome measure: Pattern of change from baseline to post-treatment, % volume of activation during wrist extension.

Table 3 shows a compilation of patterns of change comparing pre-treatment to follow-up for the four subjects. A glance at Table 3 illustrates that, for the most part, Subjects 1, 2, and 4 showed a decrease in volume of activation from pre-treatment to follow-up. In contrast, Subject 3 showed an increase in three ROIs in the lesioned hemisphere and two in the non-lesioned hemisphere and she had no change in the remaining ROIs in the non-lesioned hemisphere. Brain maps in Fig 4.1–4.4 reflect these patterns of change for each subject. Individual values for each subject, each data acquisition, each ROI are in the S1 File, Section I. Outcome Measures, Section 1.2. fMRI Outcome Measures, Individual Subject Data; Table 2a-2d in S1 File.

[Figure omitted. See PDF.]

Fig 4. Brain map outcome measures acquired at pre-treatment and follow-up.

Brain maps are shown for right and left hemisphere regions: HK, BA4—HK, BA3, and BA6) reflecting the summary results in Table 3 above for pre-treatment and follow-up. (FWEc, p value = 0.05 with small volume correction, used for thresholding each ROI). Key: R = right hemisphere; LH = left hemisphere; RH = right hemisphere. Individual activation values are in the S1 File, Section I. Outcome Measures, Section 1.2. fMRI Outcome Measures, Individual Subject Data; Table 2a-2d in S1 File. 4.1. Subject 1. S1 in S1 File showed lessening of activation from pre-treatment to follow-up in all ROIs. 4.2. Subject 2. S2 in S1 File showed lessening of activation in all ROIs from pre-treatment to follow-up, except for right sensory region which remained consistent. 4.3. Subject 3. S3 in S1 File showed increases in all left lesioned hemisphere ROIs, except sensory which decreased minimally; there was an increase in right Primary Motor-Hand Knob and all other ROIs remained consistent or with only minimal change. 4.4. Subject 4. S4 in S1 File. For S4 in S1 File, from pre-treatment to post-treatment in the right lesioned hemisphere, there was a 26% decrease for the sensory region, with zero or near zero for the remaining right ROIs. For the left hemisphere, there was a decrease in all ROIs, except for a minimal increase in the Hand Knob region.

https://doi.org/10.1371/journal.pone.0250431.g004

[Figure omitted. See PDF.]

Table 3. Patterns of change from pre-treatment to 3 month follow-up*, according to percent volume of activation; wrist extension motor task.

https://doi.org/10.1371/journal.pone.0250431.t003

fNIRS outcome measures.

fNIRS outcome measure; HbO concentration values summarized for healthy adults and stroke survivors at pre-/post-treatment/fu sessions. Fig 5, Panel A shows the range of the fNIRS signal HbO for each of ten healthy control subjects. The signal HbO was calculated by subtracting resting state signal from the signal value during the wrist extension movement task. In this case for HbO (active-rest), a positive value means that oxyhemoglobin levels were higher during the active (movement or exercise) condition than during the resting condition. This was the predicted result, since increased blood flow floods the activated region with oxygenated blood seconds after the initial conversion of oxyhemoglobin to deoxyhemoglobin during brain activation. The negative values for some, may reflect the relatively minor movement tested (simple wrist extension), so that the typical increase in oxyhemoglobin in the initial seconds after a region is activated was not always recruited. Two of the control subjects (Panel A) had maximum HbO values of 0.4–0.5 μM, two of 0.3–0.4 μM, and six of about 0.1–0.27 μM.

[Figure omitted. See PDF.]

Fig 5. fNIRS: Range of values for ‘active–rest’ hemoglobin concentration during wrist extension task three conditions, all during wrist extension task.

Panel A. Healthy controls fNIRS range of values during wrist extension. Panel B. Stroke survivors during rt-fNIRS training for wrist extension. Panel C. Stroke survivors during wrist extension without neural feedback (pre-, post-treatment, follow-up. Key: Each vertical bar shows the range of HbO for a given individual, each of whom are identified on the horizontal axis. For y-axis, oxyhemoglobin concentration values were calculated as the difference between ‘active-rest’ condition. This difference variable is a change in oxyhemoglobin concentration from rest to the active movement state, derived from the fNIRS signal. The above rectangles for each participant represent the range of values for that given participant. *Data for healthy controls and S1 were acquired using Hitachi fNIRS system and data for S2, S3, and S4 were acquired using the NIRx fNIRS system.

https://doi.org/10.1371/journal.pone.0250431.g005

Panel B results are discussed below under “Performance during feedback training”.

Panel C shows the range of values observed during pre-, post-treatment, and follow-up data acquisitions (no feedback given during acquisition of Outcome Measures). During these testing sessions without feedback (Panel C), stroke survivors had maximum HbO values of 0.8 μM (S2), and near normal of 0.5 μM (S1), 0.5 μM (S3), and 0.2 μM (S4). The elevated value for S2 is consistent with abnormal autoregulation. The expectation is that for some stroke survivors, there may be reduced brain activity due to stroke damage, and thus less oxyhemoglobin is converted to deoxyhemoglobin. Disproportionate inflow of oxyhemoglobin-rich blood as hemodynamic response may result in transient hyper-elevated oxyhemoglobin.

For greater ease in considering the data for four stroke survivors as a group, we compiled those fNIRS amplitude values into a measure of the range of values exhibited within a given subject across all outcome data acquisition sessions (Panel C), in order to visually inspect whether there may be any indication of potential abnormality for the stroke survivors in terms of their range of fNIRS amplitude values during wrist extension. From Fig 5, Panel C, we can note that S2 exhibited elevated values above that shown for control subjects in Panel A. S4 exhibited the lowest signal amplitude compared to the other three stroke survivors, but still within the range of that shown for healthy control subjects.

The Fig 1a-1d in S1 File provides within-subject data for the stroke survivors. According to visual inspection of those data, there was no discernible trend of oxyhemoglobin concentrations derived from fNIRS values from baseline to post-treatment or follow-up (no feedback) for S2 and S4, whereas S1 showed a slight trend for increased HbO and S3 for decreased HbO over time. As a group, no discernible changes in HbO across treatment protocol sessions was evident.

Performance during neural feedback training

Real-time fMRI neural training, fMRI performance measure during neural training sessions.

In addition to our outcome measures, presented above, for pre, mid, post, and follow-up data collections, we acquired fMRI data during the rt-fMRI neural training to study performance during the neural training sessions, according to whether there were patterns of change in volume of brain activation across the rt-fMRI neural training sessions of wrist extension coordination (Table 4 and Fig 6).

[Figure omitted. See PDF.]

Fig 6. Brain maps across three neural feedback training sessions (rows a, b, and c from sessions 1, 2, and 3, respectively) using real time fMRI (rt-fMRI) for each of four cases (6.1, 6.2, 6.3., and 6.4, respectively).

Brain maps show functional magnetic resonance imaging (fMRI) data acquired during each of the three real-time fMRI neural feedback wrist coordination training sessions. A session-specific, t-statistic map was overlaid on the session-specific T1 anatomical image. We used a significance threshold of p < 0.05 with small volume family-wise error correction for each ROI. 6.1. for S1. Brain maps for the left lesioned hemisphere reflect small changes (≤ 10%) or consistent pattern comparing first to last of the three sessions, rows a and c. The right hemisphere showed a marked lessening of activation (19%–62% decreases; details contained in Table 3a, S1 File). 6.2. for S2. Brain maps for the left lesioned hemisphere reflect a consistent pattern of activation comparing first to last of the sessions, rows a and c, for Hand Knob and premotor, and a lessening of activation in Primary Motor-Hand Knob and sensory. The right hemisphere showed a similar pattern across the three sessions, with the exception of a 10% increase in Primary Motor-Hand Knob (details in Table 3b, S1 File). 6.3. for S3. Brain maps reflect a marked lessening of activation comparing the first and last of the three sessions for the left lesioned hemisphere (29% to 59%) and the right hemisphere (11%–56%; details in Table 3c, S1 File). 6.4. for S4. Brain maps reflect lessening of activation comparing the first and last of the three sessions, rows a and c for right lesioned hemisphere (≤ 10%) except for Hand Knob remaining constant. The left hemisphere showed lessening of activation (7%-41%), except for Hand Knob which remained constant; details in Table 3d, S1 File).

https://doi.org/10.1371/journal.pone.0250431.g006

[Figure omitted. See PDF.]

Table 4. Performance during rt-fMRI neural feedback training patterns of change in % volume of activation for specific ROIs across the first and last rt-fMRI neural training sessions.

https://doi.org/10.1371/journal.pone.0250431.t004

We compiled the individual subject data according to their respective increase, decrease or ‘no change’ for volume of activation in each ROI, from the first to the last rt-fMRI neural training session. A summary of results for each ROI is presented below according to the patterns of activation across rt-fMRI neural feedback training sessions (Table 4, below). There was no change in lesioned Hand Knob for S1, S2, S4 or for non-lesioned Hand Knob for S2 and S4. The majority of pattern changes from session 1 to 3 were decreases in activation, except for smaller increases of 5–15% in mostly non-lesioned hemisphere ROIs. Brain maps below in Fig 6.1–6.4 reflect these changes. The individual data are located in the S1 File, Section II, Performance Measures, During Neural Feedback Training; subsection 2.1., Rt-fMRI Neural Training, fMRI Performance Measure, Table 3a-3d in S1 File.

fNIRS performance measures during rt-fNIRS neural feedback training.

Successful brain signal activation control during rt-fNIRS neural feedback training sessions summarized for four stroke survivors. Fig 7 shows that the mean brain activation success rate ranged from 42% to 78% across the four subjects (defined in the methods section as number of movement attempts for which brain signal was above threshold during wrist movement divided by total movement opportunities). S3 had the overall lowest success rate, with values at the initial session of 18%, improving by session 9 to 79%. S4 performance was higher than other subjects with values ranging from 70% to 89%. Individual values varied across the 10 training sessions; these individual subject data are provided in the S1 File (Section II, Performance Measures, Subsection 2.2.1. Successful brain signal activation control during rt-fNIRS neural feedback sessions, Fig 2a-2d in S1 File).

[Figure omitted. See PDF.]

Fig 7. Brain activation mean success rate during rt-fNIRS neural feedback training, for each of four subjects.

For each stroke survivor, mean (rectangle height) success rate for the 10 rt-fNIRS neural training sessions is shown with standard deviation (gray standard deviation lines.

https://doi.org/10.1371/journal.pone.0250431.g007

fNIRS signal amplitude range during rt-fNIRS neural feedback sessions. Fig 5, Panel B (above) shows the range of ‘Active–Rest’ HbO concentration differences for each of the four stroke survivors during rt-fNIRS neural feedback training; these range values were generated from their performance across the ten sessions of rt-fNIRS neural feedback training for the wrist extension task. The stroke survivors exhibit no values below zero; that is, none where the active-state oxyhemoglobin concentration is less than that of the rest state. Across the 10 sessions of rt-fNIRS, the four stroke survivors (Panel B) had maximum HbO values of 0.28 (S1), 0.45 (S3), 0.46 (S2), and 0.5 (S4), respectively. Data for individual subjects for each of the ten training sessions are in the S1 File (Section II. Performance Measures, Subsection 2.2.2. rt-fNIRS signal amplitude values across the 10 rt-fNIRS training sessions, Fig 2a-2d in S1 File).

Discussion

Motor improvement

The motor results at post-treatment are comparable to that reported previously for the motor learning (ML) protocol without neural feedback from our own work and that of others [5–7]. There are two major differences between our prior work and the current study. First, the current study provided neural feedback for 13 sessions, but there was no neural feedback in three prior studies employing motor learning [5–7]. Second, there were fewer overall hours of treatment in the current study (13 hr of neural feedback and 94 hr motor learning, totaling 107 hr of treatment) versus 300 hr in our prior work [5, 6], 60 motor learning sessions, 5 hrs /session). Nevertheless, the current shorter protocol that included neural feedback training produced gains in the FM and AMAT comparable to the longer motor learning prior protocol [5, 6]. In making this comparison, we should note an additional potential contributing factor to the level of motor recovery. That factor is the patient:therapist ratio. In the prior work, there was a patient:therapist ratio of 3:1; whereas, in the current work, there was a 1:1 ratio of patient:therapist for all but a few sessions. Noting the recent work of others that produced high motor control gains in coordination [7], their study also employed fewer motor learning hours (90 hr), but a 1:1 ratio of patient to therapist. Considering these three studies, it is reasonable to consider that overall, 100 hours of 1:1 treatment is needed for gains in the FM ≥10 and clinically significant AMAT gains; and it can be administered either in a group at 3:1 ratio of 300 hr or a 1:1 ratio of around 100 hr. In all these instances, patients received about 100 hours of individual treatment. Taking these findings together, along with the current study design, it must be noted that the subject results in the current study could have been produced partly or solely by the motor learning sessions.

At the same time, one notable point relevant to the current study is that the allocation of 13 hours to neural feedback did not appear to diminish response to treatment in comparison to prior studies of long duration motor learning treatment [5–7], at least for the four subjects in this limited case series. Given that the current neural feedback study did not appear to prevent the motor improvements observed in other long duration studies, even in the presence of fewer hours and sessions of motor learning, the results support further study of neural feedback and its specific and unique effects on recovery of motor function in stroke survivors. Further refining the neural feedback systems and administration, may prove to further reduce the needed hours of treatment required for clinically significant recovery.

Patterns of change (fMRI volume of activation)

Sensory region.

The importance of sensory function during movement is well-known. But after stroke, there may be a progressive lessening of the normal inter-hemispheric connections associated with the sensory regions [56], with poor connectivity associated with greater disability. In the current work with four stroke survivors, there was first an initial higher fMRI volume of activation in sensory regions followed by a lessening of activation across the rt-fMRI training sessions. There was also a lessening of activation from pre-treatment to follow-up testing in sensory regions. This lessening pattern of change could reflect recovery of connections and thus, more effective and efficient volume of activation in the sensory regions.

Contralateral hemisphere.

The role of the contra-lesional hemisphere and M1 region are complex during the chronic phase after stroke. Some have reported that in a subsample of stroke survivors, contra-lesional M1 activity interfered with motor function, perhaps abnormally inhibiting activity in the lesioned M1 region [57]. This finding could be relevant in the current work with regard to S1, S2, and S4, whose activations decreased substantially in the contra-lesional hemisphere, which could have had an influence on the motor recovery that occurred. However, in contrast, in others, there seems to be correlation between abnormally increased activation in contra-lesional M1 and greater motor recovery, which could be consistent with S2 who had a 20% increase in activation for the Hand Knob at follow-up, when her functional task performance was at its highest. In fact, the role of the contra-lesional M1 in recovery is not yet well-elucidated [57].

Variability across subjects and within stroke survivors.

The patterns of change in brain activation are complex after stroke and in response to treatment. Patterns of change are variable across individuals and within an individual across ROIs. A number of variables contribute to this complexity including location of stroke, time since stroke, severity of stroke [28, 58], integrity of remaining structures such as the corticospinal tract [59], and treatment type and duration [28, 60]. Even with consistent treatment, variation across stroke survivors can occur [61], which is evident in the current case series. Additionally, each ROI within a given stroke survivors, serving a unique function, can increase or decrease in volume of activation in response to the confluence of treatment influence and its unique function [61]; in the current work, this was the case for S3, whose patterns of change varied (both increases and decreases across ROIs) from pre-treatment to follow-up.

fMRI; variability of healthy adult brain signal for wrist extension.

The healthy adult fMRI data revealed the variability in volume of activation within a given participant across the ROIs and variability across the ten healthy adults. The task was motorically easy; and on a routine basis, it is somewhat automatically performed within many everyday functional tasks, potentially explaining the lack of significance of brain signal in some healthy adult participants and some ROIs. It appears that the precision of the fMRI methods was often not fine enough to capture the few neuronal activations required for this normally, simple and automatically performed movement. Thus, ‘normal’ range of the volume of activation in this dataset ranged from zero upwards. We should note that because the data for some normal subjects exhibited zero activation in some ROIs, it was not possible to definitively ascertain the meaning within a stroke survivor of a zero score because zero for them could have been reflecting either what was close to ‘normal’ and easy motor control or an abnormal lack of brain signal. In future work, this difficulty may be resolved for this particular task with a more powerful scanner (e.g. 7T) enabling finer resolution of activations (e.g. 1mm3).

Confluence of results.

By post -treatment, three of four stroke survivors (S1, S3, S4) showed clinically significant improvement in functional task performance (AMAT-F) and upper limb joint coordination (FM). By follow-up, the fourth subject (S2) showed near clinically significant improvement, as well. These results are consistent with other work that included moderately/severely impaired chronic stroke survivors [5–7]; in the current work, all were severely impaired. In terms of neural control of wrist extension in both hemisphere ROIs, three of four subjects (S1, S2, S4) showed lessening in volume of activation from pre-treatment to follow-up; one subject (S3) showed increased or consistent volume of activation by follow-up. Both of these change patterns have been identified as response to treatment in chronic stroke survivors and posited as change that could contribute to and drive motor recovery [61]. For HbO values from fNIRS and regarding patterns of change from pre-treatment to follow-up, HbO showed variation across subjects, accompanying motor recovery, as follows: S1, slight increasing trend; S3, decreasing trend; and S1 and S4, no discernible trend.

During rt-fNIRS neural feedback training, participants were successful in modulating brain signal (means ranging 42%–78%); and HbO values varied across the 10 sessions with no overall trend. During rt-fMRI neural feedback training, in the lesioned hemisphere, patterns changes in volume of activation in ROIs were as follows: S3, all decreased; S1, no change (Hand Knob) and small changes of 5%-10% elsewhere; S2 and S4, no change (HK) or decreases. Patterns of change in volume of activation were different in the non-lesioned hemisphere ROIs as follows: S3, reduction in HK, and increases elsewhere; S1, increases; S2 and S4, no change (Hand Knob), decrease in sensory, and S2 and S4 diverged for the Primary Motor-Hand Knob and premotor (S2, increased and S4 decreased). These two most impaired stroke survivors exhibited the same patterns of change from rt-fMRI session 1 to 3 in all but the contra-lesional Primary Motor-Hand Knob and premotor. For S2, after increasing in those latter two contra-lesional motor regions during the rt-fMRI neural feedback, she ultimately recovered greater wrist extension versus S4 by post-treatment and better maintained it by follow-up. More detailed discussion for each subject is in the S1 File, Section III.

Use of the rt-fMRI and rt-fNIRS neural feedback systems

The four participants were able to successfully modify brain signal during neural training sessions (Fig 4), despite their initial severe motor limitations, including complete lack of wrist extension at baseline for S2 and S4 and limited wrist extension for S1 and S3 at baseline.

During neural training, for one subject, S4, there was higher percentage of rt-fNIRS ‘successful’ trials in modifying brain signal compared to the others. One potential reason for this is that in the first three subjects, we noted potential discouragement when too much time between ‘successful hits’ was imposed (15 s rest). Therefore, for S4, we did not impose a 15 s rest, but shortened the interval time between ‘move’ commands after successful hits. From this experience, one could conclude that it is easier to successfully activate brain signal when there is a short interval time; therefore, we recommend for future consideration, a graded approach to setting the interval time between a success and the next motor practice attempt. For example, in early training a short interval could be provided, and as motor control recovers, a longer, potentially more challenging interval time could be introduced.

We found that the fNIRS cap was comfortable enough to be tolerated for up to 70 minutes, which was the time needed to either provide the rt-fNIRS training session or to collect fNIRS data. For MRI testing and training, we allocated sufficient time to ensuring comfort in terms of the positioning of the MRI head coil and torso and limb positioning. In terms of overall relative preference, one subject preferred rt-fMRI and three preferred rt-fNIRS. Participants expressed their motivation in response to the neural training and the presentation software with the brain picture and increasingly dark purple color in the Hand Knob region when their brain activation increased. While tapping their head, participants made such statements as “I can feel right here in my brain where my wrist extension is” (S4).

Limitations

This study focused on development of the software and use of the hardware, and so sample size was necessarily constrained by funding and time, limiting generalizability. Additionally, the study design does not allow differentiation of results produced by the neural feedback versus the motor learning sessions. Nevertheless, this case series does provide valuable information regarding the use of rt-fMRI and/or rt-fNIRS neurofeedback supporting future work that could identify the benefit that spatially focused BCI training such as MRI-based and/or NIRS-based BCIs may provide.

The use of two different NIRS systems with two different wavelength pairs is a potential confound. However, this is likely to be inconsequential, as previous studies [42, 62–65] indicate that both wavelength pairs are optimal for solving the simultaneous equations for HbO and HbR concentrations accurately. Furthermore, adverse consequences of using the two different systems are minimized, given that both systems used the same number of sources and detectors.

User data from the stroke survivors revealed the importance of a more spatially fine-grained scan (e.g., 1mm3; smaller than the 3 mm dimension and 27 mm3 volume of the current study), higher power scanner (7T), and ability to custom-draw the ROI of interest desired for rt-fMRI. Considering the individual differences across participants in changes in the volume of activation during training sessions and pre-/post-treatment, it will be important to develop the methods for greater precision and customized real time neural feedback. This will be difficult because within each subject, the separate ROIs respond differentially to treatment and improvement in motor control.

Future research

BCI for stroke motor recovery is still young, with variable results reported in motor change and in brain signal change [19]. Nevertheless, there is important neuroscience research underway that may elucidate the neural mechanisms driving motor control. Others have developed hardware, software, and signal processing methods, which could enhance the use of rt-fMRI and rt-fNIRS. Success in that direction of inquiry could significantly improve signal feature selection and the efficacy of BCI in retraining motor control after stroke. For example, emerging research is reporting correlation between change in motor function and brain functional connectivity according to either MRI [62] or EEG [19]. It remains to study whether a functional connectivity signal feature can be productively provided to a stroke survivor for motor recovery. In their work, Yuan et al. (2020) reported that BCI training produced changes in information flow among motor-related brain regions, according to measures of functional connectivity. More successful future BCI training could engage that finding, providing feedback for enhancing these connections identified by Yuan et al. [66], including ipsilesional M1 and contralesional premotor and supplementary motor regions. Others are studying neural information transfer rate of acquired brain signal and classification accuracy, needed in BCIs controlling environmental and communication devices [67–70], and these discoveries may prove applicable to BCIs for stroke motor learning. For that BCI application, it is important in future work to identify those brain signal features specifically relevant to the given motor task. That work may lead to the critical development of greater customization (precision medicine) in signal processing and feedback provision, given that each individual stroke survivor exhibits quite different baseline and recovery volume of activation patterns [61]. In early days of BCI and motor control, some researchers attempted to solve that problem using software systems based on an adaptive controller, which attempted to account for changing brain signal over time within a given individual [14]; but that method did not satisfy the requirement of a brain signal feature that was a solid target of a brain pattern driving normal motor control to which the user could aspire [71]. A sophisticated approach to neural feedback would be to provide information to the user regarding the abnormal activation driving abnormal co-contraction, such as occurs during desired wrist extension when abnormal wrist flexion occurs instead. The ability to mitigate the neural drive of abnormal wrist flexion, so-called co-contraction [72] would be a major leap in neurorehabilitation.

Conclusions

It is possible for severely impaired stroke survivors to successfully engage in spatially focused BCI systems such as rt-fMRI and rt-fNIRS. The combined neural feedback with motor learning sessions (without BCI) produced motor improvement and clinically significant mean gains, according to joint movement coordination and functional task performance, commensurate with motor learning without neural feedback, supporting possible benefit. Given that the current neural feedback study did not appear to prevent the motor improvements observed in other long duration studies, even in the presence of fewer sessions of motor learning in the current work, the results support further study of neural feedback and its potential for recovery of motor function in stroke survivors. In future work, expanding the sophistication of either or both rt-fMRI and rt-fNIRS could hold the potential for further reducing the number of hours of training needed and/or the degree of recovery. Important future work could include identifying the unique contribution of the neural feedback versus the motor learning portions of the protocol.

Supporting information

S1 File. Individual subject data and discussion of individual subjects, respectively.

The S1 File provides data for each subject separately. Additionally, some discussion for a subject is presented within the given subject’s section, for greater cohesiveness in understanding each subject.

https://doi.org/10.1371/journal.pone.0250431.s001

(PDF)

S1 Protocol. Study protocol.

https://doi.org/10.1371/journal.pone.0250431.s002

(PDF)

S1 Checklist. Trend checklist.

https://doi.org/10.1371/journal.pone.0250431.s003

(PDF)

Acknowledgments

Dr. Richard Briggs provided strong support including technical expertise, data analysis guidance, and consultation. Ms. Lynn Dirk provided important editorial expertise and figure creation. Dr. Aniruddh Ravindran and Mr. Jose Alcantara, working with other study team members, assisted in acquiring data for the healthy adult subjects (different paper recently submitted) and assisted with initial experimental set-up (recently published paper).

Citation: Matarasso AK, Rieke JD, White K, Yusufali MM, Daly JJ (2021) Combined real-time fMRI and real time fNIRS brain computer interface (BCI): Training of volitional wrist extension after stroke, a case series pilot study. PLoS ONE 16(5): e0250431. https://doi.org/10.1371/journal.pone.0250431

References

1. Park K., Chang P., & Kang S. H. In Vivo Estimation of Human Forearm and Wrist Dynamic Properties. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2017; 25(5), 436–446. pmid:27249835

2. Palmer A. K., Werner F. W., Murphy D., & Glisson R. Functional wrist motion: A biomechanical study. Journal of Hand Surgery. 1985; 10(1), 39–46. pmid:3968403

3. Bertani R., Melegari C., De Cola M. C., Bramanti A., Bramanti P., & Calabrò R. S. Effects of robot-assisted upper limb rehabilitation in stroke patients: a systematic review with meta-analysis. Neurological Sciences. 2017; 38(9), 1561–1569. pmid:28540536

4. Thrasher T. A., Zivanovic V., McIlroy W., & Popovic M. R. Rehabilitation of Reaching and Grasping Function in Severe Hemiplegic Patients Using Functional Electrical Stimulation Therapy. Neurorehabilitation and Neural Repair. 2008; 22(6), 706–714. pmid:18971385

5. McCabe J, Monkiewizc M, Pundik S., Holcomb J, Daly JJ. Comparison of robotics, functional electrical stimulation, and motor learning methods for treatment of persistent upper extremity dysfunction after stroke: a randomized controlled trial. Arch Phys Med Rehabil. 2015; 96(6):981–90. pmid:25461822

6. Daly JJ, McCabe JP, Monkiewicz M, Gansen J, Rogers J. Long-Dose Neurorehabilitation necessary to produce significant gains in response to innovative upper limb function training in chronic stroke intervention. Neurorehabil Neural Repair. 2019; Jul;33(7):523–537. Epub 2019 May 25. pmid:31131743

7. Ward NS, Brander F, Kelly K. Intensive upper limb neurorehabilitation in chronic stroke: outcomes from the Queen Square programme. J Neurol Neurosurg Psychiatry. 2019; 90:498–506. pmid:30770457

8. Daly JJ, & Wolpaw J. R. Brain-computer interfaces in neurological rehabilitation. Lancet Neurol. 2008; 7(11), 1032–1043. pmid:18835541

9. Sitaram R., Ros T., Stoeckel L., Haller S., Scharnowski F., Lewis-Peacock J., et al. Closed-loop brain training: the science of neurofeedback. Nature Reviews Neuroscience. 2016; 18, 86. pmid:28003656

10. Kögel J, Schmid JR, Jox RJ, Friedrich O. Using brain-computer interfaces: a scoping review of studies employing social research methods BMC Medical Ethics. 2019; 20:18. https://doi.org/10.1186/s12910-019-0354-1 pmid:30845952

11. Belda-Lois J.-M., Mena-del Horno S., Bermejo-Bosch I., Moreno J. C., Pons J. L., Farina D., et al. Rehabilitation of gait after stroke: a review towards a top-down approach. Journal of NeuroEngineering and Rehabilitation. 2011; 8, 66–66. pmid:22165907

12. Chaudhary U., Birbaumer N., & Curado M. R. Brain-Machine Interface (BMI) in paralysis. Annals of Physical and Rehabilitation Medicine. 2015; 58(1), 9–13. https://doi.org/10.1016/j.rehab.2014.11.002 pmid:25623294

13. Ang K. K., Chua K. S., Phua K. S., Wang C., Chin Z. Y., Kuah C. W., et al. A Randomized Controlled Trial of EEG-Based Motor Imagery Brain-Computer Interface Robotic Rehabilitation for Stroke. Clin EEG Neurosci. 2015; 46(4), 310–320. pmid:24756025

14. Daly JJ, Cheng R., Rogers J., Litinas K., Hrovat K., & Dohring M. (2009). Feasibility of a new application of noninvasive Brain Computer Interface (BCI): a case study of training for recovery of volitional motor control after stroke. J Neurol Phys Ther. 2009; 3(4), 203–211. pmid:20208465

15. Pichiorri F., Morone G., Petti M., Toppi J., Pisotta I., Molinari M., et al. Brain–computer interface boosts motor imagery practice during stroke recovery. Annals of Neurology. 2015; 77(5), 851–865. pmid:25712802

16. Prasad G., Herman P., Coyle D., McDonough S., & Crosbie J. Applying a brain-computer interface to support motor imagery practice in people with stroke for upper limb recovery: a feasibility study. J Neuroeng Rehabil. 2010; 7, 60. pmid:21156054

17. Ramos-Murguialday A., Broetz D., Rea M., Läer L., Yilmaz Ö., Brasil F. L., et al. Brain-Machine-Interface in Chronic Stroke Rehabilitation: A Controlled Study. Annals of neurology. 2013; 74(1), 100–108. pmid:23494615

18. Rayegani SM, Raeissadat SA, Sedighipour L, Rezazadeh IM, Bahrami MH, Eliaspour D, et al. Effect of neurofeedback and electromyographic-biofeedback therapy on improving hand function in stroke patients. Top Stroke Rehabil. 2014; 21(2), 137–151. pmid:24710974

19. Biasiucci A., Leeb R., Iturrate I., Perdikis S., Al-Khodairy A., Corbet T., et al. Brain-actuated functional electrical stimulation elicits lasting arm motor recovery after stroke, Nature Communications. 2018; 9(1): 2421. pmid:29925890

20. Paret C, Goldway N, Zich C, Keynan JN, Hendler T, Linden D, et al. Current progress in real-time functional magnetic resonance-based neurofeedback: Methodological challenges and achievements. Neuroimage. 2019; 202:116107. Epub 2019 Aug 19. pmid:31437551

21. Blefari ML, Sulzer J, Hepp-Reymond MC, Kollias S, Gasser R. Improvement in precision grip force control with self-modulation of primary motor cortex during motor imagery. Front Behav Neurosci. 2015. 9, 18. pmid:25762907

22. Chiew M, LaConte SM, Graham SJ. Investigation of fMRI neurofeedback of differential primary motor cortex activity using kinesthetic motor imagery. 2012; NeuroImage, 61(1), 21–31. https://doi.org/10.1016/j.neuroimage.2012.02.053 pmid:22401758

23. Hui M, Zhang H, Ge R, Yao L, Long Z. Modulation of functional network with real-time fMRI feedback training of right premotor cortex activity. Neuropsychologia. 2014; 62, 111–123. https://doi.org/10.1016/j.neuropsychologia.2014.07.012 pmid:25058055

24. Marins TF, Rodrigues EC, Engel A, Hoefle S, Basílio R, Lent R, et al. Enhancing Motor Network Activity Using Real-Time Functional MRI Neurofeedback of Left Premotor Cortex. Frontiers in Behavioral Neuroscience. 2015; 9(341). pmid:26733832

25. Sitaram R, Veit R, Stevens B, Caria A, Gerloff C, Birbaumer N, et al. Acquired control of ventral premotor cortex activity by feedback training: an exploratory real-time FMRI and TMS study. Neurorehabil Neural Repair. 2012; 26(3), 256–265. pmid:21903976

26. Liew SL, Rana M, Cornelsen S, Fortunato de Barros Filho M, Birbaumer N., Sitaram R. et al. Improving Motor Corticothalamic Communication After Stroke Using Real-Time fMRI Connectivity-Based Neurofeedback. Neurorehabil Neural Repair. 2016; 30(7), 671–675. pmid:26671217

27. Yang M, Yang Z, Yuan T, Feng W and Wang P. A Systemic Review of Functional Near-Infrared Spectroscopy for Stroke: Current Application and Future Directions. Frontiers Neural Eng. 2019; pmid:30804877

28. Krakauer J and Carmichael S. Broken Movement: The Neurobiology of Motor Recovery after Stroke. Chapter 6, pages 174–176, Cambridge: MIT Press, 2017.

29. Straudi S, Baroni A, Mele S, Craighero , Manfredini F, Lamberti N, et al. Effects of a Robot-Assisted Arm Training Plus Hand Functional Electrical Stimulation on Recovery After Stroke: A Randomized Clinical Trial. Arch Phys Med Rehabil. 2020; 101(2):309–316. Epub 2019 Nov 1. pmid:31678222

30. Eraifej J, Clark W, France B, Desando S, Moore D. Effectiveness of upper limb functional electrical stimulation after stroke for the improvement of activities of daily living and motor function: a systematic review and meta-analysis. Systematic Reviews. 2017; 6:40. pmid:28245858

31. Fulk G, Martin R, Page SJ. Clinically Important Difference of the Arm Motor Ability Test in Stroke Survivors. Neurorehabil Neural Repair. 2017; 31:272–279. pmid:27885163

32. Kopp B, Kunkel A, Flor H, Platz T, Rose U, Mauritz KH, et al. The Arm Motor Ability Test: reliability, validity, and sensitivity to change of an instrument for assessing disabilities in activities of daily living. Arch Phys Med Rehabil. 1997; Jun;78(6):615–620. pmid:9196469.

33. O’Dell MW, Kim G, Rivera L, Fieo R, Christos P, Polistena C, et al. A psychometric evaluation of the Arm Motor Ability Test. J Rehabil Med. 2013; 45:519–27. pmid:23532275

34. Chae J, Labatia I, Yang G. Upper limb motor function in hemiparesis: concurrent validity of the Arm Motor Ability test. Am J Phys Med Rehabil. 2003; 82:1–8. pmid:12510178

35. Santisteban L, Térémetz M, Bleton JP, Baron JC, Maier MA, Lindberg PG. Upper limb outcome measures used in stroke rehabilitation studies: a systematic literature review. PLoS One. 2016; 11:e0154792. pmid:27152853

36. Fugl-Meyer AR, Jääskö L, Leyman I, Olsson S, Steglind S. The post-stroke hemiplegic patient. 1. method for evaluation of physical performance. Scand J Rehabil Med. 1975; 7:13–31. pmid:1135616

37. Page SJ, Fulk GD, Boyne P. Clinically importance differences for the Upper-Extremity Fugl-Meyer in minimally to moderately impaired chronic stroke. Phys Ther. 2012; 92:791–98. pmid:22282773

38. Norkin CC, White DJ. Measurement of Joint Motion: A Guide to Goniometry. Philadelphia: FA Davis Company, 1995.

39. Rieke JD, Matarasso AK, Yusufali MM, Ravindran A, White KD, Daly JJ. Development of a combined sequential real-time fMRI and fNIRS neurofeedback system to enhanc3e motor learning after stroke. J Neurosci Methods. 2020; Jul 15;341:108719. pmid:32439425

40. Fischl B, Rajendran N, Busa E, Augustinack J, Hinds O, Yeo BTT, et al. Cortical Folding Patterns and Predicting Cytoarchitecture. Cereb Cortex. 2008; Aug 1;18(8):1973–1980. pmid:18079129

41. Yousry T. Localization of the motor hand area to a knob on the precentral gyrus, A new landmark. Brain. 1997; Jan 1;120(1):141–157. [] pmid:9055804

42. Huppert TJ, Diamond SG, Franceschini MA, Boas DA. HomER: a review of time-series analysis methods for near-infrared spectroscopy of the brain. Appl Opt. 2009; 48:D280–D298. pmid:19340120

43. Klein F, Kranczioch C. Signal Processing in fNIRS: A case for the removal of systemic activity for single trial data. Frontiers in human neuroscience. 2019; 13, 331. pmid:31607880

44. Brunnstrom S. Motor testing procedures in hemiplegia: based on sequential recover stages. Phys Ther. 1966; 46:357–375. pmid:5907254

45. Twitchell TE. The restoration of motor function following hemiplegia in man. Brain. 1951; 74:443–480. pmid:14895765

46. Crow JL, Harmeling-van der Wel BC. Hierarchical properties of the motor function sections of the Fugl-Meyer assessment scale for people after stroke: a retrospective study. Phys Ther. 2008; 88(12):1554–67. pmid:18927197

47. Nudo RJ, Milliken GW, Jenkins WM, Merzenich MM. Use-dependent alterations of movement representations in primary motor cortex of adult squirrel monkeys. J. Neurosci. 1996; 16:785–807. pmid:8551360

48. Nudo RJ, Wise BM, SiFuentes F, and Milliken GW. Neural substrates for the effects of rehabilitative training on motor recovery after ischemic infarct. Science. 1996; 272: 1791–1794. pmid:8650578

49. Pascual-Leone A. and Torres F. Plasticity of the sensorimotor cortex representation of the reading finger in Braille readers. Brain. 1993;116(Pt 1):39–52. pmid:8453464

50. Butefisch C, Hummelsheim H, Denzler P, and Mauritz KH. Repetitive training of isolated movements improves the outcome of motor rehabilitation of the centrally paretic hand. J. Neurol. Sci. 1995; 130:59–68. pmid:7650532

51. Elbert T, Pantev C, Wienbruch C, Rockstroh B, and Taub E. Increased cortical representation of the fingers of the left hand in string players. Science. 1995; 270:305–307. pmid:7569982

52. Dean CM. and Shepherd RB. Task-related training improves performance of seated reaching tasks after stroke. A randomized controlled trial. Stroke. 1997; 28:722–728. pmid:9099186

53. Singer R, Lidor R, and Cauraugh JH. To be aware or not aware? What to think about while learning and performing a motor skill. Sport Psychol. 1993; 7:19–30.

54. Plautz EJ, Milliken GW, Nudo RJ. Effects of repetitive motor training on movement representations in adult squirrel monkeys: role of use versus learning. Neurobiol Learning Memory. 2000; 74:27–55. pmid:10873519

55. Woytowicz E, Rietschel J, Goodman R. Determining levels of upper extremity movement impairment by applying a cluster analysis to the Fugl-Meyer Assessment of the Upper Extremity in chronic stroke. Arch Phys Med Rehabil. 2017;98:456–62. pmid:27519928

56. Frías , Starrs F, Gisiger T, Minuk J, Thiel A, Paquette C. Interhemispheric connectivity of primary sensory cortex is associated with motor impairment after stroke. Nature. Scientific Reports. 2018; 8:12601 | pmid:30135496

57. Bütefisch CM, Kleiser R, Körber B, Müller K, Wittsack HJ, Hömberg V, et al.: Recruitment of contralesional motor cortex in patients with recovery of hand function. Neurology. 2005 IS THIS 2015??; 64: 1067–1069. pmid:15781831

58. Cramer SC, Repairing the human brain after stroke: I. Mechanisms of spontaneous recovery. Ann. Neurol. 2008; 63, 272–287. pmid:18383072

59. Nadeau SE, Kobkin B, and LEAPS Team. The effects of stroke type, locus, and extent on long-term outcome of gait rehabilitation. Neurorehabil Neural Repair. 2016; Aug 30(7) pp 6150625.

60. Winstein CJ, Stein J, Arena R, et al. Guidelines for Adult Stroke Rehabilitation and recovery: a guideline for healthcare professionals from the American Heart Association/American Stroke Association. Stroke. 2017; 48(12).

61. Pundik S, McCabe JP, Hrovat K, Fredrickson AE, Tatsuoka C, Daly JJ. Recovery of post stroke proximal arm function, driven by complex neuroplastic bilateral brain activation patterns and predicted by baseline motor dysfunction severity. Front Hum Neurosci. 2015; 9:394–399. pmid:26257623

62. Gervain J, Mehler J, Werker JF, et al. Near-infrared spectroscopy: A report from the McDonnell infant methodology consortium. Developmental Cognitive Neuroscience. 2011; 1:22–46. pmid:22436417

63. Sato H, Kiguchi M, Kawaguchi F, Makia A. Practicality of wavelength selection to improve signal-to-noise ratio in near-infrared spectroscopy. NeuroImage. 2004; 21:1554–1562. pmid:15050579

64. Strangman G, Franceschini MA, Boas DA. Factors affecting the accuracy of near-infrared spectroscopy concentration calculations for focal changes in oxygenation parameters. NeuroImage. 2003; 18:865–879. pmid:12725763

65. Boas DA, Dale AM, Franceschini MA. Diffuse optical imaging of brain activation: approaches to optimizing image sensitivity, resolution, and accuracy. NeuroImage. 2004; 23(Suppl 1):S275–S288.

66. Yuan K, Chen K, Wang X, Chu WC, Tong RK. BCI training effects on chronic stroke correlate with functional reorganization in motor-related regions: A concurrent EEG and fMRI study. journal of neural engineering. 202; pmid:33418846

67. Fazli S, Mehnert J, Steinbrink J, Curio G, Villringer A, Muller KR, et al. Enhanced performance by a hybrid NIRS-EEG brain computer interface. NeuroImage. 2012; 59(1):519–529. pmid:21840399

68. Shin J, von Lühmann A, Blankertz B, Kim D-W, Jeong J, Hwang H-J, et al. Open Access Dataset for EEG+ NIRS Single-Trial Classification. IEEE Trans Neural Syst Rehabil Eng. 2017; 25(10):1735–1745.

69. Khan MJ, Hong MJ, Hong K-S. Decoding of four movement directions using hybrid NIRS-EEG brain computer interface. Front Hum Neurosci. 2014; 8(1):244. pmid:24808844

70. Khan MJ, Hong K-S. Hybrid EEG–fNIRS-based eight-command decoding for BCI: application to quadcopter control. Front Neurorobot. 2017; 11:6. pmid:28261084

71. Daly JJ, Huggins JE. Brain-computer interface: current and emerging rehabilitation applications. Arch Phys Med Rehabil. 2015; 96(3 Suppl):S1–7. pmid:25721542

72. Dewald JPA, Pope PS, Given JD, Buchanan TS, Rymer WZ. Abnormal muscle coactivation patterns during isometric torque generation at the elbow and shoulder in hemiparetic subjects. Brain. 1995; 118(2), 495–510.