Brain–computer interface use is a skill that user and system acquire together

Abstract

A brain–computer interface (BCI) is a computer-based system that acquires, analyzes, and translates brain signals into output commands in real time. Perdikis and colleagues demonstrate superior performance in a Cybathlon BCI race using a system based on “three pillars”: machine learning, user training, and application. These results highlight the fact that BCI use is a learned skill and not simply a matter of “mind reading.”

About the Authors:

Dennis J. McFarland

* E-mail: [email protected]

Affiliation: National Center for Adaptive Neurotechnologies, Wadsworth Center, Albany, New York, United States of America

ORCID logo http://orcid.org/0000-0001-8116-1268

Jonathan R. Wolpaw

Affiliation: National Center for Adaptive Neurotechnologies, Wadsworth Center, Albany, New York, United States of America

Abstract

A brain–computer interface (BCI) is a computer-based system that acquires, analyzes, and translates brain signals into output commands in real time. Perdikis and colleagues demonstrate superior performance in a Cybathlon BCI race using a system based on “three pillars”: machine learning, user training, and application. These results highlight the fact that BCI use is a learned skill and not simply a matter of “mind reading.”

Citation: McFarland DJ, Wolpaw JR (2018) Brain–computer interface use is a skill that user and system acquire together. PLoS Biol 16(7): e2006719. https://doi.org/10.1371/journal.pbio.2006719

Published: July 2, 2018

Copyright: © 2018 McFarland, Wolpaw. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Funding: National Institute of Biomedical Imaging and Bioengineering of the National Institutes of Health (grant number 1P41EB018783). The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: The authors have declared that no competing interests exist.

Abbreviations: ALS, amyotrophic lateral sclerosis; BCI, brain–computer interface; EEG, electroencephalographic; SMR, sensorimotor rhythm

Provenance: Commissioned; externally peer reviewed.

A brain–computer interface (BCI) is a computer-based system that acquires, analyzes, and translates brain signals into output commands in real time [1]. BCIs are distinctive in that they do not use the brain’s normal output pathways of peripheral nerves and muscles. BCI studies have focused mainly on restoring communication and control to people paralyzed by chronic neuromuscular disorders, such as amyotrophic lateral sclerosis (ALS), brainstem stroke, or high-level spinal cord injury. Most of these studies have been demonstrations of proof of principle with able-bodied users. Long-term independent use of BCIs for communication or control by individuals who can actually benefit from them is limited to a handful of case reports [2–6] and one group study [7].

One prominent class of BCIs use sensorimotor rhythms (SMRs). SMRs are oscillations (i.e., mu [8–12 Hz] and beta [18–30 Hz]) in electroencephalographic (EEG) activity recorded over sensorimotor cortices; they change in amplitude with movement, imagined movement, or preparation for movement. People can also learn through a training protocol to increase or decrease SMR amplitude and can use this control to move a computer cursor or operate another device [8–10]. Furthermore, they can learn to use several different SMRs to control movement in multiple dimensions simultaneously and to support sequential mouse-like control [11–15].

The performance of BCIs that use SMRs (as well as that of BCIs that use cortical neurons [16]) improves with training. Their usage is in large part a skill that is acquired through practice [1]. The BCI user learns to encode his/her intent in brain signal features (e.g., SMRs) that the BCI can record, extract, and translate into output commands. Thus, BCI operation entails the effective cooperation of two adaptive controllers: the user, who seeks to encode his/her intent in brain signals that the BCI can recognize; and the BCI, which adapts to recognize these signals and translate them into the intended output commands.

Many BCI studies have largely ignored the user-training component of BCI operation. They have focused almost exclusively on machine learning (e.g., on the BCI learning how to recognize the brain signals associated with particular types of motor imagery). This popular trend probably results from an emphasis on the inadequate notion that BCI users are simply executing pre-existing cognitive tasks such as motor imagery [17–18] (see Box 1).

Box 1. BCI use as an acquired skill

It is often assumed that the way to train users with a sensorimotor rhythm (SMR)-based BCI is to ask them to generate specific mental states through motor imagery [18]. Different BCI commands are often linked to different imagery (e.g., imagine hand movement to move the cursor up and foot movement to move it down). However, as the user’s SMR control improves, and particularly when users advance to controlling multiple dimensions, users tend to abandon the use of imagery [14]. While motor imagery may provide a logical and effective starting point for user training, it becomes unnecessary and may even be an impediment as training progresses. For example, Kober and colleagues [23] found that those subjects who reported using no specific mental strategy after 10 SMR training sessions showed improved performance. In contrast, subjects reporting various mental strategies after 10 training sessions showed no improvement. These results suggest that successful SMR control after extended training involves implicit learning mechanisms. Thus, SMR-based BCI control after extended training resembles typical motor performance in that it tends to become automatic (i.e., implicit) with practice. Indeed, studies in animals show that BCI control involves plasticity in corticostriatal systems typically involved in motor control [24]. Viewing SMR control as similar to other forms of motor control suggests using principles of motor learning in designing SMR-based BCI applications and in training people to use them. Motor learning involves both explicit and implicit processes, which have different characteristics [25]. Explicit instructions can interfere with implicit processes, particularly in well-trained people [26].

In contrast, the article by Perdikis and colleagues [19] in this issue illustrates the importance of recognizing and facilitating both adaptive aspects of BCI operation: adaptation by the user and adaptation by the BCI. They describe their SMR-based BCI training of two people with tetraplegia to compete in the Cybathlon BCI race; these two users went on to dominate the competition. The authors ascribe this success to a mutual learning approach that gives equal importance to “three pillars”: machine learning, user training, and application. They note that the other entries in the Cybathlon relied predominately on machine learning. They assert that their success supports the view that “BCI is a skill to be learned.”

As noted above, the fact that BCI use is a skill has been emphasized previously, and Perdikis and colleagues [19] acknowledge that their conclusions are limited by the fact that they are based on an uncontrolled study of only two individuals. Nevertheless, their study is distinctive and notable because the BCI users were severely disabled and, most important, because it reports their performance in a highly competitive and stressful setting (rather than simply in a protected laboratory environment). Given these considerations, the results provide impressive new evidence of the effectiveness of a mutual learning approach.

One of the users in the Perdikis and colleagues [19] study had the best overall performance but did not do well (i.e., “choked”) in the final competition. This finding may illustrate an important feature of skilled performance: the focus of attention can be critical. Studies of motor performance show that an external focus of attention to the outcome of an action results in performance that is superior to that associated with an internal focus on the mechanics of the action [20]. Indeed, Schucker and colleagues [21] report that individuals who choked under pressure reported greater attention to the details of their movements (i.e., allocation of attention to the different body parts that were involved in movement execution). It may be the case that this BCI user was a victim of an internal focus of attention. Ultimately, the effectiveness of various strategies for BCI control should clarify the nature of this type of skill and lead to methods for improving user performance.

Perdikis and colleagues [19] also discuss the difficult problem of mutual learning: the ongoing adaptive interaction of the user’s brain, which produces the signals measured by the BCI, and the BCI itself, which translates these signals into commands [1,22]. Although there has been considerable work on the design of controllers that involve a single adaptive element, coadaptation in BCI design represents an important new challenge.

Future BCI studies would do well to consider the three pillars of mutual learning stressed by Perdikis and colleagues [19]. More emphasis on user learning and application design are likely to move this exciting area of research forward. The ultimate goal should be the development of fast and accurate BCI systems that can be reliably operated by the individuals with severe neuromuscular disabilities who can benefit from their use.

Citation: McFarland DJ, Wolpaw JR (2018) Brain–computer interface use is a skill that user and system acquire together. PLoS Biol 16(7): e2006719. https://doi.org/10.1371/journal.pbio.2006719

References

1. Wolpaw JR, Birbaumer N, McFarland DJ, Pfurtscheller G, Vaughan TM. Brain-computer interfaces for communication and control. Clin Neurophysiol. 2002;113(6):767–791. pmid:12048038

2. Birbaumer N, Ghanayim N, Hinterberger T, et al. A spelling device for the paralyzed. Nature. 1999;398:297–298. pmid:10192330

3. Neuper C, Muller GR, Kubler A, Bibaumer N, Pfurtscheller G. Clinical application of an EEG-based brain–computer interface: a case study in a patient with severe motor impairment. Clin Neurophysiol. 2003;114:399–409. pmid:12705420

4. Saeedi S, Chavarriaga R, Millan JdR. Long-term stable control of motor-imagery BCI by a locked-in user through adaptive assistance. IEEE Trans Neural Syst Rehabil Eng. 2017;25: 380–391. pmid:28055886

5. Sellers EW, Vaughan TM, Wolpaw JR 2010. A brain-computer interface for long-term independent home use. Amyotroph Lateral Scler. 2010; 11:449–455. pmid:20583947

6. Vansteensel MJ, Pels EGM, Bleichner MG, et al. Fully implanted brain-computer interface in a locked-in patient with ALS. New Engl J Med. 2016;375:2060–2066. pmid:27959736

7. Wolpaw JR, Bedlack RS, Reda DJ, et. al. Independent home use of a brain-computer interface by people with amyotrophic lateral sclerosis. Neurology (forthcoming).

8. Pfurtscheller G, McFarland DJ. BCIs that use sensorimotor rhythms. In Wolpaw JR Wolpaw EW Brain–Computer Interfaces: Principles and Practice. New York, NY: Oxford University Press, 2012; pp 227–240.

9. Perdikis S, Leeb R, Williamson J, et al. Clinical evaluation of BrainTree, a motor imagery hybrid BCI speller. J Neural Eng. 2014;11: 036003. pmid:24737114

10. Leeb R, Tonin L, Rohm M, et al. Towards independence: a BCI telepresence robot for people with severe motor disabilities. Proc IEEE. 2015;103: 969–982.

11. Wolpaw JR, McFarland DJ, Neat GW, Forneris CA An EEG-based brain-computer interface for cursor control. EEG Clin Neurophysiol. 1991; 78:252–259.

12. Kostov A and Polak M. Parallel man-machine training in development of EEG-based cursor control. IEEE Transactions on Rehabilitation Engineering. 2000; 8:203–204. pmid:10896187

13. Wolpaw JR. and McFarland DJ. Control of a two-dimensional movement signal by a non-invasive brain-computer interface. PNAS. 2004; 51:17849–17854.

14. McFarland DJ, Sarnacki WA, Wolpaw JR Electroencephalographic (EEG) control of three-dimensional movement J Neural Eng. 2010, 7: 036007. pmid:20460690

15. Meng J, Edelman BJ, Olsoe J, Jacobs G, Zhang S, Beyko A, et al. A study of the effects of electrode number and decoding algorithm on online EEG-based BCI behavioral performance. Front Neurosci. 2018; 00227.

16. Carmena JM, Lebedev MA, Crist RE, O’Doherty JE, Santucci DM, Dimitrov DF, et al. Learning to control a brain-machine interface for reaching and grasping by primates. PLoS Biol. 2003; 1:1–16.

17. Curran EA, Stokes MJ. Learning to control brain activity: A review of the production and control of EEG components for driving brain–computer interface (BCI) systems. Brain Cogn. 2003; 51: 326–336. pmid:12727187

18. Lotte F, Larrue F, Muhl C. Flaws in current human training protocols for spontaneous brain-computer interfaces: leasons learned from instructional design. Front Hum Neurosci. 2013; 7:568. pmid:24062669

19. Perdikis S, Tonin L, Saeedi S, Schneider C, Millan JdR. The Cybathlon BCI race: Successful longitudinal mutual learning with two tetraplegic users. PLoS Biol. 2018;16:e2003787. pmid:29746465

20. Lohse KR, Jones M, Healy AF, Sherwood DE. The role of attention in motor control. J Exp Psychol Gen. 2014; 143: 930–948. pmid:23647310

21. Schucker L, Hagmann N, Strauss B. Attentional processes and choking under pressure. Percept Mot Skills. 2013; 116: 671–689. pmid:24032339

22. Wolpaw JR, Wolpaw EW. Brain–computer interfaces: something new under the sun. In: Wolpaw JR, Wolpaw EW, editors. Brain–Computer Interfaces: Principles and Practice. New York, NY: Oxford University Press, 2012. p. 3–12.

23. Kober SE, Witte M, Ninaus M, Neuper C, Wood G. Learning to modulate one’s own brain activity: the effects of spontaneous mental strategies. Front Hum Neurosci. 2013; 7:695. pmid:24151462

24. Koralek AC, Jin X, Long JD, Costa RM, Carmena JM. Corticostriatal plasticity is necessary for learning intentional neuroprosthetic skills. Nature. 2012;483:331–335. pmid:22388818

25. McDougle SD, Bond KM, Taylor JA. Explicit and implicit processes constitute the fast and slow processes of sensorimotor learning. J Neurosci. 2015; 35: 9568–9579. pmid:26134640

26. Castandeda B, Gray R. Effects of focus of attention on baseball batting performance in players of differing skill levels. J Sport Exerc Psychol. 2007; 29:60–77. pmid:17556776