Hybrid BCI
What kind of assistance can BCI actually offer to disabled persons? Despite progress in AT, there is still a large number of people with severe motor disabilities who cannot fully benefit from AT due to their limited access to current assistive products (APs). For them, BCI is the solution. However, notwithstanding the impressive demonstrations of BCI technology around the world, today's state-of-the-art is such that BCI alone cannot make patients interact with and control assistive devices over long periods of time and without expert assistance. But this doesn't mean that there is no place for BCI. The solution is to use BCI as an additional channel. Such a hybrid approach, where conventional APs (operated using some residual muscular functionality) are enhanced by BCI technology, leads to what we call hybrid BCI (hBCI).
As a general definition, a hBCI is a combination of different signals including at least one BCI channel. Thus, it could be a combination of two BCI channels but, more importantly, also a combination of BCI and other biosignals [such as electromyographic (EMG), etc.] or special AT input devices (e.g., joysticks, switches, etc.). The control channels (BCI and other modalities) can operate different parts of the assistive device or all of them could be combined to allow users to smoothly switch from one control channel to the other depending on their preference and performance. An example of the former case is a neuroprostheses that uses residual movements for reaching objects and BCI for grasping. In the latter case, a muscular dystrophy patient may prefer to speak in the morning and switch to BCI in the afternoon when fatigue prevents him from being able to speak intelligibly. Moreover, in the case of progressive loss of muscular activity [as in muscular dystrophy, amyotrophic lateral sclerosis (ALS), and spinal muscular atrophies] early BCI training while the user can still exploit her/his residual motor functions will increase long-term use of APs by smoothing the transition between the hybrid assistive device and pure BCI when muscular activity is too weak to operate the APs.
An effective way for a hBCI to combine all the control channels is to merge their individual decisions – i.e., the estimation of the user's intent – by weighting the contribution of each modality. These weights reflect the reliability of the channel, or confidence/certainty the system has regarding its output. The weights can be estimated from supervision signals such as mental states [e.g., fatigue, error potentials (ErrPs)] and physiological parameters (e.g., muscular fatigue). Another source to derive the weights is to analyze the performance of the individual channels in achieving the task at hand (e.g., stability over time).
There exist a few examples of hybrid BCIs. Some are based on multiple brain signals. One of such hBCIs is the combination of motor imagery (MI)-based BCI with ErrP detection and correction of false mental commands (Ferrez and Millán, 2008b). A second example is the combination of MI with steady state visual evoked potentials (SSVEPs) explored in some offline studies (Allison et al., 2010; Brunner et al., 2010). Other hBCIs combine brain and other biosignals. For instance, Scherer et al. (2007b) combined a standard SSVEP BCI with an on/off switch controlled by heart rate variation. Here the focus is to give users the ability to use the BCI only when they want or need to use it. Alternatively, and following the idea of enhancing people's residual capabilities with a BCI, Leeb et al. (2010b) fused EMG with EEG activity, so that the subjects could achieve a good control of their hBCI independently of their level of muscular fatigue. Finally, EEG signals could be combined with eye gaze (Danoczy et al., 2008). Pfurtscheller et al. (2010) have recently reviewed preliminary attempts, and feasibility studies, to develop hBCIs combining multiple brain signals alone or with other biosignals. Finally, hybrid BCIs could exploit several brain imaging techniques simultaneously; i.e., EEG together with MEG, fMRI, NIRS, and even TMS. As mentioned above, our focus in this review paper is on principles to develop hBCI that, when coupled with existing AT used by disabled people, can effectively improve their quality of life.