Exoskeleton Control System Based on Motor-Imaginary Brain–Computer Interface

Today, one of the fastest growing areas of medical technology is development and manufacturing of neuro-integrated devices based on brain–computer interface (BCI) technology combined with the use of robots (orthoses, exoskeletons) for neurorehabilitation purposes [1, 2]. The impetus for this growth was the discovery of training-induced plastic changes in the functional topography of the primary motor cortex [3]. It is shown in the studies by Bach-Y-Rita and Taub [4, 5] that movement can be recovered even several years after stroke. This has provided new opportunities for neurorehabilitation: the strategy of intensive, regular and motivated movement training has been developed [6]. Exoskeletons appeared to be ideal technical devices for this strategy implementation. Different types of exoskeletons have already been actively used in the clinic and the number of these developments has been growing exponentially in the last 10 years [7].

Exoskeletons used successfully for neurorehabilitation involve devices reproducing the movements of the upper limb joints [8] and devices focused on simulation of walking. Among the latter, we can distinguish exoskeletons performing biomechanically correct movements in the hip and knee joints of the patient such as LOPES (lower extremity powered exoskeleton) [9], ALEX (active leg exoskeleton) [10], ReWalk (wearable robotic exoskeleton) [http://rewalk.com/], eLEGS (exoskeleton lower extremity gait system) [http://bleex.me.berkeley.edu/research/exoskeleton/elegs], Rex (robotic exoskeleton) [http://www.rexbionics.com/], MINDWALKER [11] and others.

Despite the fact that currently there are several types of BCI known [12, 13], the most popular types of non-invasive BCI to control exoskeletons are the so-called synchronous BCIs based on registration of the operator EEG response to the external stimulus environment. They involve resonance frequency BCIs based on the well-known property of spontaneous EEG rhythms to adapt the resonance frequency to external sources of rhythmic stimulation [14] and P300 wave-based BCIs operating on the principle of detecting the P300 wave of the induced potentials in response to the stimuli (symbols) intended by the user [15, 16]. The accuracy of detecting commands in such BCIs amounts to 95–97% [17], but at the same time, the operator’s attention should be always focused on the matrix stimulus environment. Examples of applying resonance frequency BCIs for controlling lower-limb robotic exoskeletons (REs) are MindWalker [11], the Korean lower-limb exoskeleton [18].

Unlike synchronous BCIs, in asynchronous motor imagery based BCI technology (MI–BCI — motor imagery based brain–computer interface), detected as commands for controlling EEG rhythm changes are invoked by the arbitrary efforts of the human operator irrespective of any external sensory stimulation. Despite the fact that this BCI type is quite difficult to master compared to resonance frequency and P300 wave-based BCIs, exactly motor imagery BCI is considered to be the most promising for training motor function [1, 19, 20]. The operating principle of motor imagery based BCI is detecting desynchronization of sensorimotor rhythms in the motor area of the cerebral cortex contralateral to the motor act when the operator elicits motor act imageries such as grasping, moving the fingers and other movements [21]. It has been shown that even patients with paresis of the extremities are able to successfully imagine various movements of paralyzed body parts [22]. BCI-RoGO [23] and NeuroRex [24] are the examples of using such BCI for controlling exoskeletons.

It is important to underline that recently there have been published successful results of the study on the rehabilitation of people with paraplegia caused by spinal cord lesions using motor imagery based BCI technology [25]. The rehabilitation protocol included comprehensive training with controlling the virtual reality scenario, lower-limb exoskeleton, tactile feedback and locomotor activity through BCI technology.

The aim of the investigation was to develop neuro-integrated control system for a lower-limb robotic exoskeleton based on recognition of EEG patterns using motor imagery brain–computer interface technology.

Materials and Methods. The experimental study involved 14 healthy subjects (6 males and 8 females aged 18 to 23 years) who were informed about the terms and conditions of the experiment before it started and provided written informed consent for participation. The study protocol was approved by the Bioethics Committee of Lobachevsky State University of Nizhni Novgorod. The experiments were carried out without wearing the exoskeleton by the operator. During the experiment, the exoskeleton was on the stand near the operator and performed movements with the left and right foot depending on the EEG pattern generated by the operator (See Media https://drive.google.com/file/d/0BzOUw8ncip3md1dPdzZmZmtka3c/view).

The developed neuro-integrated RE control system based on BCI technology consists of three main modules: EEG signal recording module, EEG signal classifier and the software for transmission of commands to RE.

EEG signal recording module. EEG signals were recorded using the certified NVX 52 amplifier (LLC “Medical Computer Systems”, Russia). Eight leads were used to record EEG (FCz, C5, C3, C1, Cz, C2, C4, C6), arranged according to the international 10-10 scheme (Figure 1). The reference electrode was placed on the left ear lobe. The grounding electrode was on the forehead. The signal digitization frequency was 500 Hz. Resistance under the electrodes did not exceed 10 kΩ. The signal was filtered in the range of 6 to 15 Hz with Notch filter of 50 Hz.

---

Figure 1. The diagram of training and testing the classifier in the contour of motor imagery brain–computer interface; layout diagram of the electrodes used in the experiment

---

EEG signal classifier. The experimental procedure of operating the BCI-controlled RE consisted of three consecutive sessions: the training session, the test session, and controlling session. Training and testing sessions were used for the initial setup and testing the classifier, respectively.

During the classifier training, the operator performed one of three commands: rest when the monitor image of the cross appeared, left or right hand motor imagery movements when “left/right” arrow was seen on the monitor. The operator was asked to choose any hand movement considered comfortable for imagery. The exemplary movements were finger movements and rotating a hand in the wrist joint. The “rest” command meant that the operator had to sit still concentrating on breathing. Each 5-second command was presented 10 times. The inter-stimulus interval was 3 s (empty screen). The stimuli were presented randomly. The classifier training duration amounted to 4 min. While presenting the stimuli, EEG was recorded using NeoRec software (LLC “Medical Computer Systems”, Russia) which transmitted the received signal according to the LSL protocol. The transmitted signal was read as a script written in the Python language (www.python.org) which controlled the synchronization between the stimuli presentation and the EEG signal.

To control the success of mastering the motor imagery technique, the degree of sensorimotor rhythm desynchronization was assessed. When the stimuli presentation was completed, spatial filtering was applied to improve the obtained EEG recording in all the channels (Surface Laplacian method [26]). Afterwards, the power in the frequency range from 6 to 15 Hz with a step of 1 Hz was calculated for each data set corresponding to the stimulus type in each channel separately. The rate of power change relative to “rest” was calculated for the records corresponding to motor imagery. The results were mapped (Figure 2). When spectral power decreased (desynchronization) by more than 50% during motor imagery, the operator was considered to have successfully mastered motor imagery technique and proceeded to the test session of the classifier. In case of several failed attempts, the procedure was repeated with the change of motor imagery type.

---

Figure 2. Exemplary topographic mapping of EEG desynchronization degree for one operator: (a) left hand motor imagery movement; (b) right hand motor imagery movement; the maps are presented for desynchronization at 13 Hz frequency; the dark color corresponds to the maximum de-synchronization, the light color means the minimum de-synchronization

---

When testing the classifier, the results of mental task recognition were provided to the operator by visual feedback: a green scale beginning at the circle in the screen center where the subject fixed the eyes filled down to the edge of the screen if the classifier recognized the task in agreement with the given command and the scale stopped filling when another task was recognized (See Figure 1).

Recognition was performed by the classifier based on linear discriminant analysis using the features identified by spatial filtering with CSP method [27] for all types of commands pairwise. The solutions of paired classifiers were summarized by “voting”.

After the classifier training and testing, the operator could start controlling the external device, i.e. the lower-limb RE [28]. The operator was asked to choose one of three commands (left or right hand motor imagery movements and rest). The classifier analyzed EEG recording every 4.5 s, made a conclusion and transmitted the command chosen by the operator to the external device.

The software for transmission of commands to robotic exoskeleton. Figure 3 shows the software operation diagram.

---

Figure 3. The diagram of robotic exoskeleton control system based on motor imagery brain–computer interface

---

BCI classifier software and RE control system software are installed on a personal computer running Windows operating system. Web service controlling the exoskeleton is run on the microcomputer built in it and powered by Linux-compatible operating system. Data transfer between the computer and the exoskeleton is carried out through a wireless Wi-Fi channel.

RE control system based on motor imagery BCI works as follows. As described above, EEG operator data are transferred from NVX 52 amplifier to the classifier in the LSL format. Every 4.5 s the number of the pattern recognized by the classifier is placed in the User Datagram Protocol (UDP) and sent via Transmission Control Protocol (TCP)/Internet Protocol (IP), to RE control software. For easy system customization, the authors have developed original RE control software. The software has a graphical user interface and allows the user to change the mapping of RE patterns and commands in the operation process (Figure 4). The program is written in C++ using the Qt library (https://www.qt.io/).

---

Figure 4. Graphical user interface of robotic exoskeleton control software

---

Using a pre-set correspondence between the pattern number and exoskeleton command, RE control software sends a request to the web service of the exoskeleton to perform the relevant command. The web service of the exoskeleton is a program run on the BeagleBoard-xM microprocessor board coming with Angstrom Linux Distribution. The web service interface is implemented using the gSOAP code generator (https://www.cs.fsu.edu/~engelen/soap.html) and provides the ability to run movement patterns of the exoskeleton limbs. A total of 10 patterns have been implemented (stand up; sit down; bend/extend the right leg; bend/extend the left leg; vertical; embryo; walk). Each of the patterns implements the algorithm of synchronous control over a group of 4 actuators. Actuator parameters are the limiting angles at which the movement in the corresponding joints is terminated and the maximum torque provided on the actuator shafts of the joints when movement is performed.

The following associations of patterns and commands were used for the exoskeleton:

pattern 1 (left limb motor imagery movement) — flexion/extension of the left leg of the exoskeleton;

pattern 2 (right limb motor imagery movement) — flexion/extension of the right leg of the exoskeleton.

Results. The results of the operator’s work were used to calculate the average accuracy of the skeleton control based on the given BCI as the ratio of the total number of correctly entered commands to the total number of attempts. Each operator was given 10 attempts for each command in each session.

The results of testing on 14 volunteers (See the Table) showed that the average control accuracy of the developed motor-imagery BCI is 73, 71, and 66%, respectively, for three commands in the sessions. Evidently, the average accuracy of choosing commands does not vary greatly in different sessions. It is also interesting to note that individual indices of accuracy in testing the classifier predict (correlation coefficient =0.8) further assessment of skeleton operation accuracy. Notably, some subjects (about one third) definitely achieved fairly high accuracy (90% and more).

---

Classification accuracy (%) in three sessions of brain–computer interface operation (sessions 1 and 2 — testing the classifier; session 3 — operating the exoskeleton)

---

In future, we plan to test the developed control system in experiments with wearing the exoskeleton by healthy people and people with severe motor disabilities and to develop individual operator training techniques to achieve higher accuracy of control.

Conclusion. The developed control system for a robotic exoskeleton (a lower-limb exoskeleton) based on motor imagery brain–computer interface offers rather high accuracy for three commands. As a robotic rehabilitation device, it can be used for active electromechanical assistance to movements in patients with severe motor disabilities, for development and training of impaired postural functions in patients with disorders affecting the functioning of motor centers and neural signal conductivity of the spinal cord with clinical presentation of partial or complete paraplegia (stroke, traumatic brain injury, spinal cord injury, cerebral palsy and other diseases).

Study Funding. The study was supported by the grant of the Russian Science Foundation (project No.15-19-20053).

Conflicts of Interest. The authors have no conflicts of interest to disclose.