Combined Use of Command-Proportional Control of External Robotic Devices Based on Electromyography Signals

The use of human brain and muscle signals for adaptive control of external robotic devices is a critical interdisciplinary task, the solution of which lies both in various fundamental sciences (neurobiology, neurotechnology), and in applied biomedicine (rehabilitations devices and methods, exoprostheses and exoskeletons, treadmills, etc.) [1–14]. Among many works there can be distinguished two main research lines. The first aims at brain-machine interfaces for a wide range of users for neurocontrol of mechanized software and hardware platforms [12–15]. The second line is focused on biomedical problems, and rehabilitation of patients with disordered motor function due to injuries or nervous diseases [8–11, 13, 16, 17].

Exoskeleton complexes can serve as a dramatic example of innovative devices combining the latest advances of control theory of both neurointerfaces and rehabilitation medicine [4–7, 17]. An integral part of these systems is an information interpreter coming from multiple biometrical sensors — a classifier of motor patterns, the realization of which requires a particular approach [18–20].

In this study we put forward a complex algorithm for a control system using electromyography (EMG) signals. The algorithm is universal, i.e. the controlled objects can be various robotic devices: mobile and humanoid robots, exoskeletons of upper and lower extremities.

Several strategies can be used to solve the problem ofcontrol of external (“additive”) devices using EMG signals. Conventional techniques are based on one-channel recording and limited to either trigger control based on detecting a threshold signal, or proportional control in case of continuous monitoring of some discriminating feature extracted from an EMG signal. It should be noted that multichannel recording significantly expands control capabilities, and entirely new signal processing techniques are used, such as classification of EMG patterns and multichannel regression [11, 14, 21].

The method of pattern classification perfectly combines with command control and can be used in cases when a controlled device is equipped with an automatic local control system able to generate macrocommands. In turn, proportional control is highly sought in the absence of such local control system, as well as if it is necessary to provide the accuracy of a movement executed.

The aim of the investigation was to develop a novel system to combine command and proportional control, in which a movement type is determined according to the classification, and the speed of motion execution at any specific time is determined based on the characteristic of EMG signals.

Materials and Methods. The experiments involved 10 healthy research subjects aged from 20 to 56 years. The study complies with the declaration of Helsinki (adopted in June, 1964 (Helsinki, Finland) and revised in October, 2000 (Edinburg, Scotland)) and was approved by the Ethics Committee of Lobachevsky State University of Nizhni Novgorod. All patients gave their written informed consent.

EMG signals were recorded by 8-channel bracelet MYO Thalmic (Thalmic Labs, Canada) (Figure 1), the electrodes in the device being situated along the circle around the forearm. To a certain extent, such localization complicates the recognition of EMG patterns compared to the more popular schemes [18], where an electrode position is determined by the localization of motor areas of the studied muscles.

---

Figure 1. Represented version of hardware and software system: MYO Thalmic bracelet, a laptop computer with program control, a mobile robot

---

We divided the data flow of each channel into windows, 200 samples in size, with consideration for sampling frequency of 2,000 Hz it corresponded to a time interval of 100 ms. Windowing was performed at a pitch of 100 samples (50 ms). We calculated a mean square of a signal for each window (root mean square, RMS), which we used further for classification as a signal feature:

For proportional control we used mean absolute value (MAV), a signal value in modulus and averaged over all the calculations in a window:where N is the number of signal values in a time window; х_n is a signal value at a time point n.

Since a window size for successful proportional control was not known a priori, and was to be varied during the experiments, and due to the fact that RMS and MAV calculations could require the windows of different sizes, in this study we calculated MAV using a formula of exponentially moving average requiring no data segmentation into windows:

where MAV_t is exponentially moving average of an absolute value at point t (current value), MAV_t–1 is a value of exponentially moving average at point t–1 (previous value); х_t is the value of EMG signal at a time point t (the most recent recorded value); a is the coefficient characterizing weight reduction rate.

The coefficient α characterizes the effect of the previous values on a current mean value, and can be expressed as a window average value:

where n is the amount of initial function values.

As motor patterns for classification we used 6 static hand gestures (Figure 2, a classification block is marked by setting in a green box). Test subjects were making 4 series consisting of 9 gestures randomly, each gesture being executed within 2–3 s. A hand was relaxed between different gestures.

---

Figure 2. The interface of a robotic device control software using EMG signals. Classification module is marked by setting in a green box, proportional control module is set in a red box, visualization module of EMG signals is marked by setting in a blue box. The insertion demonstrates the result of proportional control experiment

---

RMS values of all channels calculated in accordance with the formula (1) were sent the classifier input. As a classifier we used a multilayer network of formal neurons with a standard back propagation algorithm. For machine learning we needed 1,000–4,000 iterations (epochs) that took 15–60 s when calculated on Intel Core I3 computer. After the procedure a classifier could operate on-line, with a recognized gesture being highlighted in a program interface (Figure 2, “Command 1”).

To study proportional control and its possible combination with command control, we developed a functional realized in a Proportional control block of the principal form of software (Figure 2, marked by setting in a red box).

The nature of dependence of EMG signal amplitude on muscle contraction was studied using a hand gripper with scale marks 25, 50, 75 and 100%. The preliminary studies showed that according to EMG signal (MAV) 100% gripper corresponded to the grip of a hand dynamometer with the force of 320 H. We canceled the further use of a dynamometer due to the interrupt mode at the maximum effort. First we recorded MAV of one of EMG channels at 100% grip of a hand gripper, which was kept within 3 s. Then a test subject relaxed his hand, the procedure being repeated after a 15-second pause, the grip of a hand gripper being equal to 25, 50 and 75%. In this case we calculated MAV for the whole time period, during which a test subject was generating the force.

Then, every test subject underwent a MAV normalization procedure, during which an operator marked a signal level at a relaxed upper extremity using a SetMin command, and at maximally strained extremity using a SetMax command (a gripper was not used in this case). After that a lower slider of Proportional control (Figure 2, a red box, a blue marker) started moving online depending on MAV recorded. Test subjects were offered to repeat the protocol slider movements by changing a hand effort (Figure 2, a red marker), the slider being preset for successive demonstration of values of 75, 25, 100, 50% within 5 s for each value. The results were displayed in the form of a diagram with the curves corresponding to the task (Figure 2, “protocol”), the task being accomplished using EMG interface (Figure 2, “experiment”).

In combined command and proportional control experiments a test subject was to perform one of the six static gestures simultaneously with the abovementioned proportional control task. In this case the diagrams showed the curve areas corresponding to correct recognition.

To assess the performance efficiency of the suggested control concept we used a mobile robot assembled using LEGO NXT Mindstorms (LEGO, Denmark). The software sent online, with frequency of 10 Hz, using wireless Bluetooth interface command corresponding to a recognized gesture. And command execution speed was determined by the value calculated by a proportional control block.

Results. Preliminary findings showed that MAV of an EMG signal recorded in the flexor muscles of the forearm demonstrated linear dependence on the efforts applied (Figure 3). Such dependence type enables to use a recorded EMG signal for proportional control, when the controlled value depends on signal amplitude, and respectively, on muscle strain degree.

---

Figure 3. The dependence of an average absolute value of EMG signal recorded in forearm flexor muscles while gripping of hand gripper in 6 test subjects

---

Since MAV is a time-averaged value, first of all it was necessary to find out how the average window size or its inversely proportional value α (equation 4) influenced control quality. The findings showed (Figure 4) that α has an effect on the time when a controlled value attains a quasisteady value, and the variability degree of the value. Thus, if α=0.001 then the average time of attaining a stationary value in 10 test subjects was 2.7 s, if α=0.003 it was 1.5 s, and if α=0.009 it was equal to 0.9 s. Variability was assessed using MAV standard deviation, starting calculating from the moment of attaining a quasisteady value. Its value was 1.4, 2.9 and 6.7%, respectively. Subjectively, the test subjects noticed that if α=0.001 the controlled value changed with the more delay, and if α=0.009, it caused difficulties in keeping it at a certain level. Reasoning from this fact, for further studies we used α=0.003.

---

Figure 4. The task execution, during which a test subject is to achieve and keep 75, 25, 100, 50% of the controlled value within 5 s. A “protocol” curve is the standard task; α curves are normalized MAV values of an EMG signal at different window average coefficients

---

To adjust the recognition system of EMG patterns we carried out a study in order to determine the optimal parameters for the purposes of the accuracy of classification and time spent for machine learning. We tested three types of classifiers: a linear classifier, a one-layer perceptron, and a multilayer neural network based on back propagation algorithm. We changed the number of layers and formal neurons in case of a multilayer network, and we changed the rate of learning, as well as the number of gestures. In accordance with the findings, we chose a two-layer neuronal network based on back propagation algorithm as an optimal classifier, the number of neurons being 9+7. The rate of learning was 0.01. The number of gestures was reduced from 9 to 7, and for further study of combined command-proportionalcontrol we used 6 gestures (it was impossible to use gesture 7, a relaxed hand, for this purpose). The Table demonstrates the results of recognition system operation with determined parameters for different test subjects.

---

The classification accuracy of recognition system of EMG patterns generated when performing 7 static gestures

---

In the course of the work we tested four schemes of combined command-proportional control.

Proportional control in the easiest Scheme 1 was performed regardless of command control, for this purpose two myobracelets were used. EMG patterns were classified based on eight electrodes situated on the upper extremity, while one of the electrodes on the second bracelet was used for proportional control. The scheme enabled to achieve maximum accuracy of classification and proportional control that can be estimated visually (Figure 5). In robot steering users had difficulties neither in mode switching and keeping the mode using static gestures of one limb, nor in setting the robot speed using different degrees of strain of another forearm muscles. However, the scheme has two substantial defects: there should be used an additional myobracelet and two limbs that can be impossible in case of exoskeleton control.

---

Figure 5. The task execution according to combined command-proportional control when using Scheme 1 (the use of independent channels for command and proportional control). A “protocol” curve is the standard task; “gesture” curves are normalized MAV values of EMG signal when accomplishing the task with an appropriate gesture. Curve gaps are the moments of EMG pattern fault recognition

---

The control in the rest schemes was exercised by means of one myobracelet. In Scheme 2 seven electrodes were used for classification, and electrode 8 was used for proportional control. When making gestures, a test subject was to grip a hand gripper (or clench his hand) with different force. The scheme appeared to be inoperable, since in this case two motor patterns were executed simultaneously, one of the patterns corresponding to an executing gesture, and another — to hand clenching by different force. It resulted to the following: even at 50% force the proportion of classifier’s errors increased up to inacceptable 30–40%.

According to Scheme 3 all 8 electrodes were used for classification. For proportional control at any time point there was chosen MAV of that channel, which contribution in a gesture under classification was maximal. A test subject could execute static gestures by different force, it had no significant effect on classifier’s work, though enabled to obtain MAV gradually dependent on force (Figure 6). The scheme turned out to be well-behaved in the experiments with online robot control, though MAV calculated this way differed in strong abrupt changes resulting in difficulties in setting the mobile robot speed. The attempts to increase a temporary window, in which an averaged MAV (α reduction) was realized, resulted in leap smoothing but significantly increased system response time delay.

---

Figure 6. The task execution according to combined command-proportional control when using Scheme 3 (the usage for proportional control of a channel with maximum activity contribution to EMG pattern). A “protocol” curve is the standard task; “gesture” curves are normalized MAV values of EMG signal when accomplishing the task with an appropriate gesture. Curve gaps are the moments of EMG pattern fault recognition

---

Scheme 4 differed from the previous one in that all 8 channels were used to receive a value that gradually dependent on an effort. As this value, we used MAV arithmetic mean value of all the channels. Averaging over all the channels resulted in leap smoothing with no increase in system response time (Figure 7). As a result, a test subject could change a value of a controlled variable, and correspondingly, the robotic device speed in experiments with an online controlled robot. Subjectively, robot steering using one extremity appeared to be more difficult than using two ones (as Scheme 1 suggests), however, all users after 3–5-minute training were able to generate any of 6 commands for a robot simultaneously with mobile robot speed control.

---

Figure 7. The task execution according to combined command-proportional control when using Scheme 4 (the use of average MAV value of all channels for proportional control). A “protocol” curve is the standard task; “gesture” curves are normalized MAV values of EMG signal when accomplishing the task with an appropriate gesture. Curve gaps are the moments of EMG pattern fault recognition

---

Discussion. The development of high-hume robotic devices necessitates revolutionary changes in medical technologies. In particular, the last decade has been marked with a wide expansion of robotic systems in the sphere of technical equipment substituting missinglimbs, and rehabilitation equipment for locomotor patients.

Most advanced devices suppose the use of bioinformative indicators for effective communication with a user. In cases when cerebrospinal conduction of brain signals is partially impaired or remained unchanged, myoelectrical activity of target muscular groups is commonly considered as such indicator.

Currently, there are two control strategies of robotic devices based on the use of neuromuscular signals. One of the strategies is focused on distinguishing some predetermined motion types, while another analyzes the muscular contraction intensity.

The authors of the present study for the first time made an attempt to develop a hybrid classifier of electromyographic activity combining the capabilities of a classical binary interpreter of neuromuscular signals (into a kind of a fixed “alphabet”: a set of commands that an executive unit understands) and a proportional translator enabling to vary the activation degree of an executive device.

Among four presented schemes of combined command-proportional control, Scheme 4 appeared to be the most optimal one, since it enabled to do with a myobracelet using only one extremity for control; in addition, it demonstrated high classification accuracy and proportional control. As for the disadvantage noted, some test subjects encountered the difficulties in using certain gestures, for example, supination. In Figure 7 (“gesture” 6) an appropriate curve is highly discontinuous. It is likely to be related to the localization of muscles providing the gesture execution in relation to the electrodes controllable. For example, the contraction of musculus biceps brachii plays the major role in maintaining supination, since it cannot be recorded without an additional electrode. The disadvantage can be overcome by individual choice of gestures, or by means of total MAV normalization depending on the recognizable pattern that is going to be considered in future studies.

An anthropomorphic robotic manipulator (e.g., hand prosthesis) can serve as a striking example to demonstrate the advantages of the proposed concept of hybrid classification. Due to an algorithm, both the type of the motion executed (e.g., finger closing), and the grip strength.

Moreover, the inclusion of myoelectric activity analysis module in control circuit enables to activate biological feedback, which in its turn will make the integration degree of an executive unit and an operator reach a qualitatively new level. However, the activation of brain plasticity mechanisms promotes the recovery of lost motor functions resulted from neurological diseases and brain traumas. The combination of biofeedback mechanisms and accepted approaches of afferent stimulation (due to periodic repetitions of some fixed motion repertoire) gives rehabilitation therapy a multiplicative effect.

Thus, the advantages of the suggested hybrid control algorithm enable to expand the functional capabilities and efficiency of rehabilitation robotic complexes (in particular, exoskeletons of lower limbs).

Conclusion. We proposed a novel hardware-software system to control medical robotic devices combining command-proportional control based on the neuromuscular activity signals. The investigations carried out revealed the optimal parameters in regard to the accuracy of command classification system, as well as speed and accuracy of proportional control. Moreover, we proposed several schemes of combined command-proportional control, and analyzed their efficiency.

Study Funding. The study was supported by Ministry of Education and Science of the Russian Federation within the frame of federal target program “Research and development in priority areas of Russian scientific and technological complex development in 2014–2020”. Aid grant agreement No.14.581.21.0011 dated 01.12.2014 (unique agreement identifier of the project is RFMEFI58114X0011).

Conflicts of Interest. The authors have no conflicts of interest related to the present study.