Quantification of Human Perception Using the Motor Human-Computer Interface Hand Tracker

Here, we propose an all-instrumental objective method (hand tracking) for the real time quantification of perception of stimuli presented in various modalities.

The method is based on measuring the dynamic error of performance by a subject of motor control over a real or virtual (computer generated) stimulus with the help of a contactless manipulator. The method got its name from its previous analogue — the eye tracking; in both cases, the measurement of psychophysiological characteristics of the subject is carried out by monitoring the parameters of his/her motor activities.

The above principle has been implemented in the form of a hardware-software complex Hand tracker, which has been tested for the ability to perceive the “segment orientation angle” and “figure color against the background” virtual stimuli.

With the first stimulus, individual diagrams of perception accuracy were obtained in healthy subjects who were asked to guess the segment inclination angle. The results allowed us to identify the parameters of the neural model simulating the respective primary recognizer. With the second visual stimulus, we found that in patients with focal brain lesions, the thresholds of color differentiation were abnormally high and, unlike the normal ones, did not depend on the semantic content of the stimulus.

Since the Hand tracker provides for the simultaneous control over several (up to eight) independent parameters of audial and visual stimuli, the proposed method can be used in studying the intermodal connections and/or the collective behavior, when different team members control different parameters of the stimulus, trying to solve a common problem.

The article also addresses the prospect of implementing the hand tracking method in the form of special applications for smartphones with embedded movement sensors. Such a portable device could be used for epidemiological studies of perception (primary cognitive functions) to elucidate the age- and gender-related, regional, social and other stratification of populations.

1. Antonets V.A., Polevaya S.A., Kazakov V.V. Hand tracking. Issledovanie pervichnykhkognitivnykhfunktsiychelovekapoikhmotornymproyavleniyam. V kn.: Sovremennaya eksperimental’nayapsikhologiya. Tom 2 [Hand tracking. Study of the primary cognitive functions of a person by their motor manifestations. In: Modern experimental psychology. Vol. 2]. Pod red. Barabanshchikova V.A. M: Institut psikhologii RAN; 2011; p. 39–55.
2. Yarbus A.L. Rol’ dvizheniy glaz v protsesse zreniya [The role of eye movements in the process of vision]. Moscow: Nauka; 1965.
3. Bouguer P. Sur la meilleure méthode pour observer l’altitude des étoiles en mer et Sur la meilleure méthode pour observer la variation de la boussole en mer. Paris; 1727.
4. Weber E.H. De pulsu, resorptione, auditu et tactu. Leipzig: Koehler; 1834.
5. Weber E.H. Der Tastsinn und das Gemeingefühl. In: Wagner R. (editor). Handwörterbuch der Phisiologie. Vol. III. Brunswick: Vieweg; 1846; p. 481–588.
6. Fechner G.T. Elemente der Psychophysik. Leipzig: Breitkopf und Härtel; 1860.
7. Stevens S.S. On the psychophysical law. Psychol Rev 1957; 64(3): 153–181, https://doi.org/10.1037/h0046162.
8. Lomov B.F. Spravochnik po inzhenernoy psikhologii [Lomov B.F. Handbook of engineering psychology]. Moscow: Mashinostroenie; 1982.
9. Antonets V.A., Kovaleva E.P., Zeveke A.V., et al. Novye cheloveko-mashinnye interfeysy. V kn.: Nelineynye volny. Sinkhronizatsiya i struktury. Ch. 2 [New human-machine interfaces. In: Nonlinear waves. Synchronization and structure. Part 2]. Pod red. Rabinovicha M.I., Sushchika M.M., Shalfeeva V.D. [Rabinovich M.I., Sushchik M.M., Shalfeev V.D. (editors)]. Nizhny Novgorod: Izdatel’stvo Nizhegorodskogo gosudarstvennogo universiteta; 1995; p. 87–98.
10. Sechenov I.M. Fiziologiya nervnykh tsentrov [Physiology of the nerve centers]. Saint Petersburg; 1891.
11. Antonets V.A., Kovaleva E.P. Evaluation of the static voltage control in the skeletal muscle by its micromovements. Biofizika 1996; 41(3): 711–717.
12. Antonets V.A., Kovaleva E.P. Statistical modeling of involuntary micro oscillations in a limb. Biofizika 1996; 41(3): 704–710.
13. Antonets V.A., Anishkina N.M., Kazakov V.V., Timanin E.M. Kolichestvennaya otsenka vospriyatiya chastoty zvukov slukhovym analizatorom. V kn.: XI sessiya Rossiyskogo akusticheskogo obshchestva [Quantification of the perception of the sound frequency by the auditory analyzer. In: XI session of the Russian Acoustic Society]. Moscow; 2001; p. 180–183.
14. Antonets V.A., Anishkina N.M., Gribkov A.L., Kazakov V.V., Timanin E.M. Kolichestvennaya otsenka vospriyatiya chastoty zvukov chelovekom s ispol’zovaniem upravlyaemogo zvukovogo generatora. V kn.: XIII sessiya Rossiyskogo akusticheskogo obshchestva. Tom 3 [Quantifying human perception of the sound frequency using a controlled sound generator. In: XIII session of the Russian Acoustic Society. Vol. 3]. Moscow; 2003; p. 107–110.
15. Antonets V.A., Kazakov V.V., Anishkina N.M. Quantitative evaluation of tonal sound frequency perception by a man. Biophysics 2010; 55(1): 104–109, https://doi.org/10.1134/s0006350910010173 .
16. Antonets V.A., Antonets M.A., Kazakov V.V., Il’in N.E., Kal’vaser I.B., Kryukov A.Yu., Pogodin V.Yu., Polevaya S.A. Programmno-apparatnyy kompleks “Handtracker” dlya izucheniya pervichnykh kognitivnykh funktsiy cheloveka po ikh motornym proyavleniyam. V kn.: Eksperimental’nyy metod v strukture psikhologicheskogo znaniya [The software-hardware complex “Handtracker” for the study of the primary human cognitive functions by their motor manifestations. In: Experimental method in the structure of psychological knowledge]. Moscow: Institut psikhologii RAN; 2012; p. 317–318.
17. Antonets V.A., Antonets M.A., Pogodin V.Yu., Kryukov A.Yu., Il’in N.E. Laboratornaya i mobil’naya versii metoda handtracking dlya issledovaniya pervichnykh kognitivnykh funktsiy cheloveka po ikh motornym proyavleniyam. V kn.: Protsedury i metody eksperimental’no-psikhologicheskikh issledovaniy [Laboratory and mobile versions of the hand tracking method for studying the primary cognitive functions by their motor manifestations In: Procedures and methods of experimental psychological research]. Pod red. Barabanshchikova V.A. [Barabanshchikova V.A. (editor)]. Moscow: Institut psikhologii RAN; 2016; p. 639–645.
18. Yakhno V.G, Nuidel I.V., Ivanov A.E. Model’nye neyronopodobnye sistemy. Primery dinamicheskikh protsessov. V kn.: Nelineynye volny — 2004 [Neuron-like model systems. Examples of dynamic processes. In: Nonlinear waves – 2004]. Pod red. Gaponova-Grekhova A.V., Nekorkina V.I. [Gaponov-Grekhov A.V., Nekorkin V.I. (editors)]. Nizhny Novgorod: IPF RAN; 2004; p. 362–375.
19. Yakhno V.G., Polevaya S.A., Parin S.B. Bazovaya arkhitektura sistemy, opisyvayushchey neyrobiologicheskie mekhanizmy osoznaniya sensornykh kanalov. V kn.: Kognitivnye issledovaniya. Tom 4 [The basic architecture of the system that describes the neurobiological mechanisms of awareness of sensory channels. In: Cognitive studies. Vol. 4]. Moscow: Institut psikhologii RAN; 2010; p. 273–301.
20. Antonets V.A., Nuidel I.V., Polevaya S.A. Adaptivnyy simulyator: komp’yuternaya tekhnologiya dlya rekonstruktsii orientatsionnykh moduley zritel’noy kory cheloveka. V kn.: Materialy XVI mezhdunarodnoy konferentsii po neyrokibernetike. Tom 2 [Adaptive simulator: computer technology for the reconstruction of the orientation modules of the human visual cortex. In: Materials of the XVI International conference on neurocybernetics. Vol. 2]. Rostov-on-Don; 2012; p. 142–145.
21. Buxhoeveden D.P., Casanova M.F. The minicolumn hypothesis in neuroscience. Brain 2002; 125(5): 935–951, https://doi. org/10.1093/brain/awf110.
22. Rusak L.V., Kalina V.A., Stasevich N.A. Komp’yuternaya grafika [Computer graphics]. Minsk; 2009.
23. URL: http://ma-tec.ru/handtracker.html.

Antonets V.A. Quantification of Human Perception Using the Motor Human-Computer Interface Hand Tracker. Sovremennye tehnologii v medicine 2019; 11(1): 141, https://doi.org/10.17691/stm2019.11.1.16