Gaze Fixation Patterns Correlate with Visual Attention and Memory: the Results of a Pilot Study in Healthy Subjects

Introduction

The mechanisms of distinguishing between similar short-term memory traces (pattern separation) enable the ability of the brain to distinguish between similar ambient situations. In humans, this ability is tested in behavioral tasks of recognition of visually similar objects [1–4]. For example, when presented (with some time interval) with two similar images, a healthy individual imprints (memorizes) these images separately, which enables him/her to distinguish between them later. If the subject perceives a similar object as the one seen previously, then such an error can be interpreted as a dysfunction of the pattern separation mechanism.

In the recent decade, quite a few studies were devoted to these mechanisms in animals and humans [2–7]. In these research projects, the authors mainly described the mapping and localization of the pattern separation function throughout the hippocampus, the structure involved in the formation and extraction of new memories. Only selected reports though addressed the parameters of visual fixations as indicators of visual attention and predictors of specific pattern separation errors [6, 8]. It has been shown that the content of visual stimuli, the distribution of visual attention, the testing conditions and the memory indicators are closely related. For instance, the distribution of visual fixations may be affected by the parallel implementation of two types of activity [9]. Emotionally significant stimuli attract longer visual fixations than neutral ones [10]. The trajectories of eye movement during the initial and repeated presentation of a stimulus may be related to the completeness of the information on this stimulus stored in the memory [11]. In addition, the age-related features in visual attention, reflecting the ontogenetic changes in the nervous system, may also affect the quality of memorization and deserves careful investigation [12].

The present pilot study was conducted to assess the spatial gaze distribution (using the eye tracking method) features in healthy subjects of various ages who made specific pattern separation errors when tested for their visual attention and memory.

Materials and Methods

The study involved 45 healthy subjects who met the inclusion criteria and gave their voluntary consent to participate in this neuropsychological testing.

The inclusion criteria:

age of 18 years and older;

Russian is a native or primary language;

preserved vision and eye movements;

consent to participate in neuropsychological tests using the eye tracking technology;

acceptable scores in the basic neuropsychological tests for cognitive status (MoCA score ≥26);

the ability to discern between the concepts of an identical and similar object in the stimulus series.

The exclusion criteria:

brain disorders;

impaired vision and/or eye movements;

cognitive impairment detected at the stage of preliminary testing.

We used the original “eye tracking–attention–memory” methodology (hereinafter referred to as the ETAM) developed by O.A. Krotkova (2016), which implied the recording of eye movements before the visual attention and memory testing [13]. The study session involved three stages (Figure 1). At the first stage, the subject was presented with visual stimuli displayed on the screen (as a triplet of colored pictures arranged in a row) alternated with distractors shaped as gray screens. The exposure time of each triplet and each distractor was 10 s. The presentation began with a gray background. Before starting the test, the subject was asked to carefully examine the images and memorize them; the subject was also instructed to ignore the gray slides and relax at the time they appear on the screen. The experimental set included 5 triplets (15 pictures) and 6 identical distractors. The total duration of the presentation, therefore, was 110 s. While the examined subjects were looking at the changing images, their eye movements were recorded. The subjects did not receive any indication as to which part of the screen their gaze should focus before the presentation started and during the pauses.

---

Figure 1. Scheme of neuropsychological and eye tracking research in three stages

---

After the presentation was over, a period of interference (according to ETAM terminology) began; during that 10 min period, the subject underwent the neuropsychological test (by the method of A.R. Luria) and other psycho-physiological tests. Next, we conducted the second stage of the study.

At the second stage, the subject was asked to freely reproduce the stimuli stored in his/her memory. The subject had to recall and name the images he/she saw at the first stage, in any order. The number of freely reproduced objects was registered. After that, there was another period of interference for 15 min; during that time, the subject underwent the same neuro-psychological tests as he/she did at the first stage of the study.

The third stage (in line with the ETAM methodology) was a test for recognition of images already seen as stimuli at the first stage. This time, separate images appeared on the screen in a pseudo-random order; some of these were identical to those shown at the first stage; some were slightly different in details, color, or location in the visual field; there were also completely new images (distractors) not related to the previously shown ones. At this stage, the stimulus material consisted of 30 images: 15 previously seen objects; 10 images similar to the lateral stimuli in the triplets shown at the first stage; and 5 new distractors previously not shown. In this test for recognition, the subject reported, whether the image had been seen previously, or it was similar to that seen previously, or it was new, not seen previously. Before the study, for the purpose of training, the subject was shown the examples of identical and similar images not included in the stimulus series. The study was conducted only with those subjects who understood the meaning of “identical” and “similar”.

At the third stage of the study, the subject could make mistakes in recognizing the previously seen objects and distinguishing between similar, new and previously seen objects. The situation of a false identification by the subject of a similar object as previously seen was regarded as a pattern separation error in agreement with the experience of other researchers [1–4].

Eye movement recording was performed using an eye tracker (Eye Tribe, Denmark). The sampling rate was 30 and 60 Hz, the accuracy was 0.5–1.0°, the spatial resolution — 0.1°, the latency (hardware delay) was less than 20 ms at 60 Hz. Images were presented using the Ogama (open gaze and mouse analyzer) software. The gaze fixation points on the screen, recorded by the eye tracker, were compared with the shown images at a resolution of 1920×1080 pixels. The Matlab and R software were used to process the eye movement data for subsequent analysis.

To analyze the pattern of gaze fixations related to an individual image or its part, the respective zones of interest were set. The coordinates of the visual fixation points in these areas of interest were determined and then, the number and duration of fixations were calculated.

As part of the pilot studies, two series of tests were conducted in independent subpopulations with different stimuli. In the first series, we tested the hypothesis of a shift in the pattern of visual fixations in subjects from two age groups (<50 years and ≥50 years) who showed a pattern separation error.

In the second series of tests, we tested whether the occurrence of pattern separation errors was related to the distribution of the subject’s attention throughout the distinct object’s areas. For this purpose, the visual fixations in certain areas of interest were recorded.

The statistical analysis was performed using the R programming language and the environment (www.project.org, version 3.4.4). To assess the statistical significance of the differences in the distribution of fixation points for the compared subgroups, we used the Mann–Whitney test. Differences in the distribution of categorical variables between the subgroups, given the small number of observations, were evaluated using the Fisher exact test. The differences were considered statistically significant at p<0.05.

Results

First, studies using the ETAM methodology were conducted in subjects of group 1 (n=28), whose average age was 47.7±21.2 years; among them, 7 men (25%) and 21 women (75%). In this group, we found that an image showing a round-shaped object (e.g. plate) caused pattern separation errors most often (Figure 2). At the third stage, another plate-like image similar to the initial one was not identified by 6 subjects, and 22 subjects identified the similar plate-like image correctly. Given a smaller number of errors in pattern separation with other stimuli, the distribution of visual fixations was analyzed only for the “plates”.

---

Figure 2. Distribution of visual fixation points within the primary “plate” image in the younger (a) and older (b) age subgroups Red — visual fixation points in subjects who made pattern separation errors when viewing the given stimulus in the recognition test; green — fixation points in subjects who made no such errors

---

This group was divided by age into two subgroups (<50 and ≥50 years old); arbitrarily, we called them the “younger” and the “older” subgroups (see the Table).

---

Basic characteristics of subgroups from the 1st group of subjects (n=28)

---

In the younger subgroup (n=12), a pattern separation error was made by 1 subject, in the older subgroup (n=16) — by 5 subjects; however, the differences in the error occurrence between the two subgroups did not reach statistical significance (p=0.197).

Next, we analyzed the distribution of visual fixation points within the “plate” image in the younger and older subgroups.

Figure 2 shows the distribution of fixation points (visual attention maps) imposed on the primary “plate” image for the subjects of the younger and older subgroups. The visual fixation points in the subjects, who (at the third stage of testing) could not recognize a plate similar to the original stimulus (made a pattern separation error), are highlighted in red. The proportion of visual fixations in the subjects who made the error significantly prevailed in the older subgroup (p=0.01). In this Figure, one can see how the attention of subjects of two age subgroups was distributed and concentrated, considering the observed pattern separation error. In general, the attention focus of the older subjects was shifted to the left (p=0.03), and in the younger — in top (p<0.0001; Figure 3). There were no differences in the duration of visual fixations with the “plate” stimulus between the two age subgroups (p=0.6; Figure 4 (a)). Also, no difference in the duration of visual fixations was found between those who did and did not make a pattern separation error (p=0.1; Figure 4 (b)).

---

Figure 3. The shift of the attention focus while viewing the primary “plate” stimulus in the younger (a) and older (b) subgroups

---

Figure 4. The duration of visual fixation on the primary “plate” stimulus: (a) subjects of the two age subgroups; (b) subjects who did and did not make pattern separation errors in the recognition test

---

However, in one subject from the younger subgroup, who made a pattern separation error, the duration of visual fixations on the “plate” was significantly less (p=0.0003; Figure 5 (a)). There were no such differences in the older subgroup (Figure 5 (b)).

---

Figure 5. Duration of viewing the primary “plate” stimulus in those who did and did not make pattern separation errors in the younger (a) and older (b) age groups

---

The visual fixations patterns in those who made a pattern separation error in the older age group were significantly shifted horizontally with respect to the patterns of those who did not make this error (p=0.01; Figure 6 (a)). In the younger subgroup, we did not find such a displacement (p=0.44; Figure 6 (b)); however, given the error in only one subject, it was not possible to conduct a more reliable statistical analysis.

---

Figure 6. Positions of the visual fixation patterns with the primary “plate” stimulus in those who did and did not make pattern separation errors in the younger (a) and older (b) age groups The visual fixation points in subjects who made pattern separation errors in the recognition test are highlighted in red. The arrows indicate the shifts of visual fixation patterns in subjects who did or did not make an error relatively to each other

---

Figure 7 shows the gaze distribution density in the subjects of the younger and older age subgroups with the primary “plate” stimulus. In the older subjects, the attention focus was shifted from the rim to the white inner space of the “plate”.

---

Figure 7. The distribution density of visual attention with the primary “plate” stimulus in the younger (a) and older (b) age subgroups

---

The subjects who made a pattern separation error focused on a less specific part of the object (in its center); we then proposed that the chance of making an error depended on the way of viewing the object (attention to more important details).

To test this hypothesis in the younger subjects, we conducted an additional pilot experiment involving 17 healthy volunteers (group 2) — students of higher educational institutions aged from 19 to 25 years old (average age 20.7±1.7 years); among those were 5 men (29.4%) and 12 women (70.6%). In this group, at the third stage of the study (test for recognition), an additional recording of eye movements was performed. In the stimulus series of images, the pattern separation error occurred more often with the “key” image. Figure 8 shows an example of how the subject viewed the original stimulus at the first stage and then — a similar image in the recognition test.

---

Figure 8. A gaze trajectory in a subject who did not make a pattern separation error with the given object: (a) stimulus shown at the first stage of the test; (b) stimulus shown in the recognition test

---

A similar picture with keys (Figure 8 (b)) was correctly identified only by 6 people; the rest 11 subjects gave erroneous answers. The subjects, who gave the correct answers, paid significantly more attention (p=0.006) to the lower part of the original “key”, which was explicitly different between the two images. At the same time, there were no significant differences in the number of fixation points in other areas of interest (p>0.05).

Discussion

Identification of neurophysiological correlates of visual attention and memory is one of the topical yet difficult research tasks. Correct interpretation of memorizing and reproducing tests depends on several factors including the complexity of the shown object [14, 15]. One of the common limitations of such studies is the insufficient quantification of visual perception [14]. Another limitation is the impossibility of clear discerning the neurophysiological processes associated with the visual functions and memory [14]. In this study, we obtained results reporting on the distribution patterns of visual attention at the stage of perception, the function of memory at the stage of recall and recognition of information, as well as the relationship between visual perception and memory. Here, we used the original ETAM methodology developed by O.A. Krotkova [16, 17]. The objectification and quantitative analysis of visual attention using tracking have proved advantageous. In earlier studies conducted with this technique in healthy subjects [13], it was shown that the processes of free recall and recognition of recently memorized images significantly correlated. Along with that, an uneven distribution of visual attention with the triplet stimuli and differences in the capability of memorizing objects using different viewing strategies were found [13, 17]. According to the authors, the proposed technique generates new quantitative data related to the physiology of voluntary visual attention. In this paper, we analyze the distribution of visual fixation preceding the occurrence of a pattern separation error.

According to the reports of other authors [6, 8, 18, 19], the strategy of viewing the presented stimulus is likely related to the functional state of the memory-associated brain structures. In this study, the strategies for distributing the visual attention were selected by the subjects themselves. The neuropsychologist did not specify the order of viewing the presented images.

In general, the younger subjects demonstrated better levels of memory and attention than the subjects in the older age group: the adaptation mechanisms of visual attention and memory may be different for these subpopulations [17]. Therefore, we chose to consider the younger and older subpopulations as two different samples for the pilot statistical data analysis. The objects, for which the pattern separation errors occurred more often, but the initial amount of visual attention (the number of points of visual fixation on the object) was small, were chosen as visual stimuli for this analysis. These stimuli (“plate” in the first series and “key” in the second) occupied a lateral position in the triplets of simultaneously displayed images and received significantly less attention compared to the centrally positioned stimulus; this finding has been known from our earlier report on the ETAM methodology.

The present pilot study showed that the distribution of visual attention and the percentage of pattern separation errors differed between the younger and older subjects. At the same time, for all 5 subjects that made pattern separation errors in the older age subgroup, visual attention was concentrated on the inner, central area of the “plate”, while the rim (different from the rim of the plate used in the recognition test) was ignored (see Figure 3). In a subject of the younger group, who made a pattern separation error, the fixation points were distributed in the same way. However, this subject was viewing the object twice faster than the others in this subgroup. Previously, other authors described the dependence of the stimulus memorization on the number of visual fixations on the stimulus [6, 8]. Thus, one cannot exclude that the error of the younger subject resulted from a shorter viewing period while in the older age subgroup, the error might be caused by other factors.

In the older age subgroup, the concentration of attention is shifted toward the object’s center, that is an area less specific to the “plate”, whereas, in the younger group, the attention was concentrated on a more specific part — the rim (see Figure 7).

In this pilot study, we observed that a pattern separation error was more likely to occur when the gaze was focused on semantically less significant parts of the presented image. From this, it could be concluded that the pattern separation quality may depend on which object parts the attention was concentrated. We tested this hypothesis in the younger subjects by conducting an additional experiment. Using the “key” image, we found that the ability to distinguish between a similar object and the original one could be related to a more careful examination of distinct, meaningful parts of the original object, which is consistent with the results obtained in the group of 28 healthy subjects.

A significant limitation of this pilot study is a very small number of observations, which does not allow a priori to achieve high levels of statistical significance. Having 12–17 subjects in each subgroup, the percentage of the specific error of distinguishing a similar from a recently seen object is very small. In addition, there are very few data on visual fixations that occur during the subject’s first examination of the object. In the younger age group, only one person made a pattern separation error. To perform a statistical analysis of the visual fixation distribution in this subgroup, a much larger number of subjects is needed. However, taking into account the resource- and time-costs of such experimentation, we opted to present the obtained data “as is” to outline the trends we observed in our studies and to remind the readers about the preliminary character of these results [13, 17].

In this study, the conditional pattern separation error was determined in agreement with other authors who also used neuropsychological tests to detect it [2, 3, 5, 6]. Along with that, today there are no proven physiological criteria that would objectify this error that was originally described in animal models. That is why we, like other authors studying specific memory errors, are critical about the way it’s determined. Therefore, we focus primarily on the phenomenology, rather than on the interpretation of currently inaccessible physiological mechanisms that generate specific memory errors [6].

The known limitations of our study include the resolution of our eye tracker, which provided the opportunity to analyze the points of visual fixation, but did not allow us to analyze saccades thus decreasing the informative content of our results.

Today, it is believed that in mammals, the hippocampus plays the major role in recognizing similar objects [2, 20, 21]. In the structure of the hippocampus, continuous postnatal neurogenesis was discovered; this process is thought to ensure the memory plasticity and, possibly, the normal operation of the pattern separation function [2]. Therefore, the visual fixation distribution as an indicator of the state of visual attention and memory can potentially be a key to understanding the fundamental mechanisms of neuroplasticity, as well as a diagnostic tool in patients with cognitive impairments.

Conclusion

The results of this pilot study are consistent with the hypothesis claiming that the occurrence of pattern separation errors depends on the way the visual attention distributes. In addition, the study supports the hypothesis of the different spatial distribution of visual attention in healthy subjects of different ages. The eye tracking technology proved to be effective for objectifying the distribution of visual attention as a correlate and predictor of specific attention and memory errors.

Financial support. The research was supported by the Russian Science Foundation (grant 17-15-01426).

Conflict of interests. The authors have no conflict of interests to disclose.