Statistical Classification of Immunosignatures under Significant Reduction of the Feature Space Dimensions for Early Diagnosis of Diseases

Introduction

Today, the prevalence of cancer, infectious and other diseases remains at a high level [1]. Making the right diagnosis is, however, an expensive and lengthy procedure that suffers from many shortcomings. In cancer, the traditional diagnostic approaches would detect the disease rather at a late stage, when the chances of cure are extremely low [2]. Therefore, the medical and scientific community is now actively searching for new methods of early diagnosis of life-threatening diseases; among those novel approaches is the immunosignature technology [3].

This technology utilizes peptide microchips, where peptides with random amino acid sequences serve as disease detectors [4]. Such peptides have partial or complete similarity with the antigen epitopes. These multiple peptides, which represent the probable amino acid sequences of proteins, are able to react with antibodies; such reactions can take place even if the chip-embedded peptide does not precisely match the real antigen epitope [5]. This innovative technology is expected to help making the final diagnosis.

To date, there are several types of peptide matrices containing from 10 thousand to 350 thousand peptides [6, 7]. Because of the large number of measured characteristics, the problem of analyzing and interpreting such data arises. Even for an expert, it is difficult to approach the diagnosis by analyzing the data from a matrix with thousands of different peptides [8, 9]. A possible solution is the data automatic classification. However, classification of multiple data of large dimensions and volume is a laborious task that requires considerable computational resources. One solution to this problem is the use of the most informative peptides instead of using all available peptides. Due to this approach, the number of classification features can be reduced ten-folds while maintaining a sufficiently high quality of classification. There are many methods for selecting the most informative peptides; among them, the Student’s t-statistic is of the most practical importance [10].

The aim of the study was to explore the options of significantly reducing the feature space of immunosignature classification by selecting the most informative features while maintaining the reasonable quality of classification.

In this study, the following tasks were addressed:

to significantly reduce the feature space and select the most informative features;

to evaluate various immunosignature classifiers and find the most effective method of reducing the dimensionality.

Materials and Methods

The data. The technology based on peptide microchips was used [11]. The following digitized data were obtained from the GEO public biomedical database [12]:

the number of samples (patients) — 240;

the number of peptides — 9781;

the number of classes — 6.

Table 1 shows a fragment of a digitized peptide matrix containing several classes of diseases Y and peptides X. The names of the peptides are presented by the sequences of respective amino acids.

---

Table 1. Fragment of the peptide matrix

---

Searching for informative features. To select the most informative features, we exploited various statistical criteria and group discriminators:

the Student’s t-test [13];

the U-criterion of Mann–Whitney–Wilcoxon [14];

the Kolmogorov–Smirnov test [15];

the Jeffries–Matusita distance [16].

When using any statistical test, two opposing hypotheses are assumed the alternative hypothesis (denoted H₁), and the null hypothesis (H₀). In our study, the null hypothesis suggests that the classes presented in Table 1 are inseparable, and the alternative hypothesis suggests they are separable.

The statistical tests used in this study are aimed at a paired comparison between two different classes. Therefore, we designated the healthy individuals as a control to be compared in pairs with various classes of diseases (see Table 1). As a result, we obtained five independent data samples (the control class versus each type of cancer) for every statistical test. To evaluate the significance of each statistical criterion, a respective p-value was used.

The p-value is a measure adopted for testing statistical hypotheses. In fact, this is the probability of an error in rejecting the null hypothesis. Usually, the obtained p-value is compared with the generally accepted standard significance levels p=0.05 or p=0.01 [17]. In our study, we considered the peptide to be informative at p<0.005. Because of the large number of peptide attributes, only peptides with a minimum p-value were selected to compose the samples for statistical testing.

The following methodology was applied to select the informative peptides for statistical tests:

for each test, we created five independent samples of comparisons between each type of cancer and the control;

from each sample we select N peptides having a minimum p-value;

then we combine the samples into one set that contains each selected peptide only once.

To apply the Jeffries–Matusita distance criterion, it was necessary to select only those peptides that have the maximum value of this parameter.

The following methodology was used to select the informative peptides based on the Jeffries–Matusita distance:

we composed five independent samples of comparisons between each type of cancer and the control;

we combine the samples into one set that contains each selected peptide only once;

we select N peptides having the maximum values of the Jeffries–Matusita distance.

Classification methods. There are a lot of different methods of data classification that can be tested to solve this problem. We chose the classifiers based on different mathematical apparatus; those are briefly described below.

The Support Vector Machine (SVM) is one of the most often used supervised learning algorithms able to solve classification problems and perform regression analysis. The algorithm is part of the linear classifiers family. Because of its universality, it is commonly used in medicine, finance, pattern recognition and other areas [18].

The Naive Bayesian classifier is a probability of classifier based on the Bayes’ theorem with strict (naive) assumptions about the independence between the features [19]. Its main advantage is the ease of implementation and low computational costs in training. Its disadvantage is the low quality of classification when applied to the problems with a high feature space dimensionality.

The Random Forest is an algorithm of machine learning based on an ensemble of decision trees [20]. The main idea behind this algorithm is that a lot of different models are created using a relatively weak algorithm, the predictions by each model are averaged and the best result is selected. The main advantage of the RF is the ability to efficiently process data with a large number of features and classes.

Gradient Boosting (eXtreme Gradient Boosting, XGBoost) is a machine learning algorithm, where a linear combination of simple algorithms is created by changing the weight of the input data [21–22]. Each subsequent model (usually a decision tree) is created in such a way as to give more weight and preference to previously incorrectly predicted observations.

Evaluation of the quality of classification. To assess the quality of models and compare different machine learning algorithms, metrics are widely used. Their choice and analysis is an indispensable part of any research. To determine the basic metrics, we used the confusion matrix [23], which is presented in Table 2.

---

Table 2. Confusion matrices

---

Here ŷ is the predicted object’s class label by the classification algorithm, and y is the true label of the object’s class.

To assess the quality of classification, different metrics are used:

accuracy (the ratio of correct to total answers);

precision;

recall;

F-score.

In our study, we used the most common metric — the ratio of correct/total answers — for both binary and multiclass classification.

Methods of classification. Classification of data is carried out at several stages:

the application of the sliding control scheme [24] based on a tenfold partitioning of a set of objects, randomly dividing the initial data sample into the training and testing samples in equal proportions;

the classifier training and testing using the training and testing samples, respectively;

evaluation of the quality of classification using one of the above metrics.

Results

Normally, the processing of peptide matrices requires a lot of processor power and RAM. Figure 1 shows a graph comparing the time (logarithmic scale) needed for training of different classifiers with different numbers of peptides. It can be seen that the learning rate increases many times with a decrease in the number of peptides. As a result, the training becomes available even with a standard computer. Therefore, the reduction of immunosignature features is an important stage that allows for correct evaluation of the results.

---

Figuгe 1. Results of classification obtained with various classifiers

---

Table 3 shows the sizes of the samples selected by the indicated method. As a result, five samples with different numbers of peptides are obtained.

---

Table 3. Sample parameters

---

To verify the methods and the results, we produced smoothed probability curves for the occurrence of most and least informative peptide. In Figure 2 (a), the curves for a low informative peptide (selected by the method of Jeffries–Matusita) are depicted. The levels of luminosity are similar for all classes of diseases in question; therefore this peptide is useless for the purpose of classification, as it increases the noise in the data space. In contrast, Figure 2 (b) depicts the curves obtained for an informative peptide; the curves apparently differ for each class of the diseases. Therefore, this peptide is suitable for use in classification algorithms.

---

Figure 2. Occurrence of peptides in different diseases: (a) CSGRDTMPPHDKSAILMMIY — low informative peptide; (b) CSGRDTMPPHDKSAILMMIY — informative peptide

---

The obtained results allowed us to conclude that the selection of the most informative features is important for the analysis of immunosignatures. This selection procedure can significantly reduce the feature space dimensionality and get rid of redundant and uninformative data.

The results of classification are shown in Figure 3. To assess the performance of the classifiers, the data set No.1 from Table 3 was used; to assess the quality of classification, the accuracy was used. According to the results, all of the tested criteria for selecting the informative parameters showed a fairly good quality of classification, regardless of the type of classifier. These results imply that for the classification of immunosignatures it is possible to use a wide range of available classifiers.

---

Figure 3. Comparison of classification methods by using various significance criteria

---

The next stage of the experiment was a detailed analysis of each and every method of reducing the number of necessary characteristics. In view of the minor differences between the classifiers, we found it reasonable to focus on one of the classification algorithms, namely, the Random Forest algorithm.

In Figure 4, the graph reflects the testing results obtained with the Random Forest algorithm applied to various samples from Table 3. Each point on the graph represents the average value of the accuracy (correct/total answers ratio) after ten training sessions with the Random Forest algorithm. The results of the testing indicate that the quality of classification declines if the number of peptides is less than 24; however, the quality does not increase with the number of peptides rising to 115. Therefore, the informative area is located between these two limits.

---

Figure 4. Informative value of data processing with the Random Forest classifier at different number of peptides

---

The presented technology of selecting the informative features provides the correct result of the classification. Our study demonstrates that by using a sample with a small number of features it is possible to achieve a quality of classification comparable with that obtained with the initial large size sample.

Conclusion

In line with the rapid development of novel technologies for early diagnosis of diseases, analyzing data of high dimensionality is becoming a computational problem. These state-of-art technologies make it possible to obtain a large number of data with different diagnostic values, which necessitates the selection of the most informative ones.

In this study, we provide evidence that the most informative characteristics (peptides) can be identified using the proposed methods of dimensionality reduction. As a result, it becomes possible to reduce the data space by ~240 times without losing the quality of classification. Along with that, this time-consuming procedure of classification can be shortened 16-fold by using the Gradient Boosting and 1625-fold by using the Random Forest methods. With these approaches the classification problem can be solved with a help of standard computers.

The present study identified no obvious leader among either the classifiers or the methods of searching for informative features. In the near future, we plan to test the proposed methods of dimensionality reduction and classification in peptide matrices of higher dimensionality.

Financial support. The study was not funded by any sources

Conflict of interest. The authors declare no conflict of interest.