A New Method to Missing Value Imputation for Immunosignature Data

Introduction

Today, fast development of modern technology makes large amounts of various information available for analysis. However, there is often a problem of data representativeness unavoidable in collecting and analyzing this information. Representativeness declines mainly due to missing values under the influence of noise and the human factor when data are collected. Missing values have become a challenging issue for researchers, since all data mining methods work incorrectly or become ineffective in such situation. Therefore, missing value imputation in the sample is one of the primary tasks in data mining [1].

Currently, many cancers are potentially curable if diagnosed in an early stage. Early detection of malignant tumors requires simple, inexpensive, minimally invasive and accurate diagnostic methods [2]. The immunosignature technology is one of such methods [3].

This technology employs microarray chips of random amino acid sequence peptides to detect diseases based on the change in the profile of circulating antibodies [4]. These peptides have partial or complete similarity to antigen epitopes. Using the arrays of peptides that represent probable amino acid sequences of proteins makes it possible to determine the binding partner for many antibodies, even if there is no exact match for the epitope [5]. The data obtained are expected to help health care professionals make the final diagnosis. However, errors may occur in the process of obtaining the data due to both technological properties of the equipment and subsequent data digitization. Data representativeness may be lost because of this and some peptide luminosity values may become unavailable. This explains the urgency of solving the problem of adequate missing value imputation.

The aim of the study was to evaluate the existing methods for missing value imputation in the selected dataset and develop a new efficient method for processing such values.

To achieve this goal, it was necessary to solve the following tasks:

to evaluate the existing missing value imputation methods based on various mathematical tools;

based on the obtained knowledge, to offer the most suitable method for missing value imputation, applicable to immunosignature data;

to compare efficiency of the existing and proposed methods for missing value imputation in immunosignature data.

Materials and Methods

Materials obtained by immunosignature assay are a set of fluorescence intensity values for peptides, where the peptide names are columns (features), and class labels (samples) are rows. A set of data from the public repository of biomedical data (accession GSE52580) obtained by digitizing immunosignature data was used in the study [6, 7].

The dataset has no missing values, which allows us to control the nature and number of missing values in our set. The dataset has the following characteristics:

the number of samples — 240;

the number of features — 9781;

the number of classes — 6.

The number of samples is the same in each class, so the dataset is a balanced subset. Table 1 shows a fragment of a dataset with known disease classes and peptides. The names of peptides are presented as sequences of amino acids.

---

Table 1. Fragment of the peptide matrix

---

Missing value imputation methods under study. To date, there are a large number of different missing value imputation methods applicable to the solution of this problem [8–10], therefore it is reasonable to analyze methods based on fundamentally different mathematical tools. The following selected methods have been considered in more detail.

Simple imputation is one of the simplest and most well-known missing value imputation methods [11]. It consists of replacing the missing values of a feature with the median, mean or mode calculated from the present values of the feature. The advantage of this method is quick missing value imputation. However, if the quantity of these values is large, simple imputation will lead to significant distortion of data analysis results.

Weighted k-nearest neighbors algorithm is a simple and effective method to handle missing values based on the hypothesis stating that if the examples (samples) are close in the measured feature space, this implies they are close by unmeasured features [12]. The distance between two examples is calculated from the present feature values. Weighted average values of neighbors are used to calculate missing values [13].

Missing values can be recovered in a particular feature by predicting its values by other features using a variety of machine learning techniques. This approach implies successive presentation of each feature as a target variable, which is followed by training on examples having no missing values in the target variable and subsequent prediction of missing values of the target variable.

Since there are missing values among the features by which training is performed, they should be initially replaced by one of the simplest imputation methods, and then clarified using one of the machine learning techniques. Let us take a closer look at some of the most popular machine learning techniques.

Linear regression is a machine learning technique that involves constructing equations with a linear function of target feature dependence on one or more other features [14]. It is possible to construct the equation of dependence of one feature on another for two features Y₁ and Y₂:

Y₂=aY₁+b,

using the values known for each feature and impute the missing values using the resulting regression equation for the available values. This method is effective only if there is a certain level of linear dependence between the features.

Random forest is a machine learning technique based on a multitude of decision trees combined into an ensemble [15]. This algorithm is a universal solution and works effectively with both continuous and categorical features [16]. For categorical features, the predicted value is determined by majority voting of each individual tree in the ensemble. In turn, for numerical features, the predicted value is determined as the average between responses of each tree in the ensemble.

Proposed missing value imputation method. Missing value imputation method developed in the framework of this investigation is based on the use of such machine learning technique as gradient boosting.

Gradient boosting is a machine learning technique based on creating a linear combination of simple algorithms by changing the weight of the input data [17, 18]. Each simple algorithm (linear classifier or a decision tree) is created in such a way as to give more weight and preference to previously incorrectly predicted values. Linear combination length in simple algorithms is equal to the number of model rounds.

Since gradient boosting provides the possibility to build a linear combination of different algorithms, we can actually use two different methods — linear gradient boosting and gradient tree boosting.

To handle missing values, gradient boosting is used as follows. Missing values in the entire data set are imputed using simple imputation (in this paper, the missing values in each feature are replaced by a median calculated from the present feature values). Next, training is performed successively for each feature on samples whose values are present in this feature, followed by clarification of missing feature values. This process is repeated until the total training error for all features stops decreasing or until the maximum number of iterations is reached. The root mean squared error is employed as a training error metric [19].

Comparing the efficiency ofmissing value imputation methods. The main purpose of using the immunosignature technology is to support decision-making in diagnosis, which in the terms of data mining is reduced to the problem of classification. In this regard, to assess the quality of missing value imputation, classification results based on the data obtained by imputation procedure were applied in our investigation.

This data set was already applied in earlier studies [20] where random forest algorithm was found to show high classification results, therefore, using it in work as a classifier was quite appropriate. Various metrics are used to assess the accuracy of classification. Their selection and analysis are an indispensable part of any investigation [21]. Given that the dataset was a balanced selected subset, we applied a classification metric “proportion of correct answers” (accuracy) in our investigation.

The data set had a large number of features, which negatively affected the time spent on calculations. Therefore, to save time, only 120 most informative features were selected from this set using Student’s t-test [22].

To compare the effectiveness of methods under consideration, it was necessary to create a subset with a wide range of parameters. It was appropriate to implement this by creating missing values artificially. In multi-step analysis of the peptide microarray, missing values may appear at any stage with no consistent pattern. Therefore, it was impossible to find a method for creating missing values that imitated all possible scenarios. In this regard, the method of completely random creation of missing values (missing completely at random) was used in this study [23, 24].

The methods were compared following these steps:

1. Preparing multiple datasets with different numbers of missing values.

2. Imputation of missing values in created datasets using each of the methods under consideration in turn.

3. Classifying based on the data of each imputed set.

4. Repeating steps 1–3 thirty times.

5. Calculating the average accuracy of classification carried out using the imputed datasets for each of the methods under consideration for different numbers of missing values.

6. Evaluating the results.

The work was performed using the R programming language and software libraries available in CRAN repository (Table 2).

---

Table 2. Packages used and their parameters

---

Results

It is important to note that the investigation has revealed inability of such methods as linear regression and k-weighted nearest neighbors to process data sets, because the proportion of missing values in these cases exceeds 0.7 and 0.85, respectively. Therefore, performance assessment values are incomplete for these methods.

The results make it obvious (Figure 1) that the use of machine learning techniques to impute missing values is the most effective solution. Random forest and linear gradient boosting have shown the best results.

---

Figure 1. Classification results based on the data obtained after imputation of a variable proportion of missing values

---

Table 3 presents the maximum and minimum accuracy values. At a proportion of missing values approximating 0.7, such methods as random forest, linear gradient boosting, and gradient tree boosting maintain the same variance indices, which indicates their high efficiency.

---

Table 3. Minimum and maximum values of classification results

---

The data given in Table 3 and Figure 1 illustrate sharp worsening of the results for random forest and gradient tree boosting in datasets with a high proportion of missing values. Consequently, linear gradient boosting is more preferable in this situation. At the same time, there are no differences between the methods when the proportion of missing values is low.

Based on processing time values (Figure 2), two aspects can be emphasized in the methods.

---

Figure 2. Processing time for methods with different proportions of missing values on a log scale

---

Firstly, processing time of missing value imputation algorithms based on machine learning techniques decreases with the increase in proportion of missing values. This is due to the fact that the size of the training sample reduces with increase in the number of missing feature values and so does the training time of the algorithm.

Secondly, the method of k-weighted nearest neighbors is preferable for a small proportion of missing values as less time is required for missing value imputation.

Conclusion

The results of the investigation have revealed efficiency of the proposed method for missing value imputation based on linear gradient boosting under conditions of high proportion of missing values as compared to the analogous methods considered. At the same time, the method of k-weighted nearest neighbors is preferable for small numbers of missing values due to insignificant amount of time required for data processing and performance efficiency comparable to more complex methods.

The obtained knowledge is the basis for future research and eventual creation of software packages for pre-processing of peptide microarray data.

Study funding and conflict of interests. This study was not supported by any financial sources and the authors have no conflict of interests to disclose.