An Ensemble of Convolutional Neural Networks for the Use in Video Endoscopy

In this study, a technology for creating a classifier able to identify pathological formations in images obtained with video endoscopy using the methods of deep learning is proposed. For the training and testing of neural network models, images from the CVC-ColonDB open database and 20 colonoscopy video records from the University of Arizona (Phoenix, USA) were used. To improve the performance of the proposed classification model, noise effects inherent to video cameras were considered. In addition, a study on building the model using small data samples was conducted.

In building the classifier, we utilized the results of recent studies on convolutional neural networks used in medical diagnostics, which allows us to apply the proposed approach to designing the architecture of a convolutional neural network adapted to a given task. By generalizing the features of the successful models, we developed an approach towards creating a non-excessive convolutional neural network. According to the proposed approach, the network architecture is divided into blocks, which alternate to enable composing the most efficient architecture.

Using the proposed approach based on the recommended selection strategy and then ranking the most significant parameters, a second approach towards building an adaptive model of classifier has been proposed. It is based on the formation of an ensemble of classifiers such as the “convolutional neural network”. To ensure the stability of the model and its insensitivity to changes in the input data as well as its applicability to different classification tasks, a set of networks with different major parameters are incorporated into the ensemble.

Our experimental studies have shown that the proposed classifier can be improved by developing an ensemble of convolutional neural networks, which considers the functions proposed in the present approach. The results imply the prospective application of the developed approach for building classification models not only for medical diagnostics but also for general problems of machine vision based on small samples.

1. Varol G. Salah A.A. Efficient large-scale action recognition in videos using extreme learning machines. Expert Syst Appl 2015; 42(21): 8274–8282, https://doi.org/10.1016/j.eswa.2015.06.013.
2. Taylor F.R. Evaluation of supervised machine learning for classifying video traffic. Doctoral dissertation. Nova Southeastern University; 2016.
3. Li B., Meng M.Q.-H., Lau J.Y.W. Computer-aided small bowel tumor detection for capsule endoscopy. Artif Intell Med 2011; 52(1): 11–16, https://doi.org/10.1016/j.artmed.2011.01.003.
4. Wang Y., Tavanapong W., Wong J., Oh J., de Groen P.C. Part-based multiderivative edge cross-sectional profiles for polyp detection in colonoscopy. IEEE J Biomed Health Inform 2014; 18(4): 1379–1389, https://doi.org/10.1109/jbhi.2013.2285230.
5. Moon W.K., Shen Y.W., Bae M.S., Huang C.S., Chen J.H., Chang R.F. Computer-aided tumor detection based on multi-scale blob detection algorithm in automated breast ultrasound images. IEEE Trans Med Imaging 2013; 32(7): 1191–1200, https://doi.org/10.1109/tmi.2012.2230403.
6. Sudharani K., Sarma T.C., Prasad K.S. Advanced morphological technique for automatic brain tumor detection and evaluation of statistical parameters. Procedia Technology 2016; 24: 1374–1387, https://doi.org/10.1016/j.protcy.2016.05.153.
7. Goel R.M., Patel K.V., Borrow D., Anderson S. Video capsule endoscopy for the investigation of the small bowel: primary care diagnostic technology update. Br J Gen Pract 2014; 64(620): 154–156, https://doi.org/10.3399/bjgp14x677680.
8. Silva F.B., Dinis-Ribeiro M., Vieth M., Rabenstein T., Goda K., Kiesslich R., Haringsma J., Edebo A., Toth E., Soares J., Areia M., Lundell L., Marschall H.U. Endoscopic assessment and grading of Barrett’s esophagus using magnification endoscopy and narrow-band imaging: accuracy and interobserver agreement of different classification systems (with videos). Gastrointest Endosc 2011; 73(1): 7–14, https://doi.org/10.1016/j.gie.2010.09.023.
9. Li B., Meng M.Q., Xu L. A comparative study of shape features for polyp detection in wireless capsule endoscopy images. Conf Proc IEEE Eng Med Biol Soc 2009, https://doi.org/10.1109/iembs.2009.5334875.
10. Li B., Fan Y., Meng M.Q.-H., Qi L. Intestinal polyp recognition in capsule endoscopy images using color and shape features. International Conference on Robotics and Biomimetics (ROBIO) 2009, https://doi.org/10.1109/robio.2009.5420969.
11. Tajbakhsh N., Gurudu S.R., Liang J. A classification-enhanced vote accumulation scheme for detecting colonic polyps. Lecture Notes in Computer Science 2013; p. 53–62, https://doi.org/10.1007/978-3-642-41083-3_7.
12. Park S.Y., Sargent D., Spofford I., Vosburgh K.G., A-Rahim Y. A colon video analysis framework for polyp detection. IEEE Trans Biomed Eng 2012; 59(5): 1408–1418, https://doi.org/10.1109/TBME.2012.2188397.
13. Song S.E., Seo B.K., Cho K.R., Woo O.H., Son G.S., Kim C., Cho S.B., Kwon S.S. Computer-aided detection (CAD) system for breast MRI in assessment of local tumor extent, nodal status, and multifocality of invasive breast cancers: preliminary study. Cancer Imaging 2015; 15(1): 1, https://doi.org/10.1186/s40644-015-0036-2.
14. El-Dahshan El-S.A., Mohsen H.M., Revett K., Salem A.-B.M. Computer-aided diagnosis of human brain tumor through MRI: a survey and a new algorithm. Expert Syst Appl 2014; 41(11): 5526–5545, https://doi.org/10.1016/j.eswa.2014.01.021.
15. Liu L., Tian Z., Zhang Z., Fei B. Computer-aided detection of prostate cancer with MRI: technology and applications. Acad Radiol 2016; 23(8): 1024–1046, https://doi.org/10.1016/j.acra.2016.03.010.
16. State Scientific Center of Coloproctology of the Federal Health Service. Department of endoscopic surgery. How is a colonoscopy? URL: http://www.colonoscopy.ru/patient/procedure2.htm.
17. Uglov A.S., Zamyatin A.V. Informatsionno-programmnyy kompleks dlya resheniya zadach personalizirovannoy meditsiny s primeneniem intellektual’nogo analiza dannykh. V kn.: Informatsionnye tekhnologii i matematicheskoe modelirovanie [Information and software complex for solving problems of personalized medicine with the use of data mining. In: Information technologies and mathematical modeling]. Tomsk; 2017; p. 126–134.
18. Axyonov S., Zamyatin A., Liang J., Kostin K. Advanced pattern recognition and deep learning for colon polyp detection. In: Distributed computer and communication networks: control, computation, communications. Moscow; 2016; p. 27–34.
19. Aksenov S.V., Kostin K.A., Jianming L., Zamyatin A.V. Ispol’zovanie metodov Deep Learning dlya detektirovaniya polipov pri kolonoskopii. V kn.: Informatsionnye tekhnologii i matematicheskoe modelirovanie [The use of Deep Learning methods for polyp detection during colonoscopy. In: Information technologies and mathematical modeling]. Tomsk; 2016; p. 75–79.
20. Bernal J., Sánchez J., Vilariño F. Towards automatic polyp detection with a polyp appearance model. Pattern Recognition 2012; 45(9): 3166–3182, https://doi.org/10.1016/j.patcog.2012.03.002.
21. Nibali A., He Z., Wollersheim D. Pulmonary nodule classification with deep residual networks. Int J Comput Assist Radiol Surg 2017; 12(10): 1799–1808, https://doi.org/10.1007/s11548-017-1605-6.
22. Tajbakhsh N., Gurudu S.R., Liang J. Automatic polyp detection in colonoscopy videos using an ensemble of convolutional neural networks. IEEE 12th International Symposium on Biomedical Imaging (ISBI) 2015, https://doi.org/10.1109/isbi.2015.7163821.
23. LeCun Y., Kavukcuoglu K., Farabet C. Convolutional networks and applications in vision. Proceedings of 2010 IEEE International Symposium on Circuits and Systems 2010, https://doi.org/10.1109/iscas.2010.5537907.
24. Flach P. Mashinnoe obuchenie. Nauka i iskusstvo postroeniya algoritmov, kotorye izvlekayut znaniya iz dannykh [Machine learning: the art and science of algorithms that make sense of data]. Moscow: DMK Press; 2015.
25. CVC colon DB. URL: http://mv.cvc.uab.es/projects/colon-qa/cvccolondb.
26. Park S.Y., Sargent D., Spofford I., Vosburgh K.G., A-Rahim Y. A colon video analysis framework for polyp detection. IEEE Transactions on Biomedical Engineering 2012; 59(5): 1408–1418, https://doi.org/10.1109/tbme.2012.2188397.
27. Tajbakhsh N., Gotway M.B., Liang J. Computer-aided pulmonary embolism detection using a novel vessel-aligned multi-planar image representation and convolutional neural networks. Medical Image Computing and Computer-Assisted Intervention 2015; p. 62–69, https://doi.org/10.1007/978-3-319-24571-3_8.
28. Zhu R., Zhang R., Xue D. Lesion detection of endoscopy images based on convolutional neural network features. 8th International Congress on Image and Signal Processing (CISP) 2015, https://doi.org/10.1109/cisp.2015.7407907.
29. Kooi T., Litjens G., van Ginneken B., Gubern-Mérida A., Sánchez C.I., Mann R., den Heeten A., Karssemeijer N. Large scale deep learning for computer aided detection of mammographic lesions. Med Image Anal 2017; 35: 303–312, https://doi.org/10.1016/j.media.2016.07.007.
30. Tajbakhsh N., Gurudu S.R., Liang J. A comprehensive computer-aided polyp detection system for colonoscopy videos. Lecture Notes in Computer Science 2015; p. 327–338, https://doi.org/10.1007/978-3-319-19992-4_25.

Aksenov S.V., Kostin K.A., Ivanova A.V., Liang J., Zamyatin A.V. An Ensemble of Convolutional Neural Networks for the Use in Video Endoscopy. Sovremennye tehnologii v medicine 2018; 10(2): 7, https://doi.org/10.17691/stm2018.10.2.01