informИнформатикаInformatics1816-03012617-6963UIIP NASB10.37661/1816-0301-2020-17-4-48-60inform-1090Research ArticleОБРАБОТКА СИГНАЛОВ, ИЗОБРАЖЕНИЙ, РЕЧИ, ТЕКСТА И РАСПОЗНАВАНИЕ ОБРАЗОВSIGNAL, IMAGE, SPEECH, TEXT PROCESSING AND PATTERN RECOGNITIONКомпьютеризированная диагностика рака простаты на основе полнослайдовых гистологических изображений и методов глубокого обученияComputerized diagnosis of prostate cancer based on whole slide histology images and deep learning methodsКовалевВ. А.KovalevV. A.

Ковалев Василий Алексеевич, кандидат технических наук, заведующий лабораторией анализа биомедицинских изображений; доцент

Минск

Vassili  A.  Kovalev,  Cand.  Sci.  (Eng.),  Head  of the Laboratory of Biomedical Images Analysis; Associated Professor

Minsk

vassili.kovalev@gmail.comВойновД. М.VoynovD. M.

Войнов Дмитрий Михайлович, магистрант

Минск

Dmitry  M.  Voynov,  Undergraduate

Minsk

voynovdd@gmail.comМалышевВ. Д.MalyshauV. D.

Малышев Валерий Дмитриевич, инженер-программист лаборатории анализа биомедицинских изображений; магистрант

Минск

Valery D. Malyshau, Software Engineer of the Laboratory of Biomedical Image Analysis; Undergraduate

Minsk

malyshevalery@gmail.comЛапоЕ. Д.,LapoE. D.

Лапо Елизавета Дмитриевна, инженер-программист лаборатории анализа биомедицинских изображений; магистрант

Минск

Elizabeth D. Lapo, Software Engineer of the Laboratory of Biomedical Image Analysis; Undergraduate

Minsk

lilibetlapo@gmail.comОбъединенный институт проблем информатики Национальной академии наук; Белорусский государственный университетThe United Institute of Informatics Problems of the National Academy of Sciences of Belarus; Belarusian State UniversityБелорусский государственный университетThe United Institute of Informatics Problems of the National Academy of Sciences of Belarus; Belarusian State University2020021120201744860Copyright © Ковалев В.А., Войнов Д.М., Малышев В.Д., Лапо Е.Д., 20212021Ковалев В.А., Войнов Д.М., Малышев В.Д., Лапо Е.Д.Kovalev V.A., Voynov D.M., Malyshau V.D., Lapo E.D.Данная работа распространяется под лицензией Creative Commons Attribution 4.0.This work is licensed under a Creative Commons Attribution 4.0 License.https://inf.grid.by/jour/article/view/1090

Представлены результаты экспериментальных исследований и разработки средств автоматического анализа и распознавания гистологических изображений с целью получения количественных оценок наличия и степени агрессивности рака простаты в общепринятых шкалах Глисона  и ISUP. В качестве исходных данных использовались 10 616 полнослайдовых гистологических изображений с размером большей стороны до 100 000 пикселов и 22 089 их фрагментов размером 256×256 пикселов. Проведена оценка эффективности решения задачи с применением как традиционных методов, так и методов глубокого обучения. В качестве финальных выбраны два решения. Первое решение основано на последовательном анализе фрагментов изображений и включает выделение признаков с использованием сети ResNet50 и последующим обобщением частных результатов распознавания с помощью небольшой сверточной сети. Второе решение базируется на одновременном анализе отобранных информативных участков, представленных в виде промежуточного псевдоизображения, и последующем его распознавании с использованием ансамбля из четырех вариантов сверточных сетей с архитектурой EfficientNetB0. В результате независимого тестирования на закрытом наборе изображений, недоступных авторам, достигнута точность предсказания финальной оценки по шкале ISUP, равная 0,9277.

This paper presents the results of an experimental study and the development of tools for automatic analysis and recognition of histological images in order to obtain quantitative estimates of the presence and degree of aggressiveness of prostate cancer in the commonly used Gleason and ISUP scales. The input data consisted of 10 616 whole-slide histological images with the size of the largest side up to 100 000 pixels and

22 089 of their image tiles of 256×256 pixels in size. Two solutions were chosen as the final ones. The first solution is based on sequential analysis of image fragments and includes feature extraction using the ResNet50 network and the subsequent generalization of particular recognition results using a small convolutional network. The second solution is based on the simultaneous analysis of the selected informative sections, presented in the form of an intermediate pseudo-image, and its subsequent recognition using an ensemble of four variants of convolutional networks with the EfficientNetB0 architecture. Being independently tested on an unknown image dataset that was not available for authors, these approaches achieved the prediction accuracy of 0,9277 according to the ISUP scale.

рак простатыгистологияполнослайдовые изображенияглубокое обучениесверточные нейронные сетиprostate cancerhistologywhole slide histologydeep learningconvolutional neural networksРабота была выполнена при частичной финансовой поддержке проекта ГКНТ Беларуси, договор № 225/4/2019.The work was carried out with partial financial support from the project of the State Committee for Science and Technology of Belarus, contract no. 225/4/2019.ReferencesRawla P. Epidemiology of prostate cancer. World Journal of Oncology, 2019, vol. 10, no. 2, pp. 63–89.Rawla P. Epidemiology of prostate cancer. World Journal of Oncology, 2019, vol. 10, no. 2, pp. 63–89.Epstein J. I, Allsbrook W. C. Jr, Amin M. B., Egevad L. L. ISUP Grading Committee. The 2005 international society of urological pathology (ISUP) consensus conference on gleason grading of prostatic carcinoma. The American Journal of Surgical Pathology, 2005, vol. 29, iss. 9, pp. 1228–1242.Epstein J. I, Allsbrook W. C. Jr, Amin M. B., Egevad L. L. ISUP Grading Committee. The 2005 international society of urological pathology (ISUP) consensus conference on gleason grading of prostatic carcinoma. The American Journal of Surgical Pathology, 2005, vol. 29, iss. 9, pp. 1228–1242.Camparo P., Egevad L., Algaba F., Berney D. M., Boccon-Gibod L., …, Varma M. Utility of whole slide imaging and virtual microscopy in prostate pathology. Acta Pathologica, Microbiologica, et Immunologica Scandinavica, 2012, vol. 120, iss. 4, pp. 298–304.Camparo P., Egevad L., Algaba F., Berney D. M., Boccon-Gibod L., …, Varma M. Utility of whole slide imaging and virtual microscopy in prostate pathology. Acta Pathologica, Microbiologica, et Immunologica Scandinavica, 2012, vol. 120, iss. 4, pp. 298–304.Goldenberg S. L., Nir G., Salcudean S. E. A new era: artificial intelligence and machine learning in prostate cancer. Nature Reviews Urology, 2019, vol. 16, pp. 391–403.Goldenberg S. L., Nir G., Salcudean S. E. A new era: artificial intelligence and machine learning in prostate cancer. Nature Reviews Urology, 2019, vol. 16, pp. 391–403.Ström P., Kartasalo K., Olsson H., Solorzano L., Delahunt B., …, Eklund M. Artificial intelligence for diagnosis and grading of prostate cancer in biopsies: a population-based, diagnostic study. The Lancet Oncology, 2020, vol. 21, iss. 2, pp. 222–232.Ström P., Kartasalo K., Olsson H., Solorzano L., Delahunt B., …, Eklund M. Artificial intelligence for diagnosis and grading of prostate cancer in biopsies: a population-based, diagnostic study. The Lancet Oncology, 2020, vol. 21, iss. 2, pp. 222–232.Pantanowitz L., Quiroga-Garza G., Bien L., Heled R., Laifenfeld D., …, Dhir R. An artificial intelligence algorithm for prostate cancer diagnosis in whole slide images of core needle biopsies: a blinded clinical validation and deployment study. The Lancet Digital Health, 2020, vol. 2, iss. 8, pp. e407–e416.Pantanowitz L., Quiroga-Garza G., Bien L., Heled R., Laifenfeld D., …, Dhir R. An artificial intelligence algorithm for prostate cancer diagnosis in whole slide images of core needle biopsies: a blinded clinical validation and deployment study. The Lancet Digital Health, 2020, vol. 2, iss. 8, pp. e407–e416.Bulten W., Balkenhol M., Belinga J.-J. А., Brilhante A., Çakır A., …, Litjens G. Artificial intelligence assistance significantly improves Gleason grading of prostate biopsies by pathologists. Modern Pathology, 2020. Available at: https://arxiv.org/abs/2002.04500 (accessed 06.08.2020).Bulten W., Balkenhol M., Belinga J.-J. А., Brilhante A., Çakır A., …, Litjens G. Artificial intelligence assistance significantly improves Gleason grading of prostate biopsies by pathologists. Modern Pathology, 2020. Available at: https://arxiv.org/abs/2002.04500 (accessed 06.08.2020).Nagpal K., Foote D., Liu Y., Chen P.-H. C., Wulczyn E., …, Stumpe M. C. Development and validation of a deep learning algorithm for improving Gleason scoring of prostate cancer. Nature Partner Journal Digital Medicine, 2019, vol. 2, iss. 48, pp. 1–10.Nagpal K., Foote D., Liu Y., Chen P.-H. C., Wulczyn E., …, Stumpe M. C. Development and validation of a deep learning algorithm for improving Gleason scoring of prostate cancer. Nature Partner Journal Digital Medicine, 2019, vol. 2, iss. 48, pp. 1–10.Schuster C. A note on the interpretation of Weighted Kappa and its relations to other rater agreement statistics for metric scales. Educational and Psychological Measurement, April 2004, vol. 64, no. 2, pp. 243–253.Schuster C. A note on the interpretation of Weighted Kappa and its relations to other rater agreement statistics for metric scales. Educational and Psychological Measurement, April 2004, vol. 64, no. 2, pp. 243–253.Luan S., Chen C., Zhang B., Han J., Liu J. Gabor convolutional networks. IEEE Transactions on Image Processing, 2018, vol. 27, no. 9, pp. 4357–4366.Luan S., Chen C., Zhang B., Han J., Liu J. Gabor convolutional networks. IEEE Transactions on Image Processing, 2018, vol. 27, no. 9, pp. 4357–4366.Kovalev V., Volmer S. Color co-occurrence descriptors for querying-by-example. International Conference on Multimedia Modeling, Lausanne, Switzerland, 12–15 October 1998. Lausanne, 1998, pp. 32–38.Kovalev V., Volmer S. Color co-occurrence descriptors for querying-by-example. International Conference on Multimedia Modeling, Lausanne, Switzerland, 12–15 October 1998. Lausanne, 1998, pp. 32–38.Achanta R., Shaji A., Smith R., Lucchi A., Fua P., Susstrunk S. SLIC superpixels compared to state-ofthe-art superpixel methods. IEEE Transactions on PAMI, 2012, vol. 34, no. 11, pp. 2274–2282.Achanta R., Shaji A., Smith R., Lucchi A., Fua P., Susstrunk S. SLIC superpixels compared to state-ofthe-art superpixel methods. IEEE Transactions on PAMI, 2012, vol. 34, no. 11, pp. 2274–2282.Horn R. A., Johnson C. R. Matrix Analysis. Part 5. Norms for Vectors and Matrices. England, Cambridge University Press, 1990.Horn R. A., Johnson C. R. Matrix Analysis. Part 5. Norms for Vectors and Matrices. England, Cambridge University Press, 1990.Macenko M., Niethammer M., Marron J., Borland D., Woosley J. T., …, Thomas N. E. A method for normalizing histology slides for quantitative analysis. 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro, Boston, MA, USA, 28 June – 1 July 2009. Boston, 2009, pp. 1107–1110.Macenko M., Niethammer M., Marron J., Borland D., Woosley J. T., …, Thomas N. E. A method for normalizing histology slides for quantitative analysis. 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro, Boston, MA, USA, 28 June – 1 July 2009. Boston, 2009, pp. 1107–1110.Gulli A., Sujit P. Deep learning with Keras. Packt Publishing Ltd, 2017, 318 р.Gulli A., Sujit P. Deep learning with Keras. Packt Publishing Ltd, 2017, 318 р.Rubinstein R. Y., Kroese D. P. The Cross Entropy Method: a Unified Approach to Combinatorial Optimization, Monte-Carlo Simulation (Information Science and Statistics). Berlin, Heidelberg, Springer-Verlag. 2004, 321 р.Rubinstein R. Y., Kroese D. P. The Cross Entropy Method: a Unified Approach to Combinatorial Optimization, Monte-Carlo Simulation (Information Science and Statistics). Berlin, Heidelberg, Springer-Verlag. 2004, 321 р.

The authors declare that there are no conflicts of interest present.