Modelling environment for analyzing the algorithms for 3D reconstruction of videoendoscopic research objects
https://doi.org/10.37661/1816-0301-2020-17-1-18-28
Abstract
Three-dimensional reconstruction based on the results of video endoscopic examination is a promising area for supporting medical diagnostics and treatment planning for a wide range of pathologies. Nevertheless, the assessment of the results of such reconstruction and verification of the correspondence of the obtained three-dimensional model to the original scene is significantly challenging. As a solution to this problem, the possibility of using a modelling environment to emulate the process of obtaining source video endoscopic data from the generated scene is suggested. The problem of three-dimensional modelling of the esophagus using the Autodesk 3ds Max environment and the Arnold visualization engine is considered. The paper describes the procedural generation of textures for the model and proposes the using Periodic Spatial Generative Adversarial Network models based on convolutional neural networks. To compare the result of reconstruction with a scene, generated using the proposed modelling environment, an optimality criterion is introduced, by which the individual stages of the three-dimensional reconstruction algorithm are compared when the model is optimized using the bundle adjustment method.
About the Authors
A. F. Chernyavsky
Belarusian State University
Belarus
Aleksandr F. Chernyavsky, Academician of the National Academy of Sciences of Belarus, Dr. Sci. (Eng.), Professor, Head of the Research Laboratory of Specialized Computing Systems, A. N. Sevchenko Institute of Applied Physical Problems
Belarus
Aleksandr F. Chernyavsky, Academician of the National Academy of Sciences of Belarus, Dr. Sci. (Eng.), Professor, Head of the Research Laboratory of Specialized Computing Systems, A. N. Sevchenko Institute of Applied Physical Problems
K. A. Halavataya
Belarusian State University
Belarus
Katsiaryna A. Halavataya, Postgraduate Student, Senior Lecturer, Department of Intelligent Systems, Faculty of Radiophysics and Computer Technologies
Belarus
Katsiaryna A. Halavataya, Postgraduate Student, Senior Lecturer, Department of Intelligent Systems, Faculty of Radiophysics and Computer Technologies
V. S. Sadau
Belarusian State University
Belarus
Vasili S. Sadau, Cand. Sci. (Eng.), Associate Professor, Professor of the Department of Intelligent Systems, Faculty of Radiophysics and Computer Technologies
Belarus
Vasili S. Sadau, Cand. Sci. (Eng.), Associate Professor, Professor of the Department of Intelligent Systems, Faculty of Radiophysics and Computer Technologies
References
1. Shinomiya A., Shindo A., Kawanishi M., Miyake K., Nakamura T., ..., Tamiya T. Usefulness of the 3D virtual visualization surgical planning simulation and 3D model for endoscopic endonasal transsphenoidal surgery of pituitary adenoma: technical report and review of literature. Interdisciplinary Neurosurgery, 2018, vol. 13, pp. 13–19. https://doi.org/10.1016/j.inat.2018.02.002
2. Gatys L. A., Ecker A. S., Bethge M. Texture synthesis using convolutional neural networks. Advances in Neural Information Processing Systems, 2015, vol. 28, pp. 262–270.
3. Mamgain P. Autodesk 3ds Max 2019: a Detailed Guide to Arnold Renderer. Padexi Academic, 2018, 192 p.
4. Simonyan K., Zisserman A. Very deep convolutional networks for large-scale image recognition. International Conference on Learning Representations 2014 (ICLR 2014), Banff, Canada, 14–16 April 2014. Banff, 2014, рр. 1–14.
5. Bergmann U., Jetchev N., Vollgraf R. Learning texture manifolds with the periodic spatial GAN. Proceedings of the 34th International Conference on Machine Learning, Sydney, Australia, 6–11 August 2017. Sydney, 2017, vol. 70, рр. 469–477.
6. Agarwal S., Snavely N., Seitz S. M., Szelisk R. Bundle adjustment in the large. Proceedings of the 11th European Conference on Computer Vision (ECCV 2010), Heraklion, Greece, 5–11 September 2010. Heraklion, 2010, рр. 29–42.
7. Halavataya K. А., Sadau V. S. Model formirovaniya izobrazheniy dlya trekhmernoy rekonstrukcii scen po dannym videoendoscopicheskyh issledovanii [Image formation model for three-dimensional reconstruction of scenes according to video endoscopic studies]. Vestnik Polotskogo gosudarstvennogo universiteta. Ser. S. Fundamental'nyye nauki [Vestnik of Polotsk State University. Ser. C. Basic Sciences], 2019, no. 12, pp. 43–49 (in Russian).
8. Lowe, D. G. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 2004, vol. 60, no. 2, pp. 91–110.
9. Rosten E., Drummond T. Machine learning for high-speed corner detection. Proceedings of the 9th European Conference on Computer Vision (ECCV 2006), Graz, Austria, 7–13 May 2006. Graz, 2006, рр. 430–443.
10. Calonder M., Lepetit V., Strecha Ch., Fua P. BRIEF: binary robust independent elementary features. Proceedings of the 11th European Conference on Computer Vision (ECCV 2010), Heraklion, Greece, 5–11 September 2010. Heraklion, 2010, рр. 778–792.
11. Rublee E., Rabaud V., Konolige K., Bradski G. ORB: an efficient alternative to SIFT or SURF. IEEE International Conference on Computer Vision (ICCV 2011), Barcelona, 6–13 November 2011. Barcelona, 2011, рр. 2564–2571.
12. Halavataya K., Sadov V. Optimizing local feature description and matching for realtime video sequence object detection. Open Semantic Technologies for Intelligent Systems. Minsk, 2019, pp. 269–272.
Review
For citations:
Chernyavsky A.F., Halavataya K.A., Sadau V.S. Modelling environment for analyzing the algorithms for 3D reconstruction of videoendoscopic research objects. Informatics. 2020;17(1):18-28. (In Russ.) https://doi.org/10.37661/1816-0301-2020-17-1-18-28
Views: 864
Email this article 