Modelling environment for analyzing the algorithms for 3D reconstruction of videoendoscopic research objects

Abstract

Three-dimensional reconstruction based on the results of video endoscopic examination is a promising area for supporting medical diagnostics and treatment planning for a wide range of pathologies. Nevertheless, the assessment of the results of such reconstruction and verification of the correspondence of the obtained three-dimensional model to the original scene is significantly challenging. As a solution to this problem, the possibility of using a modelling environment to emulate the process of obtaining source video endoscopic data from the generated scene is suggested. The problem of three-dimensional modelling of the esophagus using the Autodesk 3ds Max environment and the Arnold visualization engine is considered. The paper describes the procedural generation of textures for the model and proposes the using Periodic Spatial Generative Adversarial Network models based on convolutional neural networks. To compare the result of  reconstruction with a scene, generated using the proposed modelling environment, an optimality criterion is introduced, by which the individual stages of the three-dimensional reconstruction algorithm are compared when the model is optimized using the bundle adjustment method.