Real-to-virtual domain transfer-based depth estimation for real-time 3D annotation in transnasal surgery: a study of annotation accuracy and stability

Autor: Jason Y. K. Chan, Zhiyu Liu, Ka-Wai Kwok, Hon-Sing Tong, Po-Ling Chan, Justin D. L. Ho, Yui-Lun Ng
Rok vydání: 2021
Předmět:
Computer science
Transnasal surgery
0206 medical engineering
Video Recording
Biomedical Engineering
Stability (learning theory)
Health Informatics
Augmented reality
Surgical annotation
02 engineering and technology
Imaging phantom
030218 nuclear medicine & medical imaging
Domain (software engineering)
03 medical and health sciences
Annotation
Imaging
Three-Dimensional
0302 clinical medicine
Monitoring
Intraoperative
Cadaver
Image Processing
Computer-Assisted
Humans
Radiology
Nuclear Medicine and imaging
Computer vision
Point (geometry)
Ground truth
Monocular
Phantoms
Imaging
business.industry
Reproducibility of Results
Endoscopy
General Medicine
020601 biomedical engineering
Computer Graphics and Computer-Aided Design
Computer Science Applications
Calibration
Original Article
Surgery
Domain transfer learning
Computer Vision and Pattern Recognition
Artificial intelligence
Tomography
X-Ray Computed
business
Monocular depth estimation
Zdroj: International Journal of Computer Assisted Radiology and Surgery
ISSN: 1861-6429
1861-6410
DOI: 10.1007/s11548-021-02346-9
Popis: PurposeSurgical annotation promotes effective communication between medical personnel during surgical procedures. However, existing approaches to 2D annotations are mostly static with respect to a display. In this work, we propose a method to achieve 3D annotations that anchor rigidly and stably to target structures upon camera movement in a transnasal endoscopic surgery setting.MethodsThis is accomplished through intra-operative endoscope tracking and monocular depth estimation. A virtual endoscopic environment is utilized to train a supervised depth estimation network. An adversarial network transfers the style from the real endoscopic view to a synthetic-like view for input into the depth estimation network, wherein framewise depth can be obtained in real time.Results(1) Accuracy: Framewise depth was predicted from images captured from within a nasal airway phantom and compared with ground truth, achieving a SSIM value of 0.8310 ± 0.0655. (2) Stability: mean absolute error (MAE) between reference and predicted depth of a target point was 1.1330 ± 0.9957 mm.ConclusionBoth the accuracy and stability evaluations demonstrated the feasibility and practicality of our proposed method for achieving 3D annotations.
Databáze: OpenAIRE
Externí odkaz: