Pose Estimation and Non-rigid Registration for Augmented Reality during Neurosurgery
| Autor: | Michael Nercessian, Alexandra J. Golby, Nazim Haouchine, Sarah F. Frisken, Sandy Wells, Parikshit Juvekar |
|---|---|
| Přispěvatelé: | Harvard Medical School [Boston] (HMS), Brigham & Women’s Hospital [Boston] (BWH), Computational Anatomy and Simulation for Medicine (MIMESIS), Inria Nancy - Grand Est, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Laboratoire des sciences de l'ingénieur, de l'informatique et de l'imagerie (ICube), École Nationale du Génie de l'Eau et de l'Environnement de Strasbourg (ENGEES)-Université de Strasbourg (UNISTRA)-Institut National des Sciences Appliquées - Strasbourg (INSA Strasbourg), Institut National des Sciences Appliquées (INSA)-Institut National des Sciences Appliquées (INSA)-Institut National de Recherche en Informatique et en Automatique (Inria)-Les Hôpitaux Universitaires de Strasbourg (HUS)-Centre National de la Recherche Scientifique (CNRS)-Matériaux et Nanosciences Grand-Est (MNGE), Université de Strasbourg (UNISTRA)-Université de Haute-Alsace (UHA) Mulhouse - Colmar (Université de Haute-Alsace (UHA))-Institut National de la Santé et de la Recherche Médicale (INSERM)-Institut de Chimie du CNRS (INC)-Centre National de la Recherche Scientifique (CNRS)-Université de Strasbourg (UNISTRA)-Université de Haute-Alsace (UHA) Mulhouse - Colmar (Université de Haute-Alsace (UHA))-Institut National de la Santé et de la Recherche Médicale (INSERM)-Institut de Chimie du CNRS (INC)-Centre National de la Recherche Scientifique (CNRS)-Réseau nanophotonique et optique, Université de Strasbourg (UNISTRA)-Université de Haute-Alsace (UHA) Mulhouse - Colmar (Université de Haute-Alsace (UHA))-Centre National de la Recherche Scientifique (CNRS)-Université de Strasbourg (UNISTRA)-Centre National de la Recherche Scientifique (CNRS)-École Nationale du Génie de l'Eau et de l'Environnement de Strasbourg (ENGEES)-Université de Strasbourg (UNISTRA)-Institut National des Sciences Appliquées - Strasbourg (INSA Strasbourg), Institut National des Sciences Appliquées (INSA)-Institut National des Sciences Appliquées (INSA)-Les Hôpitaux Universitaires de Strasbourg (HUS)-Centre National de la Recherche Scientifique (CNRS)-Matériaux et Nanosciences Grand-Est (MNGE), Université de Strasbourg (UNISTRA)-Université de Haute-Alsace (UHA) Mulhouse - Colmar (Université de Haute-Alsace (UHA))-Centre National de la Recherche Scientifique (CNRS)-Université de Strasbourg (UNISTRA)-Centre National de la Recherche Scientifique (CNRS), Cornell University [New York], Massachusetts Institute of Technology (MIT), This work was supported in part by the National Institute of Health under Grant R01 EB027134-01, R01 NS049251 and BWH Radiology Department Research Pilot Grant Award. |
| Jazyk: | angličtina |
| Rok vydání: | 2022 |
| Předmět: |
Registration
medicine.medical_treatment Biomedical Engineering Neurosurgery Article Compensation (engineering) Pose Estimation Imaging Three-Dimensional medicine Polygon mesh Computer vision [INFO]Computer Science [cs] Pose Craniotomy Retrospective Studies Augmented Reality business.industry Estimator Brain Magnetic Resonance Imaging Visualization Surgery Computer-Assisted Feature (computer vision) Image-guided Intervention Augmented reality Artificial intelligence business |
| Zdroj: | IEEE Transactions on Biomedical Engineering IEEE Transactions on Biomedical Engineering, 2022, 69 (4), pp.1310-1317. ⟨10.1109/TBME.2021.3113841⟩ IEEE Trans Biomed Eng |
| ISSN: | 0018-9294 |
| Popis: | International audience; Objective: A craniotomy is the removal of a part of the skull to allow surgeons to have access to the brain and treat tumors. When accessing the brain, a tissue deformation occurs and can negatively influence the surgical procedure outcome. In this work, we present a novel Augmented Reality neurosurgical system to superimpose pre-operative 3D meshes derived from MRI onto a view of the brain surface acquired during surgery. Methods: Our method uses cortical vessels as main features to drive a rigid then non-rigid 3D/2D registration. We first use a feature extractor network to produce probability maps that are fed to a pose estimator network to infer the 6-DoF rigid pose. Then, to account for brain deformation, we add a nonrigid refinement step formulated as a Shape-from-Template problem using physics-based constraints that helps propagate the deformation to sub-cortical level and update tumor location. Results: We tested our method retrospectively on 6 clinical datasets and obtained low pose error, and showed using synthetic dataset that considerable brain shift compensation and low TRE can be achieved at cortical and sub-cortical levels. Conclusion: The results show that our solution achieved accuracy below the actual clinical errors demonstrating the feasibility of practical use of our system. Significance: This work shows that we can provide coherent Augmented Reality visualization of 3D cortical vessels observed through the craniotomy using a single camera view and that cortical vessels provide strong features for performing both rigid and non-rigid registration. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|