Towards Human in the Loop Analysis of Complex Point Clouds: Advanced Visualizations, Quantifications, and Communication Features in Virtual Reality.

Autor:	Blanc T; Laboratoire Physico-Chimie, Institut Curie, PSL Research University, CNRS UMR168, Paris, France.; Sorbonne Universités, UPMC Univ Paris 06, Paris, France., Verdier H; Decision and Bayesian Computation, CNRS USR 3756, Department of Computational Biology and Neuroscience, CNRS UMR 3571, Université de Paris, Institut Pasteur, Université de Paris, Paris, France., Regnier L; Laboratoire Physico-Chimie, Institut Curie, PSL Research University, CNRS UMR168, Paris, France.; Sorbonne Universités, UPMC Univ Paris 06, Paris, France., Planchon G; Decision and Bayesian Computation, CNRS USR 3756, Department of Computational Biology and Neuroscience, CNRS UMR 3571, Université de Paris, Institut Pasteur, Université de Paris, Paris, France., Guérinot C; Decision and Bayesian Computation, CNRS USR 3756, Department of Computational Biology and Neuroscience, CNRS UMR 3571, Université de Paris, Institut Pasteur, Université de Paris, Paris, France.; Sorbonne Universités, Collège Doctoral, Paris, France., El Beheiry M; Decision and Bayesian Computation, CNRS USR 3756, Department of Computational Biology and Neuroscience, CNRS UMR 3571, Université de Paris, Institut Pasteur, Université de Paris, Paris, France., Masson JB; Decision and Bayesian Computation, CNRS USR 3756, Department of Computational Biology and Neuroscience, CNRS UMR 3571, Université de Paris, Institut Pasteur, Université de Paris, Paris, France., Hajj B; Laboratoire Physico-Chimie, Institut Curie, PSL Research University, CNRS UMR168, Paris, France.; Sorbonne Universités, UPMC Univ Paris 06, Paris, France.
Jazyk:	angličtina
Zdroj:	Frontiers in bioinformatics [Front Bioinform] 2022 Jan 20; Vol. 1, pp. 775379. Date of Electronic Publication: 2022 Jan 20 (Print Publication: 2021).
DOI:	10.3389/fbinf.2021.775379
Abstrakt:	Multiple fields in biological and medical research produce large amounts of point cloud data with high dimensionality and complexity. In addition, a large set of experiments generate point clouds, including segmented medical data or single-molecule localization microscopy. In the latter, individual molecules are observed within their natural cellular environment. Analyzing this type of experimental data is a complex task and presents unique challenges, where providing extra physical dimensions for visualization and analysis could be beneficial. Furthermore, whether highly noisy data comes from single-molecule recordings or segmented medical data, the necessity to guide analysis with user intervention creates both an ergonomic challenge to facilitate this interaction and a computational challenge to provide fluid interactions as information is being processed. Several applications, including our software DIVA for image stack and our platform Genuage for point clouds, have leveraged Virtual Reality (VR) to visualize and interact with data in 3D. While the visualization aspects can be made compatible with different types of data, quantifications, on the other hand, are far from being standard. In addition, complex analysis can require significant computational resources, making the real-time VR experience uncomfortable. Moreover, visualization software is mainly designed to represent a set of data points but lacks flexibility in manipulating and analyzing the data. This paper introduces new libraries to enhance the interaction and human-in-the-loop analysis of point cloud data in virtual reality and integrate them into the open-source platform Genuage. We first detail a new toolbox of communication tools that enhance user experience and improve flexibility. Then, we introduce a mapping toolbox allowing the representation of physical properties in space overlaid on a 3D mesh while maintaining a point cloud dedicated shader. We introduce later a new and programmable video capture tool in VR and desktop modes for intuitive data dissemination. Finally, we highlight the protocols that allow simultaneous analysis and fluid manipulation of data with a high refresh rate. We illustrate this principle by performing real-time inference of random walk properties of recorded trajectories with a pre-trained Graph Neural Network running in Python. Competing Interests: MEB and J-BM are cofounders, shareholders and, respectively, Chief Technology Officer (CTO) and Chief Scientific Officer (CSO) of AVATAR MEDICAL SAS, a startup that commercialises software for surgery planning in virtual reality. AVATAR MEDICAL had no role in study design, data collection and analysis, funding, decision to publish, or preparation of the manuscript. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest. (Copyright © 2022 Blanc, Verdier, Regnier, Planchon, Guérinot, El Beheiry, Masson and Hajj.)
Databáze:	MEDLINE
Externí odkaz:	Zobrazit plný text záznamu