Convolutional neural network advances in demosaicing for fluorescent cancer imaging with color-near-infrared sensors.

Autor:	Jin Y; University of Illinois at Urbana-Champaign, Department of Electrical and Computer Engineering, Urbana, Illinois, United States., Kondov B; Ss. Cyril and Methodius University of Skopje, Department of Thoracic and Vascular Surgery, Skopje, North Macedonia., Kondov G; Ss. Cyril and Methodius University of Skopje, Department of Thoracic and Vascular Surgery, Skopje, North Macedonia., Singhal S; University of Pennsylvania, Perelman School of Medicine, Department of Thoracic Surgery, Philadelphia, Pennsylvania, United States., Nie S; University of Illinois at Urbana-Champaign, Department of Electrical and Computer Engineering, Urbana, Illinois, United States.; University of Illinois at Urbana-Champaign, Department of Bioengineering, Urbana, Illinois, United States.; University of Illinois at Urbana-Champaign, Beckman Institute for Advanced Science and Technology, Urbana, Illinois, United States., Gruev V; University of Illinois at Urbana-Champaign, Department of Electrical and Computer Engineering, Urbana, Illinois, United States.; University of Illinois at Urbana-Champaign, Department of Bioengineering, Urbana, Illinois, United States.; University of Illinois at Urbana-Champaign, Beckman Institute for Advanced Science and Technology, Urbana, Illinois, United States.; University of Illinois at Urbana-Champaign, Carle Illinois College of Medicine, Urbana, Illinois, United States.
Jazyk:	angličtina
Zdroj:	Journal of biomedical optics [J Biomed Opt] 2024 Jul; Vol. 29 (7), pp. 076005. Date of Electronic Publication: 2024 Jul 23.
DOI:	10.1117/1.JBO.29.7.076005
Abstrakt:	Significance: Single-chip imaging devices featuring vertically stacked photodiodes and pixelated spectral filters are advancing multi-dye imaging methods for cancer surgeries, though this innovation comes with a compromise in spatial resolution. To mitigate this drawback, we developed a deep convolutional neural network (CNN) aimed at demosaicing the color and near-infrared (NIR) channels, with its performance validated on both pre-clinical and clinical datasets. Aim: We introduce an optimized deep CNN designed for demosaicing both color and NIR images obtained using a hexachromatic imaging sensor. Approach: A residual CNN was fine-tuned and trained on a dataset of color images and subsequently assessed on a series of dual-channel, color, and NIR images to demonstrate its enhanced performance compared with traditional bilinear interpolation. Results: Our optimized CNN for demosaicing color and NIR images achieves a reduction in the mean square error by 37% for color and 40% for NIR, respectively, and enhances the structural dissimilarity index by 37% across both imaging modalities in pre-clinical data. In clinical datasets, the network improves the mean square error by 35% in color images and 42% in NIR images while enhancing the structural dissimilarity index by 39% in both imaging modalities. Conclusions: We showcase enhancements in image resolution for both color and NIR modalities through the use of an optimized CNN tailored for a hexachromatic image sensor. With the ongoing advancements in graphics card computational power, our approach delivers significant improvements in resolution that are feasible for real-time execution in surgical environments. (© 2024 The Authors.)
Databáze:	MEDLINE
Externí odkaz:	Zobrazit plný text záznamu