Segmentation-Renormalized Deep Feature Modulation for Unpaired Image Harmonization

Autor: Mengwei Ren, James Fishbaugh, Neel Dey, Guido Gerig
Jazyk: angličtina
Rok vydání: 2021
Předmět:
Normalization (statistics)
FOS: Computer and information sciences
Computer science
Computer Vision and Pattern Recognition (cs.CV)
Computer Science - Computer Vision and Pattern Recognition
Normalization (image processing)
ComputingMethodologies_IMAGEPROCESSINGANDCOMPUTERVISION
Translation (geometry)
Article
030218 nuclear medicine & medical imaging
03 medical and health sciences
0302 clinical medicine
Medical imaging
FOS: Electrical engineering
electronic engineering
information engineering
Image Processing
Computer-Assisted
Segmentation
Electrical and Electronic Engineering
Radiological and Ultrasound Technology
business.industry
Deep learning
Image and Video Processing (eess.IV)
Pattern recognition
Electrical Engineering and Systems Science - Image and Video Processing
Magnetic Resonance Imaging
Computer Science Applications
Feature (computer vision)
Image translation
Artificial intelligence
Affine transformation
business
Software
Zdroj: IEEE Trans Med Imaging
Popis: Deep networks are now ubiquitous in large-scale multi-center imaging studies. However, the direct aggregation of images across sites is contraindicated for downstream statistical and deep learning-based image analysis due to inconsistent contrast, resolution, and noise. To this end, in the absence of paired data, variations of Cycle-consistent Generative Adversarial Networks have been used to harmonize image sets between a source and target domain. Importantly, these methods are prone to instability, contrast inversion, intractable manipulation of pathology, and steganographic mappings which limit their reliable adoption in real-world medical imaging. In this work, based on an underlying assumption that morphological shape is consistent across imaging sites, we propose a segmentation-renormalized image translation framework to reduce inter-scanner heterogeneity while preserving anatomical layout. We replace the affine transformations used in the normalization layers within generative networks with trainable scale and shift parameters conditioned on jointly learned anatomical segmentation embeddings to modulate features at every level of translation. We evaluate our methodologies against recent baselines across several imaging modalities (T1w MRI, FLAIR MRI, and OCT) on datasets with and without lesions. Segmentation-renormalization for translation GANs yields superior image harmonization as quantified by Inception distances, demonstrates improved downstream utility via post-hoc segmentation accuracy, and improved robustness to translation perturbation and self-adversarial attacks.
Comment: Accepted by IEEE Transactions on Medical Imaging. Code available at https://github.com/mengweiren/segmentation-renormalized-harmonization
Databáze: OpenAIRE
Externí odkaz: