Simulating realistic fetal neurosonography images with appearance and growth change using cycle-consistent adversarial networks and an evaluation
| Autor: | Lior Drukker, Lok Hin Lee, Aris T. Papageorghiou, A. Noble, Yangdi Xu, Mohammad Yaqub |
|---|---|
| Rok vydání: | 2020 |
| Předmět: |
Unpaired Data
Adversarial network business.industry Deep learning Visibility (geometry) 030218 nuclear medicine & medical imaging Visualization 03 medical and health sciences 0302 clinical medicine Second trimester 030220 oncology & carcinogenesis Medical imaging Medicine Radiology Nuclear Medicine and imaging Computer vision Pairwise comparison Artificial intelligence Ultrasonic Imaging and Tomography business |
| Zdroj: | J Med Imaging (Bellingham) |
| ISSN: | 2329-4302 |
| DOI: | 10.1117/1.jmi.7.5.057001 |
| Popis: | Purpose: We present an original method for simulating realistic fetal neurosonography images specifically generating third-trimester pregnancy ultrasound images from second-trimester images. Our method was developed using unpaired data, as pairwise data were not available. We also report original insights on the general appearance differences between second- and third-trimester fetal head transventricular (TV) plane images. Approach: We design a cycle-consistent adversarial network (Cycle-GAN) to simulate visually realistic third-trimester images from unpaired second- and third-trimester ultrasound images. Simulation realism is evaluated qualitatively by experienced sonographers who blindly graded real and simulated images. A quantitative evaluation is also performed whereby a validated deep-learning-based image recognition algorithm (ScanNav®) acts as the expert reference to allow hundreds of real and simulated images to be automatically analyzed and compared efficiently. Results: Qualitative evaluation shows that the human expert cannot tell the difference between real and simulated third-trimester scan images. 84.2% of the simulated third-trimester images could not be distinguished from the real third-trimester images. As a quantitative baseline, on 3000 images, the visibility drop of the choroid, CSP, and mid-line falx between real second- and real third-trimester scans was computed by ScanNav® and found to be 72.5%, 61.5%, and 67%, respectively. The visibility drop of the same structures between real second-trimester and simulated third-trimester was found to be 77.5%, 57.7%, and 56.2%, respectively. Therefore, the real and simulated third-trimester images were consider to be visually similar to each other. Our evaluation also shows that the third-trimester simulation of a conventional GAN is much easier to distinguish, and the visibility drop of the structures is smaller than our proposed method. Conclusions: The results confirm that it is possible to simulate realistic third-trimester images from second-trimester images using a modified Cycle-GAN, which may be useful for deep learning researchers with a restricted availability of third-trimester scans but with access to ample second trimester images. We also show convincing simulation improvements, both qualitatively and quantitatively, using the Cycle-GAN method compared with a conventional GAN. Finally, the use of a machine learning-based reference (in the case ScanNav®) for large-scale quantitative image analysis evaluation is also a first to our knowledge. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|