AW3M: An auto-weighting and recovery framework for breast cancer diagnosis using multi-modal ultrasound
| Autor: | Ruobing Huang, Haoran Dou, YiJie Dong, Xin Yang, Jian Wang, Guangquan Zhou, Wenwen Xu, Zehui Lin, Juzheng Miao, Jianqiao Zhou, Xiaohong Jia, Dong Ni, Zihan Mei |
|---|---|
| Rok vydání: | 2020 |
| Předmět: |
Computer science
Health Informatics CAD Breast Neoplasms Machine learning computer.software_genre 030218 nuclear medicine & medical imaging 03 medical and health sciences 0302 clinical medicine medicine Reinforcement learning Humans Radiology Nuclear Medicine and imaging Diagnosis Computer-Assisted Block (data storage) Ultrasonography Modality (human–computer interaction) Radiological and Ultrasound Technology medicine.diagnostic_test business.industry Missing data Computer Graphics and Computer-Aided Design Weighting Identification (information) Elasticity Imaging Techniques Female Computer Vision and Pattern Recognition Elastography Artificial intelligence Ultrasonography Mammary business computer 030217 neurology & neurosurgery |
| Zdroj: | Medical image analysis. 72 |
| ISSN: | 1361-8423 |
| Popis: | Recently, more clinicians have realized the diagnostic value of multi-modal ultrasound in breast cancer identification and began to incorporate Doppler imaging and Elastography in the routine examination. However, accurately recognizing patterns of malignancy in different types of sonography requires expertise. Furthermore, an accurate and robust diagnosis requires proper weights of multi-modal information as well as the ability to process missing data in practice. These two aspects are often overlooked by existing computer-aided diagnosis (CAD) approaches. To overcome these challenges, we propose a novel framework (called AW3M) that utilizes four types of sonography (i.e. B-mode, Doppler, Shear-wave Elastography, and Strain Elastography) jointly to assist breast cancer diagnosis. It can extract both modality-specific and modality-invariant features using a multi-stream CNN model equipped with self-supervised consistency loss. Instead of assigning the weights of different streams empirically, AW3M automatically learns the optimal weights using reinforcement learning techniques. Furthermore, we design a light-weight recovery block that can be inserted to a trained model to handle different modality-missing scenarios. Experimental results on a large multi-modal dataset demonstrate that our method can achieve promising performance compared with state-of-the-art methods. The AW3M framework is also tested on another independent B-mode dataset to prove its efficacy in general settings. Results show that the proposed recovery block can learn from the joint distribution of multi-modal features to further boost the classification accuracy given single modality input during the test. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|
Full Text from ScienceDirect