Domain Stylization: A Fast Covariance Matching Framework Towards Domain Adaptation
| Autor: | John Zedlewski, Jan Kautz, Ting-Chun Wang, Ming-Yu Liu, Zhiding Yu, Aysegul Dundar |
|---|---|
| Rok vydání: | 2021 |
| Předmět: |
Computer science
business.industry Applied Mathematics 02 engineering and technology Image segmentation Covariance Machine learning computer.software_genre Synthetic data Rendering (computer graphics) Data modeling Computer graphics Computational Theory and Mathematics Effective domain Artificial Intelligence 0202 electrical engineering electronic engineering information engineering 020201 artificial intelligence & image processing Computer Vision and Pattern Recognition Artificial intelligence business Conditional variance computer Software |
| Zdroj: | IEEE Transactions on Pattern Analysis and Machine Intelligence. 43:2360-2372 |
| ISSN: | 1939-3539 0162-8828 |
| Popis: | Generating computer graphics (CG) rendered synthetic images has been widely used to create simulation environments for robotics/autonomous driving and generate labeled data. Yet, the problem of training models purely with synthetic data remains challenging due to the considerable domain gaps caused by current limitations on rendering. In this paper, we propose a simple yet effective domain adaptation framework towards closing such gap at image level. Unlike many GAN-based approaches, our method aims to match the covariance of the universal feature embeddings across domains, making the adaptation a fast, convenient step and avoiding the need for potentially difficult GAN training. To align domains more precisely, we further propose a conditional covariance matching framework which iteratively estimates semantic segmentation regions and conditionally matches the class-wise feature covariance given the segmentation regions. We demonstrate that both tasks can mutually refine and considerably improve each other, leading to state-of-the-art domain adaptation results. Extensive experiments under multiple synthetic-to-real settings show that our approach exceeds the performance of latest domain adaptation approaches. In addition, we offer a quantitative analysis where our framework shows considerable reduction in Frechet Inception distance between source and target domains, demonstrating the effectiveness of this work in bridging the synthetic-to-real domain gap. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|