Language-based colorization of scene sketches
| Autor: | Hongbo Fu, Chengying Gao, Haoran Mo, Changqing Zou, Ruofei Du |
|---|---|
| Rok vydání: | 2019 |
| Předmět: |
Thesaurus (information retrieval)
Generalization Interface (Java) Human–computer interaction Computer science ComputingMethodologies_IMAGEPROCESSINGANDCOMPUTERVISION 0202 electrical engineering electronic engineering information engineering 020207 software engineering 02 engineering and technology Image segmentation Object (computer science) Computer Graphics and Computer-Aided Design Sketch |
| Zdroj: | ACM Transactions on Graphics. 38:1-16 |
| ISSN: | 1557-7368 0730-0301 |
| DOI: | 10.1145/3355089.3356561 |
| Popis: | Being natural, touchless, and fun-embracing, language-based inputs have been demonstrated effective for various tasks from image generation to literacy education for children. This paper for the first time presents a language-based system for interactive colorization of scene sketches, based on semantic comprehension. The proposed system is built upon deep neural networks trained on a large-scale repository of scene sketches and cartoonstyle color images with text descriptions. Given a scene sketch, our system allows users, via language-based instructions, to interactively localize and colorize specific foreground object instances to meet various colorization requirements in a progressive way. We demonstrate the effectiveness of our approach via comprehensive experimental results including alternative studies, comparison with the state-of-the-art methods, and generalization user studies. Given the unique characteristics of language-based inputs, we envision a combination of our interface with a traditional scribble-based interface for a practical multimodal colorization system, benefiting various applications. The dataset and source code can be found at https://github.com/SketchyScene/SketchySceneColorization. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|