Parsing and Summarizing Infographics with Synthetically Trained Icon Detection
| Autor: | Adrià Recasens, Frédo Durand, Sami Alsheikh, Hanspeter Pfister, Matthew Tancik, Aude Oliva, Carolina Nobre, Zoya Bylinskii, Kimberli Zhong, Spandan Madan |
|---|---|
| Rok vydání: | 2021 |
| Předmět: |
Closed captioning
Information retrieval Parsing Computer science Infographic 020207 software engineering 02 engineering and technology computer.software_genre Automatic summarization Object detection Visualization 0202 electrical engineering electronic engineering information engineering 020201 artificial intelligence & image processing Icon Classifier (UML) computer computer.programming_language |
| Zdroj: | PacificVis |
| DOI: | 10.1109/pacificvis52677.2021.00012 |
| Popis: | Widely used in news, business, and educational media, infographics are handcrafted to effectively communicate messages about complex and often abstract topics including ‘ways to conserve the environment’ and ‘coronavirus prevention’. The computational understanding of infographics required for future applications like automatic captioning, summarization, search, and question-answering, will depend on being able to parse the visual and textual elements contained within. However, being composed of stylistically and semantically diverse visual and textual elements, infographics pose challenges for current A.I. systems. While automatic text extraction works reasonably well on infographics, standard object detection algorithms fail to identify the stand-alone visual elements in infographics that we refer to as ‘icons’. In this paper, we propose a novel approach to train an object detector using synthetically-generated data, and show that it succeeds at generalizing to detecting icons within in-the-wild infographics. We further pair our icon detection approach with an icon classifier and a state-of-the-art text detector to demonstrate three demo applications: topic prediction, multi-modal summarization, and multi-modal search. Parsing the visual and textual elements within infographics provides us with the first steps towards automatic infographic understanding. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|