PQA: Perceptual Question Answering
| Autor: | Aneeshan Sain, Yonggang Qi, Kai Zhang, Yi-Zhe Song |
|---|---|
| Jazyk: | angličtina |
| Rok vydání: | 2021 |
| Předmět: |
FOS: Computer and information sciences
Cognitive science Computer science business.industry Computer Vision and Pattern Recognition (cs.CV) media_common.quotation_subject Computer Science - Computer Vision and Pattern Recognition Visualization Data modeling Knowledge extraction Perception Human visual system model Question answering Selection (linguistics) Gestalt psychology Artificial intelligence business media_common |
| Zdroj: | CVPR |
| Popis: | Perceptual organization remains one of the very few established theories on the human visual system. It underpinned many pre-deep seminal works on segmentation and detection, yet research has seen a rapid decline since the preferential shift to learning deep models. Of the limited attempts, most aimed at interpreting complex visual scenes using perceptual organizational rules. This has however been proven to be sub-optimal, since models were unable to effectively capture the visual complexity in real-world imagery. In this paper, we rejuvenate the study of perceptual organization, by advocating two positional changes: (i) we examine purposefully generated synthetic data, instead of complex real imagery, and (ii) we ask machines to synthesize novel perceptually-valid patterns, instead of explaining existing data. Our overall answer lies with the introduction of a novel visual challenge -- the challenge of perceptual question answering (PQA). Upon observing example perceptual question-answer pairs, the goal for PQA is to solve similar questions by generating answers entirely from scratch (see Figure 1). Our first contribution is therefore the first dataset of perceptual question-answer pairs, each generated specifically for a particular Gestalt principle. We then borrow insights from human psychology to design an agent that casts perceptual organization as a self-attention problem, where a proposed grid-to-grid mapping network directly generates answer patterns from scratch. Experiments show our agent to outperform a selection of naive and strong baselines. A human study however indicates that ours uses astronomically more data to learn when compared to an average human, necessitating future research (with or without our dataset). Accepted by CVPR 2021 |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|