Deep video-to-video transformations for accessibility with an application to photosensitivity
| Autor: | Andrei Barbu, Boris Katz, Dalitso Banda |
|---|---|
| Rok vydání: | 2020 |
| Předmět: |
Computer science
media_common.quotation_subject 02 engineering and technology 01 natural sciences Rendering (computer graphics) Artificial Intelligence Human–computer interaction Perception 0103 physical sciences Signal Processing 0202 electrical engineering electronic engineering information engineering 020201 artificial intelligence & image processing Computer Vision and Pattern Recognition 010306 general physics Software media_common |
| Zdroj: | Pattern Recognition Letters. 137:99-107 |
| ISSN: | 0167-8655 |
| DOI: | 10.1016/j.patrec.2019.01.019 |
| Popis: | We demonstrate how to construct a new class of visual assistive technologies that, rather than extract symbolic information, learn to transform the visual environment to make it more accessible. We do so without engineering which transformations are useful allowing for arbitrary modifications of the visual input. As an instantiation of this idea we tackle a problem that affects and hurts millions worldwide: photosensitivity. Any time an affected person opens a website, video, or some other medium that contains an adverse visual stimulus, either intended or unintended, they might experience a seizure with potentially significant consequences. We show how a deep network can learn a video-to-video transformation rendering such stimuli harmless while otherwise preserving the video. This approach uses a specification of the adverse phenomena, the forward transformation, to learn the inverse transformation. We show how such a network generalizes to real-world videos that have triggered numerous seizures, both by mistake and in politically-motivated attacks. A number of complimentary approaches are demonstrated including using a hand-crafted generator and a GAN using a differentiable perceptual metric. Such technology can be deployed offline to protect videos before they are shown or online with assistive glasses or real-time post processing. Other applications of this general technique include helping those with limited vision, attention deficit hyperactivity disorder, and autism. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|
Full Text from ScienceDirect