PreCNet: Next-Frame Video Prediction Based on Predictive Coding

Autor:	Straka, Zdenek, Svoboda, Tomas, Hoffmann, Matej
Rok vydání:	2020
Předmět:	Computer Science - Computer Vision and Pattern Recognition Computer Science - Machine Learning Computer Science - Neural and Evolutionary Computing
Druh dokumentu:	Working Paper
DOI:	10.1109/TNNLS.2023.3240857
Popis:	Predictive coding, currently a highly influential theory in neuroscience, has not been widely adopted in machine learning yet. In this work, we transform the seminal model of Rao and Ballard (1999) into a modern deep learning framework while remaining maximally faithful to the original schema. The resulting network we propose (PreCNet) is tested on a widely used next frame video prediction benchmark, which consists of images from an urban environment recorded from a car-mounted camera, and achieves state-of-the-art performance. Performance on all measures (MSE, PSNR, SSIM) was further improved when a larger training set (2M images from BDD100k), pointing to the limitations of the KITTI training set. This work demonstrates that an architecture carefully based in a neuroscience model, without being explicitly tailored to the task at hand, can exhibit exceptional performance. Comment: Accepted by IEEE Transactions on Neural Networks and Learning Systems (TNNLS)
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2004.14878 Zobrazit plný text záznamu View this record from Arxiv