Real-World Robot Learning with Masked Visual Pre-training

Autor:	Radosavovic, Ilija, Xiao, Tete, James, Stephen, Abbeel, Pieter, Malik, Jitendra, Darrell, Trevor
Rok vydání:	2022
Předmět:	Computer Science - Robotics Computer Science - Computer Vision and Pattern Recognition Computer Science - Machine Learning
Druh dokumentu:	Working Paper
Popis:	In this work, we explore self-supervised visual pre-training on images from diverse, in-the-wild videos for real-world robotic tasks. Like prior work, our visual representations are pre-trained via a masked autoencoder (MAE), frozen, and then passed into a learnable control module. Unlike prior work, we show that the pre-trained representations are effective across a range of real-world robotic tasks and embodiments. We find that our encoder consistently outperforms CLIP (up to 75%), supervised ImageNet pre-training (up to 81%), and training from scratch (up to 81%). Finally, we train a 307M parameter vision transformer on a massive collection of 4.5M images from the Internet and egocentric videos, and demonstrate clearly the benefits of scaling visual pre-training for robot learning. Comment: CoRL 2022; Project page: https://tetexiao.com/projects/real-mvp
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2210.03109 Zobrazit plný text záznamu View this record from Arxiv