Memory Efficient Experience Replay for Streaming Learning
| Autor: | Tyler L. Hayes, Christopher Kanan, Nathan D. Cahill |
|---|---|
| Rok vydání: | 2018 |
| Předmět: |
Independent and identically distributed random variables
FOS: Computer and information sciences Computer Science - Machine Learning Forgetting Artificial neural network Computer Science - Artificial Intelligence Computer science Data stream mining business.industry Machine Learning (stat.ML) 02 engineering and technology Machine learning computer.software_genre Variety (cybernetics) Machine Learning (cs.LG) Artificial Intelligence (cs.AI) Statistics - Machine Learning 020204 information systems 0202 electrical engineering electronic engineering information engineering Robot 020201 artificial intelligence & image processing Artificial intelligence Cluster analysis business computer |
| Zdroj: | ICRA |
| DOI: | 10.48550/arxiv.1809.05922 |
| Popis: | In supervised machine learning, an agent is typically trained once and then deployed. While this works well for static settings, robots often operate in changing environments and must quickly learn new things from data streams. In this paradigm, known as streaming learning, a learner is trained online, in a single pass, from a data stream that cannot be assumed to be independent and identically distributed (iid). Streaming learning will cause conventional deep neural networks (DNNs) to fail for two reasons: 1) they need multiple passes through the entire dataset; and 2) non-iid data will cause catastrophic forgetting. An old fix to both of these issues is rehearsal. To learn a new example, rehearsal mixes it with previous examples, and then this mixture is used to update the DNN. Full rehearsal is slow and memory intensive because it stores all previously observed examples, and its effectiveness for preventing catastrophic forgetting has not been studied in modern DNNs. Here, we describe the ExStream algorithm for memory efficient rehearsal and compare it to alternatives. We find that full rehearsal can eliminate catastrophic forgetting in a variety of streaming learning settings, with ExStream performing well using far less memory and computation. Comment: To appear in the IEEE International Conference on Robotics and Automation (ICRA) 2019 |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|