TensorDIMM: A Practical Near-Memory Processing Architecture for Embeddings and Tensor Operations in Deep Learning

Autor:	Kwon, Youngeun, Lee, Yunjae, Rhu, Minsoo
Rok vydání:	2019
Předmět:	Computer Science - Machine Learning Computer Science - Hardware Architecture Computer Science - Distributed Parallel and Cluster Computing Computer Science - Neural and Evolutionary Computing
Druh dokumentu:	Working Paper
Popis:	Recent studies from several hyperscalars pinpoint to embedding layers as the most memory-intensive deep learning (DL) algorithm being deployed in today's datacenters. This paper addresses the memory capacity and bandwidth challenges of embedding layers and the associated tensor operations. We present our vertically integrated hardware/software co-design, which includes a custom DIMM module enhanced with near-data processing cores tailored for DL tensor operations. These custom DIMMs are populated inside a GPU-centric system interconnect as a remote memory pool, allowing GPUs to utilize for scalable memory bandwidth and capacity expansion. A prototype implementation of our proposal on real DL systems shows an average 6.2-17.6x performance improvement on state-of-the-art recommender systems. Comment: Accepted for publication at the 52nd IEEE/ACM International Symposium on Microarchitecture (MICRO-52), 2019
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/1908.03072 Zobrazit plný text záznamu View this record from Arxiv