Precision-aware Latency and Energy Balancing on Multi-Accelerator Platforms for DNN Inference

Autor:	Risso, Matteo, Burrello, Alessio, Sarda, Giuseppe Maria, Benini, Luca, Macii, Enrico, Poncino, Massimo, Verhelst, Marian, Pagliari, Daniele Jahier
Rok vydání:	2023
Předmět:	Computer Science - Machine Learning
Druh dokumentu:	Working Paper
Popis:	The need to execute Deep Neural Networks (DNNs) at low latency and low power at the edge has spurred the development of new heterogeneous Systems-on-Chips (SoCs) encapsulating a diverse set of hardware accelerators. How to optimally map a DNN onto such multi-accelerator systems is an open problem. We propose ODiMO, a hardware-aware tool that performs a fine-grain mapping across different accelerators on-chip, splitting individual layers and executing them in parallel, to reduce inference energy consumption or latency, while taking into account each accelerator's quantization precision to maintain accuracy. Pareto-optimal networks in the accuracy vs. energy or latency space are pursued for three popular dataset/DNN pairs, and deployed on the DIANA heterogeneous ultra-low power edge AI SoC. We show that ODiMO reduces energy/latency by up to 33%/31% with limited accuracy drop (-0.53%/-0.32%) compared to manual heuristic mappings. Comment: Accepted at 2023 ACM/IEEE International Symposium on Low Power Electronics and Design (ISLPED)
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2306.05060 Zobrazit plný text záznamu View this record from Arxiv