Rosko: Row Skipping Outer Products for Sparse Matrix Multiplication Kernels

Autor:	Natesh, Vikas, Sabot, Andrew, Kung, H. T., Ting, Mark
Rok vydání:	2023
Předmět:	Computer Science - Machine Learning Computer Science - Hardware Architecture Computer Science - Performance Computer Science - Programming Languages
Druh dokumentu:	Working Paper
Popis:	We propose Rosko -- row skipping outer products -- for deriving sparse matrix multiplication (SpMM) kernels in reducing computation and memory access requirements of deep neural networks (DNNs). Rosko allows skipping of entire row computations during program execution with low sparsity-management overheads. We analytically derive sparse CPU kernels that adapt to given hardware characteristics to effectively utilize processor cores and minimize data movement without the need for auto-tuning or search space exploration. Rosko can be integrated with other outer product scheduling methods, allowing them to leverage row skipping by using Rosko's packing format to skip unnecessary computation. Rosko kernels outperform existing auto-tuning and search-based solutions as well as state-of-the-art vendor-optimized libraries on real hardware across a variety of neural network workloads. For matrices with sparsities ranging from 65% to 99.8% typically found in machine learning, Rosko kernels achieve up to a 6.5x runtime reduction on Intel and ARM CPUs. Comment: Rosko's CPU implementation can be found at https://github.com/vnatesh/Rosko
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2307.03930 Zobrazit plný text záznamu View this record from Arxiv