Inducing Semi-Structured Sparsity by Masking for Efficient Model Inference in Convolutional Networks

Autor:	Danhofer, David A.
Rok vydání:	2024
Předmět:	Computer Science - Machine Learning Computer Science - Artificial Intelligence Computer Science - Computer Vision and Pattern Recognition Computer Science - Neural and Evolutionary Computing Computer Science - Performance C.4 I.2.6 I.2.10 I.4.m I.5.4 I.5.5
Druh dokumentu:	Working Paper
Popis:	The crucial role of convolutional models, both as standalone vision models and backbones in foundation models, necessitates effective acceleration techniques. This paper proposes a novel method to learn semi-structured sparsity patterns for convolution kernels in the form of maskings enabling the utilization of readily available hardware accelerations. The approach accelerates convolutional models more than two-fold during inference without decreasing model performance. At the same time, the original model weights and structure remain unchanged keeping the model thus easily updatable. Beyond the immediate practical use, the effect of maskings on prediction is easily quantifiable. Therefore, guarantees on model predictions under maskings are derived showing stability bounds for learned maskings even after updating the original underlying model. Comment: 15 pages, 3 figures; this work will be presented at the NeurIPS 2024 Workshop on Fine-Tuning in Modern Machine Learning: Principles and Scalability (FITML)
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2411.00288 Zobrazit plný text záznamu View this record from Arxiv