QUIC-FL: Quick Unbiased Compression for Federated Learning

Autor:	Basat, Ran Ben, Vargaftik, Shay, Portnoy, Amit, Einziger, Gil, Ben-Itzhak, Yaniv, Mitzenmacher, Michael
Rok vydání:	2022
Předmět:	Computer Science - Machine Learning Computer Science - Artificial Intelligence Computer Science - Data Structures and Algorithms Computer Science - Networking and Internet Architecture
Druh dokumentu:	Working Paper
Popis:	Distributed Mean Estimation (DME), in which $n$ clients communicate vectors to a parameter server that estimates their average, is a fundamental building block in communication-efficient federated learning. In this paper, we improve on previous DME techniques that achieve the optimal $O(1/n)$ Normalized Mean Squared Error (NMSE) guarantee by asymptotically improving the complexity for either encoding or decoding (or both). To achieve this, we formalize the problem in a novel way that allows us to use off-the-shelf mathematical solvers to design the quantization.
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2205.13341 View this record from Arxiv