Sparse Incremental Aggregation in Multi-Hop Federated Learning

Autor:	Mukherjee, Sourav, Razmi, Nasrin, Dekorsy, Armin, Popovski, Petar, Matthiesen, Bho
Rok vydání:	2024
Předmět:	Computer Science - Distributed Parallel and Cluster Computing Computer Science - Machine Learning Electrical Engineering and Systems Science - Signal Processing
Druh dokumentu:	Working Paper
Popis:	This paper investigates federated learning (FL) in a multi-hop communication setup, such as in constellations with inter-satellite links. In this setup, part of the FL clients are responsible for forwarding other client's results to the parameter server. Instead of using conventional routing, the communication efficiency can be improved significantly by using in-network model aggregation at each intermediate hop, known as incremental aggregation (IA). Prior works [1] have indicated diminishing gains for IA under gradient sparsification. Here we study this issue and propose several novel correlated sparsification methods for IA. Numerical results show that, for some of these algorithms, the full potential of IA is still available under sparsification without impairing convergence. We demonstrate a 15x improvement in communication efficiency over conventional routing and a 11x improvement over state-of-the-art (SoA) sparse IA. Comment: This paper is accepted for the 25th IEEE International Workshop on Signal Processing Advances in Wireless Communications (SPAWC) conference
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2407.18200 Zobrazit plný text záznamu View this record from Arxiv