Training Diffusion Models with Federated Learning

Autor:	de Goede, Matthijs, Cox, Bart, Decouchant, Jérémie
Rok vydání:	2024
Předmět:	Computer Science - Machine Learning Computer Science - Distributed Parallel and Cluster Computing I.2.11
Druh dokumentu:	Working Paper
Popis:	The training of diffusion-based models for image generation is predominantly controlled by a select few Big Tech companies, raising concerns about privacy, copyright, and data authority due to their lack of transparency regarding training data. To ad-dress this issue, we propose a federated diffusion model scheme that enables the independent and collaborative training of diffusion models without exposing local data. Our approach adapts the Federated Averaging (FedAvg) algorithm to train a Denoising Diffusion Model (DDPM). Through a novel utilization of the underlying UNet backbone, we achieve a significant reduction of up to 74% in the number of parameters exchanged during training,compared to the naive FedAvg approach, whilst simultaneously maintaining image quality comparable to the centralized setting, as evaluated by the FID score. Comment: Replacement of: http://resolver.tudelft.nl/uuid:49e11cf3-5a0a-40bc-9a62-1d7fe05fbe4d. Name of the algorithm has been changed slightly due to a name collision with another paper
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2406.12575 Zobrazit plný text záznamu View this record from Arxiv