GENE-FL: Gene-Driven Parameter-Efficient Dynamic Federated Learning

Abstract

Real-world Federated Learning systems often encounter Dynamic clients with Agnostic and highly heterogeneous data distributions (DAFL), which pose challenges for efficient communication and model initialization. To address these challenges, we draw inspiration from the recently proposed Learngene paradigm, which compresses the large-scale model into lightweight, cross-task meta-information fragments. Learngene effectively encapsulates and communicates core knowledge, making it particularly well-suited for DAFL, where dynamic client participation requires communication efficiency and rapid adaptation to new data distributions. Based on this insight, we propose a Gene-driven parameter-efficient dynamic Federated Learning (GENE-FL) framework. First, local models perform quadratic constraints based on parameters with high Fisher values in the global model, as these parameters are considered to encapsulate generalizable knowledge. Second, we apply the strategy of parameter sensitivity analysis in local model parameters to condense the learnGene for interaction. Finally, the server aggregates these small-scale trained learnGenes into a robust learnGene with cross-task generalization capability, facilitating the rapid initialization of dynamic agnostic client models. Extensive experimental results demonstrate that GENE-FL reduces 4 communication costs compared to FEDAVG and effectively initializes agnostic client models with only about 9.04 MB.

Tasks

Reproductions