摘要
Both Byzantine resilience and communication efficiency have attracted tremendous attention recently for their significance in edge federated learning. However, most existing algorithms may fail when dealing with real-world irregular data that behaves in a heavy-tailed manner. To address this issue, we study the stochastic convex and non-convex optimization problem for federated learning at edge and show how to handle heavy-tailed data while retaining the Byzantine resilience, communication efficiency and the optimal statistical error rates simultaneously. Specifically, we first present a Byzantine-resilient distributed gradient descent algorithm that can handle the heavy-tailed data and meanwhile converge under the standard assumptions. To reduce the communication overhead, we further propose another algorithm that incorporates gradient compression techniques to save communication costs during the learning process. Theoretical analysis shows that our algorithms achieve order-optimal statistical error rate in presence of Byzantine devices. Finally, we conduct extensive experiments on both synthetic and real-world datasets to verify the efficacy of our algorithms.
学科与关键词
- Communication-Efficient and Attack-Resistant Federated Edge Learning With Dataset DistillationZhou, Yanlin; Ma, Xiyao; Wu, Dapeng*; Li, Xiaolin*IEEE Transactions on Cloud Computing, 2023, 11(3): 2517-2528.
- Robust Communication Strategy for Federated Learning by Incorporating Self-AttentionXu, Yawen*; Li, Xiaojun; Yang, Zeyu; Song, HengjieInternational Conference on Image, Video Processing and Artificial Intelligence, Shanghai, PEOPLES R CHINA, 2020-08-21 To 2020-08-23.
- Byzantine-robust Federated Learning through Collaborative Malicious Gradient FilteringXu, Jian; Huang, Shao-Lun*; Song, Linqi; Lan, Tian42nd IEEE International Conference on Distributed Computing Systems (ICDCS), Bologna, ITALY, 2022-07-10 To 2022-07-13.被引 6
