摘要
Communication efficiency and privacy protection are two critical issues in distributed machine learning. Existing methods tackle these two issues separately and may have a high implementation complexity that constrains their application in a resource-limited environment. We propose a comprehensive quantization-based solution that could simultaneously achieve communication efficiency and privacy protection, providing new insights into the correlated nature of communication and privacy. Specifically, we demonstrate the effectiveness of our proposed solutions in the distributed stochastic gradient descent (SGD) framework by adding binomial noise to the uniformly quantized gradients to reach the desired differential privacy level but with a minor sacrifice in communication efficiency. We theoretically capture the new trade-offs between communication, privacy, and learning performance.
学科与关键词
- Communication-Efficient Federated Learning with Adaptive QuantizationMao, Yuzhu; Zhao, Zihao; Yan, Guangfeng; Liu, Yang; Lan, Tian; Song, Linqi; Ding, Wenbo*ACM Transactions on Intelligent Systems and Technology, 2022, 13(4): 67.被引 9
- Approximate to Be Great: Communication Efficient and Privacy-Preserving Large-Scale Distributed Deep Learning in Internet of ThingsDu, Wei; Li, Ang; Zhou, Pan*; Xu, Zichuan; Wang, Xiumin; Jiang, Hao; Wu, DapengIEEE Internet of Things Journal, 2020, 7(12): 11678-11692.被引 7
- Privacy-Preserving Communication-Efficient Federated Multi-Armed BanditsLi, Tan*; Song, Linqi*IEEE Journal on Selected Areas in Communications, 2022, 40(3): 773-787.被引 8
