基于联邦学习的时间序列预测算法

doi:10.6040/j.issn.1672-3961.0.2023.217

摘要/Abstract

摘要： 为应对不断升级的数据隐私保护需求,提出一种基于分布式场景下的时间序列预测算法。该算法主要改进体现在以下两个方面:在客户端模型本地训练阶段,通过正则化项约束本地模型训练方向,解决本地模型漂移问题;在全局模型聚合阶段,提出客户端贡献估计策略,根据客户端贡献程度分配权重,保护客户端协作公平性,提升全局模型泛化能力。为验证改进后算法有效性,在ETTh1数据集、ETTm1数据集和Weather数据集上将其与基线联邦学习算法FedAvg对比。试验结果表明,改进后算法在ETTh1数据集上均方误差E_MS平均降低2.99%,在ETTm1数据集上E_MS平均降低3.57%。在算法中加入正则化项和客户端贡献估计策略,E_MS分别下降0.84%和2.78%,同时加入这两个模块,E_MS降低3.03%,验证提出的算法在预测性能方面表现出更高预测准确性。

关键词: 联邦学习, 机器学习, 时间序列预测, 分布式系统, 深度学习

中图分类号:

TP181

刘新,刘冬兰,付婷,王勇,常英贤,姚洪磊,罗昕,王睿,张昊. 基于联邦学习的时间序列预测算法[J]. 山东大学学报 (工学版), 2024, 54(3): 55-63.

LIU Xin, LIU Donglan, FU Ting, WANG Yong, CHANG Yingxian, YAO Honglei, LUO Xin, WANG Rui, ZHANG Hao. Time series forecasting algorithm based on federated learning[J]. Journal of Shandong University(Engineering Science), 2024, 54(3): 55-63.

参考文献

[1] 张美英, 何杰. 时间序列预测模型研究综述[J]. 数学的实践与认识, 2011, 41(18): 189-195. ZHANG Meiying, HE Jie. A comprehensive review of time series forecasting models[J]. Mathematical Practice and Understanding, 2011, 41(18): 189-195.
[2] 李英惠,胥超. 基于时间序列模型的售电量预测方法[J].山东电力技术,2014,41(6):56-59. LI Yinghui, XU Chao. Sales Electricity forecasting method based on time series model[J]. Shandong Electric Power Technology, 2014, 41(6): 56-59.
[3] 杨海民, 潘志松, 白玮. 时间序列预测方法综述[J]. 计算机科学, 2019, 46(1): 21-28. YANG Haimin, PAN Zhisong, BAI Wei. A comprehensive review of time series forecasting methods[J]. Computer Science, 2019, 46(1): 21-28.
[4] 王连成,代桃桃. 数据驱动创新场景引领未来[J].山东电力技术,2018,45(10):22-26. WANG Liancheng, DAI Taotao. Data-driven innovation leading the future of scenarios[J]. Shandong Electric Power Technology, 2018, 45(10): 22-26.
[5] 吕秋霞,孙亮,车延华,等. 基于深度置信网络的配电网负荷预测[J]. 山东电力技术,2023,50(8):20-26. LÜ Qiuxia, SUN Liang, CHE Yanhua, et al. Distribution network load forecasting based on deep belief networks[J]. Shandong Electric Power Technology, 2023, 50(8): 20-26.
[6] TORRES J F, HADJOUT D, SEBAA A, et al. Deep learning for time series forecasting: a survey[J]. Big Data, 2021, 9(1): 3-21.
[7] LI T, SAHU A K, TALWALKAR A, et al. Federated learning: challenges, methods, and future directions[J]. IEEE Signal Processing Magazine, 2020, 37(3): 50-60.
[8] BONAWITZ K, EICHNER H, GRIESKAMP W, et al. Towards federated learning at scale: system design[J]. Machine Learning and Systems, 2019, 1: 374-388.
[9] 王健宗, 孔令炜, 黄章成,等. 联邦学习算法综述[J]. 大数据, 2020, 6(6): 64-82. WANG Jianzong, KONG Lingwei, HUANG Zhangcheng, et al. A comprehensive review of federated learning algorithms[J]. Big Data, 2020, 6(6): 64-82.
[10] 周传鑫, 孙奕, 汪德刚,等. 联邦学习研究综述[J]. 网络与信息安全学报, 2021, 7(5): 77-92. ZHOU Chuanxin, SUN Yi, WANG Degang, et al. A comprehensive review of federated learning[J]. Journal of Network and Information Security, 2021, 7(5): 77-92.
[11] 陈兵, 成翔, 张佳乐,等. 联邦学习安全与隐私保护综述[J]. 南京航空航天大学学报, 2020, 52(5): 675-684. CHEN Bing, CHENG Xiang, ZHANG Jiale, et al. A comprehensive review of security and privacy protection in federated learning[J]. Journal of Nanjing University of Aeronautics and Astronautics, 2020, 52(5): 675-684.
[12] ZHANG G P. Time series forecasting using a hybrid ARIMA and neural network model[J]. Neurocomputing, 2003, 50: 159-175.
[13] NING Y, KAZEMI H, TAHMASEBI P. A comparative machine learning study for time series oil production forecasting: ARIMA, LSTM, and Prophet[J]. Computers & Geosciences, 2022, 164: 105126.
[14] 林颖,刘萌,白德盟,等. 基于深度学习的电力设备红外可见光图像智能配准方法研究[J]. 山东电力技术,2022,49(8):22-27. LIN Ying, LIU Meng, BAI Demeng, et al. Research on intelligent registration of infrared and visible light images of power equipment based on deep learning[J]. Shandong Electric Power Technology, 2022, 49(8): 22-27.
[15] 夏瑜潞. 循环神经网络的发展综述[J]. 电脑知识与技术, 2019, 15(21): 182-184. XIA Yulu. Development overview of recurrent neural networks[J]. Computer Knowledge and Technology, 2019, 15(21): 182-184.
[16] 周飞燕, 金林鹏, 董军. 卷积神经网络研究综述[J]. 计算机学报, 2017, 40(6): 1229-1251. ZHOU Feiyan, JIN Linpeng, DONG Jun. A com-prehensive review of convolutional neural networks[J]. Chinese Journal of Computers, 2017, 40(6): 1229-1251.
[17] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[J]. Neural Information Processing Systems, 2017, 30: 5998-6008.
[18] SALINAS D, FLUNKERT V, GASTHAUS J, et al. DeepAR: probabilistic forecasting with autoregressive recurrent networks[J]. International Journal of Forecasting, 2020, 36(3): 1181-1191.
[19] 张宸嘉, 朱磊, 俞璐. 卷积神经网络中的注意力机制综述[J]. 计算机工程与应用, 2021, 57(20): 64-72. ZHANG Chenjia, ZHU Lei, YU Lu. A survey of attention mechanisms in convolutional neural networks[J]. Computer Engineering and Applications, 2021, 57(20): 64-72.
[20] 田永林, 王雨桐,王建功, 等. 视觉 Transformer 研究的关键问题: 现状及展望[J]. 自动化学报, 2022, 48(4): 957-979. TIAN Yonglin, WANG Yutong, WANG Jiangong, et al. Key issues in visual transformer research: current status and prospects[J]. Acta Automatica Sinica, 2022, 48(4): 957-979.
[21] ZHOU H, ZHANG S, PENG J, et al. Informer: Beyond efficient transformer for long sequence time-series forecasting[C] //Proceedings of the AAAI Conference on Artificial Intelligence. Palo Alto, USA: AAAI Press, 2021: 11106-11115.
[22] MCMAHAN B, MOORE E, RAMAGE D, et al. Communication efficient learning of deep networks from decentralized data[C] // Proceedings of the Artificial Intelligence and Statistics. Florence, Italy: PMLR, 2017: 1273-1282.
[23] LI T, SAHU A K, ZAHEER M, et al. Federated optimization in heterogeneous networks[C] //Procee-dings of Machine Learning and Systems. Texas, USA: PMLR, 2020: 429-450.
[24] LI Q, HE B, SONG D. Model-contrastive federated learning[C] //Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Los Alamitos, USA: IEEE, 2021: 10713-10722.
[25] MENDIETA M, YANG T, WANG P, et al. Local learning matters: rethinking data heterogeneity in federated learning[C] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Reco-gnition. New Orleans, USA: IEEE, 2022: 8397-8406.
[26] 王文鑫, 张健毅. 联邦学习公平性研究综述[J]. 北京电子科技学院学报, 2022, 30(2): 122-134. WANG Wenxin, ZHANG Jianyi. A comprehensive review of fairness in federated learning[J]. Journal of Beijing Electronic Science and Technology Institute, 2022, 30(2): 122-134.
[27] 田家会, 吕锡香, 邹仁朋,等. 一种联邦学习中的公平资源分配方案[J]. 计算机研究与发展, 2022, 59(6): 1240-1254. TIAN Jiahui, LÜ Xixiang, ZOU Renpeng, et al. A fair resource allocation scheme in federated learning[J]. Journal of Computer Research and Development, 2022, 59(6): 1240-1254.
[28] 张军阳, 王慧丽, 郭阳, 等. 深度学习相关研究综述[J]. 计算机应用研究, 2018, 35(7): 1921-1936. ZHANG Junyang, WANG Huili, GUO Yang, et al. A comprehensive review of deep learning research[J]. Journal of Computer Applications Research, 2018, 35(7): 1921-1936.
[29] LUO K, LI X, LAN Y, et al. GradMA: a gradient-memory-based accelerated federated learning with alleviated catastrophic forgetting[C] //Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Vancouver, Canada: IEEE, 2023: 3708-3717.
[30] WU H, XU J, WANG J, et al. Autoformer: decomposition transformers with auto-correlation for long-term series forecasting[J]. Neural Information Processing Systems, 2021, 34: 22419-22430.

多维度评价

Viewed

Full text

Abstract

Cited

Shared

Discussed