山东大学学报 (工学版) ›› 2024, Vol. 54 ›› Issue (3): 55-63.doi: 10.6040/j.issn.1672-3961.0.2023.217
• 机器学习与数据挖掘 • 上一篇
刘新1,2,刘冬兰1,2*,付婷3,王勇4,常英贤4,姚洪磊1,2,罗昕3,王睿1,2,张昊1,2
LIU Xin1,2, LIU Donglan1,2*, FU Ting3, WANG Yong4, CHANG Yingxian4, YAO Honglei1,2, LUO Xin3, WANG Rui1,2, ZHANG Hao1,2
摘要： 为应对不断升级的数据隐私保护需求,提出一种基于分布式场景下的时间序列预测算法。该算法主要改进体现在以下两个方面:在客户端模型本地训练阶段,通过正则化项约束本地模型训练方向,解决本地模型漂移问题;在全局模型聚合阶段,提出客户端贡献估计策略,根据客户端贡献程度分配权重,保护客户端协作公平性,提升全局模型泛化能力。为验证改进后算法有效性,在ETTh1数据集、ETTm1数据集和Weather数据集上将其与基线联邦学习算法FedAvg对比。试验结果表明,改进后算法在ETTh1数据集上均方误差EMS平均降低2.99%,在ETTm1数据集上EMS平均降低3.57%。在算法中加入正则化项和客户端贡献估计策略,EMS分别下降0.84%和2.78%,同时加入这两个模块,EMS降低3.03%,验证提出的算法在预测性能方面表现出更高预测准确性。
中图分类号:
| [1] 张美英, 何杰. 时间序列预测模型研究综述[J]. 数学的实践与认识, 2011, 41(18): 189-195. ZHANG Meiying, HE Jie. A comprehensive review of time series forecasting models[J]. Mathematical Practice and Understanding, 2011, 41(18): 189-195. [2] 李英惠,胥超. 基于时间序列模型的售电量预测方法[J].山东电力技术,2014,41(6):56-59. LI Yinghui, XU Chao. Sales Electricity forecasting method based on time series model[J]. Shandong Electric Power Technology, 2014, 41(6): 56-59. [3] 杨海民, 潘志松, 白玮. 时间序列预测方法综述[J]. 计算机科学, 2019, 46(1): 21-28. YANG Haimin, PAN Zhisong, BAI Wei. A comprehensive review of time series forecasting methods[J]. Computer Science, 2019, 46(1): 21-28. [4] 王连成,代桃桃. 数据驱动创新场景引领未来[J].山东电力技术,2018,45(10):22-26. WANG Liancheng, DAI Taotao. Data-driven innovation leading the future of scenarios[J]. Shandong Electric Power Technology, 2018, 45(10): 22-26. [5] 吕秋霞,孙亮,车延华,等. 基于深度置信网络的配电网负荷预测[J]. 山东电力技术,2023,50(8):20-26. LÜ Qiuxia, SUN Liang, CHE Yanhua, et al. Distribution network load forecasting based on deep belief networks[J]. Shandong Electric Power Technology, 2023, 50(8): 20-26. [6] TORRES J F, HADJOUT D, SEBAA A, et al. Deep learning for time series forecasting: a survey[J]. Big Data, 2021, 9(1): 3-21. [7] LI T, SAHU A K, TALWALKAR A, et al. Federated learning: challenges, methods, and future directions[J]. IEEE Signal Processing Magazine, 2020, 37(3): 50-60. [8] BONAWITZ K, EICHNER H, GRIESKAMP W, et al. Towards federated learning at scale: system design[J]. Machine Learning and Systems, 2019, 1: 374-388. [9] 王健宗, 孔令炜, 黄章成,等. 联邦学习算法综述[J]. 大数据, 2020, 6(6): 64-82. WANG Jianzong, KONG Lingwei, HUANG Zhangcheng, et al. A comprehensive review of federated learning algorithms[J]. Big Data, 2020, 6(6): 64-82. [10] 周传鑫, 孙奕, 汪德刚,等. 联邦学习研究综述[J]. 网络与信息安全学报, 2021, 7(5): 77-92. ZHOU Chuanxin, SUN Yi, WANG Degang, et al. A comprehensive review of federated learning[J]. Journal of Network and Information Security, 2021, 7(5): 77-92. [11] 陈兵, 成翔, 张佳乐,等. 联邦学习安全与隐私保护综述[J]. 南京航空航天大学学报, 2020, 52(5): 675-684. CHEN Bing, CHENG Xiang, ZHANG Jiale, et al. A comprehensive review of security and privacy protection in federated learning[J]. Journal of Nanjing University of Aeronautics and Astronautics, 2020, 52(5): 675-684. [12] ZHANG G P. Time series forecasting using a hybrid ARIMA and neural network model[J]. Neurocomputing, 2003, 50: 159-175. [13] NING Y, KAZEMI H, TAHMASEBI P. A comparative machine learning study for time series oil production forecasting: ARIMA, LSTM, and Prophet[J]. Computers & Geosciences, 2022, 164: 105126. [14] 林颖,刘萌,白德盟,等. 基于深度学习的电力设备红外可见光图像智能配准方法研究[J]. 山东电力技术,2022,49(8):22-27. LIN Ying, LIU Meng, BAI Demeng, et al. Research on intelligent registration of infrared and visible light images of power equipment based on deep learning[J]. Shandong Electric Power Technology, 2022, 49(8): 22-27. [15] 夏瑜潞. 循环神经网络的发展综述[J]. 电脑知识与技术, 2019, 15(21): 182-184. XIA Yulu. Development overview of recurrent neural networks[J]. Computer Knowledge and Technology, 2019, 15(21): 182-184. [16] 周飞燕, 金林鹏, 董军. 卷积神经网络研究综述[J]. 计算机学报, 2017, 40(6): 1229-1251. ZHOU Feiyan, JIN Linpeng, DONG Jun. A com-prehensive review of convolutional neural networks[J]. Chinese Journal of Computers, 2017, 40(6): 1229-1251. [17] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[J]. Neural Information Processing Systems, 2017, 30: 5998-6008. [18] SALINAS D, FLUNKERT V, GASTHAUS J, et al. DeepAR: probabilistic forecasting with autoregressive recurrent networks[J]. International Journal of Forecasting, 2020, 36(3): 1181-1191. [19] 张宸嘉, 朱磊, 俞璐. 卷积神经网络中的注意力机制综述[J]. 计算机工程与应用, 2021, 57(20): 64-72. ZHANG Chenjia, ZHU Lei, YU Lu. A survey of attention mechanisms in convolutional neural networks[J]. Computer Engineering and Applications, 2021, 57(20): 64-72. [20] 田永林, 王雨桐,王建功, 等. 视觉 Transformer 研究的关键问题: 现状及展望[J]. 自动化学报, 2022, 48(4): 957-979. TIAN Yonglin, WANG Yutong, WANG Jiangong, et al. Key issues in visual transformer research: current status and prospects[J]. Acta Automatica Sinica, 2022, 48(4): 957-979. [21] ZHOU H, ZHANG S, PENG J, et al. Informer: Beyond efficient transformer for long sequence time-series forecasting[C] //Proceedings of the AAAI Conference on Artificial Intelligence. Palo Alto, USA: AAAI Press, 2021: 11106-11115. [22] MCMAHAN B, MOORE E, RAMAGE D, et al. Communication efficient learning of deep networks from decentralized data[C] // Proceedings of the Artificial Intelligence and Statistics. Florence, Italy: PMLR, 2017: 1273-1282. [23] LI T, SAHU A K, ZAHEER M, et al. Federated optimization in heterogeneous networks[C] //Procee-dings of Machine Learning and Systems. Texas, USA: PMLR, 2020: 429-450. [24] LI Q, HE B, SONG D. Model-contrastive federated learning[C] //Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Los Alamitos, USA: IEEE, 2021: 10713-10722. [25] MENDIETA M, YANG T, WANG P, et al. Local learning matters: rethinking data heterogeneity in federated learning[C] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Reco-gnition. New Orleans, USA: IEEE, 2022: 8397-8406. [26] 王文鑫, 张健毅. 联邦学习公平性研究综述[J]. 北京电子科技学院学报, 2022, 30(2): 122-134. WANG Wenxin, ZHANG Jianyi. A comprehensive review of fairness in federated learning[J]. Journal of Beijing Electronic Science and Technology Institute, 2022, 30(2): 122-134. [27] 田家会, 吕锡香, 邹仁朋,等. 一种联邦学习中的公平资源分配方案[J]. 计算机研究与发展, 2022, 59(6): 1240-1254. TIAN Jiahui, LÜ Xixiang, ZOU Renpeng, et al. A fair resource allocation scheme in federated learning[J]. Journal of Computer Research and Development, 2022, 59(6): 1240-1254. [28] 张军阳, 王慧丽, 郭阳, 等. 深度学习相关研究综述[J]. 计算机应用研究, 2018, 35(7): 1921-1936. ZHANG Junyang, WANG Huili, GUO Yang, et al. A comprehensive review of deep learning research[J]. Journal of Computer Applications Research, 2018, 35(7): 1921-1936. [29] LUO K, LI X, LAN Y, et al. GradMA: a gradient-memory-based accelerated federated learning with alleviated catastrophic forgetting[C] //Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Vancouver, Canada: IEEE, 2023: 3708-3717. [30] WU H, XU J, WANG J, et al. Autoformer: decomposition transformers with auto-correlation for long-term series forecasting[J]. Neural Information Processing Systems, 2021, 34: 22419-22430. |
| [1] | 聂秀山,巩蕊,董飞,郭杰,马玉玲. 短视频场景分类方法综述[J]. 山东大学学报 (工学版), 2024, 54(3): 1-11. |
| [2] | 宋辉,张轶哲,张功萱,孟元. 基于类权重和最小化预测熵的测试时集成方法[J]. 山东大学学报 (工学版), 2024, 54(3): 36-43. |
| [3] | 索大翔,李波. 基于Gromov-Wasserstein最优传输的输电线路小目标检测方法[J]. 山东大学学报 (工学版), 2024, 54(3): 22-29. |
| [4] | 高泽文,王建,魏本征. 基于混合偏移轴向自注意力机制的脑胶质瘤分割算法[J]. 山东大学学报 (工学版), 2024, 54(2): 80-89. |
| [5] | 李璐,张志军,范钰敏,王星,袁卫华. 面向冷启动用户的元学习与图转移学习序列推荐[J]. 山东大学学报 (工学版), 2024, 54(2): 69-79. |
| [6] | 陈成,董永权,贾瑞,刘源. 基于交互序列特征相关性的可解释知识追踪[J]. 山东大学学报 (工学版), 2024, 54(1): 100-108. |
| [7] | 李鸿钊,张庆松,刘人太,陈新,辛勤,石乐乐. 浅埋地铁车站施工期地表变形风险预警[J]. 山东大学学报 (工学版), 2023, 53(6): 82-91. |
| [8] | 李家春,李博文,常建波. 一种高效且轻量的RGB单帧人脸反欺诈模型[J]. 山东大学学报 (工学版), 2023, 53(6): 1-7. |
| [9] | 王旭晴,魏伟波,杨光宇,宋金涛,吕婷,潘振宽. 基于算法展开的图像盲去模糊深度学习网络[J]. 山东大学学报 (工学版), 2023, 53(6): 35-46. |
| [10] | 卞小曼,王小琴,蓝如师,刘振丙,罗笑南. 基于相似性保持和判别性分析的快速视频哈希算法[J]. 山东大学学报 (工学版), 2023, 53(6): 63-69. |
| [11] | 王碧瑶,韩毅,崔航滨,刘毅超,任铭然,高维勇,陈姝廷,刘嘉巍,崔洋. 基于图像的道路语义分割检测方法[J]. 山东大学学报 (工学版), 2023, 53(5): 37-47. |
| [12] | 周晓昕,廖祝华,刘毅志,赵肄江,方艺洁. 融合历史与当前交通流量的信号控制方法[J]. 山东大学学报 (工学版), 2023, 53(4): 48-55. |
| [13] | 于畅,伍星,邓秋菊. 基于深度学习的多视角螺钉缺失智能检测算法[J]. 山东大学学报 (工学版), 2023, 53(4): 104-112. |
| [14] | 宋佳芮,陈艳平,王凯,黄瑞章,秦永彬. 基于Affix-Attention的命名实体识别语义补充方法[J]. 山东大学学报 (工学版), 2023, 53(2): 70-76. |
| [15] | 李旭涛,杨寒玉,卢业飞,张玮. 基于深度学习的遥感图像道路分割[J]. 山东大学学报 (工学版), 2022, 52(6): 139-145. |
|
||
