基于模块化网络的自适应加权联邦持续学习方法

doi:10.6040/j.issn.1672-3961.0.2025.074

摘要/Abstract

摘要： 针对资源受限环境下联邦持续学习(federated continual learning, FCL)中的横向与纵向灾难性遗忘问题,提出一种基于模块化网络的自适应加权联邦持续学习(modular-based adaptive weighted federated continual learning, MAWFCL)方法,有效应对任务演化引发的模型知识保持与任务适应性挑战。通过构建可组合的基础参数模块与自适应控制参数,实现个性化模型构建与任务适配;引入模块相似度度量机制,提升知识复用效率;结合参数容量感知的精准遗忘策略,有效控制模型复杂度;设计基于参数距离的自适应聚合算法,缓解聚合过程中的知识冲突。试验结果表明,MAWFCL方法在准确率、灾难性遗忘抑制和通信效率方面优于现有方法,在CIFAR100数据集上的表现明显优于联邦生成重放学习(federated generative replay learning, FedGReL)方法和基于提示词的双重知识迁移(prompt-based dual knowledge transfer, Powder)方法,测试准确率分别提升10.93百分点和10.17百分点,在复杂任务中展现出显著优势。

关键词: 联邦持续学习, 参数隔离, 神经网络, 迁移学习, 自适应算法

Abstract: To address the challenges of horizontal and vertical catastrophic forgetting in resource-constrained federated continual learning(FCL)environments, a modular-based adaptive weighted federated continual learning(MAWFCL)method was proposed, which effectively addressed the difficulties of model knowledge retention and task adaptability caused by continuously evolving tasks. Personalized models were constructed by combining composable base parameter modules with adaptive control parameters to achieve adaptation to specific tasks. A module similarity-based reuse mechanism was introduced to enhance the efficiency of knowledge reuse. A parameter capacity-aware precision forgetting strategy was incorporated to prune low-contribution modules and maintain a compact model structure. An adaptive aggregation algorithm based on parameter distance was designed to alleviate knowledge conflicts during global model aggregation. Experimental results showed that MAWFCL method outperformed existing methods in terms of accuracy, catastrophic forgetting mitigation, and communication efficiency. On the CIFAR-100 dataset, MAWFCL method improved test accuracy over federated generative replay learning(FedGReL)and prompt-based dual knowledge transfer(Powder)by 10.93 percentage points and 10.17 percentage points, respectively, demonstrating significant advantages in complex tasks.

Key words: federated continual learning, parameter isolation, neural network, transfer learning, adaptive algorithm

中图分类号:

TP18

周志刚,孙博洋,戴隆政,白增亮,苗钧重. 基于模块化网络的自适应加权联邦持续学习方法[J]. 山东大学学报 (工学版), 2026, 56(2): 19-34.

ZHOU Zhigang, SUN Boyang, DAI Longzheng, BAI Zengliang, MIAO Junzhong. Modular-based adaptive weighted federated continual learning method[J]. Journal of Shandong University(Engineering Science), 2026, 56(2): 19-34.

参考文献

[1] WANG L Y, ZHANG X X, SU H, et al. A comprehensive survey of continual learning: theory, method and application[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024, 46(8): 5362-5383.
[2] 王文晟, 谭宁, 黄凯, 等. 基于大模型的具身智能系统综述[J]. 自动化学报, 2025, 51(1): 1-19. WANG Wensheng, TAN Ning, HUANG Kai, et al. Embodied intelligence systems based on large models: a survey[J]. Acta Automatica Sinica, 2025, 51(1): 1-19.
[3] MCLEAN S, READ G J M, THOMPSON J, et al. The risks associated with artificial general intelligence: a systematic review[J]. Journal of Experimental & Theoretical Artificial Intelligence, 2023, 35(5): 649-663.
[4] SILVER D, HUBERT T, SCHRITTWIESER J, et al. A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play[J]. Science, 2018, 362(6419): 1140-1144.
[5] 肖雄, 唐卓, 肖斌, 等. 联邦学习的隐私保护与安全防御研究综述[J]. 计算机学报, 2023, 46(5): 1019-1044. XIAO Xiong, TANG Zhuo, XIAO Bin, et al. A survey on privacy protection and security defense in federated learning[J]. Chinese Journal of Computers, 2023, 46(5): 1019-1044.
[6] MCMAHAN H B, MOORE E, RAMAGE D, et al. Communication-efficient learning of deep networks from decentralized data[C] //Proceedings of the 20th Inter-national Conference on Artificial Intelligence and Statistics(AISTATS). Fort Lauderdale, USA: JMLR, 2017: 1273-1282.
[7] LI T, SAHU A K, ZAHEER M, et al. Federated optimization in heterogeneous networks[EB/OL].(2020-04-21)[2025-05-22]. https://arxiv.org/abs/1812.06127
[8] YU H, YANG X, GAO X, et al. Personalized federated continual learning via multi-granularity prompt[C] //Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. Barcelona, Spain: ACM, 2024: 4023-4034.
[9] 杜甜, 陈星延, 寇纲, 等. 面向云边个性化模型解耦的聚类联邦学习方法[J]. 计算机学报, 2025, 48(2): 407-432. DU Tian, CHEN Xingyan, KOU Gang, et al. Clustered federated learning with cloud-edge personalized model decoupling[J]. Chinese Journal of Computers, 2025, 48(2): 407-432.
[10] DE LANGE M, ALJUNDI R, MASANA M, et al. A continual learning survey: defying forgetting in classification tasks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(7): 3366-3385.
[11] YOON J, JEONG W, LEE G, et al. Federated continual learning with weighted inter-client transfer[C] //Proceedings of the 38th International Conference on Machine Learning(ICML).[S.l.] : PMLR, 2021: 12073-12086.
[12] YU H Z, CHEN Z K, ZHANG X, et al. FedHAR: semi-supervised online learning for personalized federated human activity recognition[J]. IEEE Tran-sactions on Mobile Computing, 2023, 22(6): 3318-3332.
[13] 王鹏飞, 魏宗正, 周东生, 等. 联邦忘却学习研究综述[J]. 计算机学报, 2024, 47(2): 396-422. WANG Pengfei, WEI Zongzheng, ZHOU Dongsheng, et al. A survey on federated unlearning[J]. Chinese Journal of Computers, 2024, 47(2): 396-422.
[14] WUERKAIXI A, CUI S, ZHANG J F, et al. Accurate forgetting for heterogeneous federated continual learning[EB/OL].(2025-02-20)[2025-05-22]. https://arxiv.org/abs/2502.14205
[15] YOON J, KIM S, YANG E, et al. Scalable and order-robust continual learning with additive parameter decomposition[EB/OL].(2020-02-15)[2025-05-22]. https://arxiv.org/abs/1902.09432
[16] MALLYA A, LAZEBNIK S. PackNet: adding multiple tasks to a single network by iterative pruning[C] //2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE, 2018: 7765-7773.
[17] KIRKPATRICK J, PASCANU R, RABINOWITZ N, et al. Overcoming catastrophic forgetting in neural networks[J]. Proceedings of the National Academy of Sciences of the United States of America, 2017, 114(13): 3521-3526.
[18] LI Z Z, HOIEM D. Learning without forgetting[C] //European Conference on Computer Vision(ECCV). Amsterdam, Netherlands: Springer, 2016: 614-629.
[19] CHAUDHRY A, RANZATO M, ROHRBACH M, et al. Efficient lifelong learning with A-GEM[EB/OL].(2019-01-09)[2025-05-22]. https://arxiv.org/abs/1812.00420
[20] YOON J, YANG E, LEE J, et al. Lifelong learning with dynamically expandable networks[EB/OL].(2018-06-11)[2025-05-22]. https://arxiv.org/abs/1708.01547
[21] ALJUNDI R, CHAKRAVARTY P, TUYTELAARS T. Expert gate: lifelong learning with a network of experts[C] //Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA: IEEE, 2017: 7120-7129.
[22] ROLNICK D, AHUJA A, SCHWARZ J, et al. Experience replay for continual learning[EB/OL].(2019-11-26)[2025-05-22]. https://arxiv.org/abs/1811.11682
[23] BUZZEGA P, BOSCHINI M, PORRELLO A, et al. Dark experience for general continual learning[C] // Proceedings of the 34th International Conference on Neural Information Processing Systems. Vancouver, Canada: ACM, 2020: 15920-15930.
[24] REBUFFI S A, KOLESNIKOV A, SPERL G, et al. iCaRL: Incremental classifier and representation learning[C] //2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Honolulu, USA: IEEE, 2017: 5533-5542.
[25] ZHU Z P, ZHAO S J, CHU C C, et al. FedPMR: federated personalized mixture representation for driver intention prediction[J]. IEEE Transactions on Intelligent Vehicles, 2025, 10(1): 627-640.
[26] HE Y T, CHEN Y Q, YANG X D, et al. Class-wise adaptive self-distillation for federated learning on Non-IID data[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2022, 36(11): 12967-12968.
[27] 姜慧, 何天流, 刘敏, 等. 面向异构流式数据的高性能联邦持续学习算法[J]. 通信学报, 2023, 44(5): 123-136. JIANG Hui, HE Tianliu, LIU Min, et al. High-performance federated continual learning algorithm for heterogeneous streaming data[J]. Journal on Commu-nications, 2023, 44(5): 123-136.
[28] DONG J H, WANG L X, FANG Z, et al. Federated class-incremental learning[C] //2022 IEEE/CVF Confe-rence on Computer Vision and Pattern Recognition(CVPR). New Orleans, USA: IEEE, 2022: 10154-10163.
[29] SAHA G, GARG I, ROY K. Gradient projection memory for continual learning[EB/OL].(2021-03-17)[2025-05-22]. https://arxiv.org/abs/2103.09762
[30] LUO P Y X, HAN R, ZHANG Q L, et al. FedKNOW: federated continual learning with signature task knowledge integration at edge[C] //2023 IEEE 39th International Conference on Data Engineering(ICDE). Anaheim, USA: IEEE, 2023: 341-354.
[31] SHOHAM N, AVIDOR T, KEREN A, et al. Overcoming forgetting in federated learning on Non-IID data[EB/OL].(2019-10-17)[2025-05-22]. https://arxiv.org/abs/1910.07796
[32] WU C H, WU F Z, QI T, et al. FedCL: federated contrastive learning for privacy-preserving recommendation[EB/OL].(2022-04-21)[2025-05-22]. https://arxiv.org/abs/2204.09850
[33] PIAO H M, WU Y C, WU D P, et al. Federated continual learning via prompt-based dual knowledge transfer[C] //Proceedings of the 41st International Conference on Machine Learning. Vienna, Austria: JMLR, 2024: 40725-40739.
[34] WANG Q, LIU B Y, LI Y W. Traceable federated continual learning[C] //2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). Seattle, USA: IEEE, 2024: 12872-12881.
[35] HE Y C, SHEN C Y, WANG X F, et al. FPPL: an efficient and Non-IID robust federated continual learning framework[C] //2024 IEEE International Conference on Big Data(BigData). Washington, DC, USA: IEEE, 2024: 3692-3701.
[36] YU H, YANG X, GAO X, et al. Personalized federated continual learning via multi granularity prompt[C] //Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. Barcelona, Spain: ACM, 2024: 4023-4034.
[37] WANG Z R, DAI Z H, PÓCZOS B, et al. Characterizing and avoiding negative transfer[C] //2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). Long Beach, USA: IEEE, 2020: 11285-11294.

多维度评价

Viewed

Full text

Abstract

Cited

Shared

Discussed