Journal of Shandong University(Engineering Science) ›› 2026, Vol. 56 ›› Issue (2): 19-34.doi: 10.6040/j.issn.1672-3961.0.2025.074

• Machine Learning & Data Mining • Previous Articles    

Modular-based adaptive weighted federated continual learning method

ZHOU Zhigang1, SUN Boyang1, DAI Longzheng1, BAI Zengliang1, MIAO Junzhong2   

  1. ZHOU Zhigang1, SUN Boyang1, DAI Longzheng1, BAI Zengliang1, MIAO Junzhong2(1. School of Information, Shanxi University of Finance and Economics, Taiyuan 030006, Shanxi, China;
    2. School of Cyberspace Science, Harbin Institute of Technology, Harbin 150006, Heilongjiang, China
  • Published:2026-04-13

Abstract: To address the challenges of horizontal and vertical catastrophic forgetting in resource-constrained federated continual learning(FCL)environments, a modular-based adaptive weighted federated continual learning(MAWFCL)method was proposed, which effectively addressed the difficulties of model knowledge retention and task adaptability caused by continuously evolving tasks. Personalized models were constructed by combining composable base parameter modules with adaptive control parameters to achieve adaptation to specific tasks. A module similarity-based reuse mechanism was introduced to enhance the efficiency of knowledge reuse. A parameter capacity-aware precision forgetting strategy was incorporated to prune low-contribution modules and maintain a compact model structure. An adaptive aggregation algorithm based on parameter distance was designed to alleviate knowledge conflicts during global model aggregation. Experimental results showed that MAWFCL method outperformed existing methods in terms of accuracy, catastrophic forgetting mitigation, and communication efficiency. On the CIFAR-100 dataset, MAWFCL method improved test accuracy over federated generative replay learning(FedGReL)and prompt-based dual knowledge transfer(Powder)by 10.93 percentage points and 10.17 percentage points, respectively, demonstrating significant advantages in complex tasks.

Key words: federated continual learning, parameter isolation, neural network, transfer learning, adaptive algorithm

CLC Number: 

  • TP18
[1] WANG L Y, ZHANG X X, SU H, et al. A comprehensive survey of continual learning: theory, method and application[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024, 46(8): 5362-5383.
[2] 王文晟, 谭宁, 黄凯, 等. 基于大模型的具身智能系统综述[J]. 自动化学报, 2025, 51(1): 1-19. WANG Wensheng, TAN Ning, HUANG Kai, et al. Embodied intelligence systems based on large models: a survey[J]. Acta Automatica Sinica, 2025, 51(1): 1-19.
[3] MCLEAN S, READ G J M, THOMPSON J, et al. The risks associated with artificial general intelligence: a systematic review[J]. Journal of Experimental & Theoretical Artificial Intelligence, 2023, 35(5): 649-663.
[4] SILVER D, HUBERT T, SCHRITTWIESER J, et al. A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play[J]. Science, 2018, 362(6419): 1140-1144.
[5] 肖雄, 唐卓, 肖斌, 等. 联邦学习的隐私保护与安全防御研究综述[J]. 计算机学报, 2023, 46(5): 1019-1044. XIAO Xiong, TANG Zhuo, XIAO Bin, et al. A survey on privacy protection and security defense in federated learning[J]. Chinese Journal of Computers, 2023, 46(5): 1019-1044.
[6] MCMAHAN H B, MOORE E, RAMAGE D, et al. Communication-efficient learning of deep networks from decentralized data[C] //Proceedings of the 20th Inter-national Conference on Artificial Intelligence and Statistics(AISTATS). Fort Lauderdale, USA: JMLR, 2017: 1273-1282.
[7] LI T, SAHU A K, ZAHEER M, et al. Federated optimization in heterogeneous networks[EB/OL].(2020-04-21)[2025-05-22]. https://arxiv.org/abs/1812.06127
[8] YU H, YANG X, GAO X, et al. Personalized federated continual learning via multi-granularity prompt[C] //Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. Barcelona, Spain: ACM, 2024: 4023-4034.
[9] 杜甜, 陈星延, 寇纲, 等. 面向云边个性化模型解耦的聚类联邦学习方法[J]. 计算机学报, 2025, 48(2): 407-432. DU Tian, CHEN Xingyan, KOU Gang, et al. Clustered federated learning with cloud-edge personalized model decoupling[J]. Chinese Journal of Computers, 2025, 48(2): 407-432.
[10] DE LANGE M, ALJUNDI R, MASANA M, et al. A continual learning survey: defying forgetting in classification tasks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(7): 3366-3385.
[11] YOON J, JEONG W, LEE G, et al. Federated continual learning with weighted inter-client transfer[C] //Proceedings of the 38th International Conference on Machine Learning(ICML).[S.l.] : PMLR, 2021: 12073-12086.
[12] YU H Z, CHEN Z K, ZHANG X, et al. FedHAR: semi-supervised online learning for personalized federated human activity recognition[J]. IEEE Tran-sactions on Mobile Computing, 2023, 22(6): 3318-3332.
[13] 王鹏飞, 魏宗正, 周东生, 等. 联邦忘却学习研究综述[J]. 计算机学报, 2024, 47(2): 396-422. WANG Pengfei, WEI Zongzheng, ZHOU Dongsheng, et al. A survey on federated unlearning[J]. Chinese Journal of Computers, 2024, 47(2): 396-422.
[14] WUERKAIXI A, CUI S, ZHANG J F, et al. Accurate forgetting for heterogeneous federated continual learning[EB/OL].(2025-02-20)[2025-05-22]. https://arxiv.org/abs/2502.14205
[15] YOON J, KIM S, YANG E, et al. Scalable and order-robust continual learning with additive parameter decomposition[EB/OL].(2020-02-15)[2025-05-22]. https://arxiv.org/abs/1902.09432
[16] MALLYA A, LAZEBNIK S. PackNet: adding multiple tasks to a single network by iterative pruning[C] //2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE, 2018: 7765-7773.
[17] KIRKPATRICK J, PASCANU R, RABINOWITZ N, et al. Overcoming catastrophic forgetting in neural networks[J]. Proceedings of the National Academy of Sciences of the United States of America, 2017, 114(13): 3521-3526.
[18] LI Z Z, HOIEM D. Learning without forgetting[C] //European Conference on Computer Vision(ECCV). Amsterdam, Netherlands: Springer, 2016: 614-629.
[19] CHAUDHRY A, RANZATO M, ROHRBACH M, et al. Efficient lifelong learning with A-GEM[EB/OL].(2019-01-09)[2025-05-22]. https://arxiv.org/abs/1812.00420
[20] YOON J, YANG E, LEE J, et al. Lifelong learning with dynamically expandable networks[EB/OL].(2018-06-11)[2025-05-22]. https://arxiv.org/abs/1708.01547
[21] ALJUNDI R, CHAKRAVARTY P, TUYTELAARS T. Expert gate: lifelong learning with a network of experts[C] //Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA: IEEE, 2017: 7120-7129.
[22] ROLNICK D, AHUJA A, SCHWARZ J, et al. Experience replay for continual learning[EB/OL].(2019-11-26)[2025-05-22]. https://arxiv.org/abs/1811.11682
[23] BUZZEGA P, BOSCHINI M, PORRELLO A, et al. Dark experience for general continual learning[C] // Proceedings of the 34th International Conference on Neural Information Processing Systems. Vancouver, Canada: ACM, 2020: 15920-15930.
[24] REBUFFI S A, KOLESNIKOV A, SPERL G, et al. iCaRL: Incremental classifier and representation learning[C] //2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Honolulu, USA: IEEE, 2017: 5533-5542.
[25] ZHU Z P, ZHAO S J, CHU C C, et al. FedPMR: federated personalized mixture representation for driver intention prediction[J]. IEEE Transactions on Intelligent Vehicles, 2025, 10(1): 627-640.
[26] HE Y T, CHEN Y Q, YANG X D, et al. Class-wise adaptive self-distillation for federated learning on Non-IID data[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2022, 36(11): 12967-12968.
[27] 姜慧, 何天流, 刘敏, 等. 面向异构流式数据的高性能联邦持续学习算法[J]. 通信学报, 2023, 44(5): 123-136. JIANG Hui, HE Tianliu, LIU Min, et al. High-performance federated continual learning algorithm for heterogeneous streaming data[J]. Journal on Commu-nications, 2023, 44(5): 123-136.
[28] DONG J H, WANG L X, FANG Z, et al. Federated class-incremental learning[C] //2022 IEEE/CVF Confe-rence on Computer Vision and Pattern Recognition(CVPR). New Orleans, USA: IEEE, 2022: 10154-10163.
[29] SAHA G, GARG I, ROY K. Gradient projection memory for continual learning[EB/OL].(2021-03-17)[2025-05-22]. https://arxiv.org/abs/2103.09762
[30] LUO P Y X, HAN R, ZHANG Q L, et al. FedKNOW: federated continual learning with signature task knowledge integration at edge[C] //2023 IEEE 39th International Conference on Data Engineering(ICDE). Anaheim, USA: IEEE, 2023: 341-354.
[31] SHOHAM N, AVIDOR T, KEREN A, et al. Overcoming forgetting in federated learning on Non-IID data[EB/OL].(2019-10-17)[2025-05-22]. https://arxiv.org/abs/1910.07796
[32] WU C H, WU F Z, QI T, et al. FedCL: federated contrastive learning for privacy-preserving recommendation[EB/OL].(2022-04-21)[2025-05-22]. https://arxiv.org/abs/2204.09850
[33] PIAO H M, WU Y C, WU D P, et al. Federated continual learning via prompt-based dual knowledge transfer[C] //Proceedings of the 41st International Conference on Machine Learning. Vienna, Austria: JMLR, 2024: 40725-40739.
[34] WANG Q, LIU B Y, LI Y W. Traceable federated continual learning[C] //2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). Seattle, USA: IEEE, 2024: 12872-12881.
[35] HE Y C, SHEN C Y, WANG X F, et al. FPPL: an efficient and Non-IID robust federated continual learning framework[C] //2024 IEEE International Conference on Big Data(BigData). Washington, DC, USA: IEEE, 2024: 3692-3701.
[36] YU H, YANG X, GAO X, et al. Personalized federated continual learning via multi granularity prompt[C] //Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. Barcelona, Spain: ACM, 2024: 4023-4034.
[37] WANG Z R, DAI Z H, PÓCZOS B, et al. Characterizing and avoiding negative transfer[C] //2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). Long Beach, USA: IEEE, 2020: 11285-11294.
[1] HUANG Fang, WANG Xin, GAO Guohai, SHEN Lingzhen, FU Xun, FANG Yu. Mining Top-k frequent patterns for graphs based on subjective and objective metrics [J]. Journal of Shandong University(Engineering Science), 2025, 55(6): 1-12.
[2] WANG Yuou, YUAN Yingchun, HE Zhenxue, HE Chen. University academic named entity recognition based on the fusion of multi-feature and multi-head self-attention mechanism [J]. Journal of Shandong University(Engineering Science), 2025, 55(6): 35-44.
[3] SHAO Mengwei, YUAN Shifei, ZHOU Hongzhi, WANG Naihua. Optimisation of finned tube structure based on BP neural network and genetic algorithm [J]. Journal of Shandong University(Engineering Science), 2025, 55(6): 76-82.
[4] DENG Bin, ZHANG Zongbao, ZHAO Wenmeng, LUO Xinhang, WU Qiuwei. Cloud-edge collaborative and graph neural network based load forecasting method for electric vehicle charging stations [J]. Journal of Shandong University(Engineering Science), 2025, 55(5): 62-69.
[5] DONG Mingshu, CHEN Liqi, MA Chuanyi, ZHANG Zhuhao, SUN Renjuan, GUAN Yanhua, ZHUANG Peizhi. Deep learning-based intelligent judgment for radar detection of pavement cracks [J]. Journal of Shandong University(Engineering Science), 2025, 55(3): 72-79.
[6] JIA Xuan, XU Jikai, REN Yijing, LIU Decai, XU Qiang, ZHANG Li. Calculation method of theoretical line loss in transformer districts based on sample expansion and data-driven [J]. Journal of Shandong University(Engineering Science), 2025, 55(3): 158-164.
[7] ZHU Ming, SHI Chenglong, LÜ Pan, LIU Xianrong, SUN Chi, CHEN Jiancheng, FAN Hongyun. Deformation prediction method and engineering application of deep foundation pit based on optimized LSTM method [J]. Journal of Shandong University(Engineering Science), 2025, 55(3): 141-148.
[8] Hong YU,Juan DU,Lin WEI,Li ZHANG. Data fitting method for electricity consumption of power market users considering behavioral characteristics [J]. Journal of Shandong University(Engineering Science), 2023, 53(4): 113-119.
[9] Haoyuan LI,Jingming YU,Guilin ZHANG,Bin ZHANG. Optimization of manufacturing parameters for optical fiber preform core based on intelligent algorithm [J]. Journal of Shandong University(Engineering Science), 2023, 53(4): 149-156.
[10] Yue YUAN,Yanli WANG,Kan LIU. Named entity recognition model based on dilated convolutional block architecture [J]. Journal of Shandong University(Engineering Science), 2022, 52(6): 105-114.
[11] DING Fei, JIANG Mingyan. Housing price prediction based on improved lion swarm algorithm and BP neural network model [J]. Journal of Shandong University(Engineering Science), 2021, 51(4): 8-16.
[12] YIN Xiaomin, MENG Xiangjian, HOU Kunming, CHEN Yaxiao, GAO Feng. Correction method for historical output data of photovoltaic power plant considering spatial correlation based on artificial neural network [J]. Journal of Shandong University(Engineering Science), 2021, 51(4): 118-123.
[13] LIAO Jinping, MO Yuchang, YAN Ke. Model and application of short-term electricity consumption forecast based on C-LSTM [J]. Journal of Shandong University(Engineering Science), 2021, 51(2): 90-97.
[14] HUO Bingqiang, ZHOU Tao, LU Huiling, DONG Yali, LIU Shan. Lung tumor benign-malignant classification based on multi-modal residual neural network and NRC algorithm [J]. Journal of Shandong University(Engineering Science), 2020, 50(6): 59-67.
[15] Zhiwei WANG,Nan GE,Chunwei LI. Fuzzy control of structure vibration mode based on BP neural network algorithm [J]. Journal of Shandong University(Engineering Science), 2020, 50(5): 13-19.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!