基于课程正则化的物理信息神经网络渐进式训练策略

doi:10.6040/j.issn.1672-3961.0.2023.155

山东大学学报 (工学版) ›› 2024, Vol. 54 ›› Issue (1): 11-24.doi: 10.6040/j.issn.1672-3961.0.2023.155

• 机器学习与数据挖掘 • 上一篇

基于课程正则化的物理信息神经网络渐进式训练策略

范黎林¹,刘士豪¹,李源^1,2*,毛文涛^1,2,陈宗涛¹

1.河南师范大学计算机与信息工程学院, 河南新乡453007;2.智慧商务与物联网技术河南省工程实验室, 河南新乡453007

发布日期:2024-02-01
作者简介:范黎林(1970— ),男,河南周口人,副教授,硕士生导师,博士,主要研究方向为智能商务、智能交通. E-mail:fanlilin@htu.edu.cn. *通信作者简介:李源(1989— ),女,河南新乡人,副教授,硕士生导师,博士,主要研究方向为计算力学、机器学习、边界元法. E-mail:liyuan2015097@163.com
基金资助:
国家自然科学基金资助项目(11702087,U1704158);河南省科技攻关重点项目(212102210103)

Progressive training strategy of physics-informed neural networks based on curriculum regularization

FAN Lilin¹, LIU Shihao¹, LI Yuan^1,2*, MAO Wentao^1,2, CHEN Zongtao¹

1. College of Computer and Information Engineering, Henan Normal University, Xinxiang 453007, Henan, China;
2. Engineering Lab of Intelligence Business &
Internet of Things, Xinxiang 453007, Henan, China

Published:2024-02-01

摘要/Abstract

摘要： 为降低物理信息神经网络(physics-informed neural networks, PINN)优化目标函数的复杂性和训练难度,提出一种基于课程正则化渐进式训练策略,在该策略中基于课程学习思想动态调整损失函数,使正则化项中偏微分方程所表征的物理信息从较平稳状态逐步过渡到变化剧烈状态,降低任务学习难度;加强损失函数中初始条件和边界条件部分的数据约束,平衡数据部分和物理信息部分损失;采用固定步长指数衰减学习率进行优化,尽可能避免目标函数陷入局部最小值。通过波动和热传导两类偏微分方程进行试验对比和分析,结果表明计算效率能够提升约50%,预测精度能够提高0.5~1个数量级。所提出方法可以有效提高PINN的数值稳定性和预测精度,加快PINN在复杂物理场学习任务中收敛速率。

关键词: 物理信息神经网络, 课程学习, 损失函数, 偏微分方程, 数值稳定性

中图分类号:

TP181

范黎林,刘士豪,李源,毛文涛,陈宗涛. 基于课程正则化的物理信息神经网络渐进式训练策略[J]. 山东大学学报 (工学版), 2024, 54(1): 11-24.

FAN Lilin, LIU Shihao, LI Yuan, MAO Wentao, CHEN Zongtao. Progressive training strategy of physics-informed neural networks based on curriculum regularization[J]. Journal of Shandong University(Engineering Science), 2024, 54(1): 11-24.

参考文献

[1] ALTERMAN Z, KARAL F C. Propagation of elastic waves in layered media by finite difference methods[J]. Bulletin of the Seismological Society of America, 1968, 58(1): 367-398.
[2] KUHLEMEYER R L, LYSMER J. Finite element method accuracy for wave propagation problems[J]. Journal of the Soil Mechanics and Foundations Division, 1973, 99(5): 421-427.
[3] JASAK H. Error analysis and estimation for the finite volume method with applications to fluid flows[D]. London, UK: Imperial College London, 1996.
[4] CHENG A H D, CHENG D T. Heritage and early history of the boundary element method[J]. Engineering Analysis with Boundary Elements, 2005, 29(3): 268-302.
[5] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. Imagenet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6): 84-90.
[6] LI H, Deep learning for natural language processing: advantages and challenges[J]. National Science Review, 2018, 5(1): 24-26.
[7] WANG X G, DENG X B, FU Q, et al. A weakly-supervised framework for COVID-19 classification and lesion localization from chest CT[J]. IEEE Transactions on Medical Imaging, 2020, 39(8): 2615-2625.
[8] LI Y, MAO W T, WANG G S, et al. A general-purpose machine learning framework for predicting singular integrals in boundary element method[J]. Engineering Analysis with Boundary Elements, 2020, 117: 41-56.
[9] RAISSI M, PERDIKARIS P, KARNIADAKIS G E. Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations[J]. Journal of Computational Physics, 2019, 378: 686-707.
[10] JIANG X C, WANG H, LI Y. A physics-data-driven bayesian method for heat conduction problems[EB/OL].(2021-09-02)[2023-07-03].https://arxiv.org/abs/2109.00996.html
[11] WANG H P, LIU Y, WANG S Z. Dense velocity reconstruction from particle image velocimetry/particle tracking velocimetry using a physics-informed neural network[J]. Physics of Fluids, 2022, 34(1): 017116.
[12] ZENG S J, ZHANG Z, ZOU Q S. Adaptive deep neural networks methods for high-dimensional partial differential equations[J]. Journal of Computational Physics, 2022, 463: 111232.
[13] DONG S C, YANG J L. Numerical approximation of partial differential equations by a variable projection method with artificial neural networks[J]. Computer Methods in Applied Mechanics and Engineering, 2022, 398: 115284.
[14] JAGTAP A D, KAWAGUCHI K J, KARNIADAKIS G E. Adaptive activation functions accelerate convergence in deep and physics-informed neural networks[J]. Journal of Computational Physics, 2020, 404: 109136.
[15] JAGTAP A D, KHARAZMI E, KARNIADAKIS G E. Conservative physics-informed neural networks on discrete domains for conservation laws: applications to forward and inverse problems[J]. Computer Methods in Applied Mechanics and Engineering, 2020, 365: 113028.
[16] ZOBEIRY N, HUMFELD K D. A physics-informed machine learning approach for solving heat transfer equation in advanced manufacturing and engineering applications[J]. Engineering Applications of Artificial Intelligence, 2021, 101: 104232.
[17] WANG S F, TENG Y J, PERDIKARIS P. Under-standing and mitigating gradient flow pathologies in physics-informed neural networks[J]. SIAM Journal on Scientific Computing, 2021, 43(5): 3055-3081.
[18] MOSELEY B, MARKHAM A, NISSEN-MEYER T. Finite basis physics-informed neural networks(FBPINNs): a scalable domain decomposition approach for solving differential equations[J]. Advances in Computational Mathematics, 2023, 49(4): 62.
[19] LI J H, CHEN J C, LI B. Gradient-optimized physics-informed neural networks(GOPINNs): a deep learning method for solving the complex modified KdV equation[J]. Nonlinear Dynamics, 2022, 107: 781-792.
[20] KHARAZMI E, ZHANG Z, KARNIADKIS G E. Variational physics-informed neural networks for solving partial differential equations[EB/OL].(2019-11-27)[2023-07-03].https://arxiv.org/abs/1912.00873.html
[21] RAISSI M, YAZDANI A, KARNIADAKIS G E. Hidden fluid mechanics: learning velocity and pressure fields from flow visualizations[J]. Science, 2020, 367(6481): 1026-1030.
[22] LU L, MENG X H, MAO Z P, et al. DeepXDE: a deep learning library for solving differential equations[J]. SIAM Review, 2021, 63(1): 208-228.
[23] XU K, DARVE E. ADCME: learning spatially-varying physical fields using deep neural networks[EB/OL].(2020-11-24)[2023-07-03]. https://arxiv.org/abs/2011.11955.html.
[24] HAGHIGHAT E, JUANES R. SciANN: A Keras/TensorFlow wrapper for scientific computations and physics-informed deep learning using artificial neural networks[J]. Computer Methods in Applied Mechanics and Engineering, 2021, 373: 113552.
[25] KRISHNAPRIYAN A, GHOLAMI A, ZHE S, et al. Characterizing possible failure modes in physics-informed neural networks[J]. Advances in Neural Information Processing Systems, 2021, 34: 26548-26560.
[26] EPELBAUM T. Deep learning: technical introduction[EB/OL].(2017-09-05)[2023-07-03]. https://arxiv.org/abs/1709.01412.html
[27] BAYDIN A G, PEARLMUTTER B A, RADUL A A, et al. Automatic differentiation in machine learning: a survey[J]. Journal of Marchine Learning Research, 2018, 18: 1-43.
[28] BENGIO Y, LOURADOUR J, COLLOBERT R. Curriculum learning[C] // Proceedings of the 26th Annual International Conference on Machine Learning. New York, USA: Association for Computing Machinery, 2009: 41-48.
[29] KINGMA D P, BA J. Adam: a method for stochastic optimization[EB/OL].(2014-12-22)[2023-07-03]. https://arxiv.org/abs/1412.6980.html.
[30] BERAHAS A S, NOCEDAL J, TAKAC M. A multi-batch L-BFGS method for machine learning[J]. Advances in Neural Information Processing Systems, 2016, 29: 1055-1063.

多维度评价

Viewed

Full text

Abstract

Cited

Shared

Discussed

基于课程正则化的物理信息神经网络渐进式训练策略

Progressive training strategy of physics-informed neural networks based on curriculum regularization

PDF (PC)

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 2

多维度评价

本文评价

推荐阅读 0

[1]	江珊珊,杨静,范丽亚. 基于PDEs的图像特征提取方法[J]. 山东大学学报(工学版), 2018, 48(4): 27-36.
[2]	戚世乐,王美清. 自适应分割弱边缘的活动轮廓模型[J]. 山东大学学报(工学版), 2013, 43(6): 17-20.