您的位置：山东大学 -> 科技期刊社 -> 《山东大学学报（工学版）》

山东大学学报 (工学版) ›› 2024, Vol. 54 ›› Issue (4): 13-20.doi: 10.6040/j.issn.1672-3961.0.2023.163

• 机器学习与数据挖掘 • 上一篇    

基于神经正切核草图的多核学习方法

王梅1,许传海2*,王伟东1,韩非3   

  1. 1.东北石油大学计算机与信息技术学院, 黑龙江 大庆 163318;2.新疆理工学院信息工程学院, 新疆 阿克苏 843100;3.东北石油大学人工智能能源研究院, 黑龙江 大庆 163318
  • 发布日期:2024-08-20
  • 作者简介:王梅(1975— ),女,河北安国人,教授,硕士生导师,博士,主要研究方向为机器学习、核方法、模型选择等. E-mail: wangmei@nepu.edu.cn. *通信作者简介:许传海(1998— ),男,黑龙江鸡西人,助教,硕士,主要研究方向为机器学习、深度核学习. E-mail: 2023266@xjit.edu.cn
  • 基金资助:
    国家自然科学基金资助项目(51774090,62073070);黑龙江省博士后科研启动金资助项目(LBH-Q20080);黑龙江省研究生精品课程建设项目(15141220103)

Multi-kernel learning method based on neural tangent kernel sketch

WANG Mei1, XU Chuanhai2*, WANG Weidong1, HAN Fei3   

  1. 1. School of Computer and Information Technology, Northeast Petroleum University, Daqing 163318, Heilongjiang, China;
    2. College of Information Engineering, Xinjiang Institute of Technology, Aksu 843100, Xinjiang, China;
    3. Artificial Intelligence Energy Research Institute, Northeast Petroleum University, Daqing 163318, Heilongjiang, China
  • Published:2024-08-20

摘要： 为提高多核学习对大规模及分布不均衡问题的处理能力,提出一种基于神经正切核草图的多核学习方法(neural tangent kernel sketch multiple kernel learning, NS-MKL )。应用神经正切核代替单层核函数作为多核学习基核函数,提高多核学习方法表示能力;使用神经正切核草图算法对神经正切核进行近似,减少神经正切核的特征数量和特征维度,提高多核学习方法计算效率;使用核目标对齐计算每个近似神经正切核的基核权重,根据权重进行多核线性组合,得到多核决策函数。在3个UCI数据集上对神经正线核(neural tangent kernel, NTK)核支持向量机(support vector machine, SVM)与传统核SVM进行比较分析,NTK核SVM比传统核SVM预测准确率最低提高1.9%,精度最低提高2.0%,召回率最低提高2.0%。在3个UCI数据集上对NS-MKL与传统核MKL进行比较分析,NS-MKL比应用传统核MKL预测准确率最低提高2.0%,运行时间最低减少9 s。NS-MKL能提高预测准确率,降低计算速度。

关键词: 多核学习, 神经正切核, 核目标对齐, 反余弦核, 草图算法

中图分类号: 

  • TP391
[1] ZHANG T. An introduction to support vector machines and other kernel-based learning methods[J]. AI Magazine, 2001, 22(2): 103-104.
[2] SOLICH P. Bayesian methods for support vector machines: Evidence and predictive class probabilities [J]. Machine Learning, 2002, 46(1): 21-52.
[3] KLOFT M, BLANCHARD G. On the convergence rate of lp-norm multiple kernel learning [J]. Journal of Machine Learning Research, 2012(1): 2465-2502.
[4] CRISTIANINI N, SHAWE-TAYLOR J, ELISSEEFF A, et al. On kernel-target alignment[C] // Proceedings of the 14th International Conference on Neural Information Processing Systems: Natural and Synthetic. Vancouver, Canada: MIT Press, 2001: 367-373.
[5] WILLIAMS C. Computing with infinite networks[C] // Proceedings of the 9th International Conference on Neural Information Processing Systems. Denver, USA: MIT Press, 1996: 295-301.
[6] LEE J, BAHRI Y, NOVAK R, et al. Deep neural networks as gaussian processes[C] // Proceedings of the 6th International Conference on Learning Representations. Vancouver, Canada: MIT Press, 2018: 1-17.
[7] JACOT A, GABRIEL F, HONGLER C. Neural tangent kernel: convergence and generalization in neural networks[C] //Proceedings of the 32nd International Conference on Neural Information Processing Systems. Montreal, Canada: MIT Press, 2018: 8580-8589.
[8] LEE J, XIAO L, SCHOENHOLZ S, et al. Wide neural networks of any depth evolve as linear models under gradient descent [EB/OL].(2019-08-18)[2019-12-08]. https://arxiv.org/abs/1902.06720.
[9] 王梅,许传海,刘勇. 基于神经正切核的多核学习方法[J].计算机应用,2021,41(12): 3462-3467. WANG Mei, XU Chuanhai, LIU Yong. Multi-kernel learning method based on neural tangent kernel[J]. Journal of Computer Applications, 2021, 41(12): 3462-3467.
[10] 王梅, 宋晓晖, 刘勇, 等.神经正切核K-Means聚类[J].计算机应用, 2022, 42(11): 3330-3336. WAND Mei, SONG Xiaohui, LIU Yong, et al. Neural tangent kernel K-Means clustering[J]. Journal of Computer Applications, 2022, 42(11): 3330-3336.
[11] ARORA S, DU S S, HU W, et al. On exact computation with an infinitely wide neural net [EB/OL].(2019-04-26)[2019-11-04]. https://arxiv. org/abs/1904.11955.
[12] CHEN L, XU S. Deep neural tangent kernel and laplace kernel have the same RKHS[EB/OL].(2020-09-22)[2021-03-18]. https://doi.org/10.48550/ arXiv.2009.10683.
[13] 张琳, 汪廷华, 周慧颖.基于群智能算法的SVR参数优化研究进展[J].计算机工程与应用, 2021, 57(16): 50-64. ZHANG Lin, WANG Tinghua, ZHOU Huiying. Research progress on parameter optimization of SVR based on swarm intelligence algorithm[J]. Computer Engineering and Applications, 2021, 57(16): 50-64.
[14] 祁祥洲, 邢红杰. 基于中心核对齐的多核单类支持向量机[J]. 计算机应用, 2022, 42(2): 349-356. QI Xiangzhou, XING Hongjie. Centered kernel alignment based multiple kernel one-class support vector machine[J]. Journal of Computer Applications, 2022, 42(2): 349-356.
[15] LANCKRIET G, CRISTIANINI N, BARTLETT P L, et al. Learning the kernel matrix with semidefinite programming[J]. Journal of Machine Learning Research, 2002, 5(1): 27-72.
[16] 侯能干. 基于特征融合和多核学习的行人检测方法研究[D]. 合肥:合肥工业大学, 2014. HOU Nenggan. Research on pedestrian detection methods based on feature fusion and multi-core learning[D]. Hefei: Hefei University of Technology, 2014.
[17] GONEN M, ALPAYDIN E. Localized multiple kernel learning[C] //Proceedings of the 25th International conference on Machine learning, Helsinki, Finland: MIT Press, 2008: 352-359.
[18] 梁俊. 基于多核学习支持向量机的货币识别[D]. 长沙:中南大学, 2014. LIANG Jun. Currency Recognition Based on Multikernel Learning Support Vector Machines[D]. Changsha: Central South University, 2014.
[19] HE Q, ZHANG Q, WANG H. Kernel-target alignment based multiple kernel one-class support vector machine[C] //Proceedings of the 2019 IEEE International Conference on Systems, Man and Cybernetics. Bari, Italy: IEEE, 2019: 2083-2088.
[20] 邵朝, 李强. 基于特征加权的多核支持向量机[J]. 西安邮电大学学报, 2017, 22(2): 84-88. SHAO Chao, LI Qiang. Multi-kernel support vector machines based on feature weighting[J]. Journal of Xi'an University of Posts and Telecommunications, 2017, 22(2): 84-88.
[21] 贾涵, 连晓峰, 潘兵. 基于模糊松弛约束的外观缺陷多核学习技术[J]. 测控技术, 2019, 38(8): 43-47. JIA Han, LIAN Xiaofeng, PAN Bing. Appearance defects multiple kernel learning technology based on fuzzy relaxation constraints [J]. Measurement and Control Technology, 2019, 38(8): 43-47.
[22] 王梅, 薛成龙, 张强. 基于秩空间差异的多核组合方法[J]. 山东大学学报(工学版), 2021, 51(1): 108-113. WANG Mei, XUE Chenglong, ZHANG Qiang. Multi-kernel combination method based on rank spatial difference[J]. Journal of Shandong University(Engineering Science), 2021, 51(1): 108-113.
[23] 李湘眷, 孙显, 王宏琦. 基于多核学习的高分辨率遥感图像目标检测方法[J]. 测绘科学, 2013, 38(5): 84-87. LI Xiangjuan, SUN Xian, WANG Hongqi. Target detection method for high-resolution remote sensing images based on multi-core learning [J]. Science of Surveying and Mapping, 2013, 38(5): 84-87.
[24] NOVAK R, XIAO L, HRON J, et al. Neural tangents: fast and easy infinite neural networks in Python[EB/OL].(2019-12-05)[2019-12-05]. https://doi.org/10.48550/arXiv.1912.02803.
[25] LIU X, LEI W, ZHU X, et al. Absent multiple kernel learning algorithms [J]. IEEE Transactions on Pattern Analysis and Machine Intelligencev, 2019, 42(6):1303-1316.
[26] MITCHEL A P, OVENEKE M C, H SAHLI. SVRG-MKL: a fast and scalable multiple kernel learning solution for features combination in multi-class classification problems[J]. IEEE Transactions on Neural Networks and Learning Systems, 2020, 31(5): 1710-1723.
[27] WANG X, WANG S, DU Y, et al. Minimum class variance multiple kernel learning [J]. Knowledge-Based Systems, 2020, 208(5): 106469.
[28] RAHIMI A, RECHT B. Random features for large-scale Kernel machines[EB/OL].(2019-05-28)[2019-05-28]. https://doi.org/10.48550/arXiv.2209.01958.
[29] HAN I, AVRON H, SHIN J. Polynomial Tensor Sketch for Element-wise Function of Low-Rank Matrix[C] // Proceedings of the 37th International Conference on Machine Learning. Vienna, Austria: MIT Press, 2020: 3942-3951.
[30] CHO Y, SAUL L. Kernel methods for deep learning[C] // In Advances in Neural Information Processing Systems 22. Vancouver, Canada: Red Hook: 2009:342-350.
[31] BIETTI A, MAIRAL J. On the inductive bias of neural tangent kernels[EB/OL].(2019-05-29)[2019-10-31]. ttp://arxiv.org/abs/1905.12173.
[32] ZANDIEH A, HAN I, AVORN H, et al. Scaling neural tangent kernels via sketching and random features [J]. Advances in Neural Information Processing Systems, 2021, 34: 1062-1073.
[33] PHAN N, PAGH R. Fast and scalable polynomial kernels via explicit feature maps[C] //Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Chicago, USA: ACM, 2013: 239-247.
[34] AIOLLI F, DONINI M. EasyMKL: a scalable multiple kernel learning algorithm[J]. Neurocomputing, 2015, 169: 215-224.
[35] TANABE H, HO T B, Nguyen C H, et al. Simple but effective methods for combining kernels in computational biology[C] //Proceedings of the 2008 IEEE International Conference on Research, Innovation and Vision for the Future in Computing and Communication Technologies. Ho Chi Minh City, Vietnam: IEEE, 2008: 71-78.
[1] 王梅,薛成龙,张强. 基于秩空间差异的多核组合方法[J]. 山东大学学报 (工学版), 2021, 51(1): 108-113.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
版权所有 © 2018 《山东大学学报(工学版)》编辑部　
地址: 济南市山大南路27号（250100）　 电话: 0531-88366735　 E-mail: xbgxb@sdu.edu.cn
本系统由北京玛格泰克科技发展有限公司设计开发