山东大学学报 (工学版) ›› 2024, Vol. 54 ›› Issue (4): 13-20.doi: 10.6040/j.issn.1672-3961.0.2023.163
王梅1,许传海2*,王伟东1,韩非3
WANG Mei1, XU Chuanhai2*, WANG Weidong1, HAN Fei3
摘要: 为提高多核学习对大规模及分布不均衡问题的处理能力,提出一种基于神经正切核草图的多核学习方法(neural tangent kernel sketch multiple kernel learning, NS-MKL )。应用神经正切核代替单层核函数作为多核学习基核函数,提高多核学习方法表示能力;使用神经正切核草图算法对神经正切核进行近似,减少神经正切核的特征数量和特征维度,提高多核学习方法计算效率;使用核目标对齐计算每个近似神经正切核的基核权重,根据权重进行多核线性组合,得到多核决策函数。在3个UCI数据集上对神经正线核(neural tangent kernel, NTK)核支持向量机(support vector machine, SVM)与传统核SVM进行比较分析,NTK核SVM比传统核SVM预测准确率最低提高1.9%,精度最低提高2.0%,召回率最低提高2.0%。在3个UCI数据集上对NS-MKL与传统核MKL进行比较分析,NS-MKL比应用传统核MKL预测准确率最低提高2.0%,运行时间最低减少9 s。NS-MKL能提高预测准确率,降低计算速度。
中图分类号:
[1] ZHANG T. An introduction to support vector machines and other kernel-based learning methods[J]. AI Magazine, 2001, 22(2): 103-104. [2] SOLICH P. Bayesian methods for support vector machines: Evidence and predictive class probabilities [J]. Machine Learning, 2002, 46(1): 21-52. [3] KLOFT M, BLANCHARD G. On the convergence rate of lp-norm multiple kernel learning [J]. Journal of Machine Learning Research, 2012(1): 2465-2502. [4] CRISTIANINI N, SHAWE-TAYLOR J, ELISSEEFF A, et al. On kernel-target alignment[C] // Proceedings of the 14th International Conference on Neural Information Processing Systems: Natural and Synthetic. Vancouver, Canada: MIT Press, 2001: 367-373. [5] WILLIAMS C. Computing with infinite networks[C] // Proceedings of the 9th International Conference on Neural Information Processing Systems. Denver, USA: MIT Press, 1996: 295-301. [6] LEE J, BAHRI Y, NOVAK R, et al. Deep neural networks as gaussian processes[C] // Proceedings of the 6th International Conference on Learning Representations. Vancouver, Canada: MIT Press, 2018: 1-17. [7] JACOT A, GABRIEL F, HONGLER C. Neural tangent kernel: convergence and generalization in neural networks[C] //Proceedings of the 32nd International Conference on Neural Information Processing Systems. Montreal, Canada: MIT Press, 2018: 8580-8589. [8] LEE J, XIAO L, SCHOENHOLZ S, et al. Wide neural networks of any depth evolve as linear models under gradient descent [EB/OL].(2019-08-18)[2019-12-08]. https://arxiv.org/abs/1902.06720. [9] 王梅,许传海,刘勇. 基于神经正切核的多核学习方法[J].计算机应用,2021,41(12): 3462-3467. WANG Mei, XU Chuanhai, LIU Yong. Multi-kernel learning method based on neural tangent kernel[J]. Journal of Computer Applications, 2021, 41(12): 3462-3467. [10] 王梅, 宋晓晖, 刘勇, 等.神经正切核K-Means聚类[J].计算机应用, 2022, 42(11): 3330-3336. WAND Mei, SONG Xiaohui, LIU Yong, et al. Neural tangent kernel K-Means clustering[J]. Journal of Computer Applications, 2022, 42(11): 3330-3336. [11] ARORA S, DU S S, HU W, et al. On exact computation with an infinitely wide neural net [EB/OL].(2019-04-26)[2019-11-04]. https://arxiv. org/abs/1904.11955. [12] CHEN L, XU S. Deep neural tangent kernel and laplace kernel have the same RKHS[EB/OL].(2020-09-22)[2021-03-18]. https://doi.org/10.48550/ arXiv.2009.10683. [13] 张琳, 汪廷华, 周慧颖.基于群智能算法的SVR参数优化研究进展[J].计算机工程与应用, 2021, 57(16): 50-64. ZHANG Lin, WANG Tinghua, ZHOU Huiying. Research progress on parameter optimization of SVR based on swarm intelligence algorithm[J]. Computer Engineering and Applications, 2021, 57(16): 50-64. [14] 祁祥洲, 邢红杰. 基于中心核对齐的多核单类支持向量机[J]. 计算机应用, 2022, 42(2): 349-356. QI Xiangzhou, XING Hongjie. Centered kernel alignment based multiple kernel one-class support vector machine[J]. Journal of Computer Applications, 2022, 42(2): 349-356. [15] LANCKRIET G, CRISTIANINI N, BARTLETT P L, et al. Learning the kernel matrix with semidefinite programming[J]. Journal of Machine Learning Research, 2002, 5(1): 27-72. [16] 侯能干. 基于特征融合和多核学习的行人检测方法研究[D]. 合肥:合肥工业大学, 2014. HOU Nenggan. Research on pedestrian detection methods based on feature fusion and multi-core learning[D]. Hefei: Hefei University of Technology, 2014. [17] GONEN M, ALPAYDIN E. Localized multiple kernel learning[C] //Proceedings of the 25th International conference on Machine learning, Helsinki, Finland: MIT Press, 2008: 352-359. [18] 梁俊. 基于多核学习支持向量机的货币识别[D]. 长沙:中南大学, 2014. LIANG Jun. Currency Recognition Based on Multikernel Learning Support Vector Machines[D]. Changsha: Central South University, 2014. [19] HE Q, ZHANG Q, WANG H. Kernel-target alignment based multiple kernel one-class support vector machine[C] //Proceedings of the 2019 IEEE International Conference on Systems, Man and Cybernetics. Bari, Italy: IEEE, 2019: 2083-2088. [20] 邵朝, 李强. 基于特征加权的多核支持向量机[J]. 西安邮电大学学报, 2017, 22(2): 84-88. SHAO Chao, LI Qiang. Multi-kernel support vector machines based on feature weighting[J]. Journal of Xi'an University of Posts and Telecommunications, 2017, 22(2): 84-88. [21] 贾涵, 连晓峰, 潘兵. 基于模糊松弛约束的外观缺陷多核学习技术[J]. 测控技术, 2019, 38(8): 43-47. JIA Han, LIAN Xiaofeng, PAN Bing. Appearance defects multiple kernel learning technology based on fuzzy relaxation constraints [J]. Measurement and Control Technology, 2019, 38(8): 43-47. [22] 王梅, 薛成龙, 张强. 基于秩空间差异的多核组合方法[J]. 山东大学学报(工学版), 2021, 51(1): 108-113. WANG Mei, XUE Chenglong, ZHANG Qiang. Multi-kernel combination method based on rank spatial difference[J]. Journal of Shandong University(Engineering Science), 2021, 51(1): 108-113. [23] 李湘眷, 孙显, 王宏琦. 基于多核学习的高分辨率遥感图像目标检测方法[J]. 测绘科学, 2013, 38(5): 84-87. LI Xiangjuan, SUN Xian, WANG Hongqi. Target detection method for high-resolution remote sensing images based on multi-core learning [J]. Science of Surveying and Mapping, 2013, 38(5): 84-87. [24] NOVAK R, XIAO L, HRON J, et al. Neural tangents: fast and easy infinite neural networks in Python[EB/OL].(2019-12-05)[2019-12-05]. https://doi.org/10.48550/arXiv.1912.02803. [25] LIU X, LEI W, ZHU X, et al. Absent multiple kernel learning algorithms [J]. IEEE Transactions on Pattern Analysis and Machine Intelligencev, 2019, 42(6):1303-1316. [26] MITCHEL A P, OVENEKE M C, H SAHLI. SVRG-MKL: a fast and scalable multiple kernel learning solution for features combination in multi-class classification problems[J]. IEEE Transactions on Neural Networks and Learning Systems, 2020, 31(5): 1710-1723. [27] WANG X, WANG S, DU Y, et al. Minimum class variance multiple kernel learning [J]. Knowledge-Based Systems, 2020, 208(5): 106469. [28] RAHIMI A, RECHT B. Random features for large-scale Kernel machines[EB/OL].(2019-05-28)[2019-05-28]. https://doi.org/10.48550/arXiv.2209.01958. [29] HAN I, AVRON H, SHIN J. Polynomial Tensor Sketch for Element-wise Function of Low-Rank Matrix[C] // Proceedings of the 37th International Conference on Machine Learning. Vienna, Austria: MIT Press, 2020: 3942-3951. [30] CHO Y, SAUL L. Kernel methods for deep learning[C] // In Advances in Neural Information Processing Systems 22. Vancouver, Canada: Red Hook: 2009:342-350. [31] BIETTI A, MAIRAL J. On the inductive bias of neural tangent kernels[EB/OL].(2019-05-29)[2019-10-31]. ttp://arxiv.org/abs/1905.12173. [32] ZANDIEH A, HAN I, AVORN H, et al. Scaling neural tangent kernels via sketching and random features [J]. Advances in Neural Information Processing Systems, 2021, 34: 1062-1073. [33] PHAN N, PAGH R. Fast and scalable polynomial kernels via explicit feature maps[C] //Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Chicago, USA: ACM, 2013: 239-247. [34] AIOLLI F, DONINI M. EasyMKL: a scalable multiple kernel learning algorithm[J]. Neurocomputing, 2015, 169: 215-224. [35] TANABE H, HO T B, Nguyen C H, et al. Simple but effective methods for combining kernels in computational biology[C] //Proceedings of the 2008 IEEE International Conference on Research, Innovation and Vision for the Future in Computing and Communication Technologies. Ho Chi Minh City, Vietnam: IEEE, 2008: 71-78. |
[1] | 王梅,薛成龙,张强. 基于秩空间差异的多核组合方法[J]. 山东大学学报 (工学版), 2021, 51(1): 108-113. |
|