Journal of Shandong University(Engineering Science) ›› 2024, Vol. 54 ›› Issue (4): 13-20.doi: 10.6040/j.issn.1672-3961.0.2023.163

• Machine Learning & Data Mining • Previous Articles     Next Articles

Multi-kernel learning method based on neural tangent kernel sketch

WANG Mei1, XU Chuanhai2*, WANG Weidong1, HAN Fei3   

  1. 1. School of Computer and Information Technology, Northeast Petroleum University, Daqing 163318, Heilongjiang, China;
    2. College of Information Engineering, Xinjiang Institute of Technology, Aksu 843100, Xinjiang, China;
    3. Artificial Intelligence Energy Research Institute, Northeast Petroleum University, Daqing 163318, Heilongjiang, China
  • Published:2024-08-20

PDF (PC)

21

CLC Number: 

  • TP391
[1] ZHANG T. An introduction to support vector machines and other kernel-based learning methods[J]. AI Magazine, 2001, 22(2): 103-104.
[2] SOLICH P. Bayesian methods for support vector machines: Evidence and predictive class probabilities [J]. Machine Learning, 2002, 46(1): 21-52.
[3] KLOFT M, BLANCHARD G. On the convergence rate of lp-norm multiple kernel learning [J]. Journal of Machine Learning Research, 2012(1): 2465-2502.
[4] CRISTIANINI N, SHAWE-TAYLOR J, ELISSEEFF A, et al. On kernel-target alignment[C] // Proceedings of the 14th International Conference on Neural Information Processing Systems: Natural and Synthetic. Vancouver, Canada: MIT Press, 2001: 367-373.
[5] WILLIAMS C. Computing with infinite networks[C] // Proceedings of the 9th International Conference on Neural Information Processing Systems. Denver, USA: MIT Press, 1996: 295-301.
[6] LEE J, BAHRI Y, NOVAK R, et al. Deep neural networks as gaussian processes[C] // Proceedings of the 6th International Conference on Learning Representations. Vancouver, Canada: MIT Press, 2018: 1-17.
[7] JACOT A, GABRIEL F, HONGLER C. Neural tangent kernel: convergence and generalization in neural networks[C] //Proceedings of the 32nd International Conference on Neural Information Processing Systems. Montreal, Canada: MIT Press, 2018: 8580-8589.
[8] LEE J, XIAO L, SCHOENHOLZ S, et al. Wide neural networks of any depth evolve as linear models under gradient descent [EB/OL].(2019-08-18)[2019-12-08]. https://arxiv.org/abs/1902.06720.
[9] 王梅,许传海,刘勇. 基于神经正切核的多核学习方法[J].计算机应用,2021,41(12): 3462-3467. WANG Mei, XU Chuanhai, LIU Yong. Multi-kernel learning method based on neural tangent kernel[J]. Journal of Computer Applications, 2021, 41(12): 3462-3467.
[10] 王梅, 宋晓晖, 刘勇, 等.神经正切核K-Means聚类[J].计算机应用, 2022, 42(11): 3330-3336. WAND Mei, SONG Xiaohui, LIU Yong, et al. Neural tangent kernel K-Means clustering[J]. Journal of Computer Applications, 2022, 42(11): 3330-3336.
[11] ARORA S, DU S S, HU W, et al. On exact computation with an infinitely wide neural net [EB/OL].(2019-04-26)[2019-11-04]. https://arxiv. org/abs/1904.11955.
[12] CHEN L, XU S. Deep neural tangent kernel and laplace kernel have the same RKHS[EB/OL].(2020-09-22)[2021-03-18]. https://doi.org/10.48550/ arXiv.2009.10683.
[13] 张琳, 汪廷华, 周慧颖.基于群智能算法的SVR参数优化研究进展[J].计算机工程与应用, 2021, 57(16): 50-64. ZHANG Lin, WANG Tinghua, ZHOU Huiying. Research progress on parameter optimization of SVR based on swarm intelligence algorithm[J]. Computer Engineering and Applications, 2021, 57(16): 50-64.
[14] 祁祥洲, 邢红杰. 基于中心核对齐的多核单类支持向量机[J]. 计算机应用, 2022, 42(2): 349-356. QI Xiangzhou, XING Hongjie. Centered kernel alignment based multiple kernel one-class support vector machine[J]. Journal of Computer Applications, 2022, 42(2): 349-356.
[15] LANCKRIET G, CRISTIANINI N, BARTLETT P L, et al. Learning the kernel matrix with semidefinite programming[J]. Journal of Machine Learning Research, 2002, 5(1): 27-72.
[16] 侯能干. 基于特征融合和多核学习的行人检测方法研究[D]. 合肥:合肥工业大学, 2014. HOU Nenggan. Research on pedestrian detection methods based on feature fusion and multi-core learning[D]. Hefei: Hefei University of Technology, 2014.
[17] GONEN M, ALPAYDIN E. Localized multiple kernel learning[C] //Proceedings of the 25th International conference on Machine learning, Helsinki, Finland: MIT Press, 2008: 352-359.
[18] 梁俊. 基于多核学习支持向量机的货币识别[D]. 长沙:中南大学, 2014. LIANG Jun. Currency Recognition Based on Multikernel Learning Support Vector Machines[D]. Changsha: Central South University, 2014.
[19] HE Q, ZHANG Q, WANG H. Kernel-target alignment based multiple kernel one-class support vector machine[C] //Proceedings of the 2019 IEEE International Conference on Systems, Man and Cybernetics. Bari, Italy: IEEE, 2019: 2083-2088.
[20] 邵朝, 李强. 基于特征加权的多核支持向量机[J]. 西安邮电大学学报, 2017, 22(2): 84-88. SHAO Chao, LI Qiang. Multi-kernel support vector machines based on feature weighting[J]. Journal of Xi'an University of Posts and Telecommunications, 2017, 22(2): 84-88.
[21] 贾涵, 连晓峰, 潘兵. 基于模糊松弛约束的外观缺陷多核学习技术[J]. 测控技术, 2019, 38(8): 43-47. JIA Han, LIAN Xiaofeng, PAN Bing. Appearance defects multiple kernel learning technology based on fuzzy relaxation constraints [J]. Measurement and Control Technology, 2019, 38(8): 43-47.
[22] 王梅, 薛成龙, 张强. 基于秩空间差异的多核组合方法[J]. 山东大学学报(工学版), 2021, 51(1): 108-113. WANG Mei, XUE Chenglong, ZHANG Qiang. Multi-kernel combination method based on rank spatial difference[J]. Journal of Shandong University(Engineering Science), 2021, 51(1): 108-113.
[23] 李湘眷, 孙显, 王宏琦. 基于多核学习的高分辨率遥感图像目标检测方法[J]. 测绘科学, 2013, 38(5): 84-87. LI Xiangjuan, SUN Xian, WANG Hongqi. Target detection method for high-resolution remote sensing images based on multi-core learning [J]. Science of Surveying and Mapping, 2013, 38(5): 84-87.
[24] NOVAK R, XIAO L, HRON J, et al. Neural tangents: fast and easy infinite neural networks in Python[EB/OL].(2019-12-05)[2019-12-05]. https://doi.org/10.48550/arXiv.1912.02803.
[25] LIU X, LEI W, ZHU X, et al. Absent multiple kernel learning algorithms [J]. IEEE Transactions on Pattern Analysis and Machine Intelligencev, 2019, 42(6):1303-1316.
[26] MITCHEL A P, OVENEKE M C, H SAHLI. SVRG-MKL: a fast and scalable multiple kernel learning solution for features combination in multi-class classification problems[J]. IEEE Transactions on Neural Networks and Learning Systems, 2020, 31(5): 1710-1723.
[27] WANG X, WANG S, DU Y, et al. Minimum class variance multiple kernel learning [J]. Knowledge-Based Systems, 2020, 208(5): 106469.
[28] RAHIMI A, RECHT B. Random features for large-scale Kernel machines[EB/OL].(2019-05-28)[2019-05-28]. https://doi.org/10.48550/arXiv.2209.01958.
[29] HAN I, AVRON H, SHIN J. Polynomial Tensor Sketch for Element-wise Function of Low-Rank Matrix[C] // Proceedings of the 37th International Conference on Machine Learning. Vienna, Austria: MIT Press, 2020: 3942-3951.
[30] CHO Y, SAUL L. Kernel methods for deep learning[C] // In Advances in Neural Information Processing Systems 22. Vancouver, Canada: Red Hook: 2009:342-350.
[31] BIETTI A, MAIRAL J. On the inductive bias of neural tangent kernels[EB/OL].(2019-05-29)[2019-10-31]. ttp://arxiv.org/abs/1905.12173.
[32] ZANDIEH A, HAN I, AVORN H, et al. Scaling neural tangent kernels via sketching and random features [J]. Advances in Neural Information Processing Systems, 2021, 34: 1062-1073.
[33] PHAN N, PAGH R. Fast and scalable polynomial kernels via explicit feature maps[C] //Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Chicago, USA: ACM, 2013: 239-247.
[34] AIOLLI F, DONINI M. EasyMKL: a scalable multiple kernel learning algorithm[J]. Neurocomputing, 2015, 169: 215-224.
[35] TANABE H, HO T B, Nguyen C H, et al. Simple but effective methods for combining kernels in computational biology[C] //Proceedings of the 2008 IEEE International Conference on Research, Innovation and Vision for the Future in Computing and Communication Technologies. Ho Chi Minh City, Vietnam: IEEE, 2008: 71-78.
[1] YANG Jucheng, WEI Feng, LIN Liang, JIA Qingxiang, LIU Jianzheng. A research survey of driver drowsiness driving detection [J]. Journal of Shandong University(Engineering Science), 2024, 54(2): 1-12.
[2] XIAO Wei, ZHENG Gengsheng, CHEN Yujia. Named entity recognition method combined with self-training model [J]. Journal of Shandong University(Engineering Science), 2024, 54(2): 96-102.
[3] Gang HU, Lemeng WANG, Zhiyu LU, Qin WANG, Xiang XU. Importance identification method based on multi-order neighborhood hierarchical association contribution of nodes [J]. Journal of Shandong University(Engineering Science), 2024, 54(1): 1-10.
[4] Jiachun LI,Bowen LI,Jianbo CHANG. An efficient and lightweight RGB frame-level face anti-spoofing model [J]. Journal of Shandong University(Engineering Science), 2023, 53(6): 1-7.
[5] Yujiang FAN,Huanhuan HUANG,Jiaxiong DING,Kai LIAO,Binshan YU. Resilience evaluation system of the old community based on cloud model [J]. Journal of Shandong University(Engineering Science), 2023, 53(5): 1-9, 19.
[6] Ying LI,Jiankun WANG. The classification of mild cognitive impairment based on supervised graph regularization and information fusion [J]. Journal of Shandong University(Engineering Science), 2023, 53(4): 65-73.
[7] YU Yixuan, YANG Geng, GENG Hua. Multimodal hierarchical keyframe extraction method for continuous combined motion [J]. Journal of Shandong University(Engineering Science), 2023, 53(2): 42-50.
[8] ZHANG Hao, LI Ziling, LIU Tong, ZHANG Dawei, TAO Jianhua. A technology prediction model based on fuzzy Bayesian networks with sociological factors [J]. Journal of Shandong University(Engineering Science), 2023, 53(2): 23-33.
[9] WU Yanli, LIU Shuwei, HE Dongxiao, WANG Xiaobao, JIN Di. Poisson-gamma topic model of describing multiple underlying relationships [J]. Journal of Shandong University(Engineering Science), 2023, 53(2): 51-60.
[10] YU Mingjun, DIAO Hongjun, LING Xinghong. Online multi-object tracking method based on trajectory mask [J]. Journal of Shandong University(Engineering Science), 2023, 53(2): 61-69.
[11] HUANG Huajuan, CHENG Qian, WEI Xiuxi, YU Chuchu. Adaptive crow search algorithm with Jaya algorithm and Gaussian mutation [J]. Journal of Shandong University(Engineering Science), 2023, 53(2): 11-22.
[12] LIU Fangxu, WANG Jian, WEI Benzheng. Auxiliary diagnosis algorithm for pediatric pneumonia based on multi-spatial attention [J]. Journal of Shandong University(Engineering Science), 2023, 53(2): 135-142.
[13] LIU Xing, YANG Lu, HAO Fanchang. Finger vein image retrieval based on multi-feature fusion [J]. Journal of Shandong University(Engineering Science), 2023, 53(2): 118-126.
[14] Yue YUAN,Yanli WANG,Kan LIU. Named entity recognition model based on dilated convolutional block architecture [J]. Journal of Shandong University(Engineering Science), 2022, 52(6): 105-114.
[15] Xiaobin XU,Qi WANG,Bin GAO,Zhiyu SUN,Zhongjun LIANG,Shangguang WANG. Pre-allocation of resources based on trajectory prediction in heterogeneous networks [J]. Journal of Shandong University(Engineering Science), 2022, 52(4): 12-19.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
[1] ZHANG Hong-Bo, HUANG Mao-Song, SONG Xiu-Guang. A  double hardening constitutive model for fine sands based on both the dilatancy feature and strain softening[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2008, 38(6): 55 -60 .
[2] REN Yuling, LU Wen, XU Hongqiang, HE Lihuo. An image quality assessment method based on Shearlet transform[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2015, 45(3): 15 -21 .
[3] HUANG Bin. Discrete variable control systems based on a discrete reaching law[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2011, 41(1): 45 -48 .
[4] JIAO Jianmin 1, SUN Xiaojun 1, WU Baowei 2. Guaranteed cost control for uncertain singular timedelay systems
with nonlinear perturbation[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(2): 64 -69 .
[5] MEI Qinglin, ZHANG Huaxiang. A neighborhood preserving embedding algorithm based on global distance and label information[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2016, 46(1): 10 -14 .
[6] ZHANG Jianming, LIU Quansheng, TANG Zhicheng, ZHAN Ting, JIANG Yalong. New peak shear strength criterion with inclusion of shear action history[J]. Journal of Shandong University(Engineering Science), 2015, 45(5): 77 -81 .
[7] DIAO Wen-Zhong, SHI Jun. Study of power system probability security assessment incomplex power injection space[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(6): 135 -138 .
[8] SUN Guofu1,2, LI Shucai2, LU Wei2, CHEN Lei3. Experimental research and numerical simulation of the concrete-filled
steel tube arch rib cross-section temperature during the hydration
process of a complex binder[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2011, 41(3): 106 -111 .
[9] CAO Hong-gen1, YUAN Bao-hua1, ZHU Hui-sheng2. Recognition of intersected face based on contrast information and  local binary pattern[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2012, 42(4): 29 -34 .
[10] HE Shan, YOU Xiao-Chun. Research on supporting scheme and failure mechanism of roadway surrounding rock under complicated conditions[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(4): 121 -124 .
Copyright 2018 © Editorial office of Journal of Shandong University (Engineering Science)
Supported by Beijing Magtech Co.Ltd