基于LS-SVM与模糊补准则的特征选择方法

doi:10.6040/j.issn.1672-3961.0.2016.308

摘要/Abstract

摘要： 针对传统特征选择算法采用单一度量的方式难以兼顾泛化性能和降维性能的不足,提出新的特征选择算法(least squares support vector machines and fuzzy supplementary criterion, LS-SVM-FSC)。通过核化的最小二乘支持向量机(least squares support vector machines, LS-SVM)对每个特征的样本进行分类,使用新的模糊隶属度函数获得每个样本对其所属类的模糊隶属度,使用模糊补准则选择具有最小冗余最大相关的特征子集。试验表明:与其他10个特征选择方法与7个隶属度决定方法相比,所提算法在9个数据集上都具有很高的分类准确率和很强的降维性能,且在高维数据集中的学习速度依然很快。

关键词: 最小二乘支持向量机, 模糊隶属度函数, 分类, 模糊补准则, 特征选择

Abstract: Traditional feature selection algorithm used a single scalar metric such that it might become difficult to achieve a trade-off between generalization performance and dimension reduction at the same time. A new feature selection algorithm called LS-SVM-FSC was proposed to circumvent this shortcoming. The kernel-based least squares support vector machines was used to train a set of binary classifiers on each single feature and a kind of new fuzzy membership function was used to obtain fuzzy membership value of each pattern belonging to its class. Based on a new fuzzy supplementary criterion, the features with minimal redundancy and maximal relevance was selected. Experiments indicated that the proposed algorithm had high classification accuracy and strong dimension reduction capability on nine datasets. In particular, it still kept fast learning speed for high-dimensional datasets, in contrast to other ten feature selection methods and seven degree determination methods.

Key words: feature selection, fuzzy supplementary criterion, least squares support vector machines, classification, fuzzy membership degree function

中图分类号:

TP181

李素姝,王士同,李滔. 基于LS-SVM与模糊补准则的特征选择方法[J]. 山东大学学报(工学版), 2017, 47(3): 34-42.

LI Sushu, WANG Shitong, LI Tao. A feature selection method based on LS-SVM and fuzzy supplementary criterion[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2017, 47(3): 34-42.

参考文献

[1] JAIN A, ZONGKER D. Feature selection: evaluation, application, and small sample performance[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 1997, 19(2):153-158.
[2] TAN M, PU J, ZHENG B. Optimization of breast mass classification using sequential forward floating selection(SFFS)and a support vector machine(SVM)model[J]. International Journal of Computer Assisted Radiology & Surgery, 2014, 9(6):76-82.
[3] NARENDRA P M, FUKUNAGA K. A branch and bound algorithm for feature subset selection[J]. Electronics Letters, 2010, 26(9):917-922.
[4] ROBNIK-SIKONJA M, KONONENKO I. Theoretical and empirical analysis of ReliefF and RReliefF[J]. Machine Learning, 2003, 53(1-2):23-69.
[5] MITRA P, MURTHY C A, PAL S K, et al. Unsupervised feature selection using feature similarity[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2002, 24(3):301-312.
[6] LI D, PEDRYCZ W, PIZZI N J. Fuzzy wavelet packet based feature extraction method and its application to biomedical signal classification[J]. IEEE Transactions on Bio-medical Engineering, 2005, 52(6):1132-1139.
[7] OOI C H, CHETTY M, TENG S W. Differential prioritization in feature selection and classifier aggregation for multiclass microarray datasets[J]. Data Mining & Knowledge Discovery, 2007, 14(3):329-366.
[8] ZHANG D, CHEN S, ZHOU Z H. Constraint score:a new filter method for feature selection with pairwise constraints[J]. Pattern Recognition, 2008, 41(5):1440-1451.
[9] MOUSTAKIDIS S P, THEOCHARIS J B. SVM-FuzCoC: a novel SVM-based feature selection method using a fuzzy complementary criterion[J]. Pattern Recognition, 2010, 43(11):3712-3729.
[10] CHANG C C, LIN C J. LIBSVM: A library for support vector machines[J]. Acm Transactions on Intelligent Systems & Technology, 2011, 2(3):389-396.
[11] SUYKENS J, VANDEWALLE J. Least squares support vector machine classifiers[J]. Neural Processing Letters,1999,9(3):293-300.
[12] ZHANG N, ZHOU Y, HUANG T, et al. Discriminating between lysine sumoylation and lysine acetylation using mRMR feature selection and analysis[J]. Plos One, 2014, 9(9):e107464.
[13] 张战成,王士同,邓赵红,等. 支持向量机的一种快速分类算法[J]. 电子与信息学报, 2011, 33(9):2181-2186. ZHANG Zhancheng, WANG Shitong, DENG Zhaohong, et al. Fast decision using SVM for incoming samples[J]. Journal of Electronics and Information Technolog, 2011, 33(9):2181-2186.
[14] 李欢,王士同. 适合多观测样本的基于LS-SVM的新分类算法[J]. 计算机工程与应用, 2016, 52(1):113-119. LI Huan, WANG Shitong. Novel LS-SVM based classification algorithm for multi-observation sets[J]. Computer Engineering and Applications, 2016, 52(1):113-119.
[15] 苟博,黄贤武. 支持向量机多类分类方法[J]. 数据采集与处理, 2006, 21(3):334-339. GOU Bo, HUANG Xianwu. SVM multi-class classification[J]. Journal of Data Acquisition and Processing, 2006, 21(3):334-339.
[16] AZADEH A, ARYAEE M, ZARRIN M, et al. A novel performance measurement approach based on trust context using fuzzy T-norm and S-norm operators: the case study of energy consumption[J]. Energy Exploration & Exploitation, 2016, 34(4):561-585.
[17] DERELI T, BAYKASOGLU A, ALTUN K, et al. Industrial applications of type-2 fuzzy sets and systems: a concise review[J]. Computers in Industry, 2011, 62(2):125-137.
[18] BHATT R B, GOPAL M. On the extension of functional dependency degree from crisp to fuzzy partitions[J]. Pattern Recognition Letters, 2006, 27(5):487-491.
[19] PLATT J C. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods[J]. Advances in Large Margin Classifiers, 2000, 10(4):61-74.
[20] MADEVSKA-BOGDANOVA A, NIKOLIK D, CURFS L. Probabilistic SVM outputs for pattern recognition using analytical geometry[J]. Neurocomputing, 2004, 62(1):293-303.
[21] LIU Y, GUO J, HU G, et al. Gene prediction in metagenomic fragments based on the SVM algorithm[J]. Bmc Bioinformatics, 2013, 14(2):1738-1742.
[22] BOUCHAFFRA D, GOVINDARAJU V, SRIHARI S. A methodology for mapping scores to probabilities[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 1999, 21(9):923-927.
[23] STOUT Q F. Isotonic regression via partitioning[J]. Algorithmica, 2013, 66(1):93-112.

相关文章 15

[1]	唐杰烽,张佳,龙锦益. 基于全局冗余最小的快速多标签特征选择方法[J]. 山东大学学报 (工学版), 2025, 55(6): 21-34.
[2]	吴正健,吾尔尼沙·买买提,杨耀威,阿力木江·艾沙,库尔班·吾布力. 基于DRCoALTP的印刷体文档图像多文种识别方法[J]. 山东大学学报 (工学版), 2025, 55(1): 51-57.
[3]	白琳,俱通,王浩,雷明珠,潘晓英. 面向不平衡数据的提升均衡集成学习算法[J]. 山东大学学报 (工学版), 2024, 54(4): 59-66.
[4]	陈晓江,杨晓奇,陈广豪,刘伍颖. 混合BERT和宽度学习的低时间复杂度短文本分类[J]. 山东大学学报 (工学版), 2024, 54(4): 51-58.
[5]	宋辉,张轶哲,张功萱,孟元. 基于类权重和最小化预测熵的测试时集成方法[J]. 山东大学学报 (工学版), 2024, 54(3): 36-43.
[6]	聂秀山,巩蕊,董飞,郭杰,马玉玲. 短视频场景分类方法综述[J]. 山东大学学报 (工学版), 2024, 54(3): 1-11.
[7]	徐金华,罗义凯,李昱燃,李岩. 基于时频分解与深度学习的轨道客流预测[J]. 山东大学学报 (工学版), 2024, 54(2): 60-68.
[8]	马坤,刘筱云,李乐平,纪科,陈贞翔,杨波. 用于意图识别的自适应多标签信息学习模型[J]. 山东大学学报 (工学版), 2024, 54(1): 45-51.
[9]	于泓,杜娟,魏琳,张利. 计及行为特征的市场化用户电量数据拟合方法[J]. 山东大学学报 (工学版), 2023, 53(4): 113-119.
[10]	李颖,王建坤. 基于监督图正则化和信息融合的轻度认知障碍分类方法[J]. 山东大学学报 (工学版), 2023, 53(4): 65-73.
[11]	张喜龙,韩萌,陈志强,武红鑫,李慕航. 动态集成选择的不平衡漂移数据流Boosting分类算法[J]. 山东大学学报 (工学版), 2023, 53(4): 83-92.
[12]	刘财辉,周琪,叶晓文. 一种基于改进ReliefF算法的入侵检测模型[J]. 山东大学学报 (工学版), 2023, 53(2): 1-10.
[13]	许传臻,袭肖明,李维翠,孙仪,杨璐. 基于自适应多分辨率特征学习的CNV分型网络[J]. 山东大学学报 (工学版), 2022, 52(4): 69-75.
[14]	袁高腾,周晓峰,郭宏乐. 基于特征选择算法的ECG信号分类[J]. 山东大学学报 (工学版), 2022, 52(4): 38-44.
[15]	孟令灿,聂秀山,张雪. 基于遮挡目标去除的公交车拥挤度分类算法[J]. 山东大学学报 (工学版), 2022, 52(4): 83-88.

多维度评价

Viewed

Full text

Abstract

Cited

Shared

Discussed