JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE) ›› 2017, Vol. 47 ›› Issue (3): 34-42.doi: 10.6040/j.issn.1672-3961.0.2016.308

Previous Articles     Next Articles

A feature selection method based on LS-SVM and fuzzy supplementary criterion

LI Sushu, WANG Shitong, LI Tao   

  1. School of Digital Media, Jiangnan University, Wuxi 214122, Jiangsu, China
  • Received:2016-11-07 Online:2017-06-20 Published:2016-11-07

Abstract: Traditional feature selection algorithm used a single scalar metric such that it might become difficult to achieve a trade-off between generalization performance and dimension reduction at the same time. A new feature selection algorithm called LS-SVM-FSC was proposed to circumvent this shortcoming. The kernel-based least squares support vector machines was used to train a set of binary classifiers on each single feature and a kind of new fuzzy membership function was used to obtain fuzzy membership value of each pattern belonging to its class. Based on a new fuzzy supplementary criterion, the features with minimal redundancy and maximal relevance was selected. Experiments indicated that the proposed algorithm had high classification accuracy and strong dimension reduction capability on nine datasets. In particular, it still kept fast learning speed for high-dimensional datasets, in contrast to other ten feature selection methods and seven degree determination methods.

Key words: feature selection, fuzzy supplementary criterion, least squares support vector machines, classification, fuzzy membership degree function

CLC Number: 

  • TP181
[1] JAIN A, ZONGKER D. Feature selection: evaluation, application, and small sample performance[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 1997, 19(2):153-158.
[2] TAN M, PU J, ZHENG B. Optimization of breast mass classification using sequential forward floating selection(SFFS)and a support vector machine(SVM)model[J]. International Journal of Computer Assisted Radiology & Surgery, 2014, 9(6):76-82.
[3] NARENDRA P M, FUKUNAGA K. A branch and bound algorithm for feature subset selection[J]. Electronics Letters, 2010, 26(9):917-922.
[4] ROBNIK-SIKONJA M, KONONENKO I. Theoretical and empirical analysis of ReliefF and RReliefF[J]. Machine Learning, 2003, 53(1-2):23-69.
[5] MITRA P, MURTHY C A, PAL S K, et al. Unsupervised feature selection using feature similarity[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2002, 24(3):301-312.
[6] LI D, PEDRYCZ W, PIZZI N J. Fuzzy wavelet packet based feature extraction method and its application to biomedical signal classification[J]. IEEE Transactions on Bio-medical Engineering, 2005, 52(6):1132-1139.
[7] OOI C H, CHETTY M, TENG S W. Differential prioritization in feature selection and classifier aggregation for multiclass microarray datasets[J]. Data Mining & Knowledge Discovery, 2007, 14(3):329-366.
[8] ZHANG D, CHEN S, ZHOU Z H. Constraint score:a new filter method for feature selection with pairwise constraints[J]. Pattern Recognition, 2008, 41(5):1440-1451.
[9] MOUSTAKIDIS S P, THEOCHARIS J B. SVM-FuzCoC: a novel SVM-based feature selection method using a fuzzy complementary criterion[J]. Pattern Recognition, 2010, 43(11):3712-3729.
[10] CHANG C C, LIN C J. LIBSVM: A library for support vector machines[J]. Acm Transactions on Intelligent Systems & Technology, 2011, 2(3):389-396.
[11] SUYKENS J, VANDEWALLE J. Least squares support vector machine classifiers[J]. Neural Processing Letters,1999,9(3):293-300.
[12] ZHANG N, ZHOU Y, HUANG T, et al. Discriminating between lysine sumoylation and lysine acetylation using mRMR feature selection and analysis[J]. Plos One, 2014, 9(9):e107464.
[13] 张战成,王士同,邓赵红,等. 支持向量机的一种快速分类算法[J]. 电子与信息学报, 2011, 33(9):2181-2186. ZHANG Zhancheng, WANG Shitong, DENG Zhaohong, et al. Fast decision using SVM for incoming samples[J]. Journal of Electronics and Information Technolog, 2011, 33(9):2181-2186.
[14] 李欢,王士同. 适合多观测样本的基于LS-SVM的新分类算法[J]. 计算机工程与应用, 2016, 52(1):113-119. LI Huan, WANG Shitong. Novel LS-SVM based classification algorithm for multi-observation sets[J]. Computer Engineering and Applications, 2016, 52(1):113-119.
[15] 苟博,黄贤武. 支持向量机多类分类方法[J]. 数据采集与处理, 2006, 21(3):334-339. GOU Bo, HUANG Xianwu. SVM multi-class classification[J]. Journal of Data Acquisition and Processing, 2006, 21(3):334-339.
[16] AZADEH A, ARYAEE M, ZARRIN M, et al. A novel performance measurement approach based on trust context using fuzzy T-norm and S-norm operators: the case study of energy consumption[J]. Energy Exploration & Exploitation, 2016, 34(4):561-585.
[17] DERELI T, BAYKASOGLU A, ALTUN K, et al. Industrial applications of type-2 fuzzy sets and systems: a concise review[J]. Computers in Industry, 2011, 62(2):125-137.
[18] BHATT R B, GOPAL M. On the extension of functional dependency degree from crisp to fuzzy partitions[J]. Pattern Recognition Letters, 2006, 27(5):487-491.
[19] PLATT J C. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods[J]. Advances in Large Margin Classifiers, 2000, 10(4):61-74.
[20] MADEVSKA-BOGDANOVA A, NIKOLIK D, CURFS L. Probabilistic SVM outputs for pattern recognition using analytical geometry[J]. Neurocomputing, 2004, 62(1):293-303.
[21] LIU Y, GUO J, HU G, et al. Gene prediction in metagenomic fragments based on the SVM algorithm[J]. Bmc Bioinformatics, 2013, 14(2):1738-1742.
[22] BOUCHAFFRA D, GOVINDARAJU V, SRIHARI S. A methodology for mapping scores to probabilities[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 1999, 21(9):923-927.
[23] STOUT Q F. Isotonic regression via partitioning[J]. Algorithmica, 2013, 66(1):93-112.
[1] MOU Lianming. Weighted k sub-convex-hull classifier based on adaptive feature selection [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(5): 32-37.
[2] ZHANG Pu, LIU Chang, WANG Yong. Suggestion sentence classification model based on feature fusion and ensemble learning [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(5): 47-54.
[3] CAO Ya, DENG Zhaohong, WANG Shitong. An radial basis function neural network model based on monotonic constraints [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(3): 127-133.
[4] XIE Zhifeng, WU Jiaping, MA Lizhuang. Chinese financial news classification method based on convolutional neural network [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(3): 34-39.
[5] WANG Tingting, ZHAI Junhai, ZHANG Mingyang, HAO Pu. K-NN algorithm for big data based on HBase and SimHash [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(3): 54-59.
[6] CHEN Jiajie, WANG Jinfeng. Method for solving Choquet integral model based on ant colony algorithm [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(3): 81-87.
[7] WANG Huan, ZHOU Zhongmei. An over sampling algorithm based on clustering [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(3): 134-139.
[8] YE Mingquan, GAO Lingyun, WAN Chunyuan. Gene expression data classification based on artificial bee colony and SVM [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(3): 10-16.
[9] LI Wei, WANG Zhechao, LI Shucai, DING Wantao, WANG Qi, ZONG Zhi, LIU Keqi. The mechanical properties of the silty clay and the advanced support method in Harbin Metro [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(2): 61-71.
[10] WANG Lei, DENG Xiaogang, CAO Yuping, TIAN Xuemin. Multiblock local Fisher discriminant analysis for chemical process fault classification [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2017, 47(5): 179-186.
[11] HE Qijia, LIU Zhenbing, XU Tao, JIANG Shujie. MR image classification based on LBP and extreme learning machine [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2017, 47(2): 86-93.
[12] GUO Chao, YANG Yan, JIANG Yongquan, SONG Yi. Condition recognition of high-speed train based on multi-view classification ensemble [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2017, 47(1): 7-14.
[13] FANG Hao, LI Yun. Random undersampling and POSS method for software defect prediction [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2017, 47(1): 15-21.
[14] MO Xiaoyong, PAN Zhisong, QIU Junyang, YU Yajun, JIANG Mingchu. Anomaly detection in network traffic based on online feature selection [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2016, 46(4): 21-27.
[15] WANG Bin, CHANG Faliang, LIU Chunsheng. Traffic sign classification based on multi-feature fusion [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2016, 46(4): 34-40.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!