Journal of Shandong University(Engineering Science) ›› 2018, Vol. 48 ›› Issue (6): 27-36.doi: 10.6040/j.issn.1672-3961.0.2018.264

• Machine Learning & Data Mining • Previous Articles     Next Articles

Multi-label feature selection algorithm based on correntropy andmanifold learning

Hong CHEN(),Xiaofei YANG*(),Qing WAN,Yingcang MA   

  1. School of Science, Xi′an Polytechnic University, Xi′an 710048, Shaanxi, China
  • Received:2018-07-03 Online:2018-12-20 Published:2018-12-26
  • Contact: Xiaofei YANG E-mail:13572959949@163.com;yangxiaofei2002@163.com
  • Supported by:
    国家自然科学基金资助项目(11501435);中国纺织工业联合会科技指导性项目(2016073);陕西省教育厅科研计划项目(18JK0360)

RichHTML

11

PDF (PC)

366

Abstract:

A sparse regularization method based on correntropy and feature manifold learning was proposed to solve the problem of multi-label feature selection. A regression model of multi-label feature selection was presented by means of correntropy. The sparse regularized multi-label feature selection model, combing ?2, 1 norm and feature manifold learning, was established. An iterative algorithm was proposed for the above model. The convergence of the algorithm was proved and the effectiveness of the given algorithm was verified through experiments.

Key words: correntropy, sparse regularization, feature manifold learning, multi label, feature selection

CLC Number: 

  • TP18

Table 1

Detailed information of the data sets"

数据集 特征数 标签数 训练集样本数 测试集样本数
Image 294 5 400 200
Scene 294 6 1 211 1 196
Emotion 72 6 391 202
Yeast 103 14 1 500 917
Enron 1 001 53 1 123 579

Table 2

Running time of different algorithms on different data sets"

s
数据集 CMLS PMU MDMR FIMF
Image 2.099 285 22.728 828 28.008 186 1.278 762
Scene 23.808 170 78.261 445 89.529 816 10.112 000
Emotion 1.834 695 4.801 576 4.902 648 1.368 992
Yeast 25.201 876 68.878 388 70.245 280 17.266 886
Enron 17.149 738 2 648.267 084 2 564.446 634 6.799 612

Table 3

Average precision of different algorithms on different data sets"

算法 CMLS PMU MDMR FIMF Baseline
Image 0.750 8 0.659 2 0.693 4 0.679 1 0.721 4
Scene 0.831 7 0.803 4 0.763 3 0.690 6 0.851 2
Emotion 0.769 1 0.712 6 0.755 1 0.751 0 0.693 8
Yeast 0.766 3 0.756 3 0.758 0 0.755 2 0.758 5
Enron 0.671 3 0.648 3 0.656 6 0.654 8 0.623 2

Table 4

Hamming loss of different algorithms on different data sets"

算法 CMLS PMU MDMR FIMF Baseline
Image 0.203 0 0.230 0 0.224 0 0.234 0 0.213 0
Scene 0.108 6 0.113 7 0.134 8 0.158 7 0.098 9
Emotion 0.234 3 0.267 3 0.240 9 0.225 2 0.293 7
Yeast 0.195 6 0.200 6 0.199 9 0.202 1 0.198 0
Enron 0.048 7 0.050 5 0.050 5 0.050 1 0.052 0

Table 5

Ranking loss of different algorithms on different data sets"

算法 CMLS PMU MDMR FIMF Baseline
Image 0.213 3 0.297 5 0.271 3 0.266 3 0.233 3
Scene 0.101 0 0.129 0 0.144 4 0.199 4 0.093 1
Emotion 0.176 4 0.258 4 0.199 4 0.201 2 0.282 9
Yeast 0.169 3 0.172 3 0.171 0 0.174 7 0.171 5
Enron 0.088 6 0.094 9 0.094 4 0.093 5 0.093 8

Table 6

One error of different algorithms on different data sets"

算法 CMLS PMU MDMR FIMF Baseline
Image 0.390 0 0.525 0 0.460 0 0.500 0 0.435 0
Scene 0.275 9 0.309 4 0.390 5 0.498 3 0.242 5
Emotion 0.331 7 0.361 4 0.356 4 0.351 5 0.405 9
Yeast 0.231 2 0.236 6 0.236 6 0.236 6 0.234 5
Enron 0.231 4 0.274 6 0.243 5 0.245 3 0.304 0

Table 7

Coverage of different algorithms on different data sets"

算法 CMLS PMU MDMR FIMF Baseline
Image 1.125 0 1.460 0 1.365 0 1.355 0 1.215 0
Scene 0.607 0 0.749 2 0.825 3 1.095 3 0.568 6
Emotion 2.024 8 2.405 9 2.089 1 2.054 5 2.490 1
Yeast 6.347 9 6.370 8 6.364 2 6.374 0 6.414 4
Enron 12.753 0 13.412 8 13.160 6 13.203 8 13.205 5

Fig.1

Average precision of several different feature selection algorithms"

Fig.2

Hamming loss of several different feature selection algorithms"

Fig.3

One error of several different feature selection algorithms"

Fig.4

Coverage of several different feature selection algorithms"

Fig.5

Ranking loss of several different feature selection algorithms"

1 KONG X N , YU P S . GMLC: a multi-label feature selection framework for graph classification[J]. Knowledge & Information Systems, 2012, 31 (2): 281- 305.
2 OZONAT K, YOUNG D. Towards a universal marketplace over the web: statistical multi-label classification of service provider forms with simulated annealing[C]//ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York, USA: ACM, 2009: 1295-1304.
3 LIU W , POKHAREL P P , PRINCIPE J C . Correntropy: properties and applications in non-gaussian signal processing[J]. IEEE Transactions on Signal Processing, 2007, 55 (11): 5286- 5298.
doi: 10.1109/TSP.2007.896065
4 李素姝, 王士同, 李滔, 等. 基于LS-SVM与模糊补准则的特征选择方法[J]. 山东大学学报(工学版), 2017, 47 (3): 34- 42.
LI Sushu , WANG Shitong , LI Tao , et al. Feature selection method based on LS-SVM and fuzzy complement criterion[J]. Journal of Shandong University(Engineering Edition), 2017, 47 (3): 34- 42.
5 HE R , ZHENG W S , HU B G . Maximum correntropy criterion for robust face recognition[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2011, 33 (8): 1561- 1576.
6 HE R , HU B G , ZHENG W S , et al. Robust principal component analysis based on maximum correntropy criterion[J]. IEEE Transactions on Image Processing, 2011, 20 (6): 1485- 1494.
doi: 10.1109/TIP.2010.2103949
7 CHEN X B , YANG J , LIANG J , et al. Recursive robust least squares support vector regression based on maximum correntropy criterion[J]. Neurocomputing, 2012, 97 (1): 63- 73.
8 LEE J , LIM H , KIM D W . Approximating mutual information for multi-label feature selection[J]. Electronics Letters, 2012, 48 (15): 929- 930.
doi: 10.1049/el.2012.1600
9 ZHENG W S, WANG L, TAN T, et al. L2, 1 regularized correntropy for robust feature selection[J]. 2012, 157(10): 2504-2511.
10 莫小勇, 潘志松, 邱俊洋, 等. 基于在线特征选择的网络流异常检测[J]. 山东大学学报(工学版), 2016, 46 (4): 21- 27.
MO Xiaoyong , PAN Zhisong , QIU Junyang , et al. Network flow anomaly detection based on online feature selection[J]. Journal of Shandong University(Engineering Edition), 2017, 47 (3): 34- 42.
11 NIE F P, HUANG H, CAI X, et al. Efficient and robust feature selection via joint L2, 1-norms minimization[C]//International Conference on Neural Information Processing Systems. Sydney, Australia: Curran Associates Inc, 2010: 1813-1821.
12 YANG Y, SHEN H T, MA Z, et al. L2, 1-norm regularized discriminative feature selection for unsupervised learning[C]//International Joint Conference on Artificial Intelligence. Menlo Park, USA: AAAI Press, 2011: 1589-1594.
13 BELKIN M , NIYOGI P . Laplacian eigenmaps and spectral techniques for embedding and clustering[J]. Advances in Neural Information Processing Systems, 2001, 14 (6): 585- 591.
14 GU Q Q, ZHOU J. Co-clustering on manifolds[C]//ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York, USA: ACM, 2009: 359-368.
15 CHEN B D , XING L , WANG X , et al. Robust learning with kernel mean p-power error loss[J]. IEEE Transactions on Cybernetics, 2018, 48 (7): 2101- 2113.
doi: 10.1109/TCYB.2017.2727278
16 ZHANG M L , ZHOU Z H . ML-KNN: A lazy learning approach to multi-label learning[J]. Pattern Recognition, 2007, 40 (7): 2038- 2048.
doi: 10.1016/j.patcog.2006.12.019
17 BOUTELL M R , LUO J , SHEN X , et al. Learning multi-label scene classification[J]. Pattern Recognition, 2004, 37 (9): 1757- 1771.
doi: 10.1016/j.patcog.2004.03.009
18 TROHIDIS K , TSOUMAKAS G , KALLIRIS G , et al. Multilabel classification of music into emotions[J]. Blood, 2008, 90 (9): 3438- 3443.
19 ELISSEEFF A, WESTON J. A kernel method for multi-labelled classification[C]//International Conference on Neural Information Processing Systems: Natural and Synthetic. Cambridge, USA: MIT Press, 2001: 681-687.
20 KLIMT B, YANG Y. The enron corpus: a new dataset for email classification research[C]//European Conference on Machine Learning. Berlin, Germany: Springer-Verlag, 2004: 217-226.
21 LEE J , KIM D W . Feature selection for multi-label classification using multivariate mutual information[J]. Pattern Recognition Letters, 2013, 34 (3): 349- 357.
doi: 10.1016/j.patrec.2012.10.005
22 LIN Y J , HU Q H , LIU J H , et al. Multi-label feature selection based on max-dependency and min-redundancy[J]. Neurocomputing, 2015, 168 (C): 92- 103.
23 LEE J , KIM D W . Fast multi-label feature selection based on information-theoretic feature ranking[J]. Pattern Recognition, 2015, 48 (9): 2761- 2771.
doi: 10.1016/j.patcog.2015.04.009
24 DUDA J. Supervised and unsupervised discretization of continuous features[C]//Twelfth International Conference on Machine Learning. Sydney, Australia: ICML, 1995: 194-202.
[1] Lianming MOU. Weighted k sub-convex-hull classifier based on adaptive feature selection [J]. Journal of Shandong University(Engineering Science), 2018, 48(5): 32-37.
[2] LI Sushu, WANG Shitong, LI Tao. A feature selection method based on LS-SVM and fuzzy supplementary criterion [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2017, 47(3): 34-42.
[3] FANG Hao, LI Yun. Random undersampling and POSS method for software defect prediction [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2017, 47(1): 15-21.
[4] MO Xiaoyong, PAN Zhisong, QIU Junyang, YU Yajun, JIANG Mingchu. Anomaly detection in network traffic based on online feature selection [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2016, 46(4): 21-27.
[5] WEI Xiaomin, XU Bin, GUAN Jihong. Prediction of protein energy hot spots based on recursion feature elimination [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2014, 44(2): 12-20.
[6] PAN Dong-yin, ZHU Fa, XU Sheng, YE Ning*. Feature selection of gene expression profiles of colon cancer [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2012, 42(2): 23-29.
[7] LI Guo-he1,2, YUE Xiang1,2, LI Xue3, WU Wei-jiang1,2, LI Hong-qi1. A method of feature selection for continuous attributes [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2011, 41(6): 1-6.
[8] LI Xia1, WANG Lian-xi2, JIANG Sheng-yi1. Ensemble learning based feature selection for imbalanced problems [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2011, 41(3): 7-11.
[9] YOU Ming-yu, CHEN Yan, LI Guo-zheng. Im-IG: A novel feature selection method for imbalanced problems [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2010, 40(5): 123-128.
[10] YANG Ai-min1, ZHOU Yong-mei1, DENG He2, ZHOU Jian-feng3. Method of feature generation and selection for network traffic classification [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2010, 40(5): 1-7.
[11] TAN Tai-zhe, LIANG Ying-yi, LIU Fu-chun. Application of ReliefF feature evaluation in un-supervised manifold learning [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2010, 40(5): 66-71.
[12] DAI Ping, LI Ning*. A fast SVM-based feature selection method [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2010, 40(5): 60-65.
[13] WANG Fa-bo, XU Xin-shun. A new feature selection method for text categorization [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2010, 40(4): 8-11.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
[1] LI Ke,LIU Chang-chun,LI Tong-lei . Medical registration approach using improved maximization of mutual information[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(2): 107 -110 .
[2] CHENG Daizhan, LI Zhiqiang. A survey on linearization of nonlinear systems[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(2): 26 -36 .
[3] WANG Yong, XIE Yudong. Gas control technology of largeflow pipe[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(2): 70 -74 .
[4] LIU Xin 1, SONG Sili 1, WANG Xinhong 2. [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(2): 98 -100 .
[5] HU Tian-liang,LI Peng,ZHANG Cheng-rui,ZUO Yi . Design of a QEP decode counter based on VHDL[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2008, 38(3): 10 -13 .
[6] . [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(2): 104 -107 .
[7] CHEN Huaxin, CHEN Shuanfa, WANG Binggang. The aging behavior and mechanism of base asphalts[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(2): 125 -130 .
[8] . [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(2): 131 -136 .
[9] LI Shijin, WANG Shengte, HUANG Leping. Change detection with remote sensing images based on forward-backward heterogenicity[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(3): 1 -9 .
[10] WANG Ru-gui,CAI Gan-wei . Sub-harmonic resonance analysis of 2-DOF controllable plane linkage mechanism electromechanical coupling system[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2008, 38(3): 58 -63 .
Copyright 2018 © Editorial office of Journal of Shandong University (Engineering Science)
Supported by Beijing Magtech Co.Ltd