Journal of Shandong University(Engineering Science) ›› 2018, Vol. 48 ›› Issue (5): 32-37.doi: 10.6040/j.issn.1672-3961.0.2017.415

• Machine Learning & Data Mining • Previous Articles     Next Articles

Weighted k sub-convex-hull classifier based on adaptive feature selection

Lianming MOU1,2()   

  1. 1. College of Mathematics and Information Science, Neijiang Normal University, Neijiang 641100, Sichuan, China
    2. Data Recovery Key Laboratory of Sichuan Province, Neijiang 641100, Sichuan, China
  • Received:2017-05-09 Online:2018-10-01 Published:2017-05-09
  • Supported by:
    国家自然科学基金资助项目(10872085);四川省科技厅科技计划重点资助项目(2017JY0199);四川教育厅自然科学重点项目基金资助项目(13ZA0008);2015内江市科技支撑计划资助项目

RichHTML

14

PDF (PC)

600

Abstract:

Because of the increase of the dimension of the problem and the effect of different features on classifier, the performance of the k sub-convex-hull classifier was seriously reduced. An adaptive feature selection weighted k sub-convex-hull classifier was designed (AWCH). A weighted k sub-convex-hull classifier was designed according to the shortcomings of conventional convex hull distance. By applying the distance metric learning and regularization technique in the k neighborhood of the test sample, an adaptive feature selection method was designed and seamlessly integrated into the optimization model on the weighted k sub-convex-hull. Through these efforts, for different test samples, an adaptive feature space in different categories could be extracted, and a valid weighted k sub-convex-hull distance could be obtained. Experimental results showed that the AWCH not only reduced the dimension of the problem, but also was significantly superior to similar classifiers.

Key words: weighted k sub-convex-hull classifier, distance metric learning, regularization, feature selection, adaptive

CLC Number: 

  • TP391

Table 1

Experimental data sets"

编号 数据集 样本数 属性 类别
1 artificial 6 000 7 10
2 austra 690 15 2
3 balance-scale 625 4 3
4 BCI 400 117 2
5 breast-w 699 9 2
6 bupa 345 6 2
7 clean1 476 166 2
8 diabetes 768 8 2
9 digits 1 797 64 10
10 ecoli 336 7 8
11 ethn 2 630 30 2
12 glass 214 9 7
13 heart-statlog 270 13 2
14 ionosphere 351 34 2
15 iris 150 4 3
16 isolet_norm 600 51 2
17 letter 20 000 16 26
18 LIBRASMovement 360 90 15
19 machine 209 7 8
20 mfeat-factors 2 000 216 10
21 mfeat-fourier 2 000 76 10
22 mfeat-karhunen 2 000 64 10
23 mfeat-zernike 2 000 47 10
24 musk 476 166 2
25 optdigits 5 620 64 10
26 page-blocks 5 473 10 5
27 Yale_32x32_face 165 1024 15
28 Yale_64x64_face 165 4 096 15
29 ORL_32x32_face 400 1 024 40
30 ORL_64x64_face 400 4 096 40

Table 2

Comparison of classification error rate and standard deviation"

数据集CKNN kCH RLHC AWCH
Err Std Err Std Err Std Err Std
artificial 0.433 0 0.004 4 0.429 8 0.003 2 0.377 4 0.003 1 0.322 6 0.002 4
austra 0.163 5 0.006 8 0.155 1 0.005 7 0.156 1 0.005 3 0.135 0 0.004 7
balance-scale 0.110 7 0.005 7 0.103 4 0.005 0 0.103 7 0.007 7 0.067 5 0.0043
BCI 0.445 5 0.016 0 0.460 5 0.014 7 0.390 5 0.020 0 0.338 8 0.007 3
breast-w 0.032 2 0.001 5 0.031 9 0.000 9 0.026 8 0.0028 0.020 0 0.001 2
bupa 0.368 4 0.011 6 0.357 1 0.009 0 0.333 6 0.016 5 0.284 3 0.014 9
clean1 0.247 1 0.009 8 0.140 2 0.009 3 0.183 2 0.010 7 0.082 8 0.008 5
diabetes 0.260 9 0.007 7 0.259 8 0.008 1 0.257 6 0.007 9 0.202 0 0.007 4
digits 0.023 5 0.001 7 0.021 2 0.001 5 0.009 5 0.000 7 0.007 7 0.000 6
ecoli 0.145 6 0.004 9 0.146 0 0.007 8 0.132 4 0.009 1 0.127 7 0.001 3
ethn 0.046 7 0.001 9 0.033 9 0.001 6 0.021 7 0.001 5 0.010 3 0.001 4
glass 0.375 3 0.009 8 0.329 9 0.016 5 0.308 9 0.015 5 0.268 1 0.011 8
heart-statlog 0.180 0 0.006 8 0.180 4 0.005 0 0.210 7 0.019 5 0.111 5 0.017 6
ionosphere 0.167 8 0.007 4 0.161 6 0.005 9 0.105 3 0.008 2 0.078 4 0.007 4
iris 0.049 3 0.005 8 0.044 0 0.003 4 0.0427 0.004 7 0.040 7 0.004 7
isolet_norm 0.006 2 0.001 9 0.008 0 0.001 9 0.00 43 0.001 4 0.003 8 0.001 2
letter 0.063 0 0.001 0 0.060 9 0.001 1 0.034 5 0.000 6 0.028 3 0.000 4
LIBRASMovement 0.446 5 0.025 9 0.460 5 0.017 4 0.125 1 0.009 4 0.116 7 0.008 8
machine 0.233 9 0.014 5 0.233 7 0.014 7 0.088 3 0.011 5 0.083 0 0.010 6
mfeat-factors 0.047 1 0.001 3 0.044 5 0.001 2 0.030 4 0.001 5 0.030 0 0.001 1
mfeat-fourier 0.188 7 0.004 6 0.190 8 0.002 8 0.170 1 0.004 0 0.169 8 0.002 1
mfeat-karhunen 0.052 5 0.001 6 0.050 2 0.001 2 0.026 1 0.002 8 0.022 5 0.001 1
mfeat-zernike 0.185 5 0.006 7 0.179 1 0.005 0 0.165 4 0.004 6 0.149 4 0.003 7
musk 0.247 1 0.009 8 0.140 2 0.008 3 0.103 2 0.010 7 0.082 8 0.008 5
optdigits 0.017 1 0.000 5 0.016 2 0.000 4 0.008 1 0.000 4 0.008 0 0.000 3
page-blocks 0.050 8 0.000 9 0.049 4 0.001 1 0.042 4 0.002 4 0.034 7 0.000 9
Yale_32x32_face 0.486 2 0.013 9 0.475 1 0.024 3 0.278 2 0.011 6 0.229 1 0.010 0
Yale_64x64_face 0.363 1 0.017 7 0.385 3 0.016 5 0.271 3 0.008 7 0.254 0 0.008 1
ORL_32x32_face 0.354 5 0.016 8 0.367 0 0.016 1 0.034 3 0.005 8 0.030 3 0.003 3
ORL_64x64_face 0.360 3 0.012 7 0.362 5 0.012 1 0.034 8 0.006 2 0.030 2 0.005 0
1 VINCENT P , BENGIO Y . K-local hyperplane and convex distance nearest neighbor algorithms[J]. In Advances in Neural Information Processing Systems, 2002, 14 (1): 985- 992.
2 ZHANG Y, TANG Z M, LI Y P, et al. Ensemble learning and optimizing KNN method for speaker recognition[C]//Proceedings of the Fourth International Conference on Fuzzy System and Knowledge Discovery (FSKD). Haikou, China: [S.l.], 2007: 285-289.
3 YANG T , KECMAN V . Adaptive local hyperplane classification[J]. Neurocomputing, 2008, (71): 3001- 3004.
4 YANG T , KECMAN V . Face recognition with adaptive local hyperplane algorithm[J]. Pattern Anal Applic, 2010, (13): 79- 83.
5 WEN Guihua , JIANG Lijun , WEN Jun , et al. Perceptual relativity-based local hyperplane classification[J]. Neurocomputing, 2012, (97): 155- 163.
6 XU Jie , YANG Jian , LAI Zhihui . K-local hyperplane distance nearest neighbor classifier oriented local discriminant analysis[J]. Information Sciences, 2013, 232, 11- 26.
doi: 10.1016/j.ins.2012.12.045
7 牟廉明. k子凸包分类[J]. 山西大学学报(自然科学版), 2011, 34 (3): 374- 380.
MOU Lianming . A k sub-convex-hull classifier[J]. Journal of Shanxi University (Nat Sci Ed), 2011, 34 (3): 374- 380.
8 牟廉明. 选择性自适应k子凸包分类方法[J]. 南京大学学报(自然科学), 2013, 49 (4): 410- 416.
MOU Lianming . Selective adaptive k sub-convex-hullclassifier[J]. Journal of Nanjing University(Natural Sciences), 2013, 49 (4): 410- 416.
9 李素姝, 王士同, 李滔. 基于LS-SVM与模糊补准则的特征选择方法[J]. 山东大学学报(工学版), 2017, 47 (3): 34- 42.
LI Sushu , WANG Shitong , LI Tao . A feature selection method based on LS-SVM and fuzzy supplementary criterion[J]. Journal of Shandong University(Engineering Science), 2017, 47 (3): 34- 42.
10 王法波, 许信顺. 文本分类中一种新的特征选择方法[J]. 山东大学学报(工学版), 2017, 40 (4): 8- 11, 18.
WANG Fabo , XU Xinshun . A new feature selection method for text categorization[J]. Journal of Shandong University(Engineering Science), 2017, 40 (4): 8- 11, 18.
11 戴平, 李宁. 一种基于SVM的快速特征选择方法[J]. 山东大学学报(工学版), 2017, 40 (5): 60- 65.
DAI Ping , LI Ning . A fast SVM-based feature selection method[J]. Journal of Shandong University (Engineering Science), 2017, 40 (5): 60- 65.
12 赵佳, 王士同. 特征加权距离的半监督模糊子空间聚类算法[J]. 小型微型计算机系统, 2017, 38 (2): 405- 410.
ZHAO Jia , WANG Shitong . Semi-supervised fuzzy subspace clustering algorithm based on feature weighted distance[J]. Journal of Chinese Computer Systems, 2017, 38 (2): 405- 410.
13 贾隆嘉, 孙铁利, 杨凤芹, 等. 基于类空间密度的文本分类特征加权算法[J]. 吉林大学学报(信息科学版), 2017, 35 (1): 92- 97.
doi: 10.3969/j.issn.1671-5896.2017.01.015
JIA Longjia , SUN Tieli , YANG Fengqin , et al. Class space density based weighting scheme for automated text categorization[J]. Journal of Jilin University(Information Science Edition), 2017, 35 (1): 92- 97.
doi: 10.3969/j.issn.1671-5896.2017.01.015
14 KILIAN Weinberger . Distance metric learning for large margin nearest neighbor classification[J]. Journal of Machine Learning Research, 2009, 10, 207- 244.
15 BAR-HILLEL A , HERTZ T , SHENTAL N , et al. Learning a mahalanobis metric from equivalence constraints[J]. Journal of Machine Learning Research, 2006, 6 (1): 937- 965.
16 钱强, 陈松灿. 基于矩阵正态分布似然比测试的矩阵度量学习算法[J]. 山东大学学报(工学版), 2017, 42 (6): 37- 42.
QIAN Qiang , CHEN Songcan . Matrix metric learning algorithm based on likelihood ratio test with matrix normal distribution[J]. Journal of Shandong University(Engineering Science), 2017, 42 (6): 37- 42.
17 郭文, 游思思, 高君宇, 等. 深度相对度量学习的视觉跟踪[J]. 中国科学:信息科学, 2018, 48 (1): 60- 78.
GUO Wen , YOU Sisi , GAO Junyu , et al. Deep relative metric learning for visual tracking[J]. Scientia Sinica (Informationis), 2018, 48 (1): 60- 78.
18 酆勇, 熊庆宇, 石为人, 等. 深度非线性度量学习在说话人确认中的应用[J]. 声学学报, 2018, 43 (1): 112- 120.
FENG Yong , XIONG Qingyu , SHI Weiren , et al. Deep nonlinear metric learning for speaker verification[J]. Acta Acustica, 2018, 43 (1): 112- 120.
19 CAI Deng, HE Xiaofei. Face databases[EB/OL]. (2015-03-15)[2017-05-10]. http://www.zjucadcg.cn/dengcai/Data/FaceData.html.
20 ASUNCION A, NEWMAN D J. UCI machine learning repository[EB/OL]. (1995-03-01)[2017-05-10]. http://www.ics.uci.edu/~mlearn/MLR-epository.html.
[1] MA Chicheng, GUO Zonghe, LIU Canchang, DAI Xiangjun, ZHANG Xinong, MAO Boyong. Dynamics charactersitics of flexible beams undergoing time varying mass [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(4): 78-87.
[2] CAO Ya, DENG Zhaohong, WANG Shitong. An radial basis function neural network model based on monotonic constraints [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(3): 127-133.
[3] MA Hanjie, LIN Xia, XU Xiaohui, ZHANG Jian, ZHANG Zhisheng. Load optimization model of smart home management system based on adaptive particle swarm optimization [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2017, 47(6): 57-62.
[4] YE Dan, ZHANG Tianyu, LI Kui. Adaptive fault-tolerant containment control for multi-agent systems with unknown global information [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2017, 47(5): 1-6.
[5] CHU Zhenzhong, ZHU Daqi. Fault-tolerant control of autonomous underwater vehicle based on adaptive region tracking [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2017, 47(5): 57-63.
[6] LI Zhenwei, CUI Guozhong, GUO Congzhou, YU Changhao. Blind image restoration using alternating direction method of multipliers [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2017, 47(4): 14-18.
[7] LI Sushu, WANG Shitong, LI Tao. A feature selection method based on LS-SVM and fuzzy supplementary criterion [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2017, 47(3): 34-42.
[8] REN Yongfeng, DONG Xueyu. An image saliency object detection algorithm based on adaptive manifold similarity [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2017, 47(3): 56-62.
[9] FANG Hao, LI Yun. Random undersampling and POSS method for software defect prediction [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2017, 47(1): 15-21.
[10] WANG Mei, ZENG Zhaohu, SUN Yingqi, YANG Erlong, SONG Kaoping. Bayesian combination of SVR on regularization path based on KNN of input [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2016, 46(6): 8-14.
[11] TANG Qingshun, JIN Lu, LI Guodong, WU Chunfu. Robotic manipulators tracking control based on adaptive terminal sliding mode controller [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2016, 46(5): 45-53.
[12] MO Xiaoyong, PAN Zhisong, QIU Junyang, YU Yajun, JIANG Mingchu. Anomaly detection in network traffic based on online feature selection [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2016, 46(4): 21-27.
[13] SUN Meimei, HU Yun'an, WEI Jianming. Synchronization of multiwing hyperchaotic systems via adaptive sliding mode control [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2015, 45(6): 45-51.
[14] WEI Xiaomin, XU Bin, GUAN Jihong. Prediction of protein energy hot spots based on recursion feature elimination [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2014, 44(2): 12-20.
[15] YANG Xiulin1, HUANG Shuo2*, DENG Miao1, ZHANG Jihong1,3. Image fusion method based on saliency computation and adaptive PCNN [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2014, 44(2): 35-42.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
[1] HE Dongzhi, ZHANG Jifeng, ZHAO Pengfei. Parallel implementing probabilistic spreading algorithm using MapReduce programming mode[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 0, (): 22 -28 .
[2] ZHANG Jianming, LIU Quansheng, TANG Zhicheng, ZHAN Ting, JIANG Yalong. New peak shear strength criterion with inclusion of shear action history[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 0, (): 77 -81 .
[3] WANG Huan, ZHOU Zhongmei. An over sampling algorithm based on clustering[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(3): 134 -139 .
[4] YANG Ai-min1, ZHOU Yong-mei1, DENG He2, ZHOU Jian-feng3. Method of feature generation and selection for network traffic classification[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2010, 40(5): 1 -7 .
[5] YOU Ming-yu, CHEN Yan, LI Guo-zheng. Im-IG: A novel feature selection method for imbalanced problems[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2010, 40(5): 123 -128 .
[6] WU Guo-yao1, MA Li-yong2. A method based on FFD B-spline registration of the iris image fusion[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2010, 40(5): 24 -27 .
[7] XIAO Qiao, PEI Jihong, WANG Lixia, GONG Zhicheng. Ship detection in remote sensing image based on the fuzzy fusion of multi-channel Gabor filtering[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 0, (): 29 -35 .
[8] MA Xiangming, SUN Xia, ZHANG Qiang. Construction and analysis on typical working cycle of wheel loader[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 0, (): 82 -87 .
[9] WANG Lanzhong, MENG Wenjie. Video perceptual encryption algorithm in remote education receiver[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2015, 45(4): 40 -44 .
[10] LIANG Zehua, CUI Yaodong, ZHANG Yu. The one-dimensional cutting stock problem with sequence-dependent cut losses[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(3): 75 -80 .
Copyright 2018 © Editorial office of Journal of Shandong University (Engineering Science)
Supported by Beijing Magtech Co.Ltd