Journal of Shandong University(Engineering Science) ›› 2018, Vol. 48 ›› Issue (5): 32-37.doi: 10.6040/j.issn.1672-3961.0.2017.415

• Machine Learning & Data Mining • Previous Articles     Next Articles

Weighted k sub-convex-hull classifier based on adaptive feature selection

Lianming MOU1,2()   

  1. 1. College of Mathematics and Information Science, Neijiang Normal University, Neijiang 641100, Sichuan, China
    2. Data Recovery Key Laboratory of Sichuan Province, Neijiang 641100, Sichuan, China
  • Received:2017-05-09 Online:2018-10-01 Published:2017-05-09
  • Supported by:
    国家自然科学基金资助项目(10872085);四川省科技厅科技计划重点资助项目(2017JY0199);四川教育厅自然科学重点项目基金资助项目(13ZA0008);2015内江市科技支撑计划资助项目

RichHTML

15

PDF (PC)

1566

Abstract:

Because of the increase of the dimension of the problem and the effect of different features on classifier, the performance of the k sub-convex-hull classifier was seriously reduced. An adaptive feature selection weighted k sub-convex-hull classifier was designed (AWCH). A weighted k sub-convex-hull classifier was designed according to the shortcomings of conventional convex hull distance. By applying the distance metric learning and regularization technique in the k neighborhood of the test sample, an adaptive feature selection method was designed and seamlessly integrated into the optimization model on the weighted k sub-convex-hull. Through these efforts, for different test samples, an adaptive feature space in different categories could be extracted, and a valid weighted k sub-convex-hull distance could be obtained. Experimental results showed that the AWCH not only reduced the dimension of the problem, but also was significantly superior to similar classifiers.

Key words: weighted k sub-convex-hull classifier, distance metric learning, regularization, feature selection, adaptive

CLC Number: 

  • TP391

Table 1

Experimental data sets"

编号 数据集 样本数 属性 类别
1 artificial 6 000 7 10
2 austra 690 15 2
3 balance-scale 625 4 3
4 BCI 400 117 2
5 breast-w 699 9 2
6 bupa 345 6 2
7 clean1 476 166 2
8 diabetes 768 8 2
9 digits 1 797 64 10
10 ecoli 336 7 8
11 ethn 2 630 30 2
12 glass 214 9 7
13 heart-statlog 270 13 2
14 ionosphere 351 34 2
15 iris 150 4 3
16 isolet_norm 600 51 2
17 letter 20 000 16 26
18 LIBRASMovement 360 90 15
19 machine 209 7 8
20 mfeat-factors 2 000 216 10
21 mfeat-fourier 2 000 76 10
22 mfeat-karhunen 2 000 64 10
23 mfeat-zernike 2 000 47 10
24 musk 476 166 2
25 optdigits 5 620 64 10
26 page-blocks 5 473 10 5
27 Yale_32x32_face 165 1024 15
28 Yale_64x64_face 165 4 096 15
29 ORL_32x32_face 400 1 024 40
30 ORL_64x64_face 400 4 096 40

Table 2

Comparison of classification error rate and standard deviation"

数据集CKNN kCH RLHC AWCH
Err Std Err Std Err Std Err Std
artificial 0.433 0 0.004 4 0.429 8 0.003 2 0.377 4 0.003 1 0.322 6 0.002 4
austra 0.163 5 0.006 8 0.155 1 0.005 7 0.156 1 0.005 3 0.135 0 0.004 7
balance-scale 0.110 7 0.005 7 0.103 4 0.005 0 0.103 7 0.007 7 0.067 5 0.0043
BCI 0.445 5 0.016 0 0.460 5 0.014 7 0.390 5 0.020 0 0.338 8 0.007 3
breast-w 0.032 2 0.001 5 0.031 9 0.000 9 0.026 8 0.0028 0.020 0 0.001 2
bupa 0.368 4 0.011 6 0.357 1 0.009 0 0.333 6 0.016 5 0.284 3 0.014 9
clean1 0.247 1 0.009 8 0.140 2 0.009 3 0.183 2 0.010 7 0.082 8 0.008 5
diabetes 0.260 9 0.007 7 0.259 8 0.008 1 0.257 6 0.007 9 0.202 0 0.007 4
digits 0.023 5 0.001 7 0.021 2 0.001 5 0.009 5 0.000 7 0.007 7 0.000 6
ecoli 0.145 6 0.004 9 0.146 0 0.007 8 0.132 4 0.009 1 0.127 7 0.001 3
ethn 0.046 7 0.001 9 0.033 9 0.001 6 0.021 7 0.001 5 0.010 3 0.001 4
glass 0.375 3 0.009 8 0.329 9 0.016 5 0.308 9 0.015 5 0.268 1 0.011 8
heart-statlog 0.180 0 0.006 8 0.180 4 0.005 0 0.210 7 0.019 5 0.111 5 0.017 6
ionosphere 0.167 8 0.007 4 0.161 6 0.005 9 0.105 3 0.008 2 0.078 4 0.007 4
iris 0.049 3 0.005 8 0.044 0 0.003 4 0.0427 0.004 7 0.040 7 0.004 7
isolet_norm 0.006 2 0.001 9 0.008 0 0.001 9 0.00 43 0.001 4 0.003 8 0.001 2
letter 0.063 0 0.001 0 0.060 9 0.001 1 0.034 5 0.000 6 0.028 3 0.000 4
LIBRASMovement 0.446 5 0.025 9 0.460 5 0.017 4 0.125 1 0.009 4 0.116 7 0.008 8
machine 0.233 9 0.014 5 0.233 7 0.014 7 0.088 3 0.011 5 0.083 0 0.010 6
mfeat-factors 0.047 1 0.001 3 0.044 5 0.001 2 0.030 4 0.001 5 0.030 0 0.001 1
mfeat-fourier 0.188 7 0.004 6 0.190 8 0.002 8 0.170 1 0.004 0 0.169 8 0.002 1
mfeat-karhunen 0.052 5 0.001 6 0.050 2 0.001 2 0.026 1 0.002 8 0.022 5 0.001 1
mfeat-zernike 0.185 5 0.006 7 0.179 1 0.005 0 0.165 4 0.004 6 0.149 4 0.003 7
musk 0.247 1 0.009 8 0.140 2 0.008 3 0.103 2 0.010 7 0.082 8 0.008 5
optdigits 0.017 1 0.000 5 0.016 2 0.000 4 0.008 1 0.000 4 0.008 0 0.000 3
page-blocks 0.050 8 0.000 9 0.049 4 0.001 1 0.042 4 0.002 4 0.034 7 0.000 9
Yale_32x32_face 0.486 2 0.013 9 0.475 1 0.024 3 0.278 2 0.011 6 0.229 1 0.010 0
Yale_64x64_face 0.363 1 0.017 7 0.385 3 0.016 5 0.271 3 0.008 7 0.254 0 0.008 1
ORL_32x32_face 0.354 5 0.016 8 0.367 0 0.016 1 0.034 3 0.005 8 0.030 3 0.003 3
ORL_64x64_face 0.360 3 0.012 7 0.362 5 0.012 1 0.034 8 0.006 2 0.030 2 0.005 0
1 VINCENT P , BENGIO Y . K-local hyperplane and convex distance nearest neighbor algorithms[J]. In Advances in Neural Information Processing Systems, 2002, 14 (1): 985- 992.
2 ZHANG Y, TANG Z M, LI Y P, et al. Ensemble learning and optimizing KNN method for speaker recognition[C]//Proceedings of the Fourth International Conference on Fuzzy System and Knowledge Discovery (FSKD). Haikou, China: [S.l.], 2007: 285-289.
3 YANG T , KECMAN V . Adaptive local hyperplane classification[J]. Neurocomputing, 2008, (71): 3001- 3004.
4 YANG T , KECMAN V . Face recognition with adaptive local hyperplane algorithm[J]. Pattern Anal Applic, 2010, (13): 79- 83.
5 WEN Guihua , JIANG Lijun , WEN Jun , et al. Perceptual relativity-based local hyperplane classification[J]. Neurocomputing, 2012, (97): 155- 163.
6 XU Jie , YANG Jian , LAI Zhihui . K-local hyperplane distance nearest neighbor classifier oriented local discriminant analysis[J]. Information Sciences, 2013, 232, 11- 26.
doi: 10.1016/j.ins.2012.12.045
7 牟廉明. k子凸包分类[J]. 山西大学学报(自然科学版), 2011, 34 (3): 374- 380.
MOU Lianming . A k sub-convex-hull classifier[J]. Journal of Shanxi University (Nat Sci Ed), 2011, 34 (3): 374- 380.
8 牟廉明. 选择性自适应k子凸包分类方法[J]. 南京大学学报(自然科学), 2013, 49 (4): 410- 416.
MOU Lianming . Selective adaptive k sub-convex-hullclassifier[J]. Journal of Nanjing University(Natural Sciences), 2013, 49 (4): 410- 416.
9 李素姝, 王士同, 李滔. 基于LS-SVM与模糊补准则的特征选择方法[J]. 山东大学学报(工学版), 2017, 47 (3): 34- 42.
LI Sushu , WANG Shitong , LI Tao . A feature selection method based on LS-SVM and fuzzy supplementary criterion[J]. Journal of Shandong University(Engineering Science), 2017, 47 (3): 34- 42.
10 王法波, 许信顺. 文本分类中一种新的特征选择方法[J]. 山东大学学报(工学版), 2017, 40 (4): 8- 11, 18.
WANG Fabo , XU Xinshun . A new feature selection method for text categorization[J]. Journal of Shandong University(Engineering Science), 2017, 40 (4): 8- 11, 18.
11 戴平, 李宁. 一种基于SVM的快速特征选择方法[J]. 山东大学学报(工学版), 2017, 40 (5): 60- 65.
DAI Ping , LI Ning . A fast SVM-based feature selection method[J]. Journal of Shandong University (Engineering Science), 2017, 40 (5): 60- 65.
12 赵佳, 王士同. 特征加权距离的半监督模糊子空间聚类算法[J]. 小型微型计算机系统, 2017, 38 (2): 405- 410.
ZHAO Jia , WANG Shitong . Semi-supervised fuzzy subspace clustering algorithm based on feature weighted distance[J]. Journal of Chinese Computer Systems, 2017, 38 (2): 405- 410.
13 贾隆嘉, 孙铁利, 杨凤芹, 等. 基于类空间密度的文本分类特征加权算法[J]. 吉林大学学报(信息科学版), 2017, 35 (1): 92- 97.
doi: 10.3969/j.issn.1671-5896.2017.01.015
JIA Longjia , SUN Tieli , YANG Fengqin , et al. Class space density based weighting scheme for automated text categorization[J]. Journal of Jilin University(Information Science Edition), 2017, 35 (1): 92- 97.
doi: 10.3969/j.issn.1671-5896.2017.01.015
14 KILIAN Weinberger . Distance metric learning for large margin nearest neighbor classification[J]. Journal of Machine Learning Research, 2009, 10, 207- 244.
15 BAR-HILLEL A , HERTZ T , SHENTAL N , et al. Learning a mahalanobis metric from equivalence constraints[J]. Journal of Machine Learning Research, 2006, 6 (1): 937- 965.
16 钱强, 陈松灿. 基于矩阵正态分布似然比测试的矩阵度量学习算法[J]. 山东大学学报(工学版), 2017, 42 (6): 37- 42.
QIAN Qiang , CHEN Songcan . Matrix metric learning algorithm based on likelihood ratio test with matrix normal distribution[J]. Journal of Shandong University(Engineering Science), 2017, 42 (6): 37- 42.
17 郭文, 游思思, 高君宇, 等. 深度相对度量学习的视觉跟踪[J]. 中国科学:信息科学, 2018, 48 (1): 60- 78.
GUO Wen , YOU Sisi , GAO Junyu , et al. Deep relative metric learning for visual tracking[J]. Scientia Sinica (Informationis), 2018, 48 (1): 60- 78.
18 酆勇, 熊庆宇, 石为人, 等. 深度非线性度量学习在说话人确认中的应用[J]. 声学学报, 2018, 43 (1): 112- 120.
FENG Yong , XIONG Qingyu , SHI Weiren , et al. Deep nonlinear metric learning for speaker verification[J]. Acta Acustica, 2018, 43 (1): 112- 120.
19 CAI Deng, HE Xiaofei. Face databases[EB/OL]. (2015-03-15)[2017-05-10]. http://www.zjucadcg.cn/dengcai/Data/FaceData.html.
20 ASUNCION A, NEWMAN D J. UCI machine learning repository[EB/OL]. (1995-03-01)[2017-05-10]. http://www.ics.uci.edu/~mlearn/MLR-epository.html.
[1] TANG Jiefeng, ZHANG Jia, LONG Jinyi. Fast multi-label feature selection method based on global redundancy minimization [J]. Journal of Shandong University(Engineering Science), 2025, 55(6): 21-34.
[2] ZHOU Qian, LI Qun, ZHU Dandan, LI Yibo. Coordinated inertia response control for offshore low frequency wind power system based on adaptive virtual inertia of M3C [J]. Journal of Shandong University(Engineering Science), 2025, 55(5): 30-39.
[3] LI Xiaohui, LIU Xiaofei, SUN Weitong, ZHAO Yi, DONG Yuan, JIN Yinli. An inspection task assignment and path planning algorithm based on vehicles-UAVs collaboration [J]. Journal of Shandong University(Engineering Science), 2025, 55(5): 101-109.
[4] ZHENG Xiao, CHEN He, ZHOU Dongao, GONG Yongshun. Video anomaly detection method based on video caption augmentation and dual-stream feature fusion [J]. Journal of Shandong University(Engineering Science), 2025, 55(5): 110-119.
[5] GAO Junjian, LIAO Zhuhua, LIU Yizhi, ZHAO Yijiang. Hierarchical multi-agent reinforcement learning based route guidance method combining personalization and signal control [J]. Journal of Shandong University(Engineering Science), 2025, 55(3): 34-45.
[6] Ying LI,Jiankun WANG. The classification of mild cognitive impairment based on supervised graph regularization and information fusion [J]. Journal of Shandong University(Engineering Science), 2023, 53(4): 65-73.
[7] Caihui LIU,Qi ZHOU,Xiaowen YE. An intrusion detection model based on improved ReliefF algorithm [J]. Journal of Shandong University(Engineering Science), 2023, 53(2): 1-10.
[8] Haigen MIN,Yukun FANG,Xia WU,Wuqi WANG. Fault diagnosis of vehicle-to-vehicle communication in networked traffic environment [J]. Journal of Shandong University(Engineering Science), 2021, 51(6): 84-92.
[9] YANG Xiuyuan, PENG Tao, YANG Liang, LIN Hongfei. Adaptive multi-domain sentiment analysis based on knowledge distillation [J]. Journal of Shandong University(Engineering Science), 2021, 51(3): 15-21.
[10] LIANG Qixing, LI Bin, LI Zhi, ZHANG Hui, RONG Xuewen, FAN Yong. Algorithm of adaptive slope adjustment of quadruped robot based on model predictive control and its application [J]. Journal of Shandong University(Engineering Science), 2021, 51(3): 37-44.
[11] LIN Xiaowei, CHEN Lifei. Community detection using nonnegative matrix factorization with structure extension [J]. Journal of Shandong University(Engineering Science), 2021, 51(2): 57-64.
[12] ZHOU Kaiqing, LI Hangcheng, MO Liping. Adaptive harmony search algorithm based on global optimization [J]. Journal of Shandong University(Engineering Science), 2021, 51(2): 47-56.
[13] Chunrui CHENG,Beixing MAO. Adaptive sliding mode synchronization of a class of nonlinear chaotic systems [J]. Journal of Shandong University(Engineering Science), 2020, 50(5): 1-6.
[14] Yan PENG,Tingting FENG,Jie WANG. An integrated learning approach for O3 mass concentration prediction model [J]. Journal of Shandong University(Engineering Science), 2020, 50(4): 1-7.
[15] WANG Chunyan, DI Jinhong, MAO Beixing. Sliding mode synchronization of fractional-order Rucklidge systems with unknown parameters based on a new type of reaching law [J]. Journal of Shandong University(Engineering Science), 2020, 50(4): 40-45.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
[1] WANG Su-yu,<\sup>,AI Xing<\sup>,ZHAO Jun<\sup>,LI Zuo-li<\sup>,LIU Zeng-wen<\sup> . Milling force prediction model for highspeed end milling 3Cr2Mo steel[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(1): 1 -5 .
[2] ZHANG Yong-hua,WANG An-ling,LIU Fu-ping . The reflected phase angle of low frequent inhomogeneous[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(2): 22 -25 .
[3] LI Kan . Empolder and implement of the embedded weld control system[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2008, 38(4): 37 -41 .
[4] KONG Xiang-zhen,LIU Yan-jun,WANG Yong,ZHAO Xiu-hua . Compensation and simulation for the deadband of the pneumatic proportional valve[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(1): 99 -102 .
[5] LAI Xiang . The global domain of attraction for a kind of MKdV equations[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(1): 87 -92 .
[6] YU Jia yuan1, TIAN Jin ting1, ZHU Qiang zhong2. Computational intelligence and its application in psychology[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(1): 1 -5 .
[7] CHEN Rui, LI Hongwei, TIAN Jing. The relationship between the number of magnetic poles and the bearing capacity of radial magnetic bearing[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(2): 81 -85 .
[8] LI Ke,LIU Chang-chun,LI Tong-lei . Medical registration approach using improved maximization of mutual information[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(2): 107 -110 .
[9] JI Tao,GAO Xu/sup>,SUN Tong-jing,XUE Yong-duan/sup>,XU Bing-yin/sup> . Characteristic analysis of fault generated traveling waves in 10 Kv automatic blocking and continuous power transmission lines[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(2): 111 -116 .
[10] . [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(1): 27 -32 .
Copyright 2018 © Editorial office of Journal of Shandong University (Engineering Science)
Supported by Beijing Magtech Co.Ltd