自适应特征选择加权k子凸包分类

doi:10.6040/j.issn.1672-3961.0.2017.415

摘要/Abstract

摘要：

针对问题维数的增加以及不同特征对分类的作用往往不一样,导致k子凸包分类性能降低等问题,设计自适应特征选择加权k子凸包分类方法。根据传统凸包距离存在的不足引入加权k子凸包距离,在测试样本的k邻域内引入距离度量学习技术和正则化技术进行自适应的特征选择,并将自适应特征选择无缝嵌入加权k子凸包优化模型中,这样就能为不同的测试样本在不同的类别中学习自适应特征空间,得到有效的加权k子凸包距离计算方法。试验结果表明,该方法不仅能够进行降维,而且具有明显的分类性能优势。

关键词: 加权k子凸包, 度量学习, 正则化, 特征选择, 自适应

Abstract:

Because of the increase of the dimension of the problem and the effect of different features on classifier, the performance of the k sub-convex-hull classifier was seriously reduced. An adaptive feature selection weighted k sub-convex-hull classifier was designed (AWCH). A weighted k sub-convex-hull classifier was designed according to the shortcomings of conventional convex hull distance. By applying the distance metric learning and regularization technique in the k neighborhood of the test sample, an adaptive feature selection method was designed and seamlessly integrated into the optimization model on the weighted k sub-convex-hull. Through these efforts, for different test samples, an adaptive feature space in different categories could be extracted, and a valid weighted k sub-convex-hull distance could be obtained. Experimental results showed that the AWCH not only reduced the dimension of the problem, but also was significantly superior to similar classifiers.

Key words: weighted k sub-convex-hull classifier, distance metric learning, regularization, feature selection, adaptive

中图分类号:

TP391

牟廉明. 自适应特征选择加权k子凸包分类[J]. 山东大学学报 (工学版), 2018, 48(5): 32-37.

Lianming MOU. Weighted k sub-convex-hull classifier based on adaptive feature selection[J]. Journal of Shandong University(Engineering Science), 2018, 48(5): 32-37.

图/表 2

表1

表2

分类错误率和标准偏差比较"

数据集	CKNN		kCH		RLHC		AWCH
数据集	Err	Std	Err	Std	Err	Std	Err	Std
artificial	0.433 0	0.004 4	0.429 8	0.003 2	0.377 4	0.003 1	0.322 6	0.002 4
austra	0.163 5	0.006 8	0.155 1	0.005 7	0.156 1	0.005 3	0.135 0	0.004 7
balance-scale	0.110 7	0.005 7	0.103 4	0.005 0	0.103 7	0.007 7	0.067 5	0.0043
BCI	0.445 5	0.016 0	0.460 5	0.014 7	0.390 5	0.020 0	0.338 8	0.007 3
breast-w	0.032 2	0.001 5	0.031 9	0.000 9	0.026 8	0.0028	0.020 0	0.001 2
bupa	0.368 4	0.011 6	0.357 1	0.009 0	0.333 6	0.016 5	0.284 3	0.014 9
clean1	0.247 1	0.009 8	0.140 2	0.009 3	0.183 2	0.010 7	0.082 8	0.008 5
diabetes	0.260 9	0.007 7	0.259 8	0.008 1	0.257 6	0.007 9	0.202 0	0.007 4
digits	0.023 5	0.001 7	0.021 2	0.001 5	0.009 5	0.000 7	0.007 7	0.000 6
ecoli	0.145 6	0.004 9	0.146 0	0.007 8	0.132 4	0.009 1	0.127 7	0.001 3
ethn	0.046 7	0.001 9	0.033 9	0.001 6	0.021 7	0.001 5	0.010 3	0.001 4
glass	0.375 3	0.009 8	0.329 9	0.016 5	0.308 9	0.015 5	0.268 1	0.011 8
heart-statlog	0.180 0	0.006 8	0.180 4	0.005 0	0.210 7	0.019 5	0.111 5	0.017 6
ionosphere	0.167 8	0.007 4	0.161 6	0.005 9	0.105 3	0.008 2	0.078 4	0.007 4
iris	0.049 3	0.005 8	0.044 0	0.003 4	0.0427	0.004 7	0.040 7	0.004 7
isolet_norm	0.006 2	0.001 9	0.008 0	0.001 9	0.00 43	0.001 4	0.003 8	0.001 2
letter	0.063 0	0.001 0	0.060 9	0.001 1	0.034 5	0.000 6	0.028 3	0.000 4
LIBRASMovement	0.446 5	0.025 9	0.460 5	0.017 4	0.125 1	0.009 4	0.116 7	0.008 8
machine	0.233 9	0.014 5	0.233 7	0.014 7	0.088 3	0.011 5	0.083 0	0.010 6
mfeat-factors	0.047 1	0.001 3	0.044 5	0.001 2	0.030 4	0.001 5	0.030 0	0.001 1
mfeat-fourier	0.188 7	0.004 6	0.190 8	0.002 8	0.170 1	0.004 0	0.169 8	0.002 1
mfeat-karhunen	0.052 5	0.001 6	0.050 2	0.001 2	0.026 1	0.002 8	0.022 5	0.001 1
mfeat-zernike	0.185 5	0.006 7	0.179 1	0.005 0	0.165 4	0.004 6	0.149 4	0.003 7
musk	0.247 1	0.009 8	0.140 2	0.008 3	0.103 2	0.010 7	0.082 8	0.008 5
optdigits	0.017 1	0.000 5	0.016 2	0.000 4	0.008 1	0.000 4	0.008 0	0.000 3
page-blocks	0.050 8	0.000 9	0.049 4	0.001 1	0.042 4	0.002 4	0.034 7	0.000 9
Yale_32x32_face	0.486 2	0.013 9	0.475 1	0.024 3	0.278 2	0.011 6	0.229 1	0.010 0
Yale_64x64_face	0.363 1	0.017 7	0.385 3	0.016 5	0.271 3	0.008 7	0.254 0	0.008 1
ORL_32x32_face	0.354 5	0.016 8	0.367 0	0.016 1	0.034 3	0.005 8	0.030 3	0.003 3
ORL_64x64_face	0.360 3	0.012 7	0.362 5	0.012 1	0.034 8	0.006 2	0.030 2	0.005 0

表2

参考文献 20

1	VINCENT P , BENGIO Y . K-local hyperplane and convex distance nearest neighbor algorithms[J]. In Advances in Neural Information Processing Systems, 2002, 14 (1): 985- 992.
2	ZHANG Y, TANG Z M, LI Y P, et al. Ensemble learning and optimizing KNN method for speaker recognition[C]//Proceedings of the Fourth International Conference on Fuzzy System and Knowledge Discovery (FSKD). Haikou, China: [S.l.], 2007: 285-289.
3	YANG T , KECMAN V . Adaptive local hyperplane classification[J]. Neurocomputing, 2008, (71): 3001- 3004.
4	YANG T , KECMAN V . Face recognition with adaptive local hyperplane algorithm[J]. Pattern Anal Applic, 2010, (13): 79- 83.
5	WEN Guihua , JIANG Lijun , WEN Jun , et al. Perceptual relativity-based local hyperplane classification[J]. Neurocomputing, 2012, (97): 155- 163.
6	XU Jie , YANG Jian , LAI Zhihui . K-local hyperplane distance nearest neighbor classifier oriented local discriminant analysis[J]. Information Sciences, 2013, 232, 11- 26. doi: 10.1016/j.ins.2012.12.045
7	牟廉明. k子凸包分类[J]. 山西大学学报(自然科学版), 2011, 34 (3): 374- 380.
	MOU Lianming . A k sub-convex-hull classifier[J]. Journal of Shanxi University (Nat Sci Ed), 2011, 34 (3): 374- 380.
8	牟廉明. 选择性自适应k子凸包分类方法[J]. 南京大学学报(自然科学), 2013, 49 (4): 410- 416.
	MOU Lianming . Selective adaptive k sub-convex-hullclassifier[J]. Journal of Nanjing University(Natural Sciences), 2013, 49 (4): 410- 416.
9	李素姝, 王士同, 李滔. 基于LS-SVM与模糊补准则的特征选择方法[J]. 山东大学学报(工学版), 2017, 47 (3): 34- 42.
	LI Sushu , WANG Shitong , LI Tao . A feature selection method based on LS-SVM and fuzzy supplementary criterion[J]. Journal of Shandong University(Engineering Science), 2017, 47 (3): 34- 42.
10	王法波, 许信顺. 文本分类中一种新的特征选择方法[J]. 山东大学学报(工学版), 2017, 40 (4): 8- 11, 18.
	WANG Fabo , XU Xinshun . A new feature selection method for text categorization[J]. Journal of Shandong University(Engineering Science), 2017, 40 (4): 8- 11, 18.
11	戴平, 李宁. 一种基于SVM的快速特征选择方法[J]. 山东大学学报(工学版), 2017, 40 (5): 60- 65.
	DAI Ping , LI Ning . A fast SVM-based feature selection method[J]. Journal of Shandong University (Engineering Science), 2017, 40 (5): 60- 65.
12	赵佳, 王士同. 特征加权距离的半监督模糊子空间聚类算法[J]. 小型微型计算机系统, 2017, 38 (2): 405- 410.
	ZHAO Jia , WANG Shitong . Semi-supervised fuzzy subspace clustering algorithm based on feature weighted distance[J]. Journal of Chinese Computer Systems, 2017, 38 (2): 405- 410.
13	贾隆嘉, 孙铁利, 杨凤芹, 等. 基于类空间密度的文本分类特征加权算法[J]. 吉林大学学报(信息科学版), 2017, 35 (1): 92- 97. doi: 10.3969/j.issn.1671-5896.2017.01.015
	JIA Longjia , SUN Tieli , YANG Fengqin , et al. Class space density based weighting scheme for automated text categorization[J]. Journal of Jilin University(Information Science Edition), 2017, 35 (1): 92- 97. doi: 10.3969/j.issn.1671-5896.2017.01.015
14	KILIAN Weinberger . Distance metric learning for large margin nearest neighbor classification[J]. Journal of Machine Learning Research, 2009, 10, 207- 244.
15	BAR-HILLEL A , HERTZ T , SHENTAL N , et al. Learning a mahalanobis metric from equivalence constraints[J]. Journal of Machine Learning Research, 2006, 6 (1): 937- 965.
16	钱强, 陈松灿. 基于矩阵正态分布似然比测试的矩阵度量学习算法[J]. 山东大学学报(工学版), 2017, 42 (6): 37- 42.
	QIAN Qiang , CHEN Songcan . Matrix metric learning algorithm based on likelihood ratio test with matrix normal distribution[J]. Journal of Shandong University(Engineering Science), 2017, 42 (6): 37- 42.
17	郭文, 游思思, 高君宇, 等. 深度相对度量学习的视觉跟踪[J]. 中国科学:信息科学, 2018, 48 (1): 60- 78.
	GUO Wen , YOU Sisi , GAO Junyu , et al. Deep relative metric learning for visual tracking[J]. Scientia Sinica (Informationis), 2018, 48 (1): 60- 78.
18	酆勇, 熊庆宇, 石为人, 等. 深度非线性度量学习在说话人确认中的应用[J]. 声学学报, 2018, 43 (1): 112- 120.
	FENG Yong , XIONG Qingyu , SHI Weiren , et al. Deep nonlinear metric learning for speaker verification[J]. Acta Acustica, 2018, 43 (1): 112- 120.
19	CAI Deng, HE Xiaofei. Face databases[EB/OL]. (2015-03-15)[2017-05-10]. http://www.zjucadcg.cn/dengcai/Data/FaceData.html.
20	ASUNCION A, NEWMAN D J. UCI machine learning repository[EB/OL]. (1995-03-01)[2017-05-10]. http://www.ics.uci.edu/~mlearn/MLR-epository.html.

多维度评价

Viewed

Full text

Abstract

Cited

Shared

Discussed

[1]	钱淑渠,武慧虹,徐国峰,金晶亮. 计及排放的动态经济调度免疫克隆演化算法[J]. 山东大学学报(工学版), 2018, 48(4): 1-9.
[2]	马驰骋,郭宗和,刘灿昌,代祥俊,张希农,毛伯永. 变质量弹性梁结构动力学特性[J]. 山东大学学报(工学版), 2018, 48(4): 78-87.
[3]	曹雅,邓赵红,王士同. 基于单调约束的径向基函数神经网络模型[J]. 山东大学学报(工学版), 2018, 48(3): 127-133.
[4]	张博涵,陈哲明,付江华,陈宝. 四轮独立驱动电动汽车自适应驱动防滑控制[J]. 山东大学学报(工学版), 2018, 48(1): 96-103.
[5]	马汉杰,林霞,胥晓晖,张健,张智晟. 基于自适应粒子群算法的智能家居管理系统负荷优化模型[J]. 山东大学学报(工学版), 2017, 47(6): 57-62.
[6]	叶丹,张天予,李奎. 全局信息未知的多智能体自适应容错包容控制[J]. 山东大学学报(工学版), 2017, 47(5): 1-6.
[7]	褚振忠,朱大奇. 基于自适应区域跟踪的自主式水下机器人容错控制[J]. 山东大学学报(工学版), 2017, 47(5): 57-63.
[8]	李真伟,崔国忠,郭从洲,虞昌浩. 基于交替方向乘子法的图像盲复原[J]. 山东大学学报(工学版), 2017, 47(4): 14-18.
[9]	李素姝,王士同,李滔. 基于LS-SVM与模糊补准则的特征选择方法[J]. 山东大学学报(工学版), 2017, 47(3): 34-42.
[10]	任永峰,董学育. 基于自适应流形相似性的图像显著性区域提取算法[J]. 山东大学学报(工学版), 2017, 47(3): 56-62.
[11]	方昊,李云. 基于多次随机欠采样和POSS方法的软件缺陷检测[J]. 山东大学学报(工学版), 2017, 47(1): 15-21.
[12]	王梅,曾昭虎,孙莺萁,杨二龙,宋考平. 基于输入K-近邻的正则化路径上SVR贝叶斯组合[J]. 山东大学学报(工学版), 2016, 46(6): 8-14.
[13]	唐庆顺,金璐,李国栋,吴春富. 基于自适应终端滑模控制器的机械手跟踪控制[J]. 山东大学学报(工学版), 2016, 46(5): 45-53.
[14]	莫小勇,潘志松,邱俊洋,余亚军,蒋铭初. 基于在线特征选择的网络流异常检测[J]. 山东大学学报(工学版), 2016, 46(4): 21-27.
[15]	孙美美, 胡云安, 韦建明. 多涡卷超混沌系统自适应滑模同步控制[J]. 山东大学学报(工学版), 2015, 45(6): 45-51.