基于鉴别流形的不相关稀疏投影非负矩阵分解

doi:10.6040/j.issn.1672-3961.2.2014.155

摘要/Abstract

摘要： 基于流形学习、稀疏表示和鉴别分析理论,提出一种基于鉴别流形的统计不相关稀疏投影非负矩阵分解(discriminative manifold—based uncorrelated sparse projective NMF, DMUPNMF)算法。该方法继承了线性投影NMF优点,充分利用了数据集的局部和非局部几何鉴别信息,能够从数据集中抽取不相关鉴别特征,且分解结果具有良好的数据局部表示和稀疏性;给出多乘更新规则求解优化算法并证明其收敛性,还给出投影梯度优化算法以提高收敛速度。为解决大规模数据处理中计算量和存储空间过大问题,提出一种从训练集选取少量代表性样本学习DMUPNMF方法。大量的实验表明,该算法优于现有的改进NMF算法。

关键词: 非负矩阵分解, 鉴别流形, 统计不相关特征, 稀疏性, 投影梯度优化

Abstract: Inspired by manifold learning, sparse representation and discriminant analysis theories, a discriminative manifold-based uncorrelated sparse projective nonnegative matrix factorization(DMUPNMF)algorithm was developed in this work. By exploiting local and nonlocal geometric discriminant information of the data and the merits of the linear projective NMF, the extracted features were approximately uncorrelated and the decomposition results of DMUPNMF were sparse and better part-based representation. Multiplicative updating rules were introduced to slove the optimization problem of DMUPNMF and its convergence proof was given as well. Moreover, projected gradient decent optimization method was developed to enhance the convergence speed of the method. An approach was proposed to select the informative data points from training dataset, which reduces the computation burden and storage space resulted from a large amount of objects for traditional NMF. Simulations demonstrated that the proposed algorithm outperforms the state-of-the-art algorithms on real-world problems.

Key words: nonnegative matrix factorization, discriminative manifold, uncorrelated features, sparseness, projected gradient optimization

中图分类号:

TP301

李新玉, 徐桂云,任世锦,杨茂云. 基于鉴别流形的不相关稀疏投影非负矩阵分解[J]. 山东大学学报 (工学版), 2015, 45(5): 1-12.

LI Xinyu, XU Guiyun, REN Shijin, YANG Maoyun. Discriminative manifold-based uncorrelated sparse projective nonnegative matrix factorization[J]. Journal of Shandong University(Engineering Science), 2015, 45(5): 1-12.

参考文献

[1] CHEN Yan, ZHANG Jiemi, CAI Deng, et al. Nonnegative local coordinate factorization for image representation[J]. IEEE Transactions on Image Processing, 2013, 22(3):969-980.
[2] GUAN Naiyang, TAO Dacheng, LU Zhigang, et al. Manifold regularized discriminative nonnegative matrix factorization with fast gradient descent[J]. IEEE Transactions on Image Processing, 2011, 20(7):2030-2048.
[3] ZHENG C H, HUANG D S, ZHANG L, et al. Tumor clustering using nonnegative matrix factorization with gene selection[J]. IEEE Transactions on Information Technology in Biomedicine, 2009, 15(2):599-607.
[4] CAI Deng, HE Xiaofei, HAN Jiawei, et al. Graph regularized non-negative matrix factorization for data representation[J]. IEEE Transactions on Patten Analysis and Machine Intelligence, 2011, 23(6):902-913.
[5] YOO Jiho, CHOI Seungjin. Orthogonal nonnegative matrix tri-factorization for co-clustering:multiplicative updates on Stiefel manifolds[J]. Information Processing and Management, 2010, 46(5):721-732.
[6] LEE Seokjin, PARK Sangha, SUNG Koengmo. Beamspace-domain multichannel nonnegative matrix factorization for audio source separation[J]. IEEE Signal Processing Letters, 2012, 19(1):43-47.
[7] ZHOU Zhou, LIANG Ruiyu, ZHAO Li, et al. Unsupervised learning of phonemes of whispered speech in a noisy environment based on convolutive non-negative matrix factorization[J]. Information Sciences, 2014(257):115-126.
[8] LU Xiaoqiang, WU Hao, YUAN Yuan, et al. Manifold regularized sparse NMF for hyperspectral unmixing[J]. IEEE Transactions on Geomscience and Remote Sensing, 2013, 51(5):2815-2926.
[9] ZHAO Miao, BU Jiajun, CHEN Chun, et al. Graph regularized sparse coding for image representation[J]. IEEE Transactions on Image Processing, 2011, 20(5):1327-1337.
[10] LIANG Z. Projected gradient method for kernel discriminant nonnegative matrix factorization and the applications[J]. Signal Process, 2010, 90(7):2150-2163.
[11] QIAN Y, JIA S, ZHOU J, et al. Hyperspectral unmixing via L1/2 sparsity-constrained nonnegative matrix factorization[J]. IEEE Transactions Geoscience Remote Sense, 2011, 49(11):4282-4297.
[12] ZHI Ruicong, FLERIl Markus, RUAN Qiuqi, et al. Graph-preserving sparse nonnegative matrix factorization with application to facial expression recognition[J]. IEEE Transactions on Systems, Man, and Cybernatics—Part B:Cybernetics, 2011, 41(1):38-50.
[13] 杜世强, 石玉清, 王维兰,等. 基于流形正则化判别的因子分解[J]. 山东大学学报:理学版, 2013, 48(5):63-69. DU Shiqiang, SHI Yuqing, WANG Weilan, et al. Manifold regularization-based nonnegative matrix factorization[J]. Journal of Shandong University:Natural Science, 2013, 48(5):63-69.
[14] ZAFEIRIOUS S, TEFAS A, BUCIU I, et al. Exploiting discriminant information in nonnegative matrix factorization with application to frontal face verification[J]. IEEE Transactions on Neural Networks, 2006, 17(3):683-695.
[15] KOTSIA I, ZAFEIRIOUS S, PITAS I. A novel discriminant nonnegative matrix factorization algorithm with applications to facial image characterization problems[J].IEEE Transaction on Information Forensics and Security, 2007, 2(3):588-595.
[16] ZOIDI Olga, TEFAS Anastasios, PITAS Ioannis. Multiplicative update rules for concurrent nonnegative matrix factorization and maximum margin classification[J]. IEEE Transactions on Neural Networks and Learning Systems, 2013, 24(3):422-434.
[17] YE Jun, JIN Zhong. Non-negative matrix factorization based on fuzzy K nearest neighbor graph and its applications[J]. The Institution of Engineering and Technology Computer Vision, 2013, 7(5):346-353.
[18] ZHI Ruicong, FLIERL Markus, RUAN Qiuqi, et al. Graph-preserving sparse nonngegative matrix factorization with application to facial expression recognition[J]. IEEE Transactions on System, Man and cybernetics-Part B:Cybernetics, 2011, 41(1):38-51.
[19] GUAN Naiyang, TAO Dacheng, LUO Zhigang, et al. Online nonnegative matrix factorization with robust stochastic approximation[J]. IEEE Transactions on Neural Networks and Learning Systems, 2012, 23(7):1087-1099.
[20] HU Lirui, DAI Liang, WU Jianguo. Convergent projective non-negative matrix factorization with Kullback-Leibler divergence[J]. Pattern Recognition Letters, 2014(36):15-21.
[21] GONG Pinghua, ZHANG Changshui. Efficient nonnegative matrix factorization via projected newton method[J]. Pattern Recognition, 2012, 45(9):3557-3565.
[22] LIAO Ruiqi, ZHANG Yifan, GUAN Jihong, et al. Cloud NMF:A map reduce implementation of nonnegative matrix factorization for large-scale biological datasets[J]. Genomics, Proteomics & Bioinformatics, 2014, 12(1):48-51.
[23] IVICA Kopriva, IVANKA Jeric. Blind separation of analyze in nuclear magnetic resonance spectroscopy:Improved model for nonnegative matrix factorization[J]. Chemometrics and Intelligent Laboratory Systems, 2014, 137:47-56.
[24] VIRTANEN T. Nonaural sound source separation by non-negative matrix factorization with temporal continuity and sparseness criteria[J]. IEEE Transactions on Audio, Speech, Language Process, 2007, 15(3):1066-1074.
[25] CICHOCKI A, SHISHKINA S L, MUSHA T, et al. EEG filtering based on blind source separation(BSS)for early detection of Alzheimers disease[J]. Clinic Neurophysiology, 2005(116):729-737.
[26] 殷海青, 刘红卫. 一种基于L1稀疏正则化和非负矩阵分解的盲源信号分离新算法[J]. 西安电子科技大学学报, 2010, 37(5):835-841. YIN Haiqing, LIU Hongwei. New blind source separation algorithm based on L1 sparse regularization and nonnegative matricx factorization[J]. Journal of Xidian University, 2010, 37(5):835-841.
[27] KIRBIZ S, GUNSEL B. Perceptually enhanced blind single-channel music source separation by non-negative matrix factorization[J]. Digital Signal Processing, 2014(23):646-658.
[28] CICJOCKI A, ZDUNCK R. Multilayer nonnegative matrix factorization using projected gradient approaches[J]. International Journal of Neural Networks, 2007, 7(6):431-436.
[29] YAN-WANG Jim Jing, GAO Xin. Beyond cross-domain learning:multiple-domain nonnegative matrix factorization[J]. Engineering Applications of Artificial Intelligence, 2014(28):181-189.
[30] PAN Binbin, LAI Jianhuang, CHEN Wensheng. Nonlinear nonnegative matrix factorization based on Mercer kernel construction[J]. Pattern Recognition, 2011, 44(10—11):2800-2810.
[31] 李乐, 章毓晋. 非负矩阵分解算法综述[J]. 电子信息学报, 2008, 36(4):737-743. LI Le, ZHANG Yujin. Riews on nonnegative matrix factorization[J]. Journal of Electronics & Information Technology, 2008, 36(4):737-743.
[32] WANG Yuxiong, ZHANG Yujin. Nonnegative matrix factorization:a comprehensive review[J]. IEEE Transactions on Knowledge and Data Engineering, 2013, 25(6):1336-1353.
[33] 王洪元,封磊,冯燕,等. 流形学习算法在中文文本分类中的应用[J]. 山东大学学报:工学版, 2013, 42(4):8-13. WANG Hongyuan, FENG Lei, FENG Yan, et al. Manifold leanrning and its appplicaiton in text classification[J]. Journal of Shandong University:Engineering Science, 2013, 42(4):8-13.
[34] LIU X, YAN S, JIN H. Projective nonnegative graph embedding[J]. IEEE Transaction on Image Processing, 2010, 19(5):1126-1137.
[35] HE Ran, ZHENG Weishi, TAN Tieniu, et al.Half-quadratic-based iterative minimization for robust sparse representation[J]. IEEE Transactions on Patterns Analysis and Machine Intelligence, 2014, 36(2):261-275.
[36] YU Xuelian, WANG Xuegang. Uncorrelated discriminant locality preserving projections[J]. IEEE Signal Processing Letters, 2008(15):361-364.
[37] 夏海英,杜海明,徐鲁辉,等. 基于自适应词典学习和稀疏表示的人脸表情识别[J]. 山东大学学报:工学版,2014,44(1):45-49. XIA Haiying, DU Haiming, XU Luhui, et al. Facial expression recognition based on adaptive dictionary learning and sparse representation[J]. Journal of Shandong University:Engineering Science, 2014, 44(1):45-49.
[38] LU Haiping, KONSTANTINOS N, PLATANIOTIS V, et al. Uncorrelated multilinear discriminant analysis with regularization and aggregation for tensor object recognition[J]. IEEE Transactions on Neural Networks, 2009, 20(1):103-123.
[39] 张少捷, 王振雷, 钱锋. 基于LTSA的FS-SVDD方法及其在化工过程监控中的应用[J]. 化工学报, 2010, 61(8):1894-1900. ZHANG Shaojie, WANG Zhenglei, QIAN Feng. FS-SVDD based on LTSA and its application in chemical process monitoring[J]. CISE Journal, 2010, 61(8):1894-1900.
[40] NIE Feiping, ZENG Zinan, TSANG Ivor W, et al. Spectral embedded clustering: a framework for in-sample and out-of-sample spectral clustering[J]. IEEE Trans Neural Network, 2011, 22(11):1796-1808.

多维度评价

Viewed

Full text

Abstract

Cited

Shared

Discussed