Journal of Shandong University(Engineering Science) ›› 2015, Vol. 45 ›› Issue (5): 1-12.doi: 10.6040/j.issn.1672-3961.2.2014.155

   

Discriminative manifold-based uncorrelated sparse projective nonnegative matrix factorization

LI Xinyu1, XU Guiyun1, REN Shijin2,3*, YANG Maoyun1,2   

  1. 1.School of Mechatronic Engineering, China University of Mining and Technology, Xuzhou 221116, Jiangsu, China;
    2. School of Computer Science and Technology, Jiangsu Normal University, Xuzhou 221116, Jiangsu, China;
    3. National Laboratory of Industrial Control Technology, Zhejiang University, Hangzhou 310027, Jiangsu, China
  • Published:2014-05-23

PDF (PC)

1044

Abstract: Inspired by manifold learning, sparse representation and discriminant analysis theories, a discriminative manifold-based uncorrelated sparse projective nonnegative matrix factorization(DMUPNMF)algorithm was developed in this work. By exploiting local and nonlocal geometric discriminant information of the data and the merits of the linear projective NMF, the extracted features were approximately uncorrelated and the decomposition results of DMUPNMF were sparse and better part-based representation. Multiplicative updating rules were introduced to slove the optimization problem of DMUPNMF and its convergence proof was given as well. Moreover, projected gradient decent optimization method was developed to enhance the convergence speed of the method. An approach was proposed to select the informative data points from training dataset, which reduces the computation burden and storage space resulted from a large amount of objects for traditional NMF. Simulations demonstrated that the proposed algorithm outperforms the state-of-the-art algorithms on real-world problems.

Key words: nonnegative matrix factorization, discriminative manifold, uncorrelated features, sparseness, projected gradient optimization

CLC Number: 

  • TP301
[1] CHEN Yan, ZHANG Jiemi, CAI Deng, et al. Nonnegative local coordinate factorization for image representation[J]. IEEE Transactions on Image Processing, 2013, 22(3):969-980.
[2] GUAN Naiyang, TAO Dacheng, LU Zhigang, et al. Manifold regularized discriminative nonnegative matrix factorization with fast gradient descent[J]. IEEE Transactions on Image Processing, 2011, 20(7):2030-2048.
[3] ZHENG C H, HUANG D S, ZHANG L, et al. Tumor clustering using nonnegative matrix factorization with gene selection[J]. IEEE Transactions on Information Technology in Biomedicine, 2009, 15(2):599-607.
[4] CAI Deng, HE Xiaofei, HAN Jiawei, et al. Graph regularized non-negative matrix factorization for data representation[J]. IEEE Transactions on Patten Analysis and Machine Intelligence, 2011, 23(6):902-913.
[5] YOO Jiho, CHOI Seungjin. Orthogonal nonnegative matrix tri-factorization for co-clustering:multiplicative updates on Stiefel manifolds[J]. Information Processing and Management, 2010, 46(5):721-732.
[6] LEE Seokjin, PARK Sangha, SUNG Koengmo. Beamspace-domain multichannel nonnegative matrix factorization for audio source separation[J]. IEEE Signal Processing Letters, 2012, 19(1):43-47.
[7] ZHOU Zhou, LIANG Ruiyu, ZHAO Li, et al. Unsupervised learning of phonemes of whispered speech in a noisy environment based on convolutive non-negative matrix factorization[J]. Information Sciences, 2014(257):115-126.
[8] LU Xiaoqiang, WU Hao, YUAN Yuan, et al. Manifold regularized sparse NMF for hyperspectral unmixing[J]. IEEE Transactions on Geomscience and Remote Sensing, 2013, 51(5):2815-2926.
[9] ZHAO Miao, BU Jiajun, CHEN Chun, et al. Graph regularized sparse coding for image representation[J]. IEEE Transactions on Image Processing, 2011, 20(5):1327-1337.
[10] LIANG Z. Projected gradient method for kernel discriminant nonnegative matrix factorization and the applications[J]. Signal Process, 2010, 90(7):2150-2163.
[11] QIAN Y, JIA S, ZHOU J, et al. Hyperspectral unmixing via L1/2 sparsity-constrained nonnegative matrix factorization[J]. IEEE Transactions Geoscience Remote Sense, 2011, 49(11):4282-4297.
[12] ZHI Ruicong, FLERIl Markus, RUAN Qiuqi, et al. Graph-preserving sparse nonnegative matrix factorization with application to facial expression recognition[J]. IEEE Transactions on Systems, Man, and Cybernatics—Part B:Cybernetics, 2011, 41(1):38-50.
[13] 杜世强, 石玉清, 王维兰,等. 基于流形正则化判别的因子分解[J]. 山东大学学报:理学版, 2013, 48(5):63-69. DU Shiqiang, SHI Yuqing, WANG Weilan, et al. Manifold regularization-based nonnegative matrix factorization[J]. Journal of Shandong University:Natural Science, 2013, 48(5):63-69.
[14] ZAFEIRIOUS S, TEFAS A, BUCIU I, et al. Exploiting discriminant information in nonnegative matrix factorization with application to frontal face verification[J]. IEEE Transactions on Neural Networks, 2006, 17(3):683-695.
[15] KOTSIA I, ZAFEIRIOUS S, PITAS I. A novel discriminant nonnegative matrix factorization algorithm with applications to facial image characterization problems[J].IEEE Transaction on Information Forensics and Security, 2007, 2(3):588-595.
[16] ZOIDI Olga, TEFAS Anastasios, PITAS Ioannis. Multiplicative update rules for concurrent nonnegative matrix factorization and maximum margin classification[J]. IEEE Transactions on Neural Networks and Learning Systems, 2013, 24(3):422-434.
[17] YE Jun, JIN Zhong. Non-negative matrix factorization based on fuzzy K nearest neighbor graph and its applications[J]. The Institution of Engineering and Technology Computer Vision, 2013, 7(5):346-353.
[18] ZHI Ruicong, FLIERL Markus, RUAN Qiuqi, et al. Graph-preserving sparse nonngegative matrix factorization with application to facial expression recognition[J]. IEEE Transactions on System, Man and cybernetics-Part B:Cybernetics, 2011, 41(1):38-51.
[19] GUAN Naiyang, TAO Dacheng, LUO Zhigang, et al. Online nonnegative matrix factorization with robust stochastic approximation[J]. IEEE Transactions on Neural Networks and Learning Systems, 2012, 23(7):1087-1099.
[20] HU Lirui, DAI Liang, WU Jianguo. Convergent projective non-negative matrix factorization with Kullback-Leibler divergence[J]. Pattern Recognition Letters, 2014(36):15-21.
[21] GONG Pinghua, ZHANG Changshui. Efficient nonnegative matrix factorization via projected newton method[J]. Pattern Recognition, 2012, 45(9):3557-3565.
[22] LIAO Ruiqi, ZHANG Yifan, GUAN Jihong, et al. Cloud NMF:A map reduce implementation of nonnegative matrix factorization for large-scale biological datasets[J]. Genomics, Proteomics & Bioinformatics, 2014, 12(1):48-51.
[23] IVICA Kopriva, IVANKA Jeric. Blind separation of analyze in nuclear magnetic resonance spectroscopy:Improved model for nonnegative matrix factorization[J]. Chemometrics and Intelligent Laboratory Systems, 2014, 137:47-56.
[24] VIRTANEN T. Nonaural sound source separation by non-negative matrix factorization with temporal continuity and sparseness criteria[J]. IEEE Transactions on Audio, Speech, Language Process, 2007, 15(3):1066-1074.
[25] CICHOCKI A, SHISHKINA S L, MUSHA T, et al. EEG filtering based on blind source separation(BSS)for early detection of Alzheimers disease[J]. Clinic Neurophysiology, 2005(116):729-737.
[26] 殷海青, 刘红卫. 一种基于L1稀疏正则化和非负矩阵分解的盲源信号分离新算法[J]. 西安电子科技大学学报, 2010, 37(5):835-841. YIN Haiqing, LIU Hongwei. New blind source separation algorithm based on L1 sparse regularization and nonnegative matricx factorization[J]. Journal of Xidian University, 2010, 37(5):835-841.
[27] KIRBIZ S, GUNSEL B. Perceptually enhanced blind single-channel music source separation by non-negative matrix factorization[J]. Digital Signal Processing, 2014(23):646-658.
[28] CICJOCKI A, ZDUNCK R. Multilayer nonnegative matrix factorization using projected gradient approaches[J]. International Journal of Neural Networks, 2007, 7(6):431-436.
[29] YAN-WANG Jim Jing, GAO Xin. Beyond cross-domain learning:multiple-domain nonnegative matrix factorization[J]. Engineering Applications of Artificial Intelligence, 2014(28):181-189.
[30] PAN Binbin, LAI Jianhuang, CHEN Wensheng. Nonlinear nonnegative matrix factorization based on Mercer kernel construction[J]. Pattern Recognition, 2011, 44(10—11):2800-2810.
[31] 李乐, 章毓晋. 非负矩阵分解算法综述[J]. 电子信息学报, 2008, 36(4):737-743. LI Le, ZHANG Yujin. Riews on nonnegative matrix factorization[J]. Journal of Electronics & Information Technology, 2008, 36(4):737-743.
[32] WANG Yuxiong, ZHANG Yujin. Nonnegative matrix factorization:a comprehensive review[J]. IEEE Transactions on Knowledge and Data Engineering, 2013, 25(6):1336-1353.
[33] 王洪元,封磊,冯燕,等. 流形学习算法在中文文本分类中的应用[J]. 山东大学学报:工学版, 2013, 42(4):8-13. WANG Hongyuan, FENG Lei, FENG Yan, et al. Manifold leanrning and its appplicaiton in text classification[J]. Journal of Shandong University:Engineering Science, 2013, 42(4):8-13.
[34] LIU X, YAN S, JIN H. Projective nonnegative graph embedding[J]. IEEE Transaction on Image Processing, 2010, 19(5):1126-1137.
[35] HE Ran, ZHENG Weishi, TAN Tieniu, et al.Half-quadratic-based iterative minimization for robust sparse representation[J]. IEEE Transactions on Patterns Analysis and Machine Intelligence, 2014, 36(2):261-275.
[36] YU Xuelian, WANG Xuegang. Uncorrelated discriminant locality preserving projections[J]. IEEE Signal Processing Letters, 2008(15):361-364.
[37] 夏海英,杜海明,徐鲁辉,等. 基于自适应词典学习和稀疏表示的人脸表情识别[J]. 山东大学学报:工学版,2014,44(1):45-49. XIA Haiying, DU Haiming, XU Luhui, et al. Facial expression recognition based on adaptive dictionary learning and sparse representation[J]. Journal of Shandong University:Engineering Science, 2014, 44(1):45-49.
[38] LU Haiping, KONSTANTINOS N, PLATANIOTIS V, et al. Uncorrelated multilinear discriminant analysis with regularization and aggregation for tensor object recognition[J]. IEEE Transactions on Neural Networks, 2009, 20(1):103-123.
[39] 张少捷, 王振雷, 钱锋. 基于LTSA的FS-SVDD方法及其在化工过程监控中的应用[J]. 化工学报, 2010, 61(8):1894-1900. ZHANG Shaojie, WANG Zhenglei, QIAN Feng. FS-SVDD based on LTSA and its application in chemical process monitoring[J]. CISE Journal, 2010, 61(8):1894-1900.
[40] NIE Feiping, ZENG Zinan, TSANG Ivor W, et al. Spectral embedded clustering: a framework for in-sample and out-of-sample spectral clustering[J]. IEEE Trans Neural Network, 2011, 22(11):1796-1808.
[1] BAI Shu-zhong ,LIU Ju,SUN Guo-xia .

An algorithm for under-determined blind source separation based on the least-mean-square error and sparse features

[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2008, 38(4): 97-101 .
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
[1] WANG Ru-gui,CAI Gan-wei . Sub-harmonic resonance analysis of 2-DOF controllable plane linkage mechanism electromechanical coupling system[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2008, 38(3): 58 -63 .
[2] JIA Chao,ZHAO Jian-yu,XU Bang-shu,YUE Chang-cheng,LI Shu-chen . Research on rock soil liquefaction of the Qingshui railway tunnel under dynamic vibration load[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2008, 38(1): 83 -87 .
[3] YU Jia yuan1, TIAN Jin ting1, ZHU Qiang zhong2. Computational intelligence and its application in psychology[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(1): 1 -5 .
[4] DENG Bin,WANG Jiang . Estimating parameters of a neuron model based on chaos synchronization and adaptive control[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2007, 37(5): 19 -23 .
[5] GAO Hou-Lei, TIAN Jia, DU Jiang, WU Zhi-Gang, LIU Chu-Min. Distributed generation—new technology in energy development[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(5): 106 -110 .
[6] JI Tao,GAO Xu/sup>,SUN Tong-jing,XUE Yong-duan/sup>,XU Bing-yin/sup> . Characteristic analysis of fault generated traveling waves in 10 Kv automatic blocking and continuous power transmission lines[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(2): 111 -116 .
[7] CHOU Wu-Sheng, WANG Shuo. Study on the adaptive algorithm of the force reflection robotic master under large stiffness of the environment[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2010, 40(1): 1 -5 .
[8] LI Guo-zheng1, SHI Miao-jing1, LI Fu-feng2, WANG Yi-qin2. An empirical study on tongue image detection[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2010, 40(5): 87 -95 .
[9] ZHANG Jianming, LIU Quansheng, TANG Zhicheng, ZHAN Ting, JIANG Yalong. New peak shear strength criterion with inclusion of shear action history[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 0, (): 77 -81 .
[10] LEI Xiao-feng1, ZHUANG Wei1, CHENG Yu1, DING Shi-fei1, XIE Kun-qing2. OPHCLUS:An order-preserving based hierarchical clustering algorithm[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2010, 40(5): 48 -55 .
Copyright 2018 © Editorial office of Journal of Shandong University (Engineering Science)
Supported by Beijing Magtech Co.Ltd