Computer Science ›› 2020, Vol. 47 ›› Issue (11): 80-87.doi: 10.11896/jsjkx.190900144

• Database & Big Data & Data Science • Previous Articles     Next Articles

Huber Loss Based Nonnegative Matrix Factorization Algorithm

WANG Li-xing1, CAO Fu-yuan1,2   

  1. 1 School of Computer and Information Technology,Shanxi University,Taiyuan 030006,China
    2 Key Laboratory of Computational Intelligence and Chinese Information Processing (Shanxi University),Ministry of Education,Taiyuan 030006,China
  • Received:2019-09-23 Revised:2020-02-08 Online:2020-11-15 Published:2020-11-05
  • About author:WANG Li-xing,born in 1992,postgradu-ate.Her main research interests include Subs pace Learning and NMF.
    CAO Fu-yuan,born in 1974,professor,is a member of China Computer Federatio.His main research interests include subspace learning and NMF.
  • Supported by:
    This work was supported by the National Natural Science Foundation (61573229,61976128), Shanxi Provincial Key Research and Development Program (201803D31022),Shanxi Scholarship Fund Project(2016-003) and Shanxi Scholarship Fund Selection Project (2016-001).

Abstract: Non-negative matrix factorization (NMF) algorithm can find a non-negative and linear matrix representation and retains the essential characteristics of the original data,it has been successfully applied to many fields.The classical NMF algorithm and its variant algorithms mostly use the mean square error function to measure the reconstruction error,which has been shown to be effective in many tasks,but it still faces some difficulties in dealing with noise-containing data.The Huber loss function performs the same penalty for the smaller residual as the mean square error loss function,and the penalty for the larger residual is linearly grown,so the Huber loss function is more robust than the mean square error loss function.It has been proved that the L2,1 norm sparse regularization term is a feature selection function in the classification and clustering model of machine learning.Therefore,combining the advantages of the two,a non-negative matrix factorization clustering model based on Huber loss function and incorporating L2,1 norm regularization term is proposed,and an effective optimization procedure based on projected gradient method to update variables is given.Compared with the classical NMF multi-clustering algorithm on multiple sets of datasets,the experimental results show the effectiveness of the proposed algorithm.

Key words: L2,1 norm, Huber loss function, Nonnegative matrix factorization, Projected gradient method

CLC Number: 

  • TP3-05
[1] LU H T,FU Z Y,SHU X.Non-negative and sparse spectralclustering [J].Pattern Recognition,2014,47(1):418-426.
[2] LEE D D,SEUNG H S.Learning the parts of objects by non-negative matrix factorization [J].Nature,1999,401:788-791.
[3] LEE D D,SEUNG H S.Algorithms for non-negative matrix factorization[C]//NIPS.2000:535-541.
[4] LI M J,XIE Q,DING Q L.Orthogonal Non-negative Matrix Factorization for K-means Clustering [J].Computer Science,2016,43(5):204-208.
[5] LIN C J.Projected gradient methods for non-negative matrixfactorization [J].Neural Computation,2007,19(10):2756-2779.
[6] HOYER P O.Non-negative matrix factorization with sparseness constraints [J].Journal of Machine Learning Research,2004,5(9):1457-1469.
[7] CAI D,HE H,HAN J.Graph regularized nonnegative matrixfactorization for data representation [J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2011,33(8):1548-1560.
[8] JIANG W,LI H,YU X,et al.Graph Regularized Non-negative Matrix Factorization with Sparseness Constraints [J].Computer Science,2013,40(1):218-220,256.
[9] LIU H,WU Z,LI X.Constrained nonnegative matrix factorization for image representation [J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2012,34(7):1299-1311.
[10] KONG D,DING C,HUANG H.Robust nonnegative matrix factorization using L21-norm[C]//Proceedings of the 20th ACM CIKM.2011:673-682.
[11] DU L,LI X,SHEN Y D.Robust nonnegative matrix factorization via half-quadratic minimization[C]//IEEE.ICDM,2012:201-210.
[12] YANG S,HOU C,ZHANG C,et al.Robust non-negative matrix factorization via joint sparse and graph regularization[C]//International Joint Conference on Neural Networks.2013:1-5.
[13] KONG D,DING C,HUANG H.Robust nonnegative matrix factorization using l21-norm[C]//The 20th ACM International Conference on Information and Knowledge Management.2011:673-682.
[14] NIE F P,HUANG H,CAI X,et al.Efficient and robust feature selection via L2,1-norms minimization [C]//Proceedings of International Conference on Neural Information Processing Systems.British,ACM,2010:1813-1821.
[15] CALAMAI P H,MORÈ J J.Projected gradient methods for linearly constrained problems [J].Mathematical Programming,1987,39(1):93-116.
[16] HUBER P J.Robust Statistics (second edition) [M].NewJersey:John Wiley & Sons,2009:1-5.
[17] ANDREW S,TSOCHANTARIDIS I T.HOFMANN. Supportvector machines for multiple-instance learning[C]//Advances in Neural Information Processing Systems.USA:The MIT Press,2003:577-584.
[18] TOLIC D,ANTULOV F N,KOPRVIA I.A nonlinear orthogonal non-negative matrix factorization approach to subspace clustering [J].Pattern Recognition,2018,82(10):40-55.
[19] NIE F P,WANG X Q,HUANG H.Clustering and projectedclustering with adaptive neighbors[C]//ACM SIGKDD Conference on Knowledge Discovery and Data Mining.New York,ACM,2014:977-986.
[20] DUA D,GRAFF C.UCI Machine Learning Repository[OL].http://archive.ics.uci.edu/ml.
[1] DUAN Fei, WANG Hui-min, ZHANG Chao. Cauchy Non-negative Matrix Factorization for Data Representation [J]. Computer Science, 2021, 48(6): 96-102.
[2] LI Xiang-li, JIA Meng-xue. Nonnegative Matrix Factorization Algorithm with Hypergraph Based on Per-treatments [J]. Computer Science, 2020, 47(7): 71-77.
[3] HE Xiao-wen, HU Yi-fei, WANG Hai-ping, CHEN Mo. Online Learning Nonnegative Matrix Factorization [J]. Computer Science, 2019, 46(6A): 473-477.
[4] HUANG Meng-ting, ZHANG Ling, JIANG Wen-chao. Multi-type Relational Data Co-clustering Approach Based on Manifold Regularization [J]. Computer Science, 2019, 46(6): 64-68.
[5] JIA Xu, SUN Fu-ming, LI Hao-jie, CAO Yu-dong. Vein Recognition Algorithm Based on Supervised NMF with Two Regularization Terms [J]. Computer Science, 2018, 45(8): 283-287.
[6] YU Xiao, NIE Xiu-shan, MA Lin-yuan and YIN Yi-long. Robust Video Hashing Algorithm Based on Short-term Spatial Variations [J]. Computer Science, 2018, 45(2): 84-89.
[7] ZOU Li, CAI Xi-biao, SUN Jing, SUN Fu-ming. Hyperspectral Unmixing Algorithm Based on Dual Graph-regularized Semi-supervised NMF [J]. Computer Science, 2018, 45(12): 251-254.
[8] SUN Jing, CAI Xi-biao, JIANG Xiao-yan and SUN Fu-ming. Graph Regularized and Incremental Nonnegative Matrix Factorization with Sparseness Constraints [J]. Computer Science, 2017, 44(6): 298-305.
[9] TANG Bing, Laurent BOBELIN and HE Hai-wu. Parallel Algorithm of Nonnegative Matrix Factorization Based on Hybrid MPI and OpenMP Programming Model [J]. Computer Science, 2017, 44(3): 51-54.
[10] JIANG Xiao-yan, SUN Fu-ming and LI Hao-jie. Semi-supervised Nonnegative Matrix Factorization Based on Graph Regularization and Sparseness Constraints [J]. Computer Science, 2016, 43(7): 77-82.
[11] LIANG Qiu-xia, HE Guang-hui, CHEN Ru-li and CHU Jian-pu. Research of Face Recognition Algorithm Based on Nonnegative Tensor Factorization [J]. Computer Science, 2016, 43(10): 312-316.
[12] HU Xue-kao, SUN Fu-ming and LI Hao-jie. Constrained Nonnegative Matrix Factorization with Sparseness for Image Representation [J]. Computer Science, 2015, 42(7): 280-284.
[13] LI Qian,JING Li-ping and YU Jian. Multi-kernel Projective Nonnegative Matrix Factorization Algorithm [J]. Computer Science, 2014, 41(2): 64-67.
[14] . Nonnegative Matrix Factorization-based IP Traffic Prediction [J]. Computer Science, 2012, 39(1): 48-52.
[15] JIANG Wei, YANG Bing-ru,SUI Hai-feng. Local Sensitive Nonnegative Matrix Factorization [J]. Computer Science, 2010, 37(12): 211-214.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!