Computer Science ›› 2022, Vol. 49 ›› Issue (4): 124-133.doi: 10.11896/jsjkx.210300078

• Database & Big Data & Data Science • Previous Articles     Next Articles

Adaptive Multimodal Robust Feature Learning Based on Dual Graph-regularization

ZHAO Liang, ZHANG Jie, CHEN Zhi-kui   

  1. School of Software Technology, Dalian University of Technology, Dalian, Liaoning 116620, China; Key Laboratory for Ubiquitous Network and Service Software of Liaoning Province, Dalian, Liaoning 116620, China
  • Received:2021-03-08 Revised:2021-07-15 Published:2022-04-01
  • About author:ZHAO Liang,born in 1988,Ph.D,associate professor.His main research in-terests include big data and AI.
  • Supported by:
    This work was supported by the National Natural Science Foundation of China(61906030),Equipment Advance Research Fund(80904010301) and Natural Science Foundation of Liaoning Province(2020-BS-063).

Abstract: In the big data era, the widespread of massive multi-modal data has caused huge changes in the data characteristics, namely wide variety and low value density.Different types of data are characterized by both function independently and complement each other.Discovering the hidden value behind multi-modal data has become the key problem in big data mining tasks.Therefore, to tackle the shortcomings of the low-quality multimodal data, this paper proposes a new multimodal robust feature learning method by introducing the modal specific error matrix.The effect of noisy information on the fusion result can thus be effectively reduced.Moreover, a dual graph-regularization mechanism for data manifolds and feature manifolds is designed to describe the spatial structure of multimodal data, which can ensure the data stability during multimodal feature learning.On six real-world multi-modal data sets, the results are compared with several classical algorithms in recent years based on three evaluation indexes, namely accuracy (ACC), normalized mutual information (NMI) and purity (PUR).Experimental results show that the proposed method is superior to all other compared algorithms, especially in network data sets Webkb containing large amounts of noise information, its ACC and NMI are improved by about 10% compared with the baseline algorithms.It can be seen that the proposed algorithm can accurately learn the sharing features of multi-modal data.

Key words: Adaptive weight, Dual graph-regularization, Multimodal data, Noisy data, Robust feature learning

CLC Number: 

  • TP181
[1] LI J,LIU H.Challenges of Feature Selection for Big Data Analytics[J].IEEE Intelligent Systems,2017,32(2):9-15.
[2] ZHANG S F,ZHAI J H,XIE B J,et al.Multimodal Representation Learning:Advances,Trends and Challenges[C]//Procee-dings of International Conference on Machine Learning and Cybernetics (ICMLC).IEEE Press,2019:1-6.
[3] XING J,NIU Z,HUANG J,et al.Towards Robust and Accurate Multi-View and Partially-Occluded Face Alignment[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,40(4):987-1001.
[4] FINLAYSON A E T.Dealing with Data:Fostering Fidelity[J].Science,2011,331(6024):1515-1515.
[5] NIAN F,BAO B K,LI T,et al.Multi-ModalKnowledge Representation Learning via Webly-Supervised Relationships Mining[C]//Proceedings of the 25th ACM International Conference on Multimedia.2017:411-419.
[6] KAN M,SHAN S,ZHANG H,et al.Multi-View Discriminant Analysis[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2015,38(1):188-194.
[7] XU C,TAO D,XU C,et al.Multi-View Intact Space Learning[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2015,37(12):2531-2544.
[8] RAD R,JAMZAD M.Image Annotation Using Multi-ViewNonnegative Matrix Factorization with Different Number of Basis Vectors[J].Journal of Visual Communication and Image Representation,2017,46:1-12.
[9] SUN S Z,GUO B H,YANG X B.Embedding Consensus Auto encoder for Cross-Modal Semantic Analysis[J].Computer Science,2021,48(7):93-98.
[10] ZHANG S Q,DU S D,ZHANG X B,et al.Social Rumor Detection Method Based on Multimodal Fusion[J].Computer Science,2021,48(5):117-123.
[11] DUAN L J,SUN Q C,QIAO Y H,et al.Attention-Awareand Semantic-Aware Network for RGB-D Indoor Semantic Segmentation[J].Chinese Journal of Computer,2021,44(2):275-291.
[12] LI Q,XIE J,ZHANG Z,et al.Research on Online Sequence Extreme Learning Machine Based on Multi-Modal[J].Computer Engineering,2021,47(7):67-73,80.
[13] ZHANG X,GAO H,LI G,et al.Multi-View Clustering Based on Graph-Regularized Nonnegative Matrix Factorization for Object Recognition[J].Information Sciences,2018,432:463-478.
[14] LI J,ZHOU G,QIU Y,et al.Deep Graph Regularized Nonnegative Matrix Factorization for Multi-View Clustering[J].Neurocomputing,2020,390:108-116.
[15] FENG L,MENG X,WANG H,et al.Multi-View Locality Low-Rank Embedding for Dimension Reduction[J].Knowledge-Based Systems,2020,191:105172.
[16] LYDIA E L,RAMYA D.Text Mining with Lucene and Hadoop:Document Clustering with Updated Rules of NMF Non-negative Matrix Factorization[J].International Journal of Pure and Applied Mathematics,2018,118(7):191-198.
[17] MAN Y,LIU G,YANG K,et al.SNFM:A Semi-SupervisedNMF Algorithm for Detecting Biological Functional Modules[J].Mathematical Biosciences and Engineering,2019,16(4):1933-1948.
[18] CHEN W S,LIU J,PAN B,et al.Face Recognition Using Nonnegative Matrix Factorization with Fractional Power Inner Pro-duct Kernel[J].Neurocomputing,2019,348:40-53.
[19] LEE D D,SEUNG H S.Algorithms for Nonnegative MatrixFactorization[C]//Advances in Neural Information Processing Systems.2001:556-562.
[20] CAI D,HE X,HAN J,et al.Graph Regularized Nonnegative Matrix Factorization for Data Representation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2010,33(8):1548-1560.
[21] SHANG F,JIAO L C,WANG F.Graph Dual RegularizationNonnegative Matrix Factorization for Co-Clustering[J].Pattern Recognition,2012,45(6):2237-2250.
[22] LIU J,WANG C,GAO J,et al.Multi-View Clustering via Joint Nonnegative Matrix Factorization[C]//Proceedings of the 2013 SIAM International Conference on Data Mining.Society for Industrial and Applied Mathematics.2013:252-260.
[23] HIDRU D,GOLDENBERG A.EquiNMF:Graph RegularizedMulti-View Nonnegative Matrix Factorization[J].arXiv:1409.4018,2014.
[24] LUO P,PENG J,GUAN Z,et al.Dual Regularized Multi-View Nonnegative Matrix Factorization for Clustering[J].Neurocomputing,2018,294:1-11.
[25] WANG J,TIAN F,YU H,et al.Diverse Nonnegative MatrixFactorization for Multi-View Data Representation[J].IEEE Transactions on Cybernetics,2017,48(9):2620-2632.
[26] OU W,LONG F,TAN Y,et al.Co-Regularized Multi-ViewNonnegative Matrix Factorization with Correlation Constraint for Representation Learning[J].Multimedia Tools and Applications,2018,77(10):12955-12978.
[27] YANG L,JING L,NG M K.Robust and Nonnegative Collective Matrix Factorization for Text-To-Image Transfer Learning[J].IEEE Transactions on Image Processing,2015,24(12):4701-4714.
[28] SIYUAN P,WEE S,BADONG C,et al.Correntropy BasedGraph Regularized Concept Factorization for Clustering[J].Neurocomputing,2018,316:34-48.
[29] PENG S,SER W,CHEN B,et al.Robust Nonnegative MatrixFactorization with Local Coordinate Constraint for Image Clustering[J].Engineering Applications of Artificial Intelligence,2020,88:103354.
[30] PENG S,SER W,CHEN B,et al.Robust Orthogonal Nonnegative Matrix Tri-Factorization for Data Representation[J].Knowledge-Based Systems,2020,201/202:106054.
[31] WU B,WANG E,ZHU Z,et al.Manifold NMF with L21 Norm for Clustering[J].Neurocomputing,2017,273:78-88.
[32] HUANG Q,YIN X,CHEN S,et al.Robust Nonnegative Matrix Factorization with Structure Regularization[J].Neurocompu-ting,2020,412:72-90.
[33] XIONG H,KONG D.Elastic Nonnegative Matrix Factorization[J].Pattern Recognition,2019,90:464-475.
[34] FENG Y,XIAO J,ZHOU K,et al.A Locally Weighted Sparse Graph Regularized Nonnegative Matrix Factorization Method[J].Neurocomputing,2015,169:68-76.
[35] ZHAO P,WANG W,LU Y,et al.Transfer Robust Sparse Co-ding Based on Graph and Joint Distribution Adaption for Image Representation[J].Knowledge-Based Systems,2018,147:1-11.
[36] MA Y,CHEN Z,QIU X,et al.Robust and Graph Regularised Nonnegative Matrix Factorisation for Heterogeneous Co-Transfer Clustering[J].International Journal of Computational Science and Engineering,2019,18(1):29-38.
[37] YIN Q,WU S,WANG L.Incomplete Multi-View Clustering via Subspace Learning[C]//Proceedings of the 24th ACM International on Conference on Information and Knowledge Management.2015:383-392.
[38] ZHAO L,CHEN Z,WANG Z J.Unsupervised Multiview Nonnegative Correlated Feature Learning for Data Clustering[J].IEEE Signal Processing Letters,2018,25(1):60-64.
[39] QIU X,CHEN Z,ZHAO L,et al.Unsupervised Multi-ViewNonnegative for Law Data Feature Learning with Dual Graph-Regularization in Smart Internet of Things[J].Future Generation Computer Systems,2019,100:523-530.
[40] ZONG L,ZHANG X,ZHAO L,et al.Multi-View Clustering via Multi-Manifold Regularized Nonnegative Matrix Factorization[J].Neural Networks the Official Journal of the International Neural Network Society,2017,88:74-89.
[41] CHANG C C,LIN C J.LIBSVM:A Library for Support Vector Machines[J].ACM Transactions on Intelligent Systems and Technology (TIST),2011,2(3):1-27.
[42] LI Y,NIE F,HUANG H,et al.Large-Scale Multi-View Spectral Clustering via Bipartite Graph[C]//Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence.2015:2750-2756.
[43] WEN J,XU Y,LIU H.Incomplete Multiview Spectral Clustering with Adaptive Graph Learning[J].IEEE Transactions on Cybernetics,2020,50(4):1418-1429.
[44] YANG M S,SINAGA K P.A Feature-Reduction Multi-View K-Means Clustering Algorithm[J].IEEE Access,2019,7:114472-114486.
[1] CAO Lin, YU Wei-wei. Adaptive Window Binocular Stereo Matching Algorithm Based on Image Segmentation [J]. Computer Science, 2021, 48(11A): 314-318.
[2] CHENG Zhong-Jian, ZHOU Shuang-e and LI Kang. Sparse Representation Target Tracking Algorithm Based on Multi-scale Adaptive Weight [J]. Computer Science, 2020, 47(6A): 181-186.
[3] YANG Ting, LUO Fei, DING Wei-chao, LU Hai-feng. Bin Packing Algorithm Based on Adaptive Optimization of Slack [J]. Computer Science, 2020, 47(4): 211-216.
[4] JIAN Song-lei, LU Kai. Survey on Representation Learning of Complex Heterogeneous Data [J]. Computer Science, 2020, 47(2): 1-9.
[5] YI Yu-gen, LI Shi-cheng, PEI Yang, CHEN Lei, DAI Jiang-yan. Feature Selection Method Combined with Multi-manifold Structures and Self-representation [J]. Computer Science, 2020, 47(11A): 474-478.
[6] LIN Min-hong, MENG Zu-qiang. Multimodal Sentiment Analysis Based on Attention Neural Network [J]. Computer Science, 2020, 47(11A): 508-514.
[7] ZHAO Qing-jie, LI Jie, YU Jun-yang, JI Hong-yuan. Bat Optimization Algorithm Based on Dynamically Adaptive Weight and Cauchy Mutation [J]. Computer Science, 2019, 46(6A): 89-92.
[8] SHEN Xian-bao, SONG Yu-qing, LIU Zhe. Adaptive Integrated Method Based on Sorting Selection Metrics [J]. Computer Science, 2019, 46(12): 237-241.
[9] YANG Liu, CHEN Li-min, YI Yu-gen. Face Recognition Method Based on Adaptively Weighted Sub-pattern Discriminant Neighborhood Projection [J]. Computer Science, 2019, 46(10): 307-310.
[10] WANG Hui-yan,ZHANG Teng-fei,MA Fu-min. Rough K-means Algorithm with Self-adaptive Weights Measurement Based on Space Distance [J]. Computer Science, 2018, 45(7): 190-196.
[11] LIAO Xing, YUAN Jing-ling and CHEN Min-cheng. Parallel PSO Container Packing Algorithm with Adaptive Weight [J]. Computer Science, 2018, 45(3): 231-234.
[12] XU Cheng-hao, GUO Bin, OUYANG Yi, ZHAI Shu-ying and YU Zhi-wen. Event Sensing and Multimodal Event Vein Generation Leveraging Social Media [J]. Computer Science, 2017, 44(Z6): 33-36.
[13] GUO Jing-lei, LI Wei and JIN Cong. Adaptive Image Retrieval Algorithm with Multi-feature of Center Block [J]. Computer Science, 2016, 43(6): 65-67.
[14] MA Shi-yu,LI Yi-cai and LAN Zhang-li. Novel Anti-noise K-means Algorithm Based on Spatial Distance Difference [J]. Computer Science, 2014, 41(Z6): 406-408.
[15] CHANG Chun,LI Shi-jin,WAN Ding-sheng and FENG Jun. Sample-specific Multiple Features Weighting-based High-resolution Remote Sensing Image Classification [J]. Computer Science, 2014, 41(2): 107-110.
Full text



No Suggested Reading articles found!