基于双图正则化的自适应多模态鲁棒特征学习

doi:10.11896/jsjkx.210300078

摘要/Abstract

摘要： 大数据时代,海量多模态数据的广泛存在使得数据特点发生了巨大变化:数据种类繁多且价值密度低。不同种类的数据既独立发挥作用又彼此相辅相成,发现多模态数据背后的隐藏价值成为大数据挖掘的关键。文中主要针对多模态数据的低质性问题,提出一种新的多模态鲁棒特征学习方法。该方法通过引入模态误差矩阵来有效降低噪声数据对融合结果的影响,使算法具备一定的鲁棒性。此外,设计数据流形与特征流形双图正则化机制,描述模态数据的双重空间结构,确保融合过程中数据的稳定性。在6个实际的多模态数据集上,基于准确性(Accuracy,ACC)、标准化互信息(Normalized Mutual Information,NMI)以及纯度(Purity,PUR)3种评价指标,将其与近年来的多种经典算法进行比较。实验结果显示,所提方法优于所有对比算法,尤其在含有大量噪声信息的网络数据集Webkb上表现突出,其ACC和NMI指标相比基线算法提升约10%,表明该算法实现了对多模态大数据共享特征的准确学习。

关键词: 多模态数据, 鲁棒特征学习, 双图正则化, 噪声数据, 自适应权重

Abstract: In the big data era, the widespread of massive multi-modal data has caused huge changes in the data characteristics, namely wide variety and low value density.Different types of data are characterized by both function independently and complement each other.Discovering the hidden value behind multi-modal data has become the key problem in big data mining tasks.Therefore, to tackle the shortcomings of the low-quality multimodal data, this paper proposes a new multimodal robust feature learning method by introducing the modal specific error matrix.The effect of noisy information on the fusion result can thus be effectively reduced.Moreover, a dual graph-regularization mechanism for data manifolds and feature manifolds is designed to describe the spatial structure of multimodal data, which can ensure the data stability during multimodal feature learning.On six real-world multi-modal data sets, the results are compared with several classical algorithms in recent years based on three evaluation indexes, namely accuracy (ACC), normalized mutual information (NMI) and purity (PUR).Experimental results show that the proposed method is superior to all other compared algorithms, especially in network data sets Webkb containing large amounts of noise information, its ACC and NMI are improved by about 10% compared with the baseline algorithms.It can be seen that the proposed algorithm can accurately learn the sharing features of multi-modal data.

Key words: Adaptive weight, Dual graph-regularization, Multimodal data, Noisy data, Robust feature learning

中图分类号:

TP181

赵亮, 张洁, 陈志奎. 基于双图正则化的自适应多模态鲁棒特征学习[J]. 计算机科学, 2022, 49(4): 124-133. https://doi.org/10.11896/jsjkx.210300078

ZHAO Liang, ZHANG Jie, CHEN Zhi-kui. Adaptive Multimodal Robust Feature Learning Based on Dual Graph-regularization[J]. Computer Science, 2022, 49(4): 124-133. https://doi.org/10.11896/jsjkx.210300078

参考文献

[1] LI J,LIU H.Challenges of Feature Selection for Big Data Analytics[J].IEEE Intelligent Systems,2017,32(2):9-15.
[2] ZHANG S F,ZHAI J H,XIE B J,et al.Multimodal Representation Learning:Advances,Trends and Challenges[C]//Procee-dings of International Conference on Machine Learning and Cybernetics (ICMLC).IEEE Press,2019:1-6.
[3] XING J,NIU Z,HUANG J,et al.Towards Robust and Accurate Multi-View and Partially-Occluded Face Alignment[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,40(4):987-1001.
[4] FINLAYSON A E T.Dealing with Data:Fostering Fidelity[J].Science,2011,331(6024):1515-1515.
[5] NIAN F,BAO B K,LI T,et al.Multi-ModalKnowledge Representation Learning via Webly-Supervised Relationships Mining[C]//Proceedings of the 25th ACM International Conference on Multimedia.2017:411-419.
[6] KAN M,SHAN S,ZHANG H,et al.Multi-View Discriminant Analysis[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2015,38(1):188-194.
[7] XU C,TAO D,XU C,et al.Multi-View Intact Space Learning[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2015,37(12):2531-2544.
[8] RAD R,JAMZAD M.Image Annotation Using Multi-ViewNonnegative Matrix Factorization with Different Number of Basis Vectors[J].Journal of Visual Communication and Image Representation,2017,46:1-12.
[9] SUN S Z,GUO B H,YANG X B.Embedding Consensus Auto encoder for Cross-Modal Semantic Analysis[J].Computer Science,2021,48(7):93-98.
[10] ZHANG S Q,DU S D,ZHANG X B,et al.Social Rumor Detection Method Based on Multimodal Fusion[J].Computer Science,2021,48(5):117-123.
[11] DUAN L J,SUN Q C,QIAO Y H,et al.Attention-Awareand Semantic-Aware Network for RGB-D Indoor Semantic Segmentation[J].Chinese Journal of Computer,2021,44(2):275-291.
[12] LI Q,XIE J,ZHANG Z,et al.Research on Online Sequence Extreme Learning Machine Based on Multi-Modal[J].Computer Engineering,2021,47(7):67-73,80.
[13] ZHANG X,GAO H,LI G,et al.Multi-View Clustering Based on Graph-Regularized Nonnegative Matrix Factorization for Object Recognition[J].Information Sciences,2018,432:463-478.
[14] LI J,ZHOU G,QIU Y,et al.Deep Graph Regularized Nonnegative Matrix Factorization for Multi-View Clustering[J].Neurocomputing,2020,390:108-116.
[15] FENG L,MENG X,WANG H,et al.Multi-View Locality Low-Rank Embedding for Dimension Reduction[J].Knowledge-Based Systems,2020,191:105172.
[16] LYDIA E L,RAMYA D.Text Mining with Lucene and Hadoop:Document Clustering with Updated Rules of NMF Non-negative Matrix Factorization[J].International Journal of Pure and Applied Mathematics,2018,118(7):191-198.
[17] MAN Y,LIU G,YANG K,et al.SNFM:A Semi-SupervisedNMF Algorithm for Detecting Biological Functional Modules[J].Mathematical Biosciences and Engineering,2019,16(4):1933-1948.
[18] CHEN W S,LIU J,PAN B,et al.Face Recognition Using Nonnegative Matrix Factorization with Fractional Power Inner Pro-duct Kernel[J].Neurocomputing,2019,348:40-53.
[19] LEE D D,SEUNG H S.Algorithms for Nonnegative MatrixFactorization[C]//Advances in Neural Information Processing Systems.2001:556-562.
[20] CAI D,HE X,HAN J,et al.Graph Regularized Nonnegative Matrix Factorization for Data Representation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2010,33(8):1548-1560.
[21] SHANG F,JIAO L C,WANG F.Graph Dual RegularizationNonnegative Matrix Factorization for Co-Clustering[J].Pattern Recognition,2012,45(6):2237-2250.
[22] LIU J,WANG C,GAO J,et al.Multi-View Clustering via Joint Nonnegative Matrix Factorization[C]//Proceedings of the 2013 SIAM International Conference on Data Mining.Society for Industrial and Applied Mathematics.2013:252-260.
[23] HIDRU D,GOLDENBERG A.EquiNMF:Graph RegularizedMulti-View Nonnegative Matrix Factorization[J].arXiv:1409.4018,2014.
[24] LUO P,PENG J,GUAN Z,et al.Dual Regularized Multi-View Nonnegative Matrix Factorization for Clustering[J].Neurocomputing,2018,294:1-11.
[25] WANG J,TIAN F,YU H,et al.Diverse Nonnegative MatrixFactorization for Multi-View Data Representation[J].IEEE Transactions on Cybernetics,2017,48(9):2620-2632.
[26] OU W,LONG F,TAN Y,et al.Co-Regularized Multi-ViewNonnegative Matrix Factorization with Correlation Constraint for Representation Learning[J].Multimedia Tools and Applications,2018,77(10):12955-12978.
[27] YANG L,JING L,NG M K.Robust and Nonnegative Collective Matrix Factorization for Text-To-Image Transfer Learning[J].IEEE Transactions on Image Processing,2015,24(12):4701-4714.
[28] SIYUAN P,WEE S,BADONG C,et al.Correntropy BasedGraph Regularized Concept Factorization for Clustering[J].Neurocomputing,2018,316:34-48.
[29] PENG S,SER W,CHEN B,et al.Robust Nonnegative MatrixFactorization with Local Coordinate Constraint for Image Clustering[J].Engineering Applications of Artificial Intelligence,2020,88:103354.
[30] PENG S,SER W,CHEN B,et al.Robust Orthogonal Nonnegative Matrix Tri-Factorization for Data Representation[J].Knowledge-Based Systems,2020,201/202:106054.
[31] WU B,WANG E,ZHU Z,et al.Manifold NMF with L21 Norm for Clustering[J].Neurocomputing,2017,273:78-88.
[32] HUANG Q,YIN X,CHEN S,et al.Robust Nonnegative Matrix Factorization with Structure Regularization[J].Neurocompu-ting,2020,412:72-90.
[33] XIONG H,KONG D.Elastic Nonnegative Matrix Factorization[J].Pattern Recognition,2019,90:464-475.
[34] FENG Y,XIAO J,ZHOU K,et al.A Locally Weighted Sparse Graph Regularized Nonnegative Matrix Factorization Method[J].Neurocomputing,2015,169:68-76.
[35] ZHAO P,WANG W,LU Y,et al.Transfer Robust Sparse Co-ding Based on Graph and Joint Distribution Adaption for Image Representation[J].Knowledge-Based Systems,2018,147:1-11.
[36] MA Y,CHEN Z,QIU X,et al.Robust and Graph Regularised Nonnegative Matrix Factorisation for Heterogeneous Co-Transfer Clustering[J].International Journal of Computational Science and Engineering,2019,18(1):29-38.
[37] YIN Q,WU S,WANG L.Incomplete Multi-View Clustering via Subspace Learning[C]//Proceedings of the 24th ACM International on Conference on Information and Knowledge Management.2015:383-392.
[38] ZHAO L,CHEN Z,WANG Z J.Unsupervised Multiview Nonnegative Correlated Feature Learning for Data Clustering[J].IEEE Signal Processing Letters,2018,25(1):60-64.
[39] QIU X,CHEN Z,ZHAO L,et al.Unsupervised Multi-ViewNonnegative for Law Data Feature Learning with Dual Graph-Regularization in Smart Internet of Things[J].Future Generation Computer Systems,2019,100:523-530.
[40] ZONG L,ZHANG X,ZHAO L,et al.Multi-View Clustering via Multi-Manifold Regularized Nonnegative Matrix Factorization[J].Neural Networks the Official Journal of the International Neural Network Society,2017,88:74-89.
[41] CHANG C C,LIN C J.LIBSVM:A Library for Support Vector Machines[J].ACM Transactions on Intelligent Systems and Technology (TIST),2011,2(3):1-27.
[42] LI Y,NIE F,HUANG H,et al.Large-Scale Multi-View Spectral Clustering via Bipartite Graph[C]//Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence.2015:2750-2756.
[43] WEN J,XU Y,LIU H.Incomplete Multiview Spectral Clustering with Adaptive Graph Learning[J].IEEE Transactions on Cybernetics,2020,50(4):1418-1429.
[44] YANG M S,SINAGA K P.A Feature-Reduction Multi-View K-Means Clustering Algorithm[J].IEEE Access,2019,7:114472-114486.

相关文章 15

[1]	周新民, 胡宜桂, 刘文洁, 孙荣俊. 基于多模态多层级数据融合方法的城市功能识别研究 Research on Urban Function Recognition Based on Multi-modal and Multi-level Data Fusion Method 计算机科学, 2021, 48(9): 50-58. https://doi.org/10.11896/jsjkx.210500220
[2]	曹林, 于威威. 基于图像分割的自适应窗口双目立体匹配算法研究 Adaptive Window Binocular Stereo Matching Algorithm Based on Image Segmentation 计算机科学, 2021, 48(11A): 314-318. https://doi.org/10.11896/jsjkx.201200264
[3]	程中建, 周双娥, 李康. 基于多尺度自适应权重的稀疏表示目标跟踪算法 Sparse Representation Target Tracking Algorithm Based on Multi-scale Adaptive Weight 计算机科学, 2020, 47(6A): 181-186. https://doi.org/10.11896/JsJkx.190500093
[4]	杨婷, 罗飞, 丁炜超, 卢海峰. 一种自适应优化松弛量的装箱算法 Bin Packing Algorithm Based on Adaptive Optimization of Slack 计算机科学, 2020, 47(4): 211-216. https://doi.org/10.11896/jsjkx.190500132
[5]	邓一姣, 张凤荔, 陈学勤, 艾擎, 余苏喆. 面向跨模态检索的协同注意力网络模型 Collaborative Attention Network Model for Cross-modal Retrieval 计算机科学, 2020, 47(4): 54-59. https://doi.org/10.11896/jsjkx.190600181
[6]	蹇松雷, 卢凯. 复杂异构数据的表征学习综述 Survey on Representation Learning of Complex Heterogeneous Data 计算机科学, 2020, 47(2): 1-9. https://doi.org/10.11896/jsjkx.190600180
[7]	林敏鸿, 蒙祖强. 基于注意力神经网络的多模态情感分析 Multimodal Sentiment Analysis Based on Attention Neural Network 计算机科学, 2020, 47(11A): 508-514. https://doi.org/10.11896/jsjkx.191100041
[8]	赵青杰, 李捷, 于俊洋, 吉宏远. 基于动态自适应权重和柯西变异的蝙蝠优化算法 Bat Optimization Algorithm Based on Dynamically Adaptive Weight and Cauchy Mutation 计算机科学, 2019, 46(6A): 89-92.
[9]	沈先宝, 宋余庆, 刘哲. 基于排序选择度量的自适应集成方法 Adaptive Integrated Method Based on Sorting Selection Metrics 计算机科学, 2019, 46(12): 237-241. https://doi.org/10.11896/jsjkx.181102173
[10]	王慧研,张腾飞,马福民. 基于空间距离自适应权重度量的粗糙K-means算法 Rough K-means Algorithm with Self-adaptive Weights Measurement Based on Space Distance 计算机科学, 2018, 45(7): 190-196. https://doi.org/10.11896／j.issn.1002-137X.2018.07.033
[11]	李香元,蔡骋,何进荣. 基于密度缩放因子的ISOMAP算法 Density Scaling Factor Based ISOMAP Algorithm 计算机科学, 2018, 45(7): 207-213. https://doi.org/10.11896／j.issn.1002-137X.2018.07.036
[12]	廖星,袁景凌,陈旻骋. 一种自适应权重的并行PSO快速装箱算法 Parallel PSO Container Packing Algorithm with Adaptive Weight 计算机科学, 2018, 45(3): 231-234. https://doi.org/10.11896/j.issn.1002-137X.2018.03.036
[13]	徐程浩,郭斌,欧阳逸,翟书颖,於志文. 基于社交媒体的事件感知与多模态事件脉络生成 Event Sensing and Multimodal Event Vein Generation Leveraging Social Media 计算机科学, 2017, 44(Z6): 33-36. https://doi.org/10.11896/j.issn.1002-137X.2017.6A.007
[14]	张春云,秦鹏达,尹义龙. 基于卷积神经网络的自适应权重multi-gram语句建模系统 Self-adaptation Multi-gram Weight Learning Strategy for Sentence Representation Based on Convolutional Neural Network 计算机科学, 2017, 44(1): 60-64. https://doi.org/10.11896/j.issn.1002-137X.2017.01.011
[15]	郭京蕾,李伟,金聪. 基于中心块的多特征自适应图像检索算法 Adaptive Image Retrieval Algorithm with Multi-feature of Center Block 计算机科学, 2016, 43(6): 65-67. https://doi.org/10.11896/j.issn.1002-137X.2016.06.013

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed