基于深度矩阵分解网络的矩阵填充方法

doi:10.11896/jsjkx.190300390

计算机科学 ›› 2019, Vol. 46 ›› Issue (10): 55-62.doi: 10.11896/jsjkx.190300390

基于深度矩阵分解网络的矩阵填充方法

邝神芬^1,2, 黄业文³, 宋杰¹, 李洽²

(韶关学院数学与统计学院广东韶关512005)¹
(中山大学数据科学与计算机学院广州510006)²
(华南理工大学广州学院计算机工程学院广州510800)³

收稿日期:2019-02-01 修回日期:2019-04-26 出版日期:2019-10-15 发布日期:2019-10-21
通讯作者: 邝神芬(1985-),男,博士生,讲师,主要研究方向为数据挖掘与机器学习,E-mail:shfkuang@163.com。
作者简介:黄业文(1979-),男,博士生,讲师,主要研究方向为深度学习、数理统计;宋杰(1974-),男,博士,教授,主要研究方向为数据挖掘、生物信息学与运筹学;李洽(1985-),男,博士,副教授,主要研究方向为计算优化及其在图像与机器学习中的应用。
基金资助:
本文受韶关学院校级项目(SY2016KJ17,SY2016KJ04),广东省教育厅青年创新人才项目(2018KQNCX233),广东省教育厅基础研究和应用基础研究重点项目(2018KZDXM065),广东省自然科学基金(2017A030307022,2018A0303100015),华南理工大学广州学院优秀骨干教师科研项目(56-CQ18YG18)资助。

Deep Matrix Factorization Network for Matrix Completion

KUANG Shen-fen^1,2, HUANG Ye-wen³, SONG Jie¹, LI Qia²

(School of Mathematics and Statistics,Shaoguan University,Shaoguan,Guangdong 512005,China)¹
(School of Data and Computer Science,Sun Yat-sen University,Guangzhou 510006,China)²
(School of Computer Engineering,Guangzhou College of South China University of Technology,Guangzhou 510800,China)³

Received:2019-02-01 Revised:2019-04-26 Online:2019-10-15 Published:2019-10-21

摘要/Abstract

摘要： 矩阵分解是矩阵填充中的流行方法,但现有的方法大多是基于浅层的线性模型,当数据矩阵变大且观测数据很少时,容易导致过拟合,性能也随之显著下降。针对这些问题,提出了一种基于深度矩阵分解网络(DMFN)的矩阵填充方法,该方法不仅能弥补传统矩阵分解的缺点,而且能处理复杂的非线性数据。首先,将输入矩阵的观测值对应的行和列向量作为输入,对其进行投影,得到其行(列)的潜在特征向量;然后,分别对行(列)的潜在特征向量构建多层感知器网络;最后,通过构建双线性池化层,将行和列的输出向量进行融合。在推荐系统数据集MovieLens及Netflix上进行测试,实验结果表明,在相同参数设置下,与主流的填充算法相比,所提方法填充预测的均方误差(RMSE)及绝对值误差(MAE)都有明显提高。

关键词: 多层感知器, 矩阵分解, 矩阵填充, 深度学习, 双线性池化

Abstract: Matrix factorization is a popular technique for matrix completion,but most of the existing methods are based on linear or shallow models,when the data matrix becomes large and the observation data is very small,it is prone to overfitting and the performance decreases significantly.To address this problem,this paper presented a Deep Matrix Factorization Network (DMFN) method,which can not only overcome the shortcoming of traditional matrix factorization,but also deal with complex non-linear data.First,by using rows and columns vectors corresponding to the observed values of the input matrices as input,the latent vector of its row (column) is projected,then the multi-layer perceptron network is constructed on the latent vector of row (column),at last the output vectors of rows and columns are fused by the bilinear pool layer.The proposed algorithm was tested on the standard recommender system dataset (MovieLens and Netflix).Experimental results show that the mean square error (RMSE) and mean absolute error (MAE) of the proposed method are significantly improved compared with the current popular methods under the same parameter setting.The RMSE is 0.830 and MAE is 0.652 on the MovieLens 1M dataset,which increases by about 2% and 6%,respectively.On the Netflix dataset,RMSE is 0.840 and MAE is 0.661,which increases by approximately 1% and 4%,respectively,and achieves optimal results.

Key words: Bilinear pooling, Deep learning, Matrix completion, Matrix factorization, Multilayer perception

中图分类号:

TP18

邝神芬, 黄业文, 宋杰, 李洽. 基于深度矩阵分解网络的矩阵填充方法[J]. 计算机科学, 2019, 46(10): 55-62. https://doi.org/10.11896/jsjkx.190300390

KUANG Shen-fen, HUANG Ye-wen, SONG Jie, LI Qia. Deep Matrix Factorization Network for Matrix Completion[J]. Computer Science, 2019, 46(10): 55-62. https://doi.org/10.11896/jsjkx.190300390

参考文献

[1]KOREN Y,BELL R M,VOLINSKY C,et al.Matrix factorization techniques for recommender systems[J].IEEE Computer,2009,42(8):30-37.
[2]PAN T T,WEN F,LIU Q R.Collaborative Filtering recommendation algorithm based on rating matrix filling and item predictability[J].Acta Automatica Sinica,2017,43(9):1597-1606.(in Chinese)
潘涛涛,文锋,刘勤让.基于矩阵填充和物品可预测性的协同过滤算法[J].自动化学报,2017,43(9):1597-1606.
[3]HU Y,ZHANG D,YE J,et al.Fast and Accurate Matrix completion via truncated nuclear norm regularization[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2013,35(9):2117-2130.
[4]WANG Y,LEE C M,CHEONG L F,et al.Practical Matrix completion and corruption recovery using proximal alternating robust subspace minimization[J].International Journal of Computer Vision,2015,111(3):315-344.
[5]CABRAL R S,LA TORRE F D,COSTEIRA J P,et al.Matrix completion for weakly-supervised multi-label Image classification[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2015,37(1):121-135.
[6]WEN Z,YIN W,ZHANG Y,et al.Solving a low-rank factorization model for matrix completion by a nonlinear successive over-relaxation algorithm[J].Mathematical Programming Computation,2012,4(4):333-361.
[7]JAIN P,NETRAPALLI P,SANGHAVI S,et al.Low-rank matrix completion using alternating minimization[C]//ACM Symposium on Theory of Computing.ACM,2013:665-674.
[8]MNIH A,SALAKHUTDINOV R.Probabilistic matrix Factorization[M]//Advances in Neural Information Processing Systems.Berlin:Springer,2007:1257-1264.
[9]CANDES E J,RECHT B.Exact matrix completion via convex optimization[J].Foundations of Computational Mathematics,2009,9(6):717-772.
[10]KRIZHEVSKY A,SUTSKEVER I,HINTON G E,et al.ImageNet classification with deep convolutional neural networks[C]//Advances in Neural Information Processing Systems,2012,141(5):1097-1105.
[11]FU W B,SUN T,LIANG J,et al.Review of principle and application of deep learning [J].Computer Science.2018,45(S1):11-15.(in Chinese)
付文博,孙涛,梁籍,等.深度学习原理及应用综述[J].计算机科学,2018,45(S1):11-15.
[12]WANG S,SUN G M,ZOU J Z,et al.Parallel collaborative filtering algorithm based on user recommended Influence[J].Computer Science,2017,44(9):250-255.(in Chinese)
王硕,孙光明,邹静昭,等.基于用户推荐影响度的并行协同过滤算法[J].计算机科学,2017,44(9):250-255.
[13]SALAKHUTDINOV R,MNIH A,HINTON G.Restricted Boltzmann machines for collaborative filtering[C]//Proceedings of the 24th International Conference on Machine Learning.ACM,2007:791-798.
[14]SEDHAIN S,MENON A K,SANNER S,et al.Autorec:Autoencoders meet collaborative filtering[C]//Proceedings of the 24th International Conference on World Wide Web.ACM,2015:111-112.
[15]LI X,SHE J.Collaborative variational autoencoder for recommender systems[C]//Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.ACM,2017:305-314.
[16]WU C Y,AHMED A,BEUTEL A,et al.Recurrent recommender networks[C]//Proceedings of the Tenth ACM International Conference on Web Search and Data Mining.ACM,2017:495-503.
[17]KIM D,PARK C,OH J,et al.Convolutional matrix factorization for document context-aware recommendation[C]//Proceedings of the 10th ACM Conference on Recommender Systems.ACM,2016:233-240.
[18]XUE H J,DAI X,ZHANG J,et al.Deep Matrix Factorization Models for Recommender Systems[C]//Twenty-Sixth International Joint Conference on Artificial Intelligence.AAAI Press,2017:3203-3209.
[19]ZHANG S,YAO L,SUN A,et al.Deep learning based recommender system:A survey and new perspectives[J].ACM Computing Surveys,2019,52(1):1-38.
[20]CANDÉS E J,TAO T.The power of convex relaxation:Near-optimal matrix completion[J].IEEE Transactions on Information Theory,2010,56(5):2053-2080.
[21]SI S,CHIANG K Y,HSIEH C J,et al.Goal-directed inductive matrix completion[C]//Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.ACM,2016:1165-1174.
[22]FAN J,CHOW T.Deep learning based matrix completion[J].Neurocomputing,2017,266(11):540-549.
[23]DZIUGAITE G K,ROY D M.Neural network matrix factorization[J].arXiv:1511.06443,2015.
[24]FAZEL M.Matrix rank minimization with applications[D].Palo Alto:Stanford University,2002.
[25]CAI J F,CANDÉS E J,SHEN Z.A singular value thresholding algorithm for matrix completion[J].SIAM Journal on Optimization,2010,20(4):1956-1982.
[26]LU C,TANG J,YAN S,et al.Generalized nonconvex nonsmooth low-rank minimization[C]//IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2014:4130-4137.
[27]SREBRO N,RENNIE J,JAAKKOLA T S.Maximum-margin matrix factorization[M]//Advances in Neural Information Processing Systems.Berlin:Springer,2005:1329-1336.
[28]LIN Z,XU C,ZHA H.Robust matrix factorization by majorization minimization[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2018,40(1):208-220.
[29]BOUMAL N,ABSIL P.RTRMC:A Riemannian trust-region method for low-rank matrix completion[M]//Advances in Neural Information Processing Systems.Berlin:Springer,2011:406-414.
[30]TRIGEORGIS G,BOUSMALIS K,ZAFEIRIOU S,et al.A deep matrix factorization method for learning attribute representations[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,39(3):417-429.
[31]FAN J,CHENG J.Matrix completion by deep matrix factorization[J].Neural Networks,2018,98(2):34-41.
[32]HE X,LIAO L,ZHANG H,et al.Neural collaborative filtering[C]//Proceedings of the 26th International Conference on World Wide Web.International World Wide Web Conferences Steering Committee,2017:173-182.

相关文章 15

[1]	饶志双, 贾真, 张凡, 李天瑞. 基于Key-Value关联记忆网络的知识图谱问答方法 Key-Value Relational Memory Networks for Question Answering over Knowledge Graph 计算机科学, 2022, 49(9): 202-207. https://doi.org/10.11896/jsjkx.220300277
[2]	汤凌韬, 王迪, 张鲁飞, 刘盛云. 基于安全多方计算和差分隐私的联邦学习方案 Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy 计算机科学, 2022, 49(9): 297-305. https://doi.org/10.11896/jsjkx.210800108
[3]	徐涌鑫, 赵俊峰, 王亚沙, 谢冰, 杨恺. 时序知识图谱表示学习 Temporal Knowledge Graph Representation Learning 计算机科学, 2022, 49(9): 162-171. https://doi.org/10.11896/jsjkx.220500204
[4]	王剑, 彭雨琦, 赵宇斐, 杨健. 基于深度学习的社交网络舆情信息抽取方法综述 Survey of Social Network Public Opinion Information Extraction Based on Deep Learning 计算机科学, 2022, 49(8): 279-293. https://doi.org/10.11896/jsjkx.220300099
[5]	郝志荣, 陈龙, 黄嘉成. 面向文本分类的类别区分式通用对抗攻击方法 Class Discriminative Universal Adversarial Attack for Text Classification 计算机科学, 2022, 49(8): 323-329. https://doi.org/10.11896/jsjkx.220200077
[6]	姜梦函, 李邵梅, 郑洪浩, 张建朋. 基于改进位置编码的谣言检测模型 Rumor Detection Model Based on Improved Position Embedding 计算机科学, 2022, 49(8): 330-335. https://doi.org/10.11896/jsjkx.210600046
[7]	孙奇, 吉根林, 张杰. 基于非局部注意力生成对抗网络的视频异常事件检测方法 Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection 计算机科学, 2022, 49(8): 172-177. https://doi.org/10.11896/jsjkx.210600061
[8]	侯钰涛, 阿布都克力木·阿布力孜, 哈里旦木·阿布都克里木. 中文预训练模型研究进展 Advances in Chinese Pre-training Models 计算机科学, 2022, 49(7): 148-163. https://doi.org/10.11896/jsjkx.211200018
[9]	周慧, 施皓晨, 屠要峰, 黄圣君. 基于主动采样的深度鲁棒神经网络学习 Robust Deep Neural Network Learning Based on Active Sampling 计算机科学, 2022, 49(7): 164-169. https://doi.org/10.11896/jsjkx.210600044
[10]	苏丹宁, 曹桂涛, 王燕楠, 王宏, 任赫. 小样本雷达辐射源识别的深度学习方法综述 Survey of Deep Learning for Radar Emitter Identification Based on Small Sample 计算机科学, 2022, 49(7): 226-235. https://doi.org/10.11896/jsjkx.210600138
[11]	胡艳羽, 赵龙, 董祥军. 一种用于癌症分类的两阶段深度特征选择提取算法 Two-stage Deep Feature Selection Extraction Algorithm for Cancer Classification 计算机科学, 2022, 49(7): 73-78. https://doi.org/10.11896/jsjkx.210500092
[12]	程成, 降爱莲. 基于多路径特征提取的实时语义分割方法 Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction 计算机科学, 2022, 49(7): 120-126. https://doi.org/10.11896/jsjkx.210500157
[13]	何亦琛, 毛宜军, 谢贤芬, 古万荣. 基于点割集图分割的矩阵变换与分解的推荐算法 Matrix Transformation and Factorization Based on Graph Partitioning by Vertex Separator for Recommendation 计算机科学, 2022, 49(6A): 272-279. https://doi.org/10.11896/jsjkx.210600159
[14]	王君锋, 刘凡, 杨赛, 吕坦悦, 陈峙宇, 许峰. 基于多源迁移学习的大坝裂缝检测 Dam Crack Detection Based on Multi-source Transfer Learning 计算机科学, 2022, 49(6A): 319-324. https://doi.org/10.11896/jsjkx.210500124
[15]	楚玉春, 龚航, 王学芳, 刘培顺. 基于YOLOv4的目标检测知识蒸馏算法研究 Study on Knowledge Distillation of Target Detection Algorithm Based on YOLOv4 计算机科学, 2022, 49(6A): 337-344. https://doi.org/10.11896/jsjkx.210600204

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

基于深度矩阵分解网络的矩阵填充方法

Deep Matrix Factorization Network for Matrix Completion

PDF (PC)

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

Metrics

本文评价

推荐阅读 0