计算机科学 ›› 2021, Vol. 48 ›› Issue (12): 188-194.doi: 10.11896/jsjkx.210100203

• 数据库&大数据&数据科学 • 上一篇    下一篇

基于全局注意力机制的属性网络表示学习

许营坤, 马放南, 杨旭华, 叶蕾   

  1. 浙江工业大学计算机科学与技术学院 杭州310023
  • 收稿日期:2021-01-26 修回日期:2021-04-08 出版日期:2021-12-15 发布日期:2021-11-26
  • 通讯作者: 叶蕾(yelei@zjut.edu.cn)
  • 作者简介:xyk@zjut.edu.cn
  • 基金资助:
    国家自然科学基金项目(61773348);浙江省自然科学基金(LY20F020029)

Attribute Network Representation Learning Based on Global Attention

XU Ying-kun, MA Fang-nan, YANG Xu-hua, YE Lei   

  1. College of Computer Science and Technology,Zhejiang University of Technology,Hangzhou 310023,China
  • Received:2021-01-26 Revised:2021-04-08 Online:2021-12-15 Published:2021-11-26
  • About author:XU Ying-kun,born in 1979,Ph.D,lecturer,is a member of China Computer Federation.His main research interests include machine learning and classification algorithm.
    YE Lei,born in 1979,Ph.D,associate professor.Her main research interests include machine learning and so on.
  • Supported by:
    National Natural Science Foundation of China(61773348) and Zhejiang Province Natural Science Foundation of China(LY20F020029).

摘要: 属性网络不仅具有复杂的拓扑结构,其节点还包含丰富的属性信息。属性网络表示学习方法同时提取网络拓扑结构和节点的属性信息来学习大型属性网络的低维向量表示,在节点分类、链路预测和社区识别等网络分析技术方面具有非常重要和广泛的应用。文中首先根据属性网络的拓扑结构得到网络的结构嵌入向量;接着通过全局注意力机制来学习相邻节点的属性信息,先用卷积神经网络对节点的属性信息作卷积操作得到隐藏向量,再对卷积的隐藏向量生成全局注意力的权重向量和相关性矩阵,进而得到节点的属性嵌入向量;最后将结构嵌入向量和属性嵌入向量连接得到同时反映网络结构和节点属性的联合嵌入向量。在3个真实数据集上,将提出的新算法与当前的8种知名网络表示学习模型在链路预测和节点分类等任务上进行比较,实验结果表明新算法具有良好的属性网络表示效果。

关键词: 全局注意力, 卷积神经网络, 结构嵌入, 属性嵌入, 联合嵌入

Abstract: The attribute network not only has complex topology,its nodes also contain rich attribute information.Attribute network represent learning methods simultaneously extracts network topology and node attribute information to learn low-dimensional vector embedding of large attribute networks.It has very important and extensive applications in network analysis techniques such as node classification,link prediction and community identification.In this paper,we first obtain the embedded vector of the network structure according to the topology of the attribute network.Then we propose to learn the attribute information of adjacent nodes through global attention mechanism,use convolutional neural network to convolve the attribute information of the node to obtain the hidden vectors,and the weight vector and correlation matrix of global attention are generated from the hidden vectors of convolution,and then the attribute embedding vector of nodes is obtained.Finally,the structure embedding vector and the attribute embedding vector are connected to obtain the joint embedding vector which reflects the network structure and the attribute simultaneously.On three real data sets,the new algorithm is compared with the current eight network embedding models on tasks such as link prediction and node classification.Experimental results show that the new algorithm has good attribute network embedding effects.

Key words: Global attention mechanism, Convolutional neural network, Structure embedding, Attribute embedding, Joint embedding

中图分类号: 

  • TP391
[1]PEROZZI B,AL-RFOU R,SKIENA S.DeepWalk:Online Learning of Social Representations[C]//Proceedings of the ACM SIGKDD International Conference on Knowledge Disco-very and Data Mining.New York:ACM,2014:701-710.
[2]KIP F T N,WELLING M.Semi-supervised Classification with Graph Convolutional Networks[C]//5th International Confe-rence on Learning Representations.ShangHai,2017.
[3]WANG C,PAN S,LONG G,et al.MGAE:Marginalized Graph Autoencoder for Graph Clustering[C]//International Confe-rence on Information and Knowledge Management.Singapore:ACM,2017:889-898.
[4]XIONG F,WANG X,PAN S,et al.Social Recommendation with Evolutionary Opinion Dynamics[J].IEEE Transactions on Systems Man Cybernetics-Systems,2018,50(10):3804-3816.
[5] ZHAO X,LI X,ZHANG Z H,et al.Community discovery algorithm combining community embedding and node embedding[J].Computer Science,2020,47(12):279-284.
[6]CAI H,ZHENG V,CHANG K.A Comprehensive Survey of Graph Embedding:Problems,Techniques and Applications[J].IEEE Transactions on Knowledge and Data Engineering,2018,30(9):1616-1637.
[7]SHI C,HU B,ZHAO W,et al.Heterogeneous Information Network Embedding for Recommendation[J].IEEE Transactions on Knowledge and Data Engineering,2019,31(2):357-370.
[8]ZHOU L E,YOU J G.Social Recommendation with Embedding of Summarized Graphs[J].Journal of Chinese Computer Systems,2021,42(1):78-84.
[9]GROVER A,LESKOVEC J.Node2vec:Scalable Feature Lear- ning for Networks[C]//Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mi-ning.San Francisco:ACM,2016:855-864.
[10]WANG D,CUI P,ZHU W.Structural Deep Network Embed- ding[C]//Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.San Francisco:ACM,2016:1225-1234.
[11]IVANOV S,BURNAEV E.Anonymous Walk Embeddings [C]//35th International Conference on Machine Learning.Stockholm:ACM,2018:3448-3457.
[12]DUTTA A,RIBA P,LIADOS J,et al.Hierarchical Stochastic Graphlet Embedding for Graph-based Pattern Recognition[J].Neural Computing & Applications,2019,32(15):11596-11597.
[13]ROWEIS S,SAUL L.Nonlinear Dimensionality Reduction by Locally Linear Embedding[J].Science,2000,290(5500):2323-2326.
[14]BELKIN M,NIYOGI P.Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering[J].Advances in Neural Information Processing Systems,2001,14:585-591.
[15]CAO S,LU W,XU Q.GraRep:Learning Graph Representations with Global Structural Information[C]//International Confe-rence on Information and Knowledge Management.Melbourne:ACM,2015:891-900.
[16]MIKOLOV T,CHEN K,CORRADO G,et al.Efficient Estimation of Word Representations in Vector Space[C]//1st International Conference on Learning Representations.Arizona,2013.
[17]CHEN H,HU Y,PEROZZI B,et al.HARP-Hierarchical Representation Learning for Networks[C]//32nd AAAI Conference on Artificial Intelligence.New Orleans:AAAI,2018:2127-2134.
[18]CAO S,LU W,XU Q.Deep Neural Networks for Learning Graph Representations[C]//Thirtieth AAAI Conference on Artificial Intelligence.Arizona:AAAI,2016:1145-1152.
[19]KIPF T N,WELLING M.Variational Graph Auto-Encoders [C]//Proceedings of NIPS.Barcelona:MIT Press,2016.
[20]YANG C,LIU Z,ZHAO D,et al.Network Representation Learning with Rich Text Information[C]//IJCAI International Joint Conference on Artificial Intelligence.Buenos Aires:Morgan Kaufmann,2015:2111-2117.
[21]TU C,ZHANG W,LIU Z,et al.Max-Margin DeepWalk:Dis- criminative learning of network representation[C]//IJCAI International Joint Conference on Artificial Intelligence.New York:Morgan Kaufmann,2016:3889-3895.
[22]TU C,HAN L,LIU Z,et al.CANE:Context-Aware Network Embedding for Relation Modeling[C]//ACL 2017-55th Annual Meeting of the Association for Computational Linguistics.Vancouver:ACL,2017:1722-1731.
[23]TU C,WANG H,ZENG X,et al.Community-enhanced Network Representation Learning for Network Analysis[J].arXiv:1611.06645,2016.
[24]TOMAS M,IIYA S,KAI C,et al.Distributed Representations of Words and Phrases and Their Compositionality[C]//Proceedings of NIPS.Lake Tahoe:MIT Press,2013:3111-3119.
[25]TANG J,QU M,WANG M,et al.LINE:Large-scale Information Network Embedding[C]//Proceedings of the 24th International Conference on World Wide Web.New York:ACM,2015:1067-1077.
[26]LAHOTI P,GARIMELLA K,GIONIS A.Joint non-negative Matrix Factorization for Learning Ideological Leaning on Twitter[C]//WSDM'18:Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining.Los Ange-les:ACM,2018:351-359.
[27]TAN C,TANG J,SUN J,et al.Social Action Tracking Via Noise Tolerant Time-varying Factor Graphs[C]//Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.Washington:ACM,2010:1049-1058.
[28]CHEN J,ZHANG Q,HUANG X.Incorporate Group Information to Enhance Network Embedding[C]//International Confe-rence on Information and Knowledge Management.Pollis:ACM,2016:1901-1904.
[29]SUN X,GUO J,DING X,et al.A General Framework for Content-enhanced Network Representation Learning[J].arXiv:1610.02906v3,2016.
[30]CHO K,VAN M,GULCEHRE C,et al.Learning Phrase Repre- sentations using RNN Encoder-Decoder for Statistical Machine Translation[C]//EMNLP 2014-2014 Conference on Empirical Methods in Natural Language Processing.Doha:ACL,2014:1724-1734.
[31]SUTSKEVER I,VINYALS O,LE Q.Sequence to Sequence Learning with Neural Networks[C]//Advances in Neural Information Processing Systems.Montréal:MIT Press,2014:3104-3112.
[32]BAHDANAU D,CHO K,BENGIO Y.Neural Machine Translation by Jointly Learning to Align and Translate[C]//3rd International Conference on Learning Representations.San Diego,2015.
[33]LUONG M,PHAM H,MANNING C.Effective approaches to attention-based neural machine translation[C]//Conference Proceedings-EMNLP 2015:Conference on Empirical Methods in Natural Language Processing.Lisbon:ACL,2015:1412-1421.
[34]VASWANI A,SHAZEER N,PARMAR N,et al.Attention Is All You Need[C]//Advances in Neural Information Processing Systems.Long Beach:MIT Press,2017:5999-6009.
[35]CHENG J,DONG L,LAPATA M.Long Short-Term Memory- Networks for Machine Reading[C]//EMNLP 2016-Conference on Empirical Methods in Natural Language Processing.Austin:ACL,2016:551-561.
[36]VELICKOVIC P,CUCURULL G,CASANOVA A,et al.Graph Attention Networks[C]//6th International Conference on Learning Representations.Vancouver,2018.
[37]LIU Z,CHEN C,LI L,et al.Graph Neural Networks with Adaptive Receptive Paths[C]//33rd AAAI Conference on Artificial Intelligence.Xi'an:AAAI,2019:4424-4431.
[38]HU B,LU Z,LI H,et al.Convolutional neural network architectures for matching natural language sentences[C]//Advances in Neural Information Processing Systems 27-28th Annual Confe-rence on Neural Information Processing Systems 2014.Mon-treal:MIT Press,2014:2042-2050.
[39]TOMAS M,IIYA S,KAI C,et al.Distributed Representations of Words and Phrases and Their Compositionality[C]//Proceedings of NIPS.Lake Tahoe:MIT Press,2013:3111-3119.
[40]KINGMA D,BA J.Adam:A Method for Stochastic Optimization[C]//Proceedings of ICLR.Montreal,2015.
[41]MCCALLUM A,NIGAM K,RENNIE J,et al.Automating the Construction of Internet Portals with Machine Learning[J].Information Retrieval,2000,3(2):127-163.
[42]JURE L,JON K,CHRISTOS F.Graphs over Time:Densification Laws,Shrinking Diameters and Possible Explanations[C]//Proceedings of the Eleventh ACM SIGKDD International Confe-rence on Knowledge Discovery in Data Mining.Chicago:ACM,2005:177-187.
[43]AIROLDI E,BLEI D,FIENBERG S,et al.Mixed membership stochastic blockmodels[C]//Advances in Neural Information Processing Systems.Whistler:MIT Press,2008:1981-2014.
[44]HANLEY J,MCNEIL B.The Meaning and Use of the Area Under a Receiver Operating Characteristic(roc) Curve[J].Ra-diology,1982,143(1):29-36.
[45]FAN R,CHANG K,HSIEH C,et al.Liblinear:A Library for Large Linear Classification[J].Journal of Machine Learning Research,2008,9(9):1871-1874.
[1] 黄颖琦, 陈红梅. 基于代价敏感卷积神经网络的非平衡问题混合方法[J]. 计算机科学, 2021, 48(9): 77-85.
[2] 徐涛, 田崇阳, 刘才华. 基于深度学习的人群异常行为检测综述[J]. 计算机科学, 2021, 48(9): 125-134.
[3] 王乐, 杨晓敏. 基于感知损失的遥感图像全色锐化反馈网络[J]. 计算机科学, 2021, 48(8): 91-98.
[4] 王炽, 常俊. 基于3D卷积神经网络的CSI跨场景手势识别方法[J]. 计算机科学, 2021, 48(8): 322-327.
[5] 程松盛, 潘金山. 基于深度学习特征匹配的视频超分辨率方法[J]. 计算机科学, 2021, 48(7): 184-189.
[6] 王栋, 周大可, 黄有达, 杨欣. 基于多尺度多粒度特征的行人重识别[J]. 计算机科学, 2021, 48(7): 238-244.
[7] 熊朝阳, 王婷. 基于卷积神经网络的建筑构件图像识别[J]. 计算机科学, 2021, 48(6A): 51-56.
[8] 胡京徽, 许鹏. 一种基于图像分类的航空紧固件产品自动分类方法[J]. 计算机科学, 2021, 48(6A): 63-66.
[9] 和青芳, 王慧, 程光. 自适应小数据集乳腺癌病理组织分类研究[J]. 计算机科学, 2021, 48(6A): 67-73.
[10] 徐少伟, 秦品乐, 曾建朝, 赵致楷, 高媛, 王丽芳. 基于多级特征和全局上下文的纵膈淋巴结分割算法[J]. 计算机科学, 2021, 48(6A): 95-100.
[11] 王建明, 黎向锋, 叶磊, 左敦稳, 张丽萍. 基于信道注意结构的生成对抗网络医学图像去模糊[J]. 计算机科学, 2021, 48(6A): 101-106.
[12] 韩斌, 曾松伟. 基于多特征融合和卷积神经网络的植物叶片识别[J]. 计算机科学, 2021, 48(6A): 113-117.
[13] 余晗青, 杨贞, 殷志坚. 基于区域激活策略的Tiny YOLOv3目标检测算法[J]. 计算机科学, 2021, 48(6A): 118-121.
[14] 刘吉华, 张梦迪, 彭红霞, 贾兴平. 基于卷积神经网络的汽车销量预测模型[J]. 计算机科学, 2021, 48(6A): 178-183.
[15] 陈扬, 王金亮, 夏炜, 杨颢, 朱润, 奚雪峰. 基于特征自动提取的足迹图像聚类方法[J]. 计算机科学, 2021, 48(6A): 255-259.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] 江梦涛,荆琦. C语言静态代码分析中的调用关系提取方法[J]. 计算机科学, 2014, 41(Z6): 442 -444 .
[2] 王 宁 任红伟. 网络表格间的快照关系发现[J]. 计算机科学, 2015, 42(7): 5 -11 .
[3] 王红霞,曹波. 基于遗传编程的中国股票市场有效性检验[J]. 计算机科学, 2016, 43(Z6): 538 -541 .
[4] 黄引豪, 马郓, 林兵, 於志勇, 陈星. 混合云环境下面向代价优化的工作流数据布局方法[J]. 计算机科学, 2019, 46(11A): 354 -358 .
[5] 李泽文, 李子铭, 费天禄, 王瑞琳, 谢在鹏. 基于残差生成对抗网络的人脸图像复原[J]. 计算机科学, 2020, 47(6A): 230 -236 .
[6] 崔阳, 刘长红. 基于PIFA的语音识别系统评测平台[J]. 计算机科学, 2020, 47(11A): 638 -641 .
[7] 朝乐门, 王锐. 数据科学平台:特征、技术及趋势[J]. 计算机科学, 2021, 48(8): 1 -12 .
[8] 潘孝勤, 芦天亮, 杜彦辉, 仝鑫. 基于深度学习的语音合成与转换技术综述[J]. 计算机科学, 2021, 48(8): 200 -208 .
[9] 王俊, 王修来, 庞威, 赵鸿飞. 面向科技前瞻预测的大数据治理研究[J]. 计算机科学, 2021, 48(9): 36 -42 .
[10] 余力, 杜启翰, 岳博妍, 向君瑶, 徐冠宇, 冷友方. 基于强化学习的推荐研究综述[J]. 计算机科学, 2021, 48(10): 1 -18 .