计算机科学

计算机科学 ›› 2023, Vol. 50 ›› Issue (11A): 221100083-8.doi: 10.11896/jsjkx.221100083

• 图像处理&多媒体技术 • 上一篇    下一篇

图像美学信息增强的视觉感知推荐系统

张凯煊1, 蔡国永2, 朱琨日2   

  1. 1 桂林电子科技大学计算机与信息安全学院 广西 桂林 541000
    2 广西可信软件重点实验室(桂林电子科技大学) 广西 桂林 541000
  • 发布日期:2023-11-09
  • 通讯作者: 蔡国永(ccgycai@guet.edu.cn)
  • 作者简介:(devinzkx@163.com)
  • 基金资助:
    国家自然科学基金(61763007);广西可信软件重点实验室项目(kx202060)

Image Aesthetics-enhanced Visual Perception Recommendation System

ZHANG Kaixuan1, CAI Guoyong2, ZHU Kunri2   

  1. 1 College of Computer and Information Security,Guilin University of Electronic Technology,Guilin,Guangxi 541000,China
    2 Key Laboratory of Guangxi Trusted Software(Guilin University of Electronic Technology),Guilin,Guangxi 541000,China
  • Published:2023-11-09
  • About author:ZHANG Kaixuan,born in 1996,postgraduate.His main research interests include recommendation system and graph deep learning.
    CAI Guoyong,born in 1971,Ph.D,professor,Ph.D supervisor,is a member of China Computer Federation.His main research interests include social media and data mining.
  • Supported by:
    National Natural Science Foundation of China(61763007) and Guangxi Key Lab of Trusted Software(kx202060).

PDF (PC)

192

摘要： 视觉感知推荐系统旨在从视觉认知角度出发,通过提取物品图像的视觉特征来增强用户和物品交互的行为特征,建模用户视觉与行为相关的偏好,从而更好地进行推荐。已有的视觉感知推荐研究中,通常使用预训练的卷积神经网络(CNN)来提取视觉对象语义特征,很少考虑物品外观图像内部隐藏的美学风格特征;其次,在视觉感知推荐中用户和物品的交互行为结构嵌入信息通常被忽视。为了解决这些问题,提出了一个融合图像美学和行为交互结构嵌入的美学特征感知视觉推荐系统(ABVR)。ABVR使用预训练ViT模型提取图像的高层视觉特征——语义类别特征,利用美学提取网络挖掘出图像中的中层美学视觉特征——物品的颜色、形状等特征,利用图卷积神经网络(GCN)模块学习用户物品交互图结点的多层图结构嵌入特征,最后将3类特征关联融合,以实现美学增强的视觉推荐。在两个真实数据集上进行了大量实验,验证了ABVR模型在视觉推荐性能提升上的有效性。

关键词: 视觉感知, 美学特征, 视觉推荐, 图卷积神经网络

Abstract: The visual perception recommendation system aims to enhance the behavioral features of user-item interaction by extracting the visual features of item images from the perspective of visual cognition,and model the user’s visual and behavior-rela-ted preferences,so as to make better recommendations.In the existing visual perception recommendation research,pre-trained convolutional neural network(CNN) is usually used to extract the semantic features of visual objects,and the hidden aesthetic style features inside the appearance image of the item are rarely considered.In addition,the embedded information of user-item interaction behavior structure is usually ignored in visual perception recommendation.To address these issues,an aesthetic feature-aware visual recommendation system is proposed that fuses image aesthetics and behavioral interaction structure embeddings(ABVR).ABVR uses the pre-trained ViT model to extract the high-level visual features of the image－semantic category features,uses the aesthetic extraction network to mine the middle-level aesthetic visual features in the image－－the color,shapes and other features of the items,and uses the graph convolution neural network(GCN) module to learn the multi-layer graph structure embedding features of user item interaction graph nodes,and finally associates and fuses the three types of features to achieve aesthetically enhanced visual recommendations.Extensive experiments are conducted on two real datasets to verify the effectiveness of the ABVR model in improving visual recommendation performance.

Key words: Visual perception, Aesthetic features, Visual recommendation, Graph convolutional neural networks

中图分类号: 

  • TP391
[1]ANELLI V W,BELLOGÍN A,FERRARA A,et al.Velliot:Design,evaluate and tune visual recommender systems[C]//Fifteenth ACM Conference on Recommender Systems.2021:768-771.
[2]ANELLI V W,DELDJOO Y,DI NOIA T,et al.A study of defensive methods to protect visual recommendation against adversarial manipulation of images[C]//Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval.2021:1094-1103.
[3]WANG H,ZHAO M,XIE X,et al.Knowledge graph convolutional networks for recommender systems[C]//The World Wide Web Conference.2019:3307-3313.
[4]WANG X,HE X,CAO Y,et al.Kgat:Knowledge graph attention network for recommendation[C]//Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Disco-very & Data Mining.2019:950-958.
[5]WANG H,ZHANG F,ZHAO M,et al.Multi-task f-eaturelearning for knowledge graph enhanced recommendation[C]//The World Wide Web Conference.2019:2000-2010.
[6]HE R,MCAULEY J.VBPR:visual bayesian personalized ran-king from implicit feedback[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2016:144-150.
[7]CHEN J,ZHANG H,HE X,et al.Attentive collaborative filtering:Multimedia recommendation with item-and component-le-vel attention[C]//Proceedings of the 40th International ACM SIGIR conference on Research and Developmentin Information Retrieval.2017:335-344.
[8]RENDLE S,FREUDENTHALER C,GANTNER Z,et al.Bayesian personalized ranking from implicit feedback[C]//Proceedings of Uncertainty in Artificial Intelligence.2014:452-461.
[9]WANG X,HE X,WANG M,et al.Neural graph collaborative filtering[C]//Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval.2019:165-174.
[10]HE X,DENG K,WANG X,et al.Lightgcn:Simp-lifying and powering graph convolution netw-ork for recommendation[C]//Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval.2020:639-648.
[11]TANG J,DU X,He X,et al.Adversarial training towards robust multimedia recommender syst-em[J].IEEE Transactions on Knowledge and Data Engineering,2019,32(5):855-867.
[12]HE X,LIAO L,ZHANG H,et al.Neural collabor-ative filtering[C]//Proceedings of the 26th International Conference on World Wide Web.2017:173-182.
[13]NIU W,CAVERLEE J,LU H.Neural personalized ranking for image recommendation[C]//Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining.2018:423-431.
[14]GONG W,KHALID L.Aesthetics,Personalization and Recommendation:A survey on Deep Learning in Fashion[J].arXiv:2101.08301,2021.
[15]KANG W C,FANG C,WANG Z,et al.Visually-aware fashion recommendation and design withgenerative image models[C]//2017 IEEE International Conference on Data Mining(ICDM).IEEE,2017:207-216.
[16]LIU Q,WU S,WANG L.Deepstyle:Learning user preferences for visual recommendation[C]//Proceedings of the 40th international ACM Sigir Conference on Research and Development in Information Retrieval.2017:841-844.
[17]WANG Z,CHANG S,DOLCOS F,et al.Brain-inspired deepnetworks for image aesthetics assessment[J].Michigan Law Review,2016,52(1):123-128.
[18]PAUL A,WU Z,LIU K,et al.Robust multi-objective visual bayesian personalized ranking for multimedia recommendation[J].Applied Intelligence,2022,52(4):3499-3510.
[19]CANNY J.A computational approach to edge detection[J].IEEE Transactions on Pattern Analysis and Machine Intelligence.1986:679-698.
[20]TANGSENG P,OKATANI T.Toward explainable fashion recom-mendation[C]//Proceedings of the IEEE/CVF Winter Confe-rence on Applications of Computer Vision.2020:2153-2162.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发