Computer Science ›› 2023, Vol. 50 ›› Issue (11A): 221100083-8.doi: 10.11896/jsjkx.221100083

• Image Processing & Multimedia Technology • Previous Articles     Next Articles

Image Aesthetics-enhanced Visual Perception Recommendation System

ZHANG Kaixuan1, CAI Guoyong2, ZHU Kunri2   

  1. 1 College of Computer and Information Security,Guilin University of Electronic Technology,Guilin,Guangxi 541000,China
    2 Key Laboratory of Guangxi Trusted Software(Guilin University of Electronic Technology),Guilin,Guangxi 541000,China
  • Published:2023-11-09
  • About author:ZHANG Kaixuan,born in 1996,postgraduate.His main research interests include recommendation system and graph deep learning.
    CAI Guoyong,born in 1971,Ph.D,professor,Ph.D supervisor,is a member of China Computer Federation.His main research interests include social media and data mining.
  • Supported by:
    National Natural Science Foundation of China(61763007) and Guangxi Key Lab of Trusted Software(kx202060).

Abstract: The visual perception recommendation system aims to enhance the behavioral features of user-item interaction by extracting the visual features of item images from the perspective of visual cognition,and model the user’s visual and behavior-rela-ted preferences,so as to make better recommendations.In the existing visual perception recommendation research,pre-trained convolutional neural network(CNN) is usually used to extract the semantic features of visual objects,and the hidden aesthetic style features inside the appearance image of the item are rarely considered.In addition,the embedded information of user-item interaction behavior structure is usually ignored in visual perception recommendation.To address these issues,an aesthetic feature-aware visual recommendation system is proposed that fuses image aesthetics and behavioral interaction structure embeddings(ABVR).ABVR uses the pre-trained ViT model to extract the high-level visual features of the image-semantic category features,uses the aesthetic extraction network to mine the middle-level aesthetic visual features in the image--the color,shapes and other features of the items,and uses the graph convolution neural network(GCN) module to learn the multi-layer graph structure embedding features of user item interaction graph nodes,and finally associates and fuses the three types of features to achieve aesthetically enhanced visual recommendations.Extensive experiments are conducted on two real datasets to verify the effectiveness of the ABVR model in improving visual recommendation performance.

Key words: Visual perception, Aesthetic features, Visual recommendation, Graph convolutional neural networks

CLC Number: 

  • TP391
[1]ANELLI V W,BELLOGÍN A,FERRARA A,et al.Velliot:Design,evaluate and tune visual recommender systems[C]//Fifteenth ACM Conference on Recommender Systems.2021:768-771.
[2]ANELLI V W,DELDJOO Y,DI NOIA T,et al.A study of defensive methods to protect visual recommendation against adversarial manipulation of images[C]//Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval.2021:1094-1103.
[3]WANG H,ZHAO M,XIE X,et al.Knowledge graph convolutional networks for recommender systems[C]//The World Wide Web Conference.2019:3307-3313.
[4]WANG X,HE X,CAO Y,et al.Kgat:Knowledge graph attention network for recommendation[C]//Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Disco-very & Data Mining.2019:950-958.
[5]WANG H,ZHANG F,ZHAO M,et al.Multi-task f-eaturelearning for knowledge graph enhanced recommendation[C]//The World Wide Web Conference.2019:2000-2010.
[6]HE R,MCAULEY J.VBPR:visual bayesian personalized ran-king from implicit feedback[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2016:144-150.
[7]CHEN J,ZHANG H,HE X,et al.Attentive collaborative filtering:Multimedia recommendation with item-and component-le-vel attention[C]//Proceedings of the 40th International ACM SIGIR conference on Research and Developmentin Information Retrieval.2017:335-344.
[8]RENDLE S,FREUDENTHALER C,GANTNER Z,et al.Bayesian personalized ranking from implicit feedback[C]//Proceedings of Uncertainty in Artificial Intelligence.2014:452-461.
[9]WANG X,HE X,WANG M,et al.Neural graph collaborative filtering[C]//Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval.2019:165-174.
[10]HE X,DENG K,WANG X,et al.Lightgcn:Simp-lifying and powering graph convolution netw-ork for recommendation[C]//Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval.2020:639-648.
[11]TANG J,DU X,He X,et al.Adversarial training towards robust multimedia recommender syst-em[J].IEEE Transactions on Knowledge and Data Engineering,2019,32(5):855-867.
[12]HE X,LIAO L,ZHANG H,et al.Neural collabor-ative filtering[C]//Proceedings of the 26th International Conference on World Wide Web.2017:173-182.
[13]NIU W,CAVERLEE J,LU H.Neural personalized ranking for image recommendation[C]//Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining.2018:423-431.
[14]GONG W,KHALID L.Aesthetics,Personalization and Recommendation:A survey on Deep Learning in Fashion[J].arXiv:2101.08301,2021.
[15]KANG W C,FANG C,WANG Z,et al.Visually-aware fashion recommendation and design withgenerative image models[C]//2017 IEEE International Conference on Data Mining(ICDM).IEEE,2017:207-216.
[16]LIU Q,WU S,WANG L.Deepstyle:Learning user preferences for visual recommendation[C]//Proceedings of the 40th international ACM Sigir Conference on Research and Development in Information Retrieval.2017:841-844.
[17]WANG Z,CHANG S,DOLCOS F,et al.Brain-inspired deepnetworks for image aesthetics assessment[J].Michigan Law Review,2016,52(1):123-128.
[18]PAUL A,WU Z,LIU K,et al.Robust multi-objective visual bayesian personalized ranking for multimedia recommendation[J].Applied Intelligence,2022,52(4):3499-3510.
[19]CANNY J.A computational approach to edge detection[J].IEEE Transactions on Pattern Analysis and Machine Intelligence.1986:679-698.
[20]TANGSENG P,OKATANI T.Toward explainable fashion recom-mendation[C]//Proceedings of the IEEE/CVF Winter Confe-rence on Applications of Computer Vision.2020:2153-2162.
[1] HUANG Yujiao, CHEN Mingkai, ZHENG Yuan, FAN Xinggang, XIAO Jie, LONG Haixia. Text Classification Based on Weakened Graph Convolutional Networks [J]. Computer Science, 2023, 50(6A): 220700039-5.
[2] GAO Chuang, LI Jian-hua, JI Xiu-yi, ZHU Cheng-long, LI Shi-liang, LI Hong-lin. Drug Target Interaction Prediction Method Based on Graph Convolutional Neural Network [J]. Computer Science, 2021, 48(10): 127-134.
[3] MA Ying-yi, LI Hong-ping, GUO Yi-feng. Visualization of Wind Vectors Using Line Integral Convolution with Visual Perception [J]. Computer Science, 2019, 46(6A): 242-245.
[4] DONG Tian-yang, YAO Jia-jie and JI Lei. Research on Three-dimensional Tree Model Simplification for Web Applications [J]. Computer Science, 2016, 43(Z6): 142-148.
[5] ZHAO Yan-ming and JI Sheng-jie. Feature Extraction Algorithms and Applications Based on Complex Visual Information of Image [J]. Computer Science, 2014, 41(Z6): 234-238.
[6] DUAN Da-gao,GONG Le,WAN Yue-liang and HAN Zhong-ming. Venation Extraction Algorithm for Insect Vein Based on Tensor Voting [J]. Computer Science, 2014, 41(12): 216-219.
[7] ZHOU Hai-ying and MU Zhi-chun. Image Understanding Model Based on Local Visual Perception and Semantic Association [J]. Computer Science, 2013, 40(7): 258-261.
[8] ZOU Xiao-lin and FENG Guo-can. Image Thresholding Segmentation Based on Visual Perception and Isoperimetric Cut [J]. Computer Science, 2013, 40(6): 303-307.
[9] ZHAO Yan-ming. Adaptive Parameters Settings Method of PCNN Based on Visual Information and its Modified Model [J]. Computer Science, 2013, 40(6): 291-294.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!