计算机科学

计算机科学 ›› 2024, Vol. 51 ›› Issue (7): 133-139.doi: 10.11896/jsjkx.231000137

• 数据库&大数据&数据科学 • 上一篇    下一篇

融合遗忘机制的多模态知识追踪模型

闫秋艳, 孙浩, 司雨晴, 袁冠   

  1. 中国矿业大学计算机科学与技术学院 江苏 徐州 221116
  • 收稿日期:2023-10-20 修回日期:2024-03-15 出版日期:2024-07-15 发布日期:2024-07-10
  • 通讯作者: 孙浩(sunh@cumt.edu.cn)
  • 作者简介:(yanqy@cumt.edu.cn)
  • 基金资助:
    国家自然科学基金(61977061,62277046)

Multimodality and Forgetting Mechanisms Model for Knowledge Tracing

YAN Qiuyan, SUN Hao, SI Yuqing, YUAN Guan   

  1. School of Computer Science & Technology,China University of Mining and Technology,Xuzhou,Jiangsu 221116,China
  • Received:2023-10-20 Revised:2024-03-15 Online:2024-07-15 Published:2024-07-10
  • About author:YAN Qiuyan,born in 1978,Ph.D,professor,is a member of CCF(No.13244S).Her main research interests include multimodal image behavior recognition,education big data analysis and time series data mining.
    SUN Hao,born in 1999,undergraduate,is a student member of CCF(No.P9283G).His main research interests include knowledge tracing and so on.
  • Supported by:
    National Natural Science Foundation of China(61977061,62277046).

PDF (PC)

511

摘要： 知识追踪是构建自适应教育系统的核心和关键,常被用以捕获学生的知识状态、预测学生的未来表现。以往的知识追踪模型仅根据结构信息对问题、技能进行建模,无法利用问题、技能的多模态信息构造其相互依赖关系。同时,关于学生的记忆水平仅以时间做量化,未考虑不同模态对记忆水平的影响。因此,提出了融合遗忘机制的多模态知识追踪模型。首先,对问题、技能节点,以图文匹配作为训练任务优化单模态嵌入,并通过计算多模态融合后节点间的相似度,获得问题和技能的关联权重从而计算生成问题节点的嵌入。其次,通过长短期记忆网络获取带有遗忘因素的学生知识状态,并将其融入学生的答题记录中生成学生节点的嵌入。最后,根据学生的答题次数和不同模态的有效记忆率计算学生和问题间的关联强度,通过图注意力网络进行信息传播,预测学生对不同问题的答题情况。在两个真实课堂自采数据集上进行了对比实验和消融实验,结果表明所提方法比其他基于图的知识追踪模型具有更好的预测精度,且针对多模态和遗忘机制的设计能有效提升原始模型的预测效果。同时,通过对一个具体案例的可视化分析,进一步说明了所提方法的实际应用效果。

关键词: 知识追踪, 多模态, 异质图, 遗忘机制

Abstract: Knowledge tracing is the core and key to build an adaptive education system,and it is often used to capture students' knowledge states and predict their future performance.Previous knowledge tracing models only model questions and skills based on structural information,unable to utilize the multimodal information of questions and skills to construct their interdependence.Additionally,the memory level of students is only quantified by time,without considering the influence of different modalities.Therefore,a multimodality and forgetting mechanisms model for knowledge tracing(MFKT)is proposed.Firstly,for question and skill nodes,a image-text matching task is used to optimize the unimodal embedding,and obtain the association weight calculation of questions and skills by calculating the similarity between nodes after multimodal fusion to generate the embedding of question nodes.Secondly,the student's knowledge state is obtained through the long short-term memory network,and forgetting factors are incorporated into their response records to generate student embeddings.Finally,the correlation strength between students and questions is calculated based on the student's response frequency and the effective memory rate of different modalities.Information propagation is performed using a graph attention network to predict the student's response to different questions.Comparative experiments and ablation experiments on two real classroom self-collected datasets show that our method has better prediction accuracy compared to other graph-based knowledge tracing models,and the design of multimodality and forgetting mechanisms effectively improves the prediction performance of the original model.At the same time,through the visual analysis of a specific case,further illustrate the practical application effect of this method.

Key words: Knowledge tracing, Multimodality, Heterogeneous graph, Forgetting mechanism

中图分类号: 

  • G40-057
[1]LI F M,YE Y W,LI X F,et al.Application of knowledge tra-cking model in Education:A Review of Relevant Studies from 2008 to 2017 [J].Distance Education in China,2019(7):86-91.
[2]SU Y,LIU Q,LIU Q,et al.Exercise-Enhanced Sequential Mo-deling for Student Performance Prediction[C]//National Confe-rence on Artificial Intelligence.2018.
[3]HOCHREITER S,SCHMIDHUBER J.Long Short-Term Me-mory[J].Neural Computation,1997,9(8):1735-1780.
[4]CHEN Z.An experimental study on the influence of information presentation style and students' cognitive style on science lear-ning in multimedia environment[D].Chongqing:Southwestern Normal University,2004.
[5]CHEN X L.Visual Computing－An Extension of Human Perception[J].Measurement&Control Technology,2000,19(5):7-14.
[6]WANG L,LI Y,HUANG J,et al.Learning Two-Branch Neural Networks for Image-Text Matching Tasks[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,201741(2):394-407.
[7]NAKAGAWA H,IWASAWA Y,MATSUO Y.Graph-basedknowledge tracing:modeling student proficiency using graph neural network[C]//IEEE/WIC/ACM International Confe-rence on Web Intelligence.2019:156-163.
[8]YANG Y,SHEN J,QU Y,et al.GIKT:a graph-based interaction model for knowledge tracing[C]//Machine Learning and Knowledge Discovery in Databases:European Conference.Springer International Publishing,2021:299-315.
[9]WU Z,HUANG L,HUANG Q,et al.SGKT:Session graph-based knowledge tracing for student performance prediction[J].Expert Systems with Applications,2022,206:117681.
[10]NGIAM J,KHOSLA A,KIM M,et al.Multimodal deep learning[C]//Proceedings of the 28th International Conference on Machine Learning(ICML-11).2011:689-696.
[11]KARPATHY A,JOULIN A,FEI-FEI L F.Deep fragment embeddings for bidirectional image sentence mapping[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems.2014:1889-1897.
[12]MALHOTRA P,RAMAKRISHNAN A,ANAND G,et al.LSTM-based encoder-decoder for multi-sensor anomaly detection[J].arXiv:1607.00148,2016.
[13]MITCHELL M,DODGE J,GOYAL A,et al.Midge:Generating image descriptions from computer vision detections[C]//Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics/2012:747-756.
[14]TENEY D,LIU L,VAN DEN HENGEL A.Graph-structuredrepresentations for visual question answering[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017.
[15]BALTRUŠAITIS T,AHUJA C,MORENCY L P.Multimodal machine learning:A survey and taxonomy[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2018,41(2):423-443.
[16]TYLENDA T,ANGELOVA R,BEDATHUR S.Towards time-aware link prediction in evolving social networks[C]//Procee-dings of the 3rd Workshop on Social Network Mining and Ana-lysis.2009.
[17]THEOCHARIDIS A,VAN DONGEN S,ENRIGHT A J,et al.Network visualization and analysis of gene expression data using BioLayout Express 3D[J].Nature Protocols,2009,4(10):1535-1550.
[18]BATTAGLIA P,PASCANU R,LAI M,et al.Interaction networks for learning about objects,relations and physics[C]//Proceedings of the 30th International Conference on Neural Information Processing Systems.2016:4509-4517.
[19]ATWOOD J,TOWSLEY D.Diffusion-convolutional neural networks[C]//Proceedings of the 30th International Conference on Neural Information Processing Systems.2016:2001-2009.
[20]SUN Z,DENG Z H,NIE J Y,et al.Rotate:Knowledge graph embedding by relational rotation in complex space[J].arXiv:1902.10197,2019.
[21]LI J H,WANG C,LI M.Knowledge Tracking Algorithm Driven by Graph Ripple Feature[J].Journal of Chinese Computer Systems.2023,44(7):1419-1427.
[22]WU S Q,DONG Y H,WANG X,et al.Learning attribute network algorithm based on high-order similarity[J].Telecommunications Science,2020,36(12):20-32.
[23]ZHOU C,LIU Y,LIU X,et al.Scalable graph embedding for asymmetric proximity[C]//Proceedings of the AAAI Confe-rence on Artificial Intelligence.2017.
[24]DONG Y,CHAWLA N V,SWAMI A.metapath2vec:Scalable representation learning for heterogeneous networks[C]//Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.2017:135-144.
[25]WANG X,JI H,SHI C,et al.Heterogeneous graph attention network[C]//The World Wide Web Conference.2019:2022-2032.
[26]LI J,SELVARAJU R,GOTMARE A,et al.Align before fuse:Vision and language representation learning with momentum distillation[J].Advances in Neural Information Processing Systems,2021,34:9694-9705.
[27]PAGE L,BRIN S,MOTWANI R,et al.The PageRank citation ranking:Bringing order to the web[R].Stanford infolab,1999.
[28]FENG H Y.Reserch of Teachers-Students's Space Distacne in the University[D].Zhengzhou:Henan University,2015.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发