Computer Science ›› 2026, Vol. 53 ›› Issue (6): 19-29.doi: 10.11896/jsjkx.250600192

• Intelligent Education Technology • Previous Articles     Next Articles

Academic Early Warning Prediction Model Based on Attention Mechanism and FeatureInteraction

LIU Ruyi, LYU Xiaohan, MIAO Qiguang, LU Zixiang, WANG Di   

  1. School of Computer Science and Technology,Xidian University,Xi'an 710126,China
  • Received:2025-06-26 Revised:2025-08-05 Online:2026-06-15 Published:2026-06-09
  • About author:LIU Ruyi,born in 1989,Ph.D,associate professor,Ph.D supervisor,is a member of CCF(No.55279S).Her main research interests include human action recognition and intelligent education.
    MIAO Qiguang,born in 1972,Ph.D,professor,Ph.D supervisor, is a member of CCF(No.09025D).His main research interests include computer vision and intelligent education.
  • Supported by:
    National Science and Technology Major Project(2022ZD0117103) and Open Fund of Key Lab of Education Blockchain and Intelligent Technology,Ministry of Education(EBME25-F-09).

Abstract: Under the “Internet+Education” context,higher education platforms have accumulated vast student behavior data,which is critical for academic early warning research.However,these data exhibit significant class imbalance.Additionally,structured student behavior data lack inherent spatial correlations among features,making it challenging for traditional deep learning methods to uncover potential feature interactions.Semantic differences among features can also lead to ineffective or misleading associations if not properly addressed,further impacting early warning accuracy.To address these challenges,a novel academic early warning model based on attention mechanisms and feature interactions is proposed.The model firstly employs a non-linear minority oversampling algorithm to augment data and mitigate class imbalance.It then uses residual connections and learnable multivariate Gaussian kernels to encode heterogeneous features into uniform vectors,reducing differences in data types and distributions.Feature interactions are modeled using a graph-based approach with semantic matching and attention mechanisms.Self-attention explores intra-sample feature relationships,while inter-sample attention captures correlations among different samples.An improved neural additive module based on Taylor formulas is introduced to provide interpretable predictions by explicitly representing model outputs as combinations of linear and non-linear feature contributions.Tensor decomposition is used to reduce computational complexity and enhance high-dimensional data processing efficiency,improving the model's generalization ability.

Key words: Academic early warning, Structured data, Feature interaction, Attention mechanism

CLC Number: 

  • TP391
[1]ZHAO H N.Application of Data Mining in the Prediction of Student Retention in Universities [D].Lanzhou:Lanzhou University of Technology,2020.
[2]ZHANG G P,QIN P Z,LI B W,et al.Research on Academic Early Warning Model in Higher Vocational Education in the “Internet Plus” Era:A Case Study of Jiangsu City Vocational College [J].Office Informatization,2024,29(3):24-26.
[3]LUO J,XU S.NCART:Neural Classification and RegressionTree for tabular data[J].Pattern Recognition,2024,154:110578.
[4]CHAWLA N V,BOWYER K W,HALL L O,et al.SMOTE:synthetic minority over-sampling technique[J].Journal of Artificial Intelligence Research,2002,16:321-357.
[5]HAN H,WANG W Y,MAO B H.Borderline-SMOTE:a newover-sampling method in imbalanced data sets learning[C]//International Conference on Intelligent Computing.Berlin:Springer,2005:878-887.
[6]HE H,BAI Y,GARCIA E A,et al.ADASYN:Adaptive syn-thetic sampling approach for imbalanced learning[C]//2008 IEEE International Joint Conference on Neural Networks(IEEE World Congress on Computational Intelligence).IEEE,2008:1322-1328.
[7]BUNKHUMPORNPAT C,LURSINSAP C.Safe-level-smote:Safe-level-synthetic minority over-sampling technique for handling the class imbalanced problem[C]//Advances in Know-ledge Discovery and Data Mining:13th Pacific-Asia Conference. Berlin:Springer,2009:475-482.
[8]FONSECA J,BACAO F.Geometric SMOTE for imbalanceddatasets with nominal and continuous features[J].Expert Systems with Applications,2023,234:121053.
[9]LAVALLEY M P.Logistic regression[J].Circulation,2008,117(18):2395-2399.
[10]YING L U.Decision tree methods:applications for classification and prediction[J].Shanghai Archives of Psychiatry,2015,27(2):130.
[11]CHEN T,GUESTRIN C.Xgboost:A scalable tree boosting system[C]//Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.2016:785-794.
[12]BELGIU M,DRÃGUŢ L.Random forest in remote sensing:A review of applications and future directions[J].ISPRS Journal of Photogrammetry and Remote Sensing,2016,114:24-31.
[13]KE G,MENG Q,FINLEY T,et al.Lightgbm:A highly efficient gradient boosting decision tree[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems.2017:3149-3157.
[14]HE K,ZHANG X,REN S,et al.Deep residual learning forimage recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:770-778.
[15]KLAMBAUER G,UNTERTHINER T,MAYR A,et al.Self-normalizing neural networks[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems.2017:972-981.
[16]POPOV S,MOROZOV S,BABENKO A.Neural oblivious decision ensembles for deep learning on tabular data[J].arXiv:1909.06312,2019.
[17]KATZIR L,ELIDAN G,EL-YANIV R.Net-dnf:Effective deep modeling of tabular data[C]//International Conference on Learning Representations.2020.
[18]KE G,XU Z,ZHANG J,et al.DeepGBM:A deep learningframework distilled by GBDT for online prediction tasks[C]//Proceedings of the 25th ACM SIGKDD International Confe-rence on Knowledge Discovery & Data Mining.2019:384-394.
[19]ARIK S Ö,PFISTER T.Tabnet:Attentive interpretable tabular learning[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2021:6679-6687.
[20]HUANG X,KHETAN A,CVITKOVIC M,et al.Tabtrans-former:Tabular data modeling using contextual embeddings[J].arXiv:2012.06678,2020.
[21]SUN L,ZHAO G,ZHENG Y,et al.Spectral-spatial feature tokenization transformer for hyperspectral image classification[J].IEEE Transactions on Geoscience and Remote Sensing,2022,60:1-14.
[22]CHEN J,LIAO K,WAN Y,et al.Danets:Deep abstract networks for tabular data classification and regression[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2022:3930-3938.
[23]HUI F G,RUI M T,YUN M Y,et al.Deepfm:A factorization-machine based neural network for CTR prediction[C]//IJCAI.2017.
[24]VILLAIZÁN-VALLELADO M,SALVATORI M,CARRO B,et al.Graph neural network contextual embedding for deep learning on tabular data[J].Neural Networks,2024,173:106180.
[25]CHEN J,LIAO K L,FANG Y,et al.Tabcaps:A capsule neural network for tabular data classification with bow routing[C]//The Eleventh International Conference on Learning Representations.2023.
[26]WU J,CHEN S,ZHAO Q,et al.Switchtab:Switched autoen-coders are effective tabular learners[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2024:15924-15933.
[27]AHAMED M A,CHENG Q.MambaTab:A plug-and-play mo-del for learning tabular data[C]//2024 IEEE 7th International Conference on Multimedia Information Processing and Retrieval(MIPR).IEEE,2024:369-375.
[1] KE Changbo, LI Tianhao, ZHANG Bolei, XIAO Fu, XU Kang. Teaching Evaluation Sentiment Analysis Method Based on Capsule Network [J]. Computer Science, 2026, 53(6): 10-18.
[2] XU Zhihong, YANG Xinlei, WANG Liqin, DONG Yongfeng, WANG Xu. Knowledge Tracing Model Based on Relational Learning Memory Network [J]. Computer Science, 2026, 53(6): 84-92.
[3] LI Zongmin, WANG Li, LI Yachuan, LIU Yujie, RONG Guangcai, LIU Weihan, MA Wenkang. High-accuracy Human Pose Estimation Combining Wavelet Analysis and Frequency-DomainAttention [J]. Computer Science, 2026, 53(5): 228-236.
[4] CHEN Boying, SHI Jie. Continuous Image Super-resolution Based on Self-attention Implicit Feature Encoding andDecoding [J]. Computer Science, 2026, 53(5): 237-246.
[5] LIU Dehua, YU Saixuan, QIAO Jinlan, HUANG Heqing, CHENG Wenhui. Denoising Diffusion Model-enhanced Algorithm for Battery Swap Demand Data Generation [J]. Computer Science, 2026, 53(4): 163-172.
[6] PENG Juhong, ZHANG Zhengyue, DING Zixu, FAN Xinyu, HU Changyu, ZHAO Mingjun. Multi-view Local Language Feature and Global Feature Fusion for Conversational Aspect-based Sentiment Quadruple Analysis [J]. Computer Science, 2026, 53(4): 384-392.
[7] ZHENG Cheng, BAN Qingqing. Knowledge-assisted and Reinforced Syntax-driven for Aspect-based Sentiment Analysis [J]. Computer Science, 2026, 53(4): 406-414.
[8] QIAN Qing, CHEN Huicheng, CUI Yunhe, TANG Ruixue, FU Jinmei. Joint Entity and Relation Extraction Method with Multi-scale Collaborative Aggregation and Axial-semantic Guidance [J]. Computer Science, 2026, 53(3): 97-106.
[9] GE Zeqing, HUANG Shengjun. Semi-supervised Learning Method for Multi-label Tabular Data [J]. Computer Science, 2026, 53(3): 151-157.
[10] WANG Xinyu, GAO Donghuai, NING Yuwen, XU Hao, QI Haonan. Student Behavior Detection Method Based on Improved YOLO Algorithm [J]. Computer Science, 2026, 53(3): 246-256.
[11] ZHUO Tienong, YING Di, ZHAO Hui. Research on Student Classroom Concentration Integrating Cross-modal Attention and Role
Interaction
[J]. Computer Science, 2026, 53(2): 67-77.
[12] XU Jingtao, YANG Yan, JIANG Yongquan. Time-Frequency Attention Based Model for Time Series Anomaly Detection [J]. Computer Science, 2026, 53(2): 161-169.
[13] HAN Lei, SHANG Haoyu, QIAN Xiaoyan, GU Yan, LIU Qingsong, WANG Chuang. Constrained Multi-loss Video Anomaly Detection with Dual-branch Feature Fusion [J]. Computer Science, 2026, 53(2): 236-244.
[14] GUO Xingxing, XIAO Yannan, WEN Peizhi, XU Zhi, HUANG Wenming. Attention-based Audio-driven Digital Face Video Generation Method [J]. Computer Science, 2026, 53(2): 245-252.
[15] JI Sai, QIAO Liwei, SUN Yajie. Semantic-guided Hybrid Cross-feature Fusion Method for Infrared and Visible Light Images [J]. Computer Science, 2026, 53(2): 253-263.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!