Computer Science ›› 2023, Vol. 50 ›› Issue (5): 38-51.doi: 10.11896/jsjkx.220900030

• Explainable AI • Previous Articles     Next Articles

Interpretable Repair Method for Event Logs Based on BERT and Weak Behavioral Profiles

LI Binghui, FANG Huan, MEI Zhenhui   

  1. School of Mathematics and Big Data,Anhui University of Science and Technology,Huainan,Anhui 232001,China
    Anhui Province Engineering Laboratory for Big Data Analysis and Early Warning Technology of Coal Mine Safety,Huainan,Anhui 232001,China
  • Received:2022-09-05 Revised:2023-01-26 Online:2023-05-15 Published:2023-05-06
  • About author:LI Binghui,born in 1998,postgraduate,is a member of China Computer Federation.His main research interests include process mining and deep learning.
    FANG Huan,born in 1982,postgraduate supervisor,professor,Ph.D.Her main research interests include Petri nets theory and application,behavioral profiles,change mining and process mi-ning.
  • Supported by:
    National Natural Science Foundation of China(61902002).

Abstract: In practical business processes,low-quality event logs due to outliers and missing values are often unavoidable.Low-quality event logs can degrade the performance of associated algorithms for process mining,which in turn interferes with the correct implementation of decisions.Under the condition that the system reference model is unknown,when performing log anomaly detection and repair work,the existing methods have the problems of needing to manually set thresholds,do not understand what behavior constraints the prediction model learns,and poor interpretability of repair results.Inspired by the fact that the pre-trained language model BERT using the masking strategy can self-supervise learning of general semantics in text through context information,combined with attention mechanism with multi-layer and multi-head,this paper proposes the model BERT4Log and weak behavioral profiles theory to perform an interpretable repair process for low-quality event logs.The proposed repair method does not need to set a threshold in advance,and only needs to perform self-supervised training once.At the same time,the method uses the weak behavioral profiles theory to quantify the degree of behavioral repair of logs.And combined with the multi-layer multi-head attention mechanism to realize the detailed interpretation process about the specific prediction results.Finally,the performance of the proposed method is evaluated on a set of public datasets,and compared with the current research with the best performance.Experimental results show that the repair performance of BERT4Log is better than the comparative research,and at the same time,the model can learn weak behavioral profiles and achieve detailed interpretation of repair results.

Key words: Event log repair, Weak behavioral profiles, BERT, Interpretable model, Attention mechanism

CLC Number: 

  • TP391
[1]WICKRAMANAYAKE B,HE Z,OUYANG C,et al.Building interpretable models for business process prediction using shared and specialised attention mechanisms[J].Knowledge-Based Systems,2022,248:108773.
[2]MOON J,PARK G,JEONG J.Pop-on:Prediction of processusing one-way language model based on nlp approach[J].Applied Sciences,2021,11(2):864.
[3]BATINI C,CAPPIELLO C,FRANCALANCI C,et al.Metho-dologies for data quality assessment and improvement[J].ACM Computing Surveys(CSUR),2009,41(3):1-52.
[4]NGUYEN H T C,LEE S,KIM J,et al.Autoencoders for improving quality of process event logs[J].Expert Systems with Applications,2019,131:132-147.
[5]BOSE R P J C,MANS R S,VAN DER AALST W M P.Wanna improve process mining results?[C]//2013 IEEE Symposium on Computational Intelligence and Data Mining(CIDM).IEEE,2013:127-134.
[6]SARZYNSKA-WAWER J,WAWER A,PAWLAK A,et al.Detecting formal thought disorder by deep contextualized word representations[J].Psychiatry Research,2021,304:114135.
[7]RADFORD A,NARASIMHAN K,SALIMANS T,et al.Improving language understanding by generative pre-training[EB/OL].(2018-12-30)[2022-07-15].https://www.cs.ubc.ca/~amuham01/LING530/papers/radfod2018improving.pdf.
[8]DEVLIN J,CHANG M W,LEE K,et al.Bert:Pre-training ofdeep bidirectional transformers for language understanding[J].arXiv:1810.04805,2018.
[9]VAN DER AALST W M P,DE MEDEIROS A K A.Process mining and security:Detecting anomalous process executions and checking process conformance[J].Electronic Notes in Theo-retical Computer Science,2005,121:3-21.
[10]GHIONNA L,GRECO G,GUZZO A,et al.Outlier detectiontechniques for process mining applications[C]//International Symposium on Methodologies for Intelligent Systems.Springer,2008:150-159.
[11]FANISANIM,ZELST S J,VAN DER AALST W M P.Repairing outlier behaviour in event logs[C]//International Confe-rence on Business Information Systems.Springer,2018:115-131.
[12]ROGGE-SOLTIA,MANS R S,VAN DER AALST W M P,et al.Improving documentation by repairing event logs[C]//IFIP Working Conference on the Practice of Enterprise Modeling.Heidelberg,Springer,2013:129-144.
[13]WANG J,SONG S,ZHU X,et al.Efficient recovery of missing events[J].Proceedings of the VLDB Endowment,2013,6(10):841-852.
[14]WANG J,SONG S,LIN X,et al.Cleaning structured eventlogs:A graph repair approach[C]//2015 IEEE 31st International Conference on Data Engineering.IEEE,2015:30-41.
[15]CHINCES D,SALOMIE I.Optimizing spaghetti process models[C]//2015 20th International Conference on Control Systems and Computer Science.IEEE,2015:506-511.
[16]LIU J,XU J,ZHANG R,et al.A repairing missing activities approach with succession relation for event logs[J].Knowledge and Information Systems,2021,63(2):477-495.
[17]SURIADI S,ANDREWS R,TER HOFSTEDE A H M,et al.Event log imperfection patterns for process mining:Towards a systematic approach to cleaning event logs[J].Information Systems,2017,64:132-150.
[18]NOLLE T,LUETTGEN S,SEELIGER A,et al.Analyzing busi-ness process anomalies using autoencoders[J].Machine Lear-ning,2018,107(11):1875-1893.
[19]NGUYEN H T C,COMUZZI M.Event log reconstruction using autoencoders[C]//International Conference on Service-Oriented Computing.Springer,2018:335-350.
[20]VASWANI A,SHAZEER N,PARMAR N,et al.Attention isall you need[C]//Advances in Neural Information Processing Systems.2017:5998-6008.
[21]VAN DER A A H,REBMANN A,LEOPOLD H.Natural language-based detection of semantic execution anomalies in event logs[J].Information Systems,2021,102:101824.
[22]RIZZI W,DI FRANCESCOMARINO C,MAGGI F M.Explaina-bility in predictive process monitoring:when understanding helps improving[C]//International Conference on Business Process Management.Springer,2020:141-158.
[23]GALANTI R,COMA-PUIG B,DE LEONI M,et al.Explainable predictive process monitoring[C]//2020 2nd International Conference on Process Mining(ICPM).IEEE,2020:1-8.
[24]WEIDLICH M,MENDLING J,WESKE M.Efficient consistency measurement based on behavioral profiles of process models[J].IEEE Transactions on Software Engineering,2010,37(3):410-429.
[25]FANG H,JIN P P,FANG X W,et al.Process variants cluster mining method based on causal behavioral profiles[J].Compu-ter Integrated Manufacturing System,2020,26(6):1538-1547.
[26]FANG H,FANG X W,WANG L L.Review of Reliability Ana-lysis Based on Petri Nets[J].Computer Science,2014,41(7):40-44.
[27]FANG H,SUN S Y,FANG X W.Behavior change mining me-thods based on incomplete logs conjoint occurrence relation[J].Computer Integrated Manufacturing System,2020,26(7):1887-1895.
[1] ZHANG Shunyao, LI Huawang, ZHANG Yonghe, WANG Xinyu, DING Guopeng. Image Retrieval Based on Independent Attention Mechanism [J]. Computer Science, 2023, 50(6A): 220300092-6.
[2] LIU Haowei, YAO Jingchi, LIU Bo, BI Xiuli, XIAO Bin. Two-stage Method for Restoration of Heritage Images Based on Muti-scale Attention Mechanism [J]. Computer Science, 2023, 50(6A): 220600129-8.
[3] LI Fan, JIA Dongli, YAO Yumin, TU Jun. Graph Neural Network Few Shot Image Classification Network Based on Residual and Self-attention Mechanism [J]. Computer Science, 2023, 50(6A): 220500104-5.
[4] SUN Kaiwei, WANG Zhihao, LIU Hu, RAN Xue. Maximum Overlap Single Target Tracking Algorithm Based on Attention Mechanism [J]. Computer Science, 2023, 50(6A): 220400023-5.
[5] WU Liuchen, ZHANG Hui, LIU Jiaxuan, ZHAO Chenyang. Defect Detection of Transmission Line Bolt Based on Region Attention Mechanism andMulti-scale Feature Fusion [J]. Computer Science, 2023, 50(6A): 220200096-7.
[6] ZHANG Shuaiyu, PENG Li, DAI Feifei. Person Re-identification Method Based on Progressive Attention Pyramid [J]. Computer Science, 2023, 50(6A): 220200084-8.
[7] LUO Huilan, LONG Jun, LIANG Miaomiao. Attentional Feature Fusion Approach for Siamese Network Based Object Tracking [J]. Computer Science, 2023, 50(6A): 220300237-9.
[8] DOU Zhi, HU Chenguang, LIANG Jingyi, ZHENG Liming, LIU Guoqi. Lightweight Target Detection Algorithm Based on Improved Yolov4-tiny [J]. Computer Science, 2023, 50(6A): 220700006-7.
[9] ZENG Zehua, LUO Huilan. Cross-dataset Learning Combining Multi-object Tracking and Human Pose Estimation [J]. Computer Science, 2023, 50(6A): 220400199-7.
[10] DAI Tianhong, SONG Jieqi. Multimodal MRI Brain Tumor Segmentation Based on Multi-encoder Architecture [J]. Computer Science, 2023, 50(6A): 220200108-6.
[11] ZHANG Guohua, YAN Xuefeng, GUAN Donghai. Anomaly Detection of Time-series Based on Multi-modal Feature Fusion [J]. Computer Science, 2023, 50(6A): 220700094-7.
[12] LIU Jiansong, KANG Yan, LI Hao, WANG Tao, WANG Hailing. Spatial-Temporal Graph-CoordAttention Network for Traffic Forecasting [J]. Computer Science, 2023, 50(6A): 220200042-7.
[13] MENG Xiangfu, XU Ruihang. City Traffic Flow Prediction Method Based on Dynamic Spatio-Temporal Neural Network [J]. Computer Science, 2023, 50(6A): 220600266-7.
[14] GAO Xiang, TANG Jiqiang, ZHU Junwu, LIANG Mingxuan, LI Yang. Study on Named Entity Recognition Method Based on Knowledge Graph Enhancement [J]. Computer Science, 2023, 50(6A): 220700153-6.
[15] ZHANG Tao, CHENG Yifei, SUN Xinxu. Graph Attention Networks Based on Causal Inference [J]. Computer Science, 2023, 50(6A): 220600230-9.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!