Computer Science

Computer Science ›› 2025, Vol. 52 ›› Issue (1): 250-258.doi: 10.11896/jsjkx.240100019

• Computer Software • Previous Articles     Next Articles

Patch Correctness Verification Method Based on CodeBERT and Stacking Ensemble Learning

HAN Wei, JIANG Shujuan, ZHOU Wei   

  1. School of Computer Science and Technology,China University of Mining and Technology,Xuzhou,Jiangsu 221116,China
    Engineering Research Center of Mine Digitalization of Ministry of Education,China University of Mining and Technology,Xuzhou,Jiangsu 221116,China
  • Received:2024-01-02 Revised:2024-06-04 Online:2025-01-15 Published:2025-01-09
  • About author:HAN Wei,born in 1998,master.His main research interest is automatic program repair.
    JIANG Shujuan,born in 1966,Ph.D,professor,Ph.D supervisor,is a member of CCF(No.15801M).Her main research interests include software analysis and test and compilation techniques.
  • Supported by:
    National Natural Science Foundation of China(61673384).

PDF (PC)

202

Abstract: In recent years,automatic program repair has become an important research topics in the field of software engineering.However,most of the existing automatic repair technologies are based on patch generation and testing,which consumes a significant amount of time and cost in the patch verification process.In addition,because the test suite is not completeness,many candidate patches can pass the test,but the test results are not consistent with the facts,which leads to the patch overfitting problem.To improve the efficiency of patch verification and alleviate patch overfitting issues,a static patch verification method is proposed.The method first uses the large pre-training model CodeBERT to automatically extract the semantic features of defect code fragments and patch code fragments,and then uses the historical defect repair patch data to train a Stacking ensemble learning model.The trained model can effectively verify the new defect repair patch.The verification ability of the proposed method is evaluated on the 1 000 patch data related to the Defects4J defect dataset.Experimental results show that the static patch verification method can effectively verify the correctness of the patch,thereby improving the efficiency of patch verification.

Key words: Automatic program repair, Patch verification, Pre-training model, Ensemble learning, Defects4J defect dataset

CLC Number: 

  • TP311
[1]LIU K,KOYUNCU A,KIM D,et al.TBar:revisiting template-based automated program repair[C]//Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis.NY:ACM,2019:31-42.
[2]WONG C,SANTIESTEBAN P,KÄSTNER C,et al.VarFix:balancing edit expressiveness and search effectiveness in automated program repair[C]//Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering.NY:ACM,2021:354-366.
[3]JIANG N,LUTELLIER T,TAN L.Cure:code-aware neuralmachine translation for automatic program repair[C]//2021 IEEE/ACM 43rd International Conference on Software Engineering.NJ:IEEE,2021:1161-1173.
[4]JIANG J J,CHEN J J,XIONG Y F.Survey of automatic program repair techniques[J].Journal of Software,2021,32(9):2665-2690.
[5]CSUVIK V,HORVÁTH D,HORVÁTH F,et al.Utilizingsource code embeddings to identify correct patches[C]//2020 IEEE 2nd International Workshop on Intelligent Bug Fixing.NJ:IEEE,2020:18-25.
[6]FRASER G,ARCURI A.Evosuite:automatic test suite generation for object-oriented software[C]//Proceedings of the 19th ACM SIGSOFT Symposium and the 13th European Conference on Foundations of Software Engineering.NY:ACM,2011:416-419.
[7]PACHECO C,ERNST M D.Randoop:feedback-directed ran-dom testing for Java[C]//Companion to the 22nd ACM SIGPLAN Conference on Object-oriented Programming Systems and Applications Companion.NY:ACM,2007:815-816.
[8]XIONG Y F,LIU X Y,ZENG M H,et al.Identifying patch correctness in test-based program repair[C]//Proceedings of the 40th International Conference on Software Engineering.NY:ACM,2018:789-799.
[9]LIN B,WANG S W,WEN M,et al.Context-aware code change embedding for better patch correctness assessment[J].ACM Transactions on Software Engineering and Methodology,2022,31(3):1-29.
[10]YE H,GU J,MARTINEZ M,et al.Automated classification of overfitting patches with statically extracted code features[J].IEEE Transactions on Software Engineering,2021,48(8):2920-2938.
[11]FENG Z Y,GUO D Y,TANG D Y,et al.Codebert:a pre-trained model for programming and natural languages[J].ar-Xiv:2002.08155,2020.
[12]VASWANI A,SHAZEER N,PARMAR N,et al.Attention isall you need[J].arXiv:1706.03762,2017.
[13]HUSAIN H,WU H H,GAZIT T,et al.Codesearchnet chal-lenge:Evaluating the state of semantic code search[J].arXiv:1909.09436,2019.
[14]KARTHIKEYAN K,WANG Z H,MAYHEW S,et al.Cross-lingual ability of multilingual bert:An empirical study[C]//International Conference on Learning Representations.OpenReview.net,2020.
[15]WOLPERT D H.Stacked generalization[J].Neural Networks,1992,5(2):241-259.
[16]DITTERRICH T G.Machine learning research:four current direction[J].Artificial Intelligence Magzine,1997,4:97-136.
[17]HOANG T,Kang H J,LO D,et al.CC2Vec:distributed representations of code changes[C]//Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering.NY:ACM,2020:518-529.
[18]TIAN H Y,LIU K,KABORÉ A K,et al.Evaluating representation learning of code changes for predicting patch correctness in program repair[C]//Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering.NJ:IEEE,2020:981-992.
[19]BREIMAN L.Random forests[J].Machine Learning,2001,45:5-32.
[20]CHEN T Q,GUESTRIN C.Xgboost:a scalable tree boosting system[C]//Proceedings of the 22nd acm sigkdd International Conference on Knowledge Discovery and Data Mining.NY:ACM,2016:785-794.
[21]KE G L,MENG Q,FINLEY T,et al.Lightgbm:A highly efficient gradient boosting decision tree[J].Neural Information Processing Systems,2017,30:3149-3157.
[22]PROKHORENKOVA L,GUSEV G,VOROBEV A,et al.CatBoost:unbiased boosting with categorical features[J].Neural Information Processing Systems,2018,31:6639-6649.
[23]LIU K,WANG S W,KOYUNCU A,et al.On the efficiency of test suite based program repair:a systematic assessment of 16 automated repair systems for java programs[C]//Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering.NY:ACM,2020:615-627.
[24]JUST R,JALALI D,ERNST M D.Defects4J:a database ofexisting faults to enable controlled testing studies for Java programs[C]//Proceedings of the 2014 International Symposium on Software Testing and Analysis.NY:ACM,2014:437-440.
[25]LE Q,MIKOLOV T.Distributed representations of sentencesand documents[C]//International Conference on Machine Learning.NY:ACM,2014:1188-1196.
[26]DEVLIN J,CHANG M W,LEE K,et al.Bert:pre-training of deep bidirectional transformers for language understanding[J].arXiv:1810.04805,2018.
[27]QI Z C,LONG F,ACHOUR S,et al.An analysis of patch plausibility and correctness for generate-and-validate patch generation systems[C]//Proceedings of the 2015 International Symposium on Software Testing and Analysis.NY:ACM,2015:24-36.
[28]LE G C,NGUYEN T V,FORREST S,et al.Genprog:a generic method for automatic software repair[J].IEEE Transactions on Software Engineering,2011,38(1):54-72.
[29]QI Y H,MAO X G,LEI Y,et al.The strength of random search on automated program repair[C]//Proceedings of the 36th International Conference on Software Engineering.NY:ACM,2014:254-265.
[30]WEIMER W,FRY Z P,FORREST S.Leveraging programequivalence for adaptive program repair:Models and first results[C]//2013 28th IEEE/ACM International Conference on Automated Software Engineering.NJ:IEEE,2013:356-366.
[31]XIN Q,REISS S P.Identifying test-suite-overfitted patchesthrough test case generation[C]//Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis.NY:ACM,2017:226-236.
[32]YANG J,ZHIKHARTSEV A,LIU Y,et al.Better test cases for better automated program repair[C]//Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering.NY:ACM,2017:831-841.
[33]TIAN H,TANG X,HABIB A,et al.Is this change the answer to that problem? Correlating descriptions of bug and code changes for evaluating patch correctness[C]//Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering.Rochester,IEEE/ACM,2022:1-13.
[34]TIAN H,LIU K,LI Y,et al.The Best of Both Worlds:Combining Learned Embeddings with Engineered Features for Accurate Prediction of Correct Patches[J].ACM Transactions on Software Engineering and Methodology,2023,32(4):1-34.
[1] ZHANG Jian, LI Hui, ZHANG Shengming, WU Jie, PENG Ying. Review of Pre-training Methods for Visually-rich Document Understanding [J]. Computer Science, 2025, 52(1): 259-276.
[2] LU Xulin, LI Zhihua. IoT Device Recognition Method Combining Multimodal IoT Device Fingerprint and Ensemble Learning [J]. Computer Science, 2024, 51(9): 371-382.
[3] LIANG Meiyan, FAN Yingying, WANG Lin. Fine-grained Colon Pathology Images Classification Based on Heterogeneous Ensemble Learningwith Multi-distance Measures [J]. Computer Science, 2024, 51(6A): 230400043-7.
[4] LI Xinrui, ZHANG Yanfang, KANG Xiaodong, LI Bo, HAN Junling. Intelligent Diagnosis of Brain Tumor with MRI Based on Ensemble Learning [J]. Computer Science, 2024, 51(6A): 230600043-7.
[5] ZHUO Peiyan, ZHANG Yaona, LIU Wei, LIU Zijin, SONG You. CTGANBoost:Credit Fraud Detection Based on CTGAN and Boosting [J]. Computer Science, 2024, 51(6A): 230600199-7.
[6] SHI Jiyun, ZHANG Chi, WANG Yuqiao, LUO Zhaojing, ZHANG Meihui. Generation of Structured Medical Reports Based on Knowledge Assistance [J]. Computer Science, 2024, 51(6): 317-324.
[7] ZHANG Zhiyuan, ZHANG Weiyan, SONG Yuqiu, RUAN Tong. Multilingual Event Detection Based on Cross-level and Multi-view Features Fusion [J]. Computer Science, 2024, 51(5): 208-215.
[8] KANG Wei, LI Lihui, WEN Yimin. Semi-supervised Classification of Data Stream with Concept Drift Based on Clustering Model Reuse [J]. Computer Science, 2024, 51(4): 124-131.
[9] BAI Jianghao, PIAO Yong. Ensemble Learning Based Open Source License Detection and Compatibility Assessment [J]. Computer Science, 2024, 51(12): 79-86.
[10] YI Liu, GENG Xinyu, BAI Jing. Hierarchical Multi-label Text Classification Algorithm Based on Parallel Convolutional Network Information Fusion [J]. Computer Science, 2023, 50(9): 278-286.
[11] ZHANG Desheng, CHEN Bo, ZHANG Jianhui, BU Youjun, SUN Chongxin, SUN Jia. Browser Fingerprint Recognition Based on Improved Self-paced Ensemble Algorithm [J]. Computer Science, 2023, 50(7): 317-324.
[12] YANG Qianlong, JIANG Lingyun. Study on Load Balancing Algorithm of Microservices Based on Machine Learning [J]. Computer Science, 2023, 50(5): 313-321.
[13] HU Zhongyuan, XUE Yu, ZHA Jiajie. Survey on Evolutionary Recurrent Neural Networks [J]. Computer Science, 2023, 50(3): 254-265.
[14] LIU Zhe, YIN Chengfeng, LI Tianrui. Chinese Spelling Check Based on BERT and Multi-feature Fusion Embedding [J]. Computer Science, 2023, 50(3): 282-290.
[15] HE Wenhao, WU Chunjiang, ZHOU Shijie, HE Chaoxin. Study on Short Text Clustering with Unsupervised SimCSE [J]. Computer Science, 2023, 50(11): 71-76.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.