面向软件错误定位与理解的测试执行离散特征筛选

doi:10.11896/j.issn.1002-137X.2016.03.034

摘要/Abstract

摘要： 软件错误定位与错误理解是软件调试过程中的重要步骤,然而调试人员利用基于覆盖分析的软件错误定位获取的可疑度,从高到低静态分析每条程序语句的检查方式,与实际软件调试过程并不相符。为了能够筛选更有助于理解错误根源的测试执行,尤其是致使程序失效的失效执行,帮助调试人员进行动态差异化分析,针对失效执行提出基于高可疑度覆盖率、揭示错误潜力和覆盖语句可疑度离散特征的3种优先级策略,针对成功执行提出加权余弦相似度匹配策略。通过将3种失效执行优先级策略与随机选择在常用错误定位技术中进行实验对比,验证了基于覆盖语句可疑度离散特征的失效执行筛选策略能够对筛选前后的错误理解工作量变化产生更强的积极影响和更弱的消极影响,并能够在相同工作量下理解更多的错误,进而更有助于将错误定位结果应用于错误根源的理解。

关键词: 软件错误定位,错误理解,覆盖分析,测试执行优先级,离散特征

Abstract: Software fault localization and understanding are key steps in software debugging,while the way that debuggers use suspicious scores calculated by coverage-based fault localization from high to low to analysis every program statement does not correspond to real debugging.To select test executions which are more helpful for fault understan-ding,especially the failed executions which lead to program failure,and help debuggers conduct differentiation analysis dynamically,three prioritization strategies which are based on coverage rate of high suspicious scores,fault exposing potential and discrete characteristic of suspicious scores of covered statements were presented respectively for failed executions,and weighted cosine similarity matching was presented for passed executions.These failed test execution prioritization strategies and random sorting were compared in general fault localization techniques.And results show that the selection based on discrete characteristic of suspicious scores of statements covered by test executions has stronger positive influence and weaker negative influence on the change of fault understanding expense before and after selection,can understand more faults under same expense,and is more helpful for the application of fault localization results to fault understanding.

Key words: Software fault localization,Fault understanding,Coverage analysis,Test execution prioritization,Discrete characteristic

刘梦冷,杨小双,赵磊,王丽娜. 面向软件错误定位与理解的测试执行离散特征筛选[J]. 计算机科学, 2016, 43(3): 179-187. https://doi.org/10.11896/j.issn.1002-137X.2016.03.034

LIU Meng-leng, YANG Xiao-shuang, ZHAO Lei and WANG Li-na. Discrete Characteristic-based Test Execution Selection for Software Fault Localization and Understanding[J]. Computer Science, 2016, 43(3): 179-187. https://doi.org/10.11896/j.issn.1002-137X.2016.03.034

参考文献

[1] Shen Chang-xiang,Zhang Huan-guo,Feng Deng-guo,et al.Surver of information security [J].Science China(E),2007,32(2):129-150(in Chinese) 沈昌祥,张焕国,冯登国,等.信息安全综述[J].中国科学E辑:信息科学,2007,32(2):129-150
[2] Duraes A,Madeira H,Duraes J.Emulation of software faults:a field data study and a practical approach [J].IEEE Transaction on Software Engineering,2006,32(11):849-867
[3] Zhao Lei,Wang Li-na,Gao Dong-ming,et al.Mining associations to improve the effectiveness of fault localization[J].Chinese Journal of Computers,2012,35(12):2528-2540(in Chinese) 赵磊,王丽娜,高东明,等.基于关联挖掘的软件错误定位方法[J].计算机学报,2012,35(12):2528-2540
[4] Hao Dan,Zhang Lu,Pan Ying,et al.On similarity-awareness in testing-based fault localization[J].Journal of Automated Software Engineering,2008,15(2):207-249
[5] Cao He-ling,Jiang Shu-juan,Ju Xiao-lin.Survey of softwarefault localization[J].Computer Science,2014,41(2):1-6(in Chinese) 曹鹤玲,姜淑娟,鞠小林.软件错误定位研究综述[J].计算机科学,2014,41(2):1-6
[6] Arumuga Nainar P,Chen T,Rosin J,et al.Statistical debugging using compound Boolean predicates[C]∥Proceedings of the 2007 ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2007).New York:ACM Press,2007:5-15
[7] Liblit B,Naik M,Zheng A X,et al.Scalable statistical bug isolation[C]∥Proceedings of the ACM SIGPLAN 2005 Conference on Programming Language Design and Implementation (PLDI 2005).2005:15-26
[8] Chilimbi T,Liblit B,Mehra K,et al.HOLMES:effective statistical debugging via efficient path profiling[C]∥Proceedings of the 31st International Conference on Software Engineering (ICSE 2009).Vancouver,Canada:2009:34-44
[9] Jones J A,Harrold M J.Empirical evaluation of the tarantula automatic fault-localization technique[C]∥Proceedings of the 20th IEEE/ACM international Conference on Automated software engineering.ACM,2005:273-282
[10] Zuddas D,Wei Jin,Pastore F,et al.MIMIC:locating and understanding bugs by analyzing mimicked executions[C]∥ Procee-dings of the 29th ACM/IEEE International Conference on Automated Software Engineering.ACM,2014:815-826
[11] Sumner W N,Bao Tao,Zhang Xiang-yu.Selecting peers for execution comparison[C]∥Proceedings of the 2011 International Symposium on Software Testing and Analysis.ACM,2011:309-319
[12] Bettenburg N,Just S,Schrter A,et al.What makes a good bug report?[C]∥Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of software engineering.New York,USA:ACM Press,2008:308-318
[13] Brumley D,Caballero J,Liang Zhen-kai,et al.Towards Auto-matic Discovery of Deviations in Binary Implementations with Applications to Error Detection and Fingerprint Generation[C]∥Proceedings of 16th USENIX Security Symposium on USENIX Security Symposium.2007
[14] Jeffrey D,Gupta N,Gupta R.Fault localization using value replacement[C]∥Proceedings of the 2008 International Symposium on Software Testing and Analysis.New York,USA:ACM Press,2008(1):167-178
[15] Evans R,Savoia A.Differential testing:a new approach tochange detection[C]∥Proceedings of the the 6th joint meeting of the European Software Enginee-ring Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering:Companion Papers.ACM,2007:549-552
[16] Wong W,Horgan J,London S,et al.A study of effective regression testing in practice[C]∥Proceedings of the International Symposium on Software Reliability Engineering.IEEE Press,1997:264-274
[17] Rothermel G,Untch R H,Chu Cheng-yun,et al.Test case prio-ritization:An empirical study[C]∥IEEE International Confe-rence on Software Maintenance,1999(ICSM’99).IEEE,1999:179-188
[18] Elbaum S,Malishevsky A G,Rothermel G.Prioritizing test cases for regression testing[C]∥Proceedings of the International Symposium on Software Testing and Analysis.ACM Press,2000:102-112
[19] Elbaum S,Malishevsky A G,Rothermel G.Test case prioritization:A family of empirical studies[J].IEEE Transactions on Software Engineering,2002,28(2):159-182
[20] Yan Sha-li,Chen Zhen-yu,Zhao Zhi-hong,et al.A dynamic test cluster sampling strategy by leveraging execution spectra information[C]∥2010 Third International Conference on Software Testing,Verification and Validation (ICST).IEEE,2010:147-154
[21] Taneja K,Xie Tao,Tillmann N,et al.Guided path exploration for regression test generation[C]∥31st International Confe-rence on Software Engineering－Companion Volume,2009.IEEE,2009:311-314
[22] Renieres M,Reiss S P.Fault localization with nearest neighbor queries[C]∥Proceedings.18th IEEE International Conference on Automated Software Engineering,2003.IEEE,2003:30-39
[23] Wang Tao,Roychoudhury A.Automated path generation forsoftware fault localization[C]∥Proceedings of the 20th IEEE/ACM International Conference on Automated Software Engineering.ACM,2005:347-351
[24] Guo Liang,Roychoudhury A,Wang Tao.Accurately choosingexecution runs for software fault localization[M]∥Compiler Construction.Springer Berlin Heidelberg,2006:80-95
[25] Jones J A,Harrold M J.Empirical evaluation of the tarantula automatic fault-localization technique[C]∥Proceedings of the 20th IEEE/ACM International Conference on Automated Software Engineering.ACM,2005:273-282
[26] Voas J M.PIE:A dynamic failure-based technique[J].IEEETransactions on Software Engineering,1992,18(8):717-727
[27] Li Jia-jing,Su Xiao-hong,Ma Pei-jun,et al.A test cases selection method oriented to fault localization requirement based on branch clustering[J].Intelligent Computer and Application,2012,2(5):16-19(in Chinese) 李佳婧,苏小红,马培军,等.面向错误定位的基于分支聚类的测试用例选择方法[J].智能计算机与应用,2012,2(5):16-19
[28] He Xiao-cong.Coefficient of variation and its application tostrength prediction of adhesively bonded joints[C]∥Measuring Technology and Mechatronics Automation,2009.ICMTMA’09.International Conference on.IEEE,2009,1:602-605
[29] Xue Li-xiang,Qiu Bao-zhi.Boundary points detection algorithm based on coefficient of variation[J].Pattern Recognition and Artificial Intelligence,2009(5):799-802(in Chinese) 薛丽香,邱保志.基于变异系数的边界点检测算法[J].模式识别与人工智能,2009(5):799-802
[30] Nierman A,Jagadish H V.Evaluating structural similarity inXML documents[C]∥Proceedings of the WebDB Workshop.Madison Wisconsin,USA:EECS,2002:61-66
[31] Lee Y K,Kim W Y,Cai Y D,et al.CoMine:efficient mining of correlated patterns[C]∥Proceedings of the Third IEEE International Conference on Data Mining.Melbourne,2003:581-584
[32] Omiecinski E R.Alternative interest measures for mining associations in databases[J].IEEE Transactions on Knowledge and Data Engineering,2003,15(1):57-69
[33] Anna K.A fast outlier detection strategy for distributed high-dimensional data sets with mixed attributes[J].Data Mining and Knowledge Discovery,2010,0(2):259-289
[34] Do H,Elbaum S,Rothermel G.Supporting controlled experi-mentation with testing techniques:An infrastructure and its potential impact[J].Empirical Software Engineering,2005,10(4):405-435
[35] Liblit B,Naik M,Zheng A X,et al.Scalable statistical bug isolation[J].ACM SIGPLAN Notices,2005,40(6):15-26
[36] Abreu R,Zoeteweij P,Van Gemund A J C.On the accuracy of spectrum-based fault localization[C]∥Testing:Academic and Industrial Conference Practice and Research Techniques－MUTATION,2007.IEEE,2007:89-98
[37] Yu Yan-bing,Jones J A,Harrold M J.An empirical study of the effects of test-suite reduction on fault localization[C]∥Procee-dings of the 30th International Conference on Software Enginee-ring.ACM,2008:201-210
[38] Jiang Ling-xiao,Su Zhen-dong.Context-aware statistical debugging:from bug predictors to faulty control flow paths[C]∥Proceedings of the 22nd IEEE International Conference on Automated Software Engineering (ASE 2007).Atlanta,USA,2007:184-193
[39] Hsu H-Y,Jones J A,Orso A.Rapid:Identifying bug signatures to support eebugging activities[C]∥2008 23rd IEEE/ACM International Conference on Automated Software Engineering.IEEE,2008(2):439-442
[40] Cheng Hong,Lo D,Zhou Yang,et al.Identifying bug signatures using discriminative graph mining[C]∥Proceedings of the eighteenth International Symposium on Software Testing and Analy-sis-ISSTA ’09.New York,USA:ACM Press,2009:141-150
[41] Rβler J,Fraser G,Zeller A,et al.Isolating failure causesthrough test case generation[C]∥Proceedings of the 2012 International Symposium on Software Testing and Analysis.ACM,2012:309-319

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed