计算机科学

计算机科学 ›› 2023, Vol. 50 ›› Issue (12): 49-57.doi: 10.11896/jsjkx.221200166

• 计算机软件 • 上一篇    下一篇

面向JavaScript引擎报错机制的类别导向模糊测试方法

卢凌1, 周志德1, 任志磊1,2, 江贺1   

  1. 1 大连理工大学软件学院 辽宁 大连 116620
    2 南京航空航天大学高安全系统的软件开发与验证技术工业和信息化部重点实验室 南京 210016
  • 收稿日期:2022-12-28 修回日期:2023-03-30 出版日期:2023-12-15 发布日期:2023-12-07
  • 通讯作者: 江贺(jianghe@dlut.edu.cn)
  • 作者简介:(lling@mail.dlut.edu.cn)

Category-directed Fuzzing Test Method for Error Reporting Mechanism in JavaScript Engines

LU Ling1, ZHOU Zhide1, REN Zhilei1,2, JIANG He1   

  1. 1 School of Software Engineering,Dalian University of Technology,Dalian,Liaoning 116620,China
    2 Key Laboratory of Safety-Critical Software Ministry of Industry and Information Technology,Nanjing University of Aeronautics and Astronautics,Nanjing 210016,China
  • Received:2022-12-28 Revised:2023-03-30 Online:2023-12-15 Published:2023-12-07
  • About author:LU Ling,born in 1998,postgraduate.His main research interests include software test and so on.
    JIANG He,born in 1980,Ph.D,professor,Ph.D supervisor,is a member of China Computer Federation.His main research interests include intelligent software engineering and industrial software testing.

PDF (PC)

1180

摘要： 报错机制是JavaScript引擎必不可少的一部分。面对错误的程序,JavaScript引擎报错机制应输出合理的错误信息,指出错误的原因和位置,帮助开发人员修复错误。然而,JavaScript引擎报错机制中存在会阻碍开发人员修复错误的缺陷。文中提出了首个面向JavaScript引擎报错机制的类别导向模糊测试方法CAFJER。给定一个种子程序,CAFJER首先为其选择一个目标类别的错误信息,并进行动态分析得到其上下文信息。其次,CAFJER根据种子程序的上下文信息生成能触发目标类别错误信息的测试用例。然后,CAFJER将生成的测试用例输入不同JavaScript引擎中进行差分测试。若输出的错误信息间有所差异,则说明其中可能存在缺陷。最后,CAFJER自动过滤重复的和无效的测试用例,有效减少了人工的参与。为了验证CAFJER的有效性,将CAFJER与目前先进的相似方法JEST和DIPROM进行比较,实验结果表明,CAFJER在JavaScript引擎报错机制中发现的独特缺陷数分别是JEST和DIPROM的2.17倍和26倍。在为期3个月的实验中,CAFJER还向开发者提交了17个缺陷报告,其中7个已被确认。

关键词: JavaScript, 报错机制, 错误信息, 差分测试, 程序变异

Abstract: Error reporting mechanism is an indispensable part of JavaScript engines.For programs with errors,the error reporting mechanism of JavaScript engines should output reasonable error message,point out location and cause of the error,help develo-pers to repair the program.However,there are defects in the JavaScript engine error reporting mechanism that will preventdeve-lopers from repairing errors.In this paper,the first category directed fuzzy testing method for JavaScript engine error reporting mechanism called CAFJER is proposed.For a given seed program,CAFJER first selects an error message of the target category for it and dynamically analyzes it to obtain its context information.Secondly,CAFJER generates test cases that can trigger target category error information according to the context information of the seed program.Thirdly,CAFJER inputs the generated test cases into different JavaScript engines for differential testing.If there are differences between error messages thrown by Java-Script engines,it indicates that there may be a defect.Finally,CAFJER automatically filters repeated and invalid test cases,effectively reducing manual participation.In order to verify the effectiveness of CAFJER,it is compared with the current advanced similar methods JEST and DIPROM.Experimental results show that the unique defects found by CAFJER in the JavaScript engine error reporting mechanism is 2.17 times and 26.00 times that of JEST and DIPROM respectively.During the three-month experiment,CAFJER also submitted 17 defect reports to developers and 7 of which have been confirmed.

Key words: JavaScript, Error reporting mechanism, Error message, Differential test, Program mutation

中图分类号: 

  • TP311
[1]RYAN DAHL.Node.js[EB/OL].(2022)[2022-12-21].https://nodejs.org/.
[2]GITHUB.Github Annual Report in 2021[EB/OL].(2022)[2022-12-21].https://octovers-e.github.com/2022/top-programming-languages.
[3]MOZILLA FOUNDATION.Spidermonkey[EB/OL].(2022)[2022-12-21].https://spidermonkey.dev/.
[4]SPIDERMONKEY.SpiderMonkey bug#1775215.[EB/OL].(2022-6)[2022-12-21]. https://bugzilla.-mozilla.org/show_bug.cgi?id=1775215.
[5]WANG J,CHEN B,WEI L,et al.Superion:Grammar-AwareGreybox Fuzzing[C]//2019 IEEE/ACM 41st International Conference on Software Engineering(ICSE).ACM,2019.
[6]LI Y,XUE Y,CHEN H,et al.Cerebro:context-aware adaptive fuzzing for effective vulnerability detection[C]//2019 27th ACM Joint Meeting.ACM,2019.
[7]PARK S,XU W,YUN I,et al.Fuzzing JavaScript Engines with Aspect-preserving Mutation[C]//2020 IEEE Symposium on Security and Privacy(SP).IEEE,2020.
[8]MATHIS B,GOPINATH R,ZELLER A.Learning input tokens for effective fuzzing[C]//29th ACM SIGSOFT International Symposium on Software Testing and Analysis(ISSTA'20).ACM,2020.
[9]HE X,XIE X,LI Y,et al.SoFi:Reflection-Augmented Fuzzing for JavaScript Engines[C]//Association for Computing Machi-nery.2021.
[10]PARK J,AN S,YOUN D,et al.Jest:N+ 1-version differential testing of both javascript engines and specification[C]//2021 IEEE/ACM 43rd International Conference on Software Engineering(ICSE).IEEE,2021:13-24.
[11]HAN H S,OH D H,CHA S K.Codealchemist:Semantics-Aware code Generation to Find Vulnerabilities in Javascript Engines[C]//The 2019 Annual Network and Distributed System Security Symposium.2019.
[12]LEE S,HAN H S,CHA S K,et al.Montage:A Neural Network Language Model-Guided JavaScript Engine Fuzzer[J].arXiv:2001.04107,2020.
[13]YE G,TANG Z,TAN S H,et al.Automated Conformance Testing for JavaScript Engines via Deep Compiler Fuzzing:,10.1145/3453483.345405-4[P].2021.
[14]ECMA-262.The specification of JavaScript language[S/OL].https://tc39.es/ecma262/,2021.
[15]Google.V8[EB/OL].(2022)[2022-12-21].https://v8.dev/.
[16]Apple.JavaScriptCore[EB/OL].(2022)[2022-12-21].https://github.com/phoboslab/JavaScript-Core-iOS.
[17]TANG Y X,JIANG H,ZHOU Z D,et al.Detecting compilerwarning defects via diversity-guided program mutation.[C]//IEEE Transactions on Software Engineering,2021.
[18]CHEN J J,HU W X,HAO D,et al.An empirical comparison of compiler testing techniques.[C]//Proceedings of the 38th International Conference on Software Engineering.2016:180-190.
[19]Grammar-based interpreter fuzz testing[D].Christian Holler:Saarland University,2011.
[20]MANÈS V J M,HAN H S,HAN C,et al.The art,science,and engineering of fuzzing:A survey[J].IEEE Transactions on Software Engineering,2019,47(11):2312-2331.
[21]BARTON P M,LARS F,BRYAN S.An empirical study of the reliability of unix utilities[J].Communications of the ACM,1990,33(12):32-44.
[22]HAN H S,SANG K C.IMF:Inferred Model-based Fuzzer[C]//Acm Sigsac Conference.ACM,2017.
[23]HOLLER C,HERZIG K,ZELLER A.Fuzzing with Code Fragments[C]//Proceedings of the 21st Usenix Security Sympo-sium.2012.
[24]YANG X,YANG C,EIDE E,et al.Finding and understanding bugs in C compilers[C]//ACM Sigplan Conference on Programming Language Design & Implementation.ACM,2011.
[25]MICHA Z.AFL[EB/OL].(2022)[2022-12-21].http://lcamtuf.coredump.cx/afl/.
[26]SUN C,LE V,SU Z.Finding and Analyzing Compiler Warning Defects[C]//IEEE/ACM International Conference on Software Engineering.IEEE,2017.
[27]SLASHDATA.State of the developer nation[EB/OL].(2021)[2022-12-21].https://slashdataweb-sitecms.s3.amazonaws.com/sample_reports/VZtJWxZw5Q9NDSAQ.pdf.
[28]ARIYA HIDAYAT.esprima[EB/OL].(2021)[2022-12-21].https://github.com/jquery/esprima.
[29]YUSUKE SUZUKI.escodegen[EB/OL].(2020)[2022-12-21].https://github.com/estools/escodegen.
[30]LYU C,JI S,ZHANG C,et al.MOPT:optimized mutationscheduling for fuzzers[C]//USENIX Security Symposium.2019.
[31]CHEN Y,SU T,SU Z.Deep differential testing of JVM implementations[C]//2019 IEEE/ACM 41st International Confe-rence on Software Engineering(ICSE).IEEE,2019.
[32]CHEN Y,SU T,SUN C,et al.Coverage-directed differentialtesting of JVM implementations[C]//ACM Sigplan Conference on Programming Language Design & Implementation.ACM,PUB27,New York,NY,USA,2016.
[33]METROPOLIS N,ROSENBLUTH A W,ROSENBLUTH MN,et al.Equation of state calculations by fast computing machines[J].The Journal of Chemical Physics,1953,21(6):1087-1092.
[34]CHEN J,BAI Y,DAN H,et al.Learning to Prioritize Test Programs for Compiler Testing[C]//IEEE/ACM International Conference on Software Engineering.IEEE Computer Society,2017.
[35]LE V,AFSHARI M,SU Z.Compiler validation via equivalence modulo inputs[J].ACM Sigplan Notices,2014,49(6):216-226.
[36]OFENBECK G,ROMPF T,PÜSCHEL M.RandIR:differential testing for embedded compilers[C]//ACM Sigplan Symposium on Scala.ACM,2016.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发