Android GUI自动化测试综述

doi:10.11896/jsjkx.210900231

摘要/Abstract

摘要： 新类型新版本的手机应用数量与日俱增,使得传统的人工测试方法已经无法负荷,因此需要研究人员提出更加有效的自动化测试方法。在自动化测试的过程中,Android应用程序的GUI(Graphical User Interface),即图形用户界面,发挥着极其重要的作用,GUI自动化测试凭借其出色的测试覆盖率和故障检测性能,成为研究人员的重点研究对象。文中对当前GUI自动化测试的相关研究进行梳理和总结,选取其中有代表性、普遍性的自动化测试框架进行详细剖析,从测试策略、探索策略、错误报告、是否支持重放、测试环境、支持的事件类型、是否使用APP源码、是否开源、系统事件识别方法几个方面来对挑选出的自动化测试工具进行相应的分类、分析与对比。同时选取部分有代表性的自动化测试框架进行对比实验,以探究测试效率以及各自的优缺点。最后提出当前研究所面临的挑战以及未来的发展前景。

关键词: Android, 自动化测试, GUI测试, 测试框架, 测试用例生成

Abstract: With the increasing number of new types and versions of mobile apps,the traditional manual testing methods can’t cater for the demand.Therefore,more effective automated testing methods need to be proposed.In the process of automated testing,the GUI (Graphical User Interface) of Android apps plays an extremely important role.GUI automated testing has become the focus of researchers because of its excellent test coverage and ability of crash detection.In this paper,the current research on GUI automated testing is sorted out and summarized,and the representative automated testing framework is chosen for detailed analysis.The selected automated testing tools are classified,analyzed and compared from the aspects of testing strategy,exploration strategy,crash report,whether to support replay,testing environment,supported event type,whether to use source code,whether open source,and system event identification method.At the same time,some representative automated testing frameworks are selected for contrast experiments to explore the testing efficiency and their advantages and disadvantages.Finally,the challenges faced by the current research and the future development prospects are proposed.

Key words: Android, Automated testing, GUI testing, Testing tools, Test case generation

中图分类号:

TP391

杨艺, 王嬉, 赵春蕾, 步志亮. Android GUI自动化测试综述[J]. 计算机科学, 2022, 49(11A): 210900231-10. https://doi.org/10.11896/jsjkx.210900231

YANG Yi, WANG Xi, ZHAO Chun-lei, BU Zhi-liang. Overview of Android GUI Automated Testing[J]. Computer Science, 2022, 49(11A): 210900231-10. https://doi.org/10.11896/jsjkx.210900231

参考文献

[1]China Internet Network Information Center.The 47th ChinaStatistical Report on Internet Development[R].Beijing:China Internet Network Information Center,2021.
[2]NASS M,ALÉGROTH E,FELDT R.Why many challengeswith GUI test automation(will) remain[J].Information and Software Technology.2021,138:106625.
[3]KONG P,LI L,GAO J,et al.Automated Testing of Android Apps:A Systematic Literature Review[J].IEEE Transactions on Reliability.2019,68(1):45-66.
[4]PRASAD K.Software Testing Tools:Covering WinRunner,Silk Test,LoadRunner,JMeter and TestDirector with case studies w/CD[M].New Delhi:Dreamtech Press,2004.
[5]Google.Ui/application exerciser monkey[EB/OL].[2021-09-26].https://developer.android.com/studio/test/monkey.html.
[6]MACHIRY A,TAHILIANI R,NAIK M.Dynodroid:an input generation system for Android apps[C]//Proceedings of the 2013 9th Joint Meeting on Foundations of Software Enginee-ring.2013:224-234.
[7]LI Y,YANG Z,GUO Y,et al.Droidbot:a lightweight ui-guided test input generator for android[C]//2017 IEEE/ACM 39th International Conference on Software Engineering Companion(ICSE-C).IEEE,2017:23-26.
[8]ARDITO L,COPPOLA R,LEONARDI S,et al.AutomatedTest Selection for Android Apps Based on APK and Activity Classification[J].IEEE Access,2020,8:187648-187670.
[9]SASNAUSKAS R,REGEHR J.Intent fuzzer:crafting intents of death[C]//Proceedings of the 2014 Joint International Workshop on Dynamic Analysis(WODA) and Software and System Performance Testing,Debugging,and Analytics(PERTEA).2014:1-5.
[10]ANAND S,NAIK M,HARROLD M J,et al.Automated concolictesting of smartphone apps[C]//Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering.2012:1-11.
[11]MAHMOOD R,MIRZAEI N,MALEK S.Evodroid:Segmented evolutionary testing of android apps[C]//Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering.2014:599-609.
[12]HAO S,LIU B,NATH S,et al.Puma:Programmable ui-automation for large-scale dynamic analysis of mobile apps[C]//Proceedings of the 12th Annual International Conference on Mobile Systems,Applications,and Services.2014:204-217.
[13]NGUYEN D M,HUYNH Q T,HA N H,et al.Automated Test Input Generation via Model Inference Based on User Story and Acceptance Criteria for Mobile Application Development[J].International Journal of Software Engineering and Knowledge Engineering,2020,30(3):399-425.
[14]SU T,MENG G,CHEN Y,et al.Guided,stochastic model-based GUI testing of Android apps[C]//Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering.2017:245-256.
[15]YANG W,PRASAD M R,XIE T.A grey-box approach for automated GUI-model generation of mobile applications[M].Berlin:Springer,2013:250-265.
[16]MAO K,HARMAN M,JIA Y.Sapienz:Multi-objective automated testing for android applications[C]//Proceedings of the 25th International Symposium on Software Testing and Analysis.2016:94-105.
[17]YASIN H N,HAMID S H A,RAJA Y R J.DroidbotX:Test Case Generation Tool for Android Applications Using Q-Learning[J].Symmetry,2021,13(2):310.
[18]YAN J,ZHOU H,DENG X,et al.Efficient testing of GUI applications by event sequence reduction[J].Science of Computer Programming,2021,201:102522.
[19]YASIN H N,HAMID S H A,YUSOF R J R,et al.An empirical analysis of test input generation tools for android apps through a sequence of events[J].Symmetry,2020,12(11):1894.
[20]DING R M,SHENG J,CHEN H T,et al.Tencent Android automated testing practice [M].Beijing:Machinery Industry Press,2016.
[21]YAN J,ZHOU H,DENG X,et al.Efficient testing of GUI applications by event sequence reduction[J].Science of Computer Programming.2021,201:102522.
[22]HUANG R,SUN W,XU Y,et al.A Survey on Adaptive Random Testing[J].IEEE Transactions on Software Engineering.2021,47(10):2052-2083.
[23]NOVELLA L,TUFO M,FIENGO G.Automatic Test Set Generation for Event-Driven Systems in the Absence of Specifications Combining Testing with Model Inference[J].Information Technology and Control,2019,48(2):316-334.
[24]CAI T Q.Fastbot:moving smart monkey[EB/OL].(2020-09-28) [2021-9-22].https://mp.weixin.qq.com/s/3t4H2bfDjei4vXkj_Cz2pg.
[25]CHOI W,NECULA G,SEN K.Guided gui testing of android apps with minimal restart and approximate learning[J].ACM Sigplan Notices,2013,48(10):623-640.
[26]AMALFITANO D,FASOLINO A R,TRAMONTANA P,et al.Using GUI ripping for automated testing of Android applications[C]//2012 Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering.IEEE,2012:258-261.
[27]AMALFITANO D,FASOLINO A R,TRAMONTANA P,et al.MobiGUITAR:Automated model-based testing of mobile apps[J].IEEE Software,2014,32(5):53-59.
[28]BAEK Y M,BAE D H.Automated model-based android gui testing using multi-level gui comparison criteria[C]//Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering.2016:238-249.
[29]LINARES-VÁSQUEZ M,WHITE M,BERNAL-CÁRDENASC,et al.Mining android app usages for generating actionable gui-based execution scenarios[C]//2015 IEEE/ACM 12th Working Conference on Mining Software Repositories.IEEE,2015:111-122.
[30]AMALFITANO D,RICCIO V,AMATUCCI N,et al.Combining automated gui exploration of android apps with capture and replay through machine learning[J].Information and Software Technology,2019,105:95-116.
[31]RAVELO-MÉNDEZ W,ESCOBAR-VELÁSQUEZ C,LIN-ARES-VÁSQUEZ M.Kraken:A framework for enabling multi-device interaction-based testing of Android apps[J].Science of Computer Programming,2021,206:102627.
[32]AZIM T,NEAMTIU I.Targeted and depth-first exploration for systematic testing of android apps[C]//Proceedings of the 2013 ACM SIGPLAN International Conference on Object Oriented Programming Systems Languages & Applications.2013:641-660.
[33]JENSEN C S,PRASAD M R,MØLLER A.Automated testing with targeted event sequence generation[C]//Proceedings of the 2013 International Symposium on Software Testing and Analysis.2013:67-77.
[34]MIRZAEI N,BAGHERI H,MAHMOOD R,et al.Sig-droid:Automated system input generation for android applications[C]//2015 IEEE 26th International Symposium on Software Reliability Engineering(ISSRE).IEEE,2015:461-471.
[35]HU G,YUAN X,TANG Y,et al.Efficiently,effectively detecting mobile app bugs with appdoctor[C]//Proceedings of the Ninth European Conference on Computer Systems.2014:1-15.
[36]LI Y,YANG Z,GUO Y,et al.A deep learning based approach to automated android app testing[J].arXiv:1901.02633,2019.
[37]PAN M,LU Y,PEI Y,et al.Effective testing of Android apps using extended IFML models[J].Journal of Systems and Software,2020,159:110433.
[38]AMALFITANO D,RICCIO V,AMATUCCI N,et al.Combining automated gui exploration of android apps with capture and replay through machine learning[J].Information and Software Technology,2019,105:95-116.
[39]PACKEVIČIUS Š,RUDIONIEN G,BAREIA E.AutomatedVisual Testing of Application User Interfaces Using Static Analysis of Screenshots[J].International Journal of Software Engineering and Knowledge Engineering,2021,31(2):167-191.
[40]WANG J,JIANG Y,XU C,et al.ComboDroid:generating high-quality test inputs for Android apps via use case combinations[C]//Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering.2020:469-480.
[41]LIU C.A compatibility testing platform for android multimedia applications[J].Multimedia Tools and Applications,2019,78(4):4885-4904.
[42]LIU Y,YAN F,XIA M,et al.TimelyRep:Timing deterministic replay for Android web applications[J].Software Testing,Verification and Reliability,2020,30(4/5):e1745.
[43]FU J,WANG Y,ZHOU Y,et al.How resource utilization influences UI responsiveness of Android software[J].Information and Software Technology,2022,141:106728.
[44]PILGUN A,GADYATSKAYA O,ZHAUNIAROVICH Y,et al.Fine-grained Code Coverage Measurement in Automated Black-box Android Testing[J].ACM Transactions on Software Engineering and Methodology,2020,29(4):1-35.
[45]MU T,ZHANG H,WANG J,et al.CoLaFUZE:Coverage-Guided and Layout-Aware Fuzzing for Android Drivers[J].IEICE Transactions on Information and Systems,2021,104(11):1902-1912.
[46]KOROGLU Y,SEN A.Functional test generation from UI test scenarios using reinforcement learning for android applications[J].Software Testing,Verification and Reliability,2020,31(3):e1752.
[47]RAVINDRANATH L,NATH S,PADHYE J,et al.Automatic and scalable fault detection for mobile applications[C]//Proceedings of the 12th Annual International Conference on Mobile Systems,Applications,and Services.2014:190-203.
[48]MORAN K,LINARES-VÁSQUEZ M,BERNAL-CÁRDENAS C,et al.Crashscope:A practical tool for automated testing of android applications[C]//2017 IEEE/ACM 39th International Conference on Software Engineering Companion(ICSE-C).IEEE,2017:15-18.
[49]Google.Google firebase test lab robo test.(2021-4-28)[2021-9-26].https://firebase.google.com/docs/test-lab/robo-ux-test.
[50]ZAEEM R N,PRASAD M R,KHURSHID S.Automated generation of oracles for testing user-interaction features of mobile apps[C]//2014 IEEE Seventh International Conference on Software Testing,Verification and Validation.IEEE,2014:183-192.
[51]VAN DER MERWE H,VAN DER MERWE B,VISSER W.Execution and property specifications for jpf-android[J].ACM SIGSOFT Software Engineering Notes,2014,39(1):1-5.
[52]WHITE M,LINARES-VÁSQUEZ M,JOHNSON P,et al.Generating reproducible and replayable bug reports from android application crashes[C]//2015 IEEE 23rd International Conference on Program Comprehension.IEEE,2015:48-59.
[53]ADAMSEN C Q,MEZZETTI G,MØLLER A.Systematic execution of android test suites in adverse conditions[C]//Procee-dings of the 2015 International Symposium on Software Testing and Analysis.2015:83-93.
[54]YE H,CHENG S,ZHANG L,et al.Droidfuzzer:Fuzzing the android apps with intent-filter tag[C]//Proceedings of International Conference on Advances in Mobile Computing & Multimedia.2013:68-74.
[55]AMALFITANO D,FASOLINO A R,TRAMONTANA P,et al.A toolset for GUI testing of Android applications[C]//2012 28th IEEE International Conference on Software Maintenance(ICSM).IEEE,2012:650-653.
[56]KOROGLU Y,SEN A,MUSLU O,et al.QBE:Q Learning-based exploration of android applications[C]//2018 IEEE 11th International Conference on Software Testing,Verification and Validation(ICST).IEEE,2018:105-115.
[57]RUIZ X,CALVET L,FERRARONS J,et al.SmartMonkey:a web browser tool for solving combinatorial optimization problems in real time[C]//International Forum for Interdisciplinary Mathematics.Cham:Springer,2015:74-86.
[58]AMALFITANO D,FASOLINO A R,TRAMONTANA P,et al.Considering context events in event-based testing of mobile applications[C]//2013 IEEE Sixth International Conference on Software Testing,Verification and Validation Workshops.IEEE,2013:126-133.
[59]JABBARVAND R,SADEGHI A,BAGHERI H,et al.Energy-aware test-suite minimization for android apps[C]//Proceedings of the 25th International Symposium on Software Testing and Analysis.2016:425-436.
[60]MIRZAEI N,GARCIA J,BAGHERI H,et al.Reducing combinatorics in GUI testing of android applications[C]//2016 IEEE/ACM 38th International Conference on Software Engineering(ICSE).IEEE,2016:559-570.

相关文章 15

[1]	王毅, 李政浩, 陈星. 基于用户场景的Android 应用服务推荐方法 Recommendation of Android Application Services via User Scenarios 计算机科学, 2022, 49(6A): 267-271. https://doi.org/10.11896/jsjkx.210700123
[2]	姚烨, 朱怡安, 钱亮, 贾耀, 张黎翔, 刘瑞亮. 一种基于异质模型融合的 Android 终端恶意软件检测方法 Android Malware Detection Method Based on Heterogeneous Model Fusion 计算机科学, 2022, 49(6A): 508-515. https://doi.org/10.11896/jsjkx.210700103
[3]	张大林, 张哲玮, 王楠, 刘吉强. AutoUnit:基于主动学习和预测引导的测试自动生成 AutoUnit:Automatic Test Generation Based on Active Learning and Prediction Guidance 计算机科学, 2022, 49(11): 39-48. https://doi.org/10.11896/jsjkx.220200086
[4]	黄双芹, 刘英博, 黄向生. 模型驱动开发工具的自动化测试技术研究 Research on Automatic Testing Technology of Model Driven Development Tools 计算机科学, 2021, 48(6A): 568-571. https://doi.org/10.11896/jsjkx.201000139
[5]	汪文轩, 胡军, 胡建成, 康介祥, 王辉, 高忠杰. 一种面向形式化表格需求模型的测试用例生成方法 Test Case Generation Method Oriented to Tabular Form Formal Requirement Model 计算机科学, 2021, 48(5): 16-24. https://doi.org/10.11896/jsjkx.201000048
[6]	吉顺慧, 张鹏程. 基于支配关系的数据流测试用例生成方法 Test Case Generation Approach for Data Flow Based on Dominance Relations 计算机科学, 2020, 47(9): 40-46. https://doi.org/10.11896/jsjkx.200700021
[7]	孙志强, 万良, 丁红卫. 基于深度自编码网络的Android恶意软件检测方法 Android Malware Detection Method Based on Deep Autoencoder Network 计算机科学, 2020, 47(4): 298-304. https://doi.org/10.11896/jsjkx.190700132
[8]	孙志刚, 王国涛, 蒋爱平, 高萌萌, 刘金钢. 基于信息融合技术的行车安全监测系统 Monitoring System of Traffic Safety Based on Information Fusion Technology 计算机科学, 2020, 47(11A): 642-650. https://doi.org/10.11896/jsjkx.200400133
[9]	黄钊,黄曙光,邓兆琨,黄晖. 基于SEH的漏洞自动检测与测试用例生成 Automatic Vulnerability Detection and Test Cases Generation Method for Vulnerabilities Caused by SEH 计算机科学, 2019, 46(7): 133-138. https://doi.org/10.11896/j.issn.1002-137X.2019.07.021
[10]	张娜,滕赛娜,吴彪,包晓安. 基于粒子群优化算法的测试用例生成方法 Test Case Generation Method Based on Particle Swarm Optimization Algorithm 计算机科学, 2019, 46(7): 146-150. https://doi.org/10.11896/j.issn.1002-137X.2019.07.023
[11]	章宗美, 桂盛霖, 任飞. 基于N-gram 的Android恶意检测 Android Malware Detection Based on N-gram 计算机科学, 2019, 46(2): 145-151. https://doi.org/10.11896/j.issn.1002-137X.2019.02.023
[12]	侯禹臣, 吴伟. 静态图像行为标注众包系统的设计与实现 Design and Implementation of Crowdsourcing System for Still Image Activity Annotation 计算机科学, 2019, 46(11A): 580-583.
[13]	叶佳, 葛红军, 曹春, 朱晋, 张营. 规则驱动的Android应用DFS测试技术 Rule-driven DFS Testing Technology for Android Application 计算机科学, 2018, 45(9): 99-103. https://doi.org/10.11896／j.issn.1002-137X.2018.09.015
[14]	包晓安, 熊子健, 张唯, 吴彪, 张娜. 一种基于改进遗传算法的路径测试用例生成方法 Approach for Path-oriented Test Cases Generation Based on Improved Genetic Algorithm 计算机科学, 2018, 45(8): 174-178. https://doi.org/10.11896/j.issn.1002-137X.2018.08.031
[15]	曾星,孙备,罗武胜,刘涛诚,鲁琴. 基于深度传感器的坐姿检测系统 Sitting Posture Detection System Based on Depth Sensor 计算机科学, 2018, 45(7): 237-242. https://doi.org/10.11896／j.issn.1002-137X.2018.07.041

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed