Computer Science

Computer Science ›› 2022, Vol. 49 ›› Issue (11A): 210900231-10.doi: 10.11896/jsjkx.210900231

• Software Engineering • Previous Articles     Next Articles

Overview of Android GUI Automated Testing

YANG Yi, WANG Xi, ZHAO Chun-lei, BU Zhi-liang   

  1. Key Laboratory of Computer Vision and System of Ministry of Education,Tianjin University of Technology,Tianjin 300384,China
    Tianjin Key Laboratory of Intelligent Computing and Novel Software Technology,Tianjin University of Technology,Tianjin 300384,China
  • Online:2022-11-10 Published:2022-11-21
  • About author:YANG Yi,born in 1998,master.Her main research interests include Android automated testing and so on.
    ZHAO Chun-lei,born in 1979,Ph.D,associate professor,is a member of China Computer Federation.Her main research interests include cybersecurity and so on.
  • Supported by:
    National Funds-joint Fund Projects(U1536122),Key Special Project of “Science and Technology Helps Economy 2020” of the Ministry of Science and Technology(SQ2020YFF0413781) and Tianjin Natural Science Youth Fund (18JCQNJC69900).

PDF (PC)

605

Abstract: With the increasing number of new types and versions of mobile apps,the traditional manual testing methods can’t cater for the demand.Therefore,more effective automated testing methods need to be proposed.In the process of automated testing,the GUI (Graphical User Interface) of Android apps plays an extremely important role.GUI automated testing has become the focus of researchers because of its excellent test coverage and ability of crash detection.In this paper,the current research on GUI automated testing is sorted out and summarized,and the representative automated testing framework is chosen for detailed analysis.The selected automated testing tools are classified,analyzed and compared from the aspects of testing strategy,exploration strategy,crash report,whether to support replay,testing environment,supported event type,whether to use source code,whether open source,and system event identification method.At the same time,some representative automated testing frameworks are selected for contrast experiments to explore the testing efficiency and their advantages and disadvantages.Finally,the challenges faced by the current research and the future development prospects are proposed.

Key words: Android, Automated testing, GUI testing, Testing tools, Test case generation

CLC Number: 

  • TP391
[1]China Internet Network Information Center.The 47th ChinaStatistical Report on Internet Development[R].Beijing:China Internet Network Information Center,2021.
[2]NASS M,ALÉGROTH E,FELDT R.Why many challengeswith GUI test automation(will) remain[J].Information and Software Technology.2021,138:106625.
[3]KONG P,LI L,GAO J,et al.Automated Testing of Android Apps:A Systematic Literature Review[J].IEEE Transactions on Reliability.2019,68(1):45-66.
[4]PRASAD K.Software Testing Tools:Covering WinRunner,Silk Test,LoadRunner,JMeter and TestDirector with case studies w/CD[M].New Delhi:Dreamtech Press,2004.
[5]Google.Ui/application exerciser monkey[EB/OL].[2021-09-26].https://developer.android.com/studio/test/monkey.html.
[6]MACHIRY A,TAHILIANI R,NAIK M.Dynodroid:an input generation system for Android apps[C]//Proceedings of the 2013 9th Joint Meeting on Foundations of Software Enginee-ring.2013:224-234.
[7]LI Y,YANG Z,GUO Y,et al.Droidbot:a lightweight ui-guided test input generator for android[C]//2017 IEEE/ACM 39th International Conference on Software Engineering Companion(ICSE-C).IEEE,2017:23-26.
[8]ARDITO L,COPPOLA R,LEONARDI S,et al.AutomatedTest Selection for Android Apps Based on APK and Activity Classification[J].IEEE Access,2020,8:187648-187670.
[9]SASNAUSKAS R,REGEHR J.Intent fuzzer:crafting intents of death[C]//Proceedings of the 2014 Joint International Workshop on Dynamic Analysis(WODA) and Software and System Performance Testing,Debugging,and Analytics(PERTEA).2014:1-5.
[10]ANAND S,NAIK M,HARROLD M J,et al.Automated concolictesting of smartphone apps[C]//Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering.2012:1-11.
[11]MAHMOOD R,MIRZAEI N,MALEK S.Evodroid:Segmented evolutionary testing of android apps[C]//Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering.2014:599-609.
[12]HAO S,LIU B,NATH S,et al.Puma:Programmable ui-automation for large-scale dynamic analysis of mobile apps[C]//Proceedings of the 12th Annual International Conference on Mobile Systems,Applications,and Services.2014:204-217.
[13]NGUYEN D M,HUYNH Q T,HA N H,et al.Automated Test Input Generation via Model Inference Based on User Story and Acceptance Criteria for Mobile Application Development[J].International Journal of Software Engineering and Knowledge Engineering,2020,30(3):399-425.
[14]SU T,MENG G,CHEN Y,et al.Guided,stochastic model-based GUI testing of Android apps[C]//Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering.2017:245-256.
[15]YANG W,PRASAD M R,XIE T.A grey-box approach for automated GUI-model generation of mobile applications[M].Berlin:Springer,2013:250-265.
[16]MAO K,HARMAN M,JIA Y.Sapienz:Multi-objective automated testing for android applications[C]//Proceedings of the 25th International Symposium on Software Testing and Analysis.2016:94-105.
[17]YASIN H N,HAMID S H A,RAJA Y R J.DroidbotX:Test Case Generation Tool for Android Applications Using Q-Learning[J].Symmetry,2021,13(2):310.
[18]YAN J,ZHOU H,DENG X,et al.Efficient testing of GUI applications by event sequence reduction[J].Science of Computer Programming,2021,201:102522.
[19]YASIN H N,HAMID S H A,YUSOF R J R,et al.An empirical analysis of test input generation tools for android apps through a sequence of events[J].Symmetry,2020,12(11):1894.
[20]DING R M,SHENG J,CHEN H T,et al.Tencent Android automated testing practice [M].Beijing:Machinery Industry Press,2016.
[21]YAN J,ZHOU H,DENG X,et al.Efficient testing of GUI applications by event sequence reduction[J].Science of Computer Programming.2021,201:102522.
[22]HUANG R,SUN W,XU Y,et al.A Survey on Adaptive Random Testing[J].IEEE Transactions on Software Engineering.2021,47(10):2052-2083.
[23]NOVELLA L,TUFO M,FIENGO G.Automatic Test Set Generation for Event-Driven Systems in the Absence of Specifications Combining Testing with Model Inference[J].Information Technology and Control,2019,48(2):316-334.
[24]CAI T Q.Fastbot:moving smart monkey[EB/OL].(2020-09-28) [2021-9-22].https://mp.weixin.qq.com/s/3t4H2bfDjei4vXkj_Cz2pg.
[25]CHOI W,NECULA G,SEN K.Guided gui testing of android apps with minimal restart and approximate learning[J].ACM Sigplan Notices,2013,48(10):623-640.
[26]AMALFITANO D,FASOLINO A R,TRAMONTANA P,et al.Using GUI ripping for automated testing of Android applications[C]//2012 Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering.IEEE,2012:258-261.
[27]AMALFITANO D,FASOLINO A R,TRAMONTANA P,et al.MobiGUITAR:Automated model-based testing of mobile apps[J].IEEE Software,2014,32(5):53-59.
[28]BAEK Y M,BAE D H.Automated model-based android gui testing using multi-level gui comparison criteria[C]//Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering.2016:238-249.
[29]LINARES-VÁSQUEZ M,WHITE M,BERNAL-CÁRDENASC,et al.Mining android app usages for generating actionable gui-based execution scenarios[C]//2015 IEEE/ACM 12th Working Conference on Mining Software Repositories.IEEE,2015:111-122.
[30]AMALFITANO D,RICCIO V,AMATUCCI N,et al.Combining automated gui exploration of android apps with capture and replay through machine learning[J].Information and Software Technology,2019,105:95-116.
[31]RAVELO-MÉNDEZ W,ESCOBAR-VELÁSQUEZ C,LIN-ARES-VÁSQUEZ M.Kraken:A framework for enabling multi-device interaction-based testing of Android apps[J].Science of Computer Programming,2021,206:102627.
[32]AZIM T,NEAMTIU I.Targeted and depth-first exploration for systematic testing of android apps[C]//Proceedings of the 2013 ACM SIGPLAN International Conference on Object Oriented Programming Systems Languages & Applications.2013:641-660.
[33]JENSEN C S,PRASAD M R,MØLLER A.Automated testing with targeted event sequence generation[C]//Proceedings of the 2013 International Symposium on Software Testing and Analysis.2013:67-77.
[34]MIRZAEI N,BAGHERI H,MAHMOOD R,et al.Sig-droid:Automated system input generation for android applications[C]//2015 IEEE 26th International Symposium on Software Reliability Engineering(ISSRE).IEEE,2015:461-471.
[35]HU G,YUAN X,TANG Y,et al.Efficiently,effectively detecting mobile app bugs with appdoctor[C]//Proceedings of the Ninth European Conference on Computer Systems.2014:1-15.
[36]LI Y,YANG Z,GUO Y,et al.A deep learning based approach to automated android app testing[J].arXiv:1901.02633,2019.
[37]PAN M,LU Y,PEI Y,et al.Effective testing of Android apps using extended IFML models[J].Journal of Systems and Software,2020,159:110433.
[38]AMALFITANO D,RICCIO V,AMATUCCI N,et al.Combining automated gui exploration of android apps with capture and replay through machine learning[J].Information and Software Technology,2019,105:95-116.
[39]PACKEVIČIUS Š,RUDIONIEN G,BAREIA E.AutomatedVisual Testing of Application User Interfaces Using Static Analysis of Screenshots[J].International Journal of Software Engineering and Knowledge Engineering,2021,31(2):167-191.
[40]WANG J,JIANG Y,XU C,et al.ComboDroid:generating high-quality test inputs for Android apps via use case combinations[C]//Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering.2020:469-480.
[41]LIU C.A compatibility testing platform for android multimedia applications[J].Multimedia Tools and Applications,2019,78(4):4885-4904.
[42]LIU Y,YAN F,XIA M,et al.TimelyRep:Timing deterministic replay for Android web applications[J].Software Testing,Verification and Reliability,2020,30(4/5):e1745.
[43]FU J,WANG Y,ZHOU Y,et al.How resource utilization influences UI responsiveness of Android software[J].Information and Software Technology,2022,141:106728.
[44]PILGUN A,GADYATSKAYA O,ZHAUNIAROVICH Y,et al.Fine-grained Code Coverage Measurement in Automated Black-box Android Testing[J].ACM Transactions on Software Engineering and Methodology,2020,29(4):1-35.
[45]MU T,ZHANG H,WANG J,et al.CoLaFUZE:Coverage-Guided and Layout-Aware Fuzzing for Android Drivers[J].IEICE Transactions on Information and Systems,2021,104(11):1902-1912.
[46]KOROGLU Y,SEN A.Functional test generation from UI test scenarios using reinforcement learning for android applications[J].Software Testing,Verification and Reliability,2020,31(3):e1752.
[47]RAVINDRANATH L,NATH S,PADHYE J,et al.Automatic and scalable fault detection for mobile applications[C]//Proceedings of the 12th Annual International Conference on Mobile Systems,Applications,and Services.2014:190-203.
[48]MORAN K,LINARES-VÁSQUEZ M,BERNAL-CÁRDENAS C,et al.Crashscope:A practical tool for automated testing of android applications[C]//2017 IEEE/ACM 39th International Conference on Software Engineering Companion(ICSE-C).IEEE,2017:15-18.
[49]Google.Google firebase test lab robo test.(2021-4-28)[2021-9-26].https://firebase.google.com/docs/test-lab/robo-ux-test.
[50]ZAEEM R N,PRASAD M R,KHURSHID S.Automated generation of oracles for testing user-interaction features of mobile apps[C]//2014 IEEE Seventh International Conference on Software Testing,Verification and Validation.IEEE,2014:183-192.
[51]VAN DER MERWE H,VAN DER MERWE B,VISSER W.Execution and property specifications for jpf-android[J].ACM SIGSOFT Software Engineering Notes,2014,39(1):1-5.
[52]WHITE M,LINARES-VÁSQUEZ M,JOHNSON P,et al.Generating reproducible and replayable bug reports from android application crashes[C]//2015 IEEE 23rd International Conference on Program Comprehension.IEEE,2015:48-59.
[53]ADAMSEN C Q,MEZZETTI G,MØLLER A.Systematic execution of android test suites in adverse conditions[C]//Procee-dings of the 2015 International Symposium on Software Testing and Analysis.2015:83-93.
[54]YE H,CHENG S,ZHANG L,et al.Droidfuzzer:Fuzzing the android apps with intent-filter tag[C]//Proceedings of International Conference on Advances in Mobile Computing & Multimedia.2013:68-74.
[55]AMALFITANO D,FASOLINO A R,TRAMONTANA P,et al.A toolset for GUI testing of Android applications[C]//2012 28th IEEE International Conference on Software Maintenance(ICSM).IEEE,2012:650-653.
[56]KOROGLU Y,SEN A,MUSLU O,et al.QBE:Q Learning-based exploration of android applications[C]//2018 IEEE 11th International Conference on Software Testing,Verification and Validation(ICST).IEEE,2018:105-115.
[57]RUIZ X,CALVET L,FERRARONS J,et al.SmartMonkey:a web browser tool for solving combinatorial optimization problems in real time[C]//International Forum for Interdisciplinary Mathematics.Cham:Springer,2015:74-86.
[58]AMALFITANO D,FASOLINO A R,TRAMONTANA P,et al.Considering context events in event-based testing of mobile applications[C]//2013 IEEE Sixth International Conference on Software Testing,Verification and Validation Workshops.IEEE,2013:126-133.
[59]JABBARVAND R,SADEGHI A,BAGHERI H,et al.Energy-aware test-suite minimization for android apps[C]//Proceedings of the 25th International Symposium on Software Testing and Analysis.2016:425-436.
[60]MIRZAEI N,GARCIA J,BAGHERI H,et al.Reducing combinatorics in GUI testing of android applications[C]//2016 IEEE/ACM 38th International Conference on Software Engineering(ICSE).IEEE,2016:559-570.
[1] YAO Ye, ZHU Yi-an, QIAN Liang, JIA Yao, ZHANG Li-xiang, LIU Rui-liang. Android Malware Detection Method Based on Heterogeneous Model Fusion [J]. Computer Science, 2022, 49(6A): 508-515.
[2] WANG Yi, LI Zheng-hao, CHEN Xing. Recommendation of Android Application Services via User Scenarios [J]. Computer Science, 2022, 49(6A): 267-271.
[3] ZHANG Da-lin, ZHANG Zhe-wei, WANG Nan, LIU Ji-qiang. AutoUnit:Automatic Test Generation Based on Active Learning and Prediction Guidance [J]. Computer Science, 2022, 49(11): 39-48.
[4] WANG Wen-xuan, HU Jun, HU Jian-cheng, KANG Jie-xiang, WANG Hui, GAO Zhong-jie. Test Case Generation Method Oriented to Tabular Form Formal Requirement Model [J]. Computer Science, 2021, 48(5): 16-24.
[5] JI Shun-hui, ZHANG Peng-cheng. Test Case Generation Approach for Data Flow Based on Dominance Relations [J]. Computer Science, 2020, 47(9): 40-46.
[6] SUN Zhi-qiang, WAN Liang, DING Hong-wei. Android Malware Detection Method Based on Deep Autoencoder Network [J]. Computer Science, 2020, 47(4): 298-304.
[7] SUN Zhi-gang, WANG Guo-tao, JIANG Ai-ping, GAO Meng-meng, LIU Jin-gang. Monitoring System of Traffic Safety Based on Information Fusion Technology [J]. Computer Science, 2020, 47(11A): 642-650.
[8] ZHANG Na,TENG Sai-na,WU Biao,BAO Xiao-an. Test Case Generation Method Based on Particle Swarm Optimization Algorithm [J]. Computer Science, 2019, 46(7): 146-150.
[9] ZHANG Jing, LI Rui-xuan, TANG Jun-wei, HAN Hong-mu, GU Xi-wu. Collusion Behavior Detection Towards Android Third-party Libraries [J]. Computer Science, 2019, 46(5): 83-91.
[10] XIE Nian-nian, ZENG Fan-ping, ZHOU Ming-song, QIN Xiao-xia, LV Cheng-cheng, CHEN Zhao. Android Malware Detection with Multi-dimensional Sensitive Features [J]. Computer Science, 2019, 46(2): 95-101.
[11] ZHANG Zong-mei, GUI Sheng-lin, REN Fei. Android Malware Detection Based on N-gram [J]. Computer Science, 2019, 46(2): 145-151.
[12] HOU Yu-chen, WU Wei. Design and Implementation of Crowdsourcing System for Still Image Activity Annotation [J]. Computer Science, 2019, 46(11A): 580-583.
[13] YE Jia, GE Hong-jun, CAO Chun, ZHU Jin, ZHANG Ying. Rule-driven DFS Testing Technology for Android Application [J]. Computer Science, 2018, 45(9): 99-103.
[14] ZENG Xing, SUN Bei ,LUO Wu-sheng, LIU Tao-cheng ,LU Qin. Sitting Posture Detection System Based on Depth Sensor [J]. Computer Science, 2018, 45(7): 237-242.
[15] HAO Jun-sheng,LI Bing-feng,CHEN Xi,GAO Wen-juan. Design and Implementation of Network Subscription System Based on Android Platform [J]. Computer Science, 2018, 45(6A): 591-594.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.