Computer Science ›› 2025, Vol. 52 ›› Issue (11): 330-338.doi: 10.11896/jsjkx.240900150

• Computer Software • Previous Articles     Next Articles

Adaptive Android Program Test Method Based on Thompson Sampling

ZHAO Yingnan, LENG Chongyang, HAN Qilong, YU Cheng   

  1. College of Computer Science and Technology,Harbin Engineering University,Harbin 150001,China
  • Received:2024-09-25 Revised:2024-12-09 Online:2025-11-15 Published:2025-11-06
  • About author:ZHAO Yingnan,born in 1992,Ph.D,associate professor,master supervisor,is a member of CCF(No.P2652M).His main research interests include reinforcement learning and intelligent software engineering.
    HAN Qilong,born in 1974,Ph.D,professor,Ph.D supervisor.His main research interests include industrial data intelligence,big data analysis and mi-ning,data security and privacy computing,knowledge graph theory and applications,and industrial Internet.
  • Supported by:
    Key Research and Development Program of Heilongjiang Province(KY10600230082).

Abstract: Recently,the research of Android graphical interface test method has attracted wide attention.At present,most test methods are developed based on reinforcement learning.However,the existing methods can explore the application by selecting parameters according to experience,and can not to change the parameter settings according to the interface changes adaptively.This paper proposes an adaptive Android testing method based on Thompson sampling,which combines Thompson sampling with the Q-learning algorithm,enabling the agent to adaptively determine the next exploration action based on the current state of the interface controls being examined so that to balance exploitation and exploration more effectively for superior testing outcomes.Firstly,events of the jump at the interface during exploration are modeled for the Beta probability distribution,then a probability distribution matrix is obtained,which is weighted-averaged with the Q-table.It can take into account both the exploration value and the exploition value of the events.At the same time,the probability distribution of operational events under the current interface is sampled,and the maximum sampling value is the exploration probability value,combined with the weighted matrix,it can guide the test more comprehensively,so as to realize the adaptive exploration of the Android application interface.Experiments executed on 13 Android applications confirm the efficacy of the proposed method through experimentalcomparison and analysis with conventional reinforcement learning testing tools.

Key words: Android GUI testing, Reinforcement learning, Q-learning, Thompson sampling

CLC Number: 

  • TP311
[1]LI Y,YANG Z,GUO Y,et al.Droidbot:a lightweight ui-guided test input generator for android[C]//2017 IEEE/ACM 39th International Conference on Software Engineering Companion.IEEE,2017:23-26.
[2]DONG Z,BÖHME M,COJOCARU L,et al.Time-travel testing of android apps[C]//Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering.2020:481-492.
[3]ADAMO D,KHAN M K,KOPPULA S,et al.Reinforcement learning for android gui testing[C]//Proceedings of the 9th ACM SIGSOFT International Workshop on Automating TEST Case Design,Selection,and Evaluation.2018:2-8.
[4]KOROGLU Y,SEN A,MUSLU O,et al.Qbe:Qlearning-based exploration of android applications[C]//2018 IEEE 11th International Conference on Software Testing,Verification and Validation.IEEE,2018:105-115.
[5]VUONG T A T,TAKADA S.A reinforcement learning based approach to automated testing of android applications[C]//Proceedings of the 9th ACM SIGSOFT International Workshop on Automating TEST Case Design,Selection,and Evaluation.2018:31-37.
[6]MAOK,HARMAN M,JIA Y.Sapienz:Multi-objective automated testing for android applications[C]//Proceedings of the 25th International Symposium on Software Testing and Analysis.New York:ACM,2016:94-105.
[7]SU T,MENG G,CHEN Y,et al.Guided,stochastic model-basedGUI testing of Android apps[C]//Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering.New York:ACM,2017:245-256.
[8]LI Y,YANG Z,GUO Y,et al.DroidBot:A Lightweight UI-Guided Test Input Generator for Android[C]//2017 IEEE/ACM 39th International Conference on Software Engineering Companion.IEEE,2017:23-26.
[9]YASIN H N,HAMID S H A,YUSOF R J R.DroidbotX:Test Case Generation Tool for Android Applications Using Q-Lear-ning[J].Symmetry,2021,13(2):310.
[10]HANH,HAN W,RUIQI D,et al.Enhancing GUI Exploration Coverage of Android Apps with Deep Link-Integrated Monkey[J],ACM Transactions on Software Engineering and Methodo-logy,2024,33(6):163.
[11]LI Y,YANG Z,GUO Y,et al.Humanoid:A Deep Learning-Based Approach to Automated Black-box Android App Testing[C]//2019 34th IEEE/ACM International Conference on Automated Software Engineering (ASE).2019:1070-1073.
[12]PAN M,HUANG A,WANG G,et al.Reinforcement learningbased curiosity-driven testing of Android applications[C]//Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis.New York:ACM,2020:153-164.
[13]MINDOM P S N,NIKANJAM A,KHOMH F.A comparison of reinforcement learning frameworks for software testing tasks[J].Empirical Software Engineering,2023,28(5):article number 111.
[14]FUJIMOTO S,VAN HOOF H,MEGER D.Addressing Function Approximation Error in Actor-Critic Methods[J].arXiv:1802.09477,2018.
[15]GRUSLYS A,AZAR M G,BELLEMARE M G,et al.The Reactor:A Sample-Efficient Actor-Critic Architecture[J].arXiv:1704.04651,2017.
[16]GU S,LILLICRAP T,GHAHRAMANI Z,et al.Q-Prop:Sample-Efficient Policy Gradient with An Off-Policy Critic[J].ar-Xiv:1611.02247,2016.
[17]LILLICRAP T P,HUNT J J,PRITZEL A,et al.Continuouscontrol with deep reinforcement learning[J].arXiv:1509.02971,2015.
[18]HAARNOJA T,ZHOU A,ABBEEL P,et al.Soft Actor-Critic:Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor[J].arXiv:1801.01290,2018.
[19]PAN M,HUANG A,WANG G,et al.Reinforcement learning based curiosity-driven testing of Android applications[C]//Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis.2020:153-164.
[20] MARIANI L,PEZZE M,RIGANELLI O,et al.AutoBlackTest:Automatic BlackBox Testing of Interactive Applications[C]//2012 IEEE Fifth International Conference on Software Testing,Verification and Validation.IEEE,2012:81-90.
[21]WATKINS C J C H,DAYAN P.Technical Note:Q-Learning[J].Machine Learning,1992,8(3/4):279-292.
[22]THOMPSON W R.On the likelihood that one unknown proba-bility exceeds another in view of the evidence of two samples[J].Biometrika,1933,25(3/4):285-294.
[23]FUJIMOTO S,VAN HOOF H,MEGER D.Addressing Func-tion Approximation Error in Actor-Critic Methods[J].arXiv:1802.09477,2018.
[24]MNIHV,KAVUKCUOGLU K,SILVER D,et al.Playing Atari with Deep Reinforcement Learning[J].arXiv:1312.5602,2013.
[25]CHOUDHARY S R,GORLA A,ORSO A.Automated Test Input Generation for Android:Are We There Yet? [C]//2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE).IEEE,2015:429-440.
[26]JaCoCo.Java code coverage library[DB/OL].https://www.jacoco.org/jacoco/.
[27]HUANG C Y,CHIU C H,LIN C H,et al.Code Coverage Meas-urement for Android Dynamic Analysis Tools[C]//2015 IEEE International Conference on Mobile Services.IEEE,2015:209-216.
[28]YANGS,HUANG S,HUI Z.Theoretical Analysis and Empirical Evaluation of Coverage Indictors for Closed Source APP Testing[J].IEEE Access,2019,7:162323-162332.
[29]ADAMO D,KHAN M K,KOPPULA S,et al.Reinforcement learning for Android GUI testing[C]//Proceedings of the 9th ACM SIGSOFT International Workshop on Automating TEST Case Design,Selection,and Evaluation.New York:ACM,2018:2-8.
[30]PILGUN A,GADYATSKAYA O,ZHAUNIAROVICH Y,et al.Fine-grained Code Coverage Measurement in Automated Black-box Android Testing[J].ACM Transactions on Software Engineering and Methodology,2020,29(4):1-35.
[31]SUT,MENG G,CHEN Y,et al.Guided,stochastic model-based GUI testing of Android apps[C]//Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering.New York:ACM,2017:245-256.
[32]KOROGLUY,SEN A,MUSLU O,et al.QBE:QLearning-Based Exploration of Android Applications[C]//2018 IEEE 11th International Conference on Software Testing,Verification and Validation.IEEE,2018.
[1] ZHU Shihao, PENG Kexing, MA Tinghuai. Graph Attention-based Grouped Multi-agent Reinforcement Learning Method [J]. Computer Science, 2025, 52(9): 330-336.
[2] CHEN Jintao, LIN Bing, LIN Song, CHEN Jing, CHEN Xing. Dynamic Pricing and Energy Scheduling Strategy for Photovoltaic Storage Charging Stations Based on Multi-agent Deep Reinforcement Learning [J]. Computer Science, 2025, 52(9): 337-345.
[3] ZHANG Yongliang, LI Ziwen, XU Jiahao, JIANG Yuchen, CUI Ying. Congestion-aware and Cached Communication for Multi-agent Pathfinding [J]. Computer Science, 2025, 52(8): 317-325.
[4] HUO Dan, YU Fuping, SHEN Di, HAN Xueyan. Research on Multi-machine Conflict Resolution Based on Deep Reinforcement Learning [J]. Computer Science, 2025, 52(7): 271-278.
[5] PIAO Mingjie, ZHANG Dongdong, LU Hu, LI Rupeng, GE Xiaoli. Study on Multi-agent Supply Chain Inventory Management Method Based on Improved Transformer [J]. Computer Science, 2025, 52(6A): 240500054-10.
[6] XU Dan, WANG Jiangtao. Design of Autonomous Decision for Trajectory Optimization of Intelligent Morphing Aircraft [J]. Computer Science, 2025, 52(6A): 240600068-7.
[7] WU Zongming, CAO Jijun, TANG Qiang. Online Parallel SDN Routing Optimization Algorithm Based on Deep Reinforcement Learning [J]. Computer Science, 2025, 52(6A): 240900018-9.
[8] ZHAO Chanchan, YANG Xingchen, SHI Bao, LYU Fei, LIU Libin. Optimization Strategy of Task Offloading Based on Meta Reinforcement Learning [J]. Computer Science, 2025, 52(6A): 240800050-8.
[9] WANG Chenyuan, ZHANG Yanmei, YUAN Guan. Class Integration Test Order Generation Approach Fused with Deep Reinforcement Learning andGraph Convolutional Neural Network [J]. Computer Science, 2025, 52(6): 58-65.
[10] ZHAO Xuejian, YE Hao, LI Hao, SUN Zhixin. Multi-AGV Path Planning Algorithm Based on Improved DDPG [J]. Computer Science, 2025, 52(6): 306-315.
[11] LI Yuanbo, HU Hongchao, YANG Xiaohan, GUO Wei, LIU Wenyan. Intrusion Tolerance Scheduling Algorithm for Microservice Workflow Based on Deep Reinforcement Learning [J]. Computer Science, 2025, 52(5): 375-383.
[12] HAN Lin, WANG Yifan, LI Jianan, GAO Wei. Automatic Scheduling Search Optimization Method Based on TVM [J]. Computer Science, 2025, 52(3): 268-276.
[13] ZHENG Longhai, XIAO Bohuai, YAO Zewei, CHEN Xing, MO Yuchang. Graph Reinforcement Learning Based Multi-edge Cooperative Load Balancing Method [J]. Computer Science, 2025, 52(3): 338-348.
[14] DU Likuan, LIU Chen, WANG Junlu, SONG Baoyan. Self-learning Star Chain Space Adaptive Allocation Method [J]. Computer Science, 2025, 52(3): 359-365.
[15] HUO Xingpeng, SHA Letian, LIU Jianwen, WU Shang, SU Ziyue. Windows Domain Penetration Testing Attack Path Generation Based on Deep Reinforcement Learning [J]. Computer Science, 2025, 52(3): 400-406.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!