Computer Science ›› 2020, Vol. 47 ›› Issue (3): 54-60.doi: 10.11896/jsjkx.191100085

• Intelligent Software Engineering • Previous Articles     Next Articles

Web Service Crowdtesting Task Assignment Approach Based onReinforcement Learning

TANG Wen-jun,ZHANG Jia-li,CHEN Rong,GUO Shi-kai   

  1. (College of Information Science and Technology, Dalian Maritime University, Dalian, Liaoning 116026, China)
  • Received:2019-09-10 Online:2020-03-15 Published:2020-03-30
  • About author:TANG Wen-jun,born in 1994,Ph.D.Her research interests include crowdsourcing workflows,crowdsourcing task assignment and web service testing. CHEN Rong,born in 1969,Ph.D,professor,is a member of the IEEE and a member of the ACM.His research interests inculde software diagnosis,collective intelligence,activity recognition,Internet and mobile computing.
  • Supported by:
    This work was supported by the National Natural Science Foundation of China (61672122, 61902050, 61602077), Fundamental Research Funds for the Central Universities of Ministry of Education of China (3132019355) and CERNET Innovation Project (NGII20190627).

Abstract: How to assign tasks to appropriate workers to get better testing results at a lower cost is an important problem.This paper modeled the CWS testing task assignment as a Markov decision process-based problem,and used Deep Q Network to learn and perform real-time online testing task assignment.The proposed approach based on reinforcement learning is named WTA-C.In addition,this paper calculated the probability of the testing worker completing the task within the duration through statistical conditional probability in accordance with the time of the worker’s historical execution of tasks,and used it as the workers’ reputation value to reflect their quality.The worker’s reputation is updated after each assignment.The experimental results show that WTA-C is superior to other real-time assignment methods based on heuristic strategies in controlling the “quality-cost” trade-off of testing tasks and ensuring worker quality,and its assignment effect is more than 18% higher than that of each heuristic strategy,which demonstrates that WTA-C can better adapt to the structure of the CWS and the characteristics of Crowdsourcing environment.

Key words: Crowdtesting, Composite Web service testing, Web service testing, Reinforcement learning

CLC Number: 

  • TP311.5
[1]HUSSAIN S,WANG Z,TOURE I,et al.Web Service Testing Tools:A Comparative Study [J],CoRR,2013,10(1):641-647.
[2]LIU X,HSIEH Y,CHEN R,et al.Distributed testing system for web service based on crowdsourcing [J].Complexity,2018:2170585:1-2170585:15.
[3]LIU D,BIAS R,LEASE M,et al.Crowdsourcing for Usability Testing[C]∥Association for Information Science and Technology.2012:1-10.
[4]RAHMAN H,ROY S,THIRUMURUGANATHAN S,et al. Task Assignment Optimization in Collaborative Crowdsourcing[C]∥IEEE International Conference on Data Mining.2015:949-954.
[5]KOMAROV S,REINECKE K,GAJOS K.Crowdsourcing per- formance evaluations of user interfaces[C]∥Computer Human Interaction.2013:207-216.
[6]IPEIROTIS P.Analyzing the Amazon Mechanical Turk Marketplace[J].CM Crossroads Student Magazine,2010,17(2):16-21.
[7]TIAN X,LI H,FENG L.Web Service Reliability Test Method Based on Log Analysis[C]∥Software Quality.Reliability and Security,2017:195-199.
[8]DU Z,MIAO H.Research review on web service composition testing[C]∥The International Workshop on Structured Object-oriented Formal Language and Method.2018:39-51.
[9]GUO S,LIU Y,CHEN R,et al.Improved SMOTE Algorithm to Deal with Imbalanced Activity Classes in Smart Homes[J].Neural Processing Letters,2019,50(2):1503-1526.
[10]GARDLO B.Quality of Experience Evaluation Methodology via Crowdsourcing[D].Žilina:University of Žilina,2012.
[11]GARDLO B,EGGER S,SEUFERT M,et al.Crowdsourcing 2.0:Enhancing execution speed and reliability of web-based QoE testing[C]∥IEEE International Conference on Communications.2014:1070-1075.
[12]BLANCO R,HALPIN H,HERZIG D,et al.Repeatable and reliable search system evaluation using crowdsourcing [C]∥International Conference on Research and Development in Information Retrieval.2011:923-932.
[13]SHERIEF N,JIANG N,HOSSEINI M,et al.Crowdsourcing software evaluation[C]∥Evaluation & Assessment in Software Engineering.2014:19:1-19:4.
[14]CHEN F,KIM S.Crowd debugging,in Joint Meeting on Foundations of Software Engineering [C]∥ESEC/SIGSOFT FSE.2015:320-332.
[15]PETRILLO F,SOH Z,KHOMH F,et al.Towards Understan- ding Interactive Debugging [C]∥IEEE International Conference on Software Quality.2016:152-163.
[16]PETRILLO F,LACERDA G,et al.Visualizing interactive and shared debugging sessions ∥IEEE Working Conference on Software Visualization.2015:140-144.
[17]GUAIANI F,MUCCINI H.Crowd and Laboratory Testing,Can They Co-exist? An Exploratory Study[C]∥International Conference on Software Engineering.2015:32-37.
[18]LIU Y,ZHANG T,LI K,et al.Evaluation model of mobile application crowdsourcing testers [J].Journal of Computer Applications,2017,37(12):3569-3573.
[19]ZHANG Z Q,PANG J S,XIE X Q,et al.Research on Crowdsourcing Quality Control Strategies and Evaluation Algorithm [J].Chinese Journal of Computers,2013,36(8):1636-1649.
[20]CHENG J,GE L Q,ZHANG T,et al.Research on factors affecting quality of mobile application crowdsourced testing [J].Journal of Computer Applications,2018,38(9):2626-2630.
[21]FENG J H,LI G L,GENG J H.A Survey on Crowdsourcing [J].Chinese Journal of Computers,2015,38(9):1713-1726.
[22]TUNG Y,TSENGE S.A novel approach to collaborative testing in a crowdsourcing environment [J].Journal of Systems and Software,2013,86(8):2143-2153.
[23]BOUTSIS I,KALOGERAKI V.On Task Assignment for Real-Time Reliable Crowdsourcing [C]∥IEEE International Conference on Distributed Computing Systems.2014:1-10.
[24]BOUTSIS I,KALOGERAKI V.Crowdsourcing under Real- Time Constraints [C]∥International Parallel (and Distributed) Processing Symposium.2013:753-764.
[25]HAN Y,SHEN Z,FAUVEL S,et al.Efficient scheduling in crowdsourcing based on workers’ mood[C]∥IEEE Internatio-nal Conference on Agents.2017:121-126.
[26]ROY S,LYKOURENTZOU I,THIRUMURUGANATHAN S,et al.Task assignment optimization in knowledge-intensive crowdsourcing [J].The VLDB Journal,2014,24(4):467-491.
[27]LI Y J,GUO J F,GOU X M.Software Task Allocation Method in Crowdsourcing [J].Computer Systems & Applications,2015,24(2):1-6.
[28]YAN M,SUN H,LIU X.iTest:testing software with mobile crowdsourcing[C]∥CrowdSoft@SIGSOFT FSE.2014:19-24.
[29]YAN M,SUN H,LIU X.Efficient Testing of Web Services with Mobile Crowdsourcing[C]∥Internetware.2015:157-165.
[30]CLAUSET A,SHALIZI C,NEWMAN M.Power-law distributions in empirical data[J].SIAM Review,2009,51(4):661-703.
[31]RUMMERY G,NIRANJAN M.On-line Q-learning using connectionist systems [D].University of Cambridge,1994.
[32]TIAN Y L,HUI L.Accelerated modify approach for initial state error iterative learning control in sense of Lebesgue-p norm[J].Control and Decision,2016,31(3):429-434.
[33]BARABÁSI A,ALBERT R.Emergence of scaling in random networks[J].Science,1999,286(5439):509-512.
[34]ZHENG Z,LYU M.Collaborative reliability prediction of service-oriented systems[C]∥ACM/IEEE International Conference on Software Engineering.2010:35-44.
[1] ANG Wei-yi,BAI Chen-jia,CAI Chao,ZHAO Ying-nan,LIU Peng. Survey on Sparse Reward in Deep Reinforcement Learning [J]. Computer Science, 2020, 47(3): 182-191.
[2] SUN Hao,CHEN Chun-lin,LIU Qiong,ZHAO Jia-bao. Traffic Signal Control Method Based on Deep Reinforcement Learning [J]. Computer Science, 2020, 47(2): 169-174.
[3] LI Li,ZHENG Jia-li,WANG Zhe,YUAN Yuan,SHI Jing. RFID Indoor Positioning Algorithm Based on Asynchronous Advantage Actor-Critic [J]. Computer Science, 2020, 47(2): 233-238.
[4] LU Hai-feng, GU Chun-hua, LUO Fei, DING Wei-chao, YUAN Ye, REN Qiang. Virtual Machine Placement Strategy with Energy Consumption Optimization under Reinforcement Learning [J]. Computer Science, 2019, 46(9): 291-297.
[5] DU Wei, DING Shi-fei. Overview on Multi-agent Reinforcement Learning [J]. Computer Science, 2019, 46(8): 1-8.
[6] ZHANG Hao-yu, XIONG Kai. Improved Deep Deterministic Policy Gradient Algorithm and Its Application in Control [J]. Computer Science, 2019, 46(6A): 555-557.
[7] LI Jian-guo, ZHAO Hai-tao, SUN Shao-yuan. KL-divergence-based Policy Optimization [J]. Computer Science, 2019, 46(6): 212-217.
[8] LI Jie, LING Xing-hong, FU Yu-chen, LIU Quan. Asynchronous Advantage Actor-Critic Algorithm with Visual Attention Mechanism [J]. Computer Science, 2019, 46(5): 169-174.
[9] JING Jie, CHEN Tan, DU Wen-li, LIU Zhi-kang, YIN Hao. Spatio-temporal Features Extraction of Traffic Based on Deep Neural Network [J]. Computer Science, 2019, 46(11A): 1-4.
[10] LAI Jian-hui. Traffic Signal Control Based on Double Deep Q-learning Network with Dueling Architecture [J]. Computer Science, 2019, 46(11A): 117-121.
[11] XU Ji-ning, ZENG Jie. Dynamic Target Following Based on Reinforcement Learning of Robot-car [J]. Computer Science, 2019, 46(11A): 94-97.
[12] CHEN Jian-ping, ZOU Feng, LIU Quan, WU Hong-jie, HU Fu-yuan, FU Qi-ming. Reinforcement Learning Algorithm Based on Generative Adversarial Networks [J]. Computer Science, 2019, 46(10): 265-272.
[13] ZHAO Xing-yu, DING Shi-fei. Research on Deep Reinforcement Learning [J]. Computer Science, 2018, 45(7): 1-6.
[14] LI Min-shuo, YAO Ming-hai. Q-learning with Feature-based Approximation for Traffic Light Control [J]. Computer Science, 2018, 45(11A): 143-145,159.
[15] ZHANG Jing-min and DONG Hong-bin. Optimized Negotiation Model Based on Reinforcement Learning of Medium Agent [J]. Computer Science, 2017, 44(1): 53-59.
Full text



[1] LI Jian-hong, WU Ya-rong and LV Ju-jian. Online Single Image Super-resolution Algorithm Based on Group Sparse Representation[J]. Computer Science, 2018, 45(4): 312 -318 .
[2] SHI Jing-qi, YANG Geng, SUN Yan-jun, BAI Shuang-jie and MIN Zhao-e. Efficient Parallel Algorithm of Fully Homomorphic Encryption Supporting Operation of Floating-point Number[J]. Computer Science, 2018, 45(5): 116 -122, 130 .
[3] ZENG Xing, SUN Bei ,LUO Wu-sheng, LIU Tao-cheng ,LU Qin. Sitting Posture Detection System Based on Depth Sensor[J]. Computer Science, 2018, 45(7): 237 -242 .
[4] HAN Xiu-ping, WANG Zhi, PEI Dan. Study on Wi-Fi Fingerprint Anonymization for Users in Wireless Networks[J]. Computer Science, 2018, 45(8): 7 -12 .
[5] LI Shan and RAO Wen-bi. Video-based Detection of Human Motion Area in Mine[J]. Computer Science, 2018, 45(4): 291 -295 .
[6] RAO Yuan, LU Shu-min. KAAS-based Service Mechanism of Technology Resources for Collaborative Innovation[J]. Computer Science, 2018, 45(7): 286 -292 .
[7] SHI Wei-wen, WANG Xue-qi, FAN Kai-yin, WANG Ming-jun. Measuring Point Selection Method of Board-level Circuit Based on Multi-signal Model and Genetic Algorithm[J]. Computer Science, 2018, 45(8): 295 -299 .
[8] LI Fang-wei HUANG Xu ZHANG Hai-bo LIU Kai-jian HE Xiao-fan. Cluster-based Radio Resource Allocation Mechanism in D2D Networks[J]. Computer Science, 2018, 45(9): 123 -128, 165 .
[9] LIU Qin. Study on Data Quality Based on Constraint in Computer Forensics[J]. Computer Science, 2018, 45(4): 169 -172 .
[10] LIU Yu-cheng, Richard·DING, ZHANG Ying-chao. Research on Pan-real-time Problem of Medical Detection Based on BPNNs Recognition Algorithm[J]. Computer Science, 2018, 45(6): 301 -307 .