Computer Science ›› 2020, Vol. 47 ›› Issue (10): 19-25.doi: 10.11896/jsjkx.191200164

Special Issue: Mobile Crowd Sensing and Computing

• Mobile Crowd Sensing and Computing • Previous Articles     Next Articles

Crowdsourcing Collaboration Process Recovery Method

WANG Kuo, WANG Zhong-jie   

  1. Research Center on Intelligent Computing for Enterprises & Services,Harbin Institute of Technology,Harbin 150001,China
  • Received:2019-12-27 Revised:2020-05-08 Online:2020-10-15 Published:2020-10-16
  • About author:WANG Kuo,born in 1990,Ph.D,is a member of China Computer Federation.His main research interests include social software engineering and crowdsourcing,software warehouse mining and service recommendation.
    WANG Zhong-jie,born in 1978,Ph.D,professor,Ph.D supervisor,is a member of China Computer Federation.His main research interests include service computing,service engineering Internet services and cloud computing,social networking services,social software engineering and crowdsourcing,software warehouse mining.
  • Supported by:
    National Natural Science Foundation of China (61772155)

Abstract: Crowdsourcing is a distributed problem solving mechanism using group intelligence.It is widely used in Internet application scenarios based on artificial intelligence activities,using large groups of users on the Internet to work together to solve complex problems that cannot be solved by one person.Taking the development and maintenance process of open source software as an example,participants jointly complete key tasks such as code writing and bug repair through specific platforms.Different from traditional business process management (BPM),collaborative processes in the crowdsourcing scenario face challenges such as undetermined process structure,and unpredictable timing and results,which bring great difficulties to the efficiency and quality control of crowdsourcing collaboration.In this paper,aiming at a series of collaborative behaviors produced by multiple participants according to the time sequence (embodied as text in the form of natural language),natural language processing and artificial intelligence are used to propose a restoration algorithm for the process of crowdsourcing collaboration.An empirical study is carried out on the case of personnel cooperation in the process of bug repair in the field of open source software development.The collaborative process of recovery is visualized,and the accuracy of process recovery algorithm is quantitatively compared.This research can help coordinators of crowdsourcing process (such as open source project managers) to understand the problem solving process more intuitively,and find the typical patterns of collaboration,so as to make an accurate prediction for the nature of the collaborative process of the new crowdsourcing task.

Key words: Bug fix, Collaborative process, Crowdsourcing, Open source software development, Process restore

CLC Number: 

  • TP315
[1]HOWE J.The rise of crowdsourcing [J].Wired Magazine,2006,14(6):1-4.
[2]ZHAO Y X,ZHU Q H.Evaluation on crowdsourcing research: current status and future direction [J].Information Systems Frontiers,2012,11(1):1-18.
[3]KOCH G,FULLER J,BRUNSWICKER S.Online crowdsourcing in the public sector: how to design open government platforms [C]//Proceedings of The 4th International Conference on Online Communities and Social Computing.Orlando,USA,2011: 203-212.
[4]KHEER J,BOSTOCK M.Crowdsourcing graphical perception: Using mechanical turk to assess visualization design[C]//Proceedings of the 28th International Conference on Human Factors in Computing Systems.Atlanta,USA,2010:203-212.
[5]PARAMESWARAN A G,GARCIA-MOLINA H,PARK H,et al.CrowdScreen:Algorithms for filtering data with humans[C]//Proceedings of the ACM SIGMOD International Conference on Management of Data.Scottsdale,USA,2012:361-372.
[6]VENETIS P,GARCIA-MOLINA H,HUANG K,et al.Maxalgorithms in crowdsourcing environments[C]//Proceedings of the 21st World Wide Web Conference.Lyon,France,2012:989-998.
[7]LAWS F,SCHEIBLE C,SCHUTZE H.Active learning withamazon mechanical turk[C]//Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing.Edinburgh,UK,2011:1546-1556.
[8]LIU X,LU M,OOI B,et al.CDAS:A crowdsourcing data analytics system[J].Proceedings of the VLDB Endowment,2012,5(10):1040-1051.
[9]KAZAI G.In search of quality in crowdsourcing for search engine evaluation[C]//Proceedings of the 33rd European Conference on IR Research.Dublin,Ireland,2011:165-176.
[10]CHAWLA S,HARTLINE J D,SIVAN B.Optimal Crowdsourcing contest[C]//Proceedings of the 23rd Annual ACM-SIAM Symposium on Discrete Algorithms.Kyoto,Japan,2012:856-868.
[11]KITTUR A,NICKERSON J V,BERNSTEIN M S,et al.The future of crowd work[C]//Proceedings of the 2013 ACM Conference on Computer Supported Cooperative Work.San Antonio,USA,2013:1301-1318.
[12]DOAN A,RAMAKRICHNAN R,HALEVY A Y.Crowdsourcing systems on the world-wide web [J].Communications of the ACM,2011,54(4):86-96.
[13]ANJALI G,NEETU S.An empirical study of non-reproducible bugs[J].International Journal of System Assurance Engineering and Management,2019,10(5):1186-1220.
[14]ZHAO Y,HE T K,CHEN Z Y.A Unified Framework for Bug Report Assignment[J].International Journal of Software Engineering and Knowledge Engineering,2019,29(4):607-628.
[15]HUI L,GAO G F,CHEN R,et al.The Influence Ranking forTesters in Bug Tracking Systems[J].International Journal of Software Engineering and Knowledge Engineering,2019,29(1):93-113.
[16]PRESSMAN R S,INCE D.Software engineering:a practitioner’sapproach[M].New York:McGraw-hill,1992.
[17]XIE T,PEI J,HASSAN A E.Mining software engineering data [C]//Proceedings of the 29th International Conference on Software Engineering.Minnesota.USA,2007:172-173.
[18]ZHOU J,ZHANG H Y,LO D.where should the bugs be fixed?more accurate information retrieval-based bug localization based on bug reports [C]//the 34th International Conference on Software Engineering.Switzerland,2012:14-24.
[1] FU Yan-ming, ZHU Jie-fu, JIANG Kan, HUANG Bao-hua, MENG Qing-wen, ZHOU Xing. Incentive Mechanism Based on Multi-constrained Worker Selection in Mobile Crowdsourcing [J]. Computer Science, 2022, 49(9): 275-282.
[2] CHEN Dan-hong, PENG Zhang-lin, WAN De-quan, YANG Shan-lin. Identification and Segmentation of User Value in Crowdsourcing Platforms:An Improved RFMModel [J]. Computer Science, 2022, 49(4): 37-42.
[3] SHEN Biao, SHEN Li-wei, LI Yi. Dynamic Task Scheduling Method for Space Crowdsourcing [J]. Computer Science, 2022, 49(2): 231-240.
[4] LAI Xiang-wei, ZHENG Wan-bo, WU Yan-qing, XIA Yun-ni, RAN Qi-hua, DONG Yin-huan. Task Collaborative Process Network Model and Time Analysis of Mine Accident Emergency Rescue Digital Plan [J]. Computer Science, 2021, 48(6A): 596-602.
[5] ZHANG Shao-jie, LU Xu-dong, GUO Wei, WANG Shi-peng, HE Wei. Prevention of Dishonest Behavior in Supply-Demand Matching [J]. Computer Science, 2021, 48(4): 303-308.
[6] ZHAO Yang, NI Zhi-wei, ZHU Xu-hui, LIU Hao, RAN Jia-min. Multi-worker and Multi-task Path Planning Based on Improved Lion Evolutionary Algorithm forSpatial Crowdsourcing Platform [J]. Computer Science, 2021, 48(11A): 30-38.
[7] LI Yu, DUAN Hong-yue, YIN Yu-yu, GAO Hong-hao. Survey of Crowdsourcing Applications in Blockchain Systems [J]. Computer Science, 2021, 48(11): 12-27.
[8] YU Dun-hui, CHENG Tao, YUAN Xu. Software Crowdsourcing Task Recommendation Algorithm Based on Learning to Rank [J]. Computer Science, 2020, 47(12): 106-113.
[9] ZHANG Guang-yuan, WANG Ning. Truth Inference Based on Confidence Interval of Small Samples in Crowdsourcing [J]. Computer Science, 2020, 47(10): 26-31.
[10] HU Ying, WANG Ying-jie, TONG Xiang-rong. Task Recommendation Model Based on Crowd Worker’s Movement Trajectory [J]. Computer Science, 2020, 47(10): 32-40.
[11] LV Jia-gao,LIANG Kui-yang,CAI Wei. Frontier Scientific Keyword Extraction Based on Bibliometric and Crowdsourcing [J]. Computer Science, 2019, 46(3): 275-282.
[12] HOU Yu-chen, WU Wei. Design and Implementation of Crowdsourcing System for Still Image Activity Annotation [J]. Computer Science, 2019, 46(11A): 580-583.
[13] ZHOU A-peng, QIN Xi-zhong, JIA Zhen-hong and NIKOLA Kasabov. Crowdsourcing-based Indoor Localization via Embedded Manifold Matching [J]. Computer Science, 2017, 44(8): 64-70.
[14] ZHANG Kai, SUN Xiao-bing, PENG Xin and ZHAO Wen-yun. Empirical Study of Reopened Security Bugs on Mozilla [J]. Computer Science, 2017, 44(11): 41-49.
[15] CHEN Hai-yan and XU Zheng. Crowdsourcing Based Description of Urban Emergency Events [J]. Computer Science, 2016, 43(5): 209-213.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!