Computer Science ›› 2023, Vol. 50 ›› Issue (10): 282-290.doi: 10.11896/jsjkx.221000133

• Computer Network • Previous Articles     Next Articles

Bidirectional Quality Control Strategies Based on CIDA and PI-cosine in Crowdsourcing

LIU Qingju, PAN Qingxian, TONG Xiangrong, YU Song, PAN Yanan   

  1. School of Computer and Control Engineering,Yantai University,Yantai,Shandong 264005,China
  • Received:2022-10-17 Revised:2023-03-21 Online:2023-10-10 Published:2023-10-10
  • About author:LIU Qingju,born in 1997,postgra-duate,is a member of China Computer Federation.Her main research interest is mobile crowdsourcing.PAN Qingxian,born in 1979,Ph.D candidate,associate professor,is a member of China Computer Federation.His main research interests include artificial intelligence and machine learning.
  • Supported by:
    National Natural Science Foundation of China(60903098,61502140,61572418,61472095,62072392),Natural Science Foundation of Heilongjiang,China(LH2020F023) and KeyResearch Project of Undergraduate Teaching Reform in Shandong Province(Z2022327).

Abstract: With the popularity of mobile smart terminals,crowdsourcing to collect large-scale perceptual data becomes easier and easier.The selfishness of crowdworkers makes them want to get the most pay with the least effort,and even collude with each other and submit crowdsourced data arbitrarily,resulting in poor quality of crowdsourced task completion.This paper proposes a jury-based quality control strategy,a mechanism that solves the data validation problem.To address the behaviors that degrade the quality of crowdsourcing,this paper uses the proposed community influence detection algorithm(CIDA) to detect conspiracy leaders and their organizations after determining the presence of spam employees and conspiracy organizations,and finally uses an improved similarity detection algorithm(PI-Cosine) to screen out for spam employees.These two aspects are used to improve the quality of crowdsourcing data.Experiments show that the proposed method improves the accuracy of 12.3% over Cosine similarity detection algorithm in accuracy and F1-score measures.

Key words: Crowdsourcing, Quality control, CIDA algorithm, PI-Cosine similarity detection, Spam

CLC Number: 

  • TP391
[1]HOWE J.The rise of crowdsourcing[J].Wired Magazine,2006,14(6):1-4.
[2]KOROVINA O,BAEZ M,CASATI F.Reliability of crowd-sourcing as a method for collecting emotions labels on pictures[J].BMC Research Notes,2019,12(1):715-715.
[3]ZHANG C,ZHU L,XU C,et al.A privacy-preserving trafficmonitoring scheme via vehicular crowdsourcing[J].Sensors(Basel),2019,19(6):1274.
[4]AHMED M,KARAGIORGOU S,PFOSER D,et al.A comparison and evaluation of map construction algorithms using vehicle tracking data[J].GeoInformatica,2015,19(3):601-632.
[5]CIRQUEIRA D,VINíCIUS L,PINHEIRO M,et al.OpinionLabel:A Gamified Crowdsourcing System for Sentiment[C]//Anais Estendidos do XXIII Simpósio Brasileiro de Sistemas Multimídia e Web.SBC,2017:209-213.
[6]HAGERER G,SZABO D,KOCH A,et al.End-to-End Annotator Bias Approximation on Crowdsourced Single-Label Sentiment Analysis[C]//Proceedings of The Fourth International Conference on Natural Language and Speech Processing(ICNLSP 2021).2021:1-10.
[7]ENNAJI F Z,FAZZIKI A E,ABDALLAOUI H,et al.ACrowdsourcing Based Framework for Sentiment Analysis:A Product Reputation[J].Journal of Communications Software and Systems,2020,16(4):285-295.
[8]ZHOU J,JIN X,YU L,et al.TruthTrust:Truth Inference-Based Trust Management Mechanism on a Crowdsourcing Platform[J].Sensors(Basel,Switzerland),2021,21(8):2578.
[9]KHUDABUKHSH A,CARBONELL J,JANSEN P.DetectingNon-Adversarial Collusion in Crowdsourcing[C]//Proceedings of the AAAI Conference on Human Computation and Crowdsourcing.2014,2:104-111.
[10]CHEN P P,SUN H L,FANG Y L,et al.Collusion-Proof Result Inference in Crowdsourcing[J].Journal of Computer Science & Technology,2018,33(2):351-365.
[11]NIAZI T M,AMINTOOSI H.Collusion-resistant worker selection in social crowdsensing systems[J].Computer and Know-ledge Engineering,2018,1(1):9-20.
[12]AKKERHUIS T S,DE MAST J.Quantifying the random component of measurement error of nominal measurements without a gold standard[J].Quality and Reliability Engineering International,2016,32(6):1993-2003.
[13]JEONG S,LEE K.Spam Classification Based on Signed Net-work Analysis[J].Applied Sciences,2020,10(24):8952.
[14]MADHAVAN V M,PANDE S,UMEKAR P,et al.Comparative Analysis of Detection of Email Spam With the Aid of Machine Learning Approaches[J].IOP Conference Series:Mate-rials Science and Engineering,2021,1022(1):012113.
[15]XU C,SHEN X,ZHU L,et al.A Collusion-Resistant and Privacy-Preserving Data Aggregation Protocol in Crowdsensing System[J].Mobile Information Systems,2017,2017:1-11.
[16]LI M,WENG J,YANG A,et al.CrowdBC:A blockchain-baseddecentralized framework for crowdsourcing[J].IEEE Transactions on Parallel and Distributed Systems,2018,30(6):1251-1266.
[17]WANG Z,HU R,CHEN Q,et al.ColluEagle:collusive reviewspammer detection using Markov random fields[J].Data Mining and Knowledge Discovery,2020,34(6):1621-1641.
[18]KUANG L,ZHANG H,SHI R,et al.A spam worker detection approach based on heterogeneous network embedding in crowdsourcing platforms[J].Computer Networks,2020,183:107587.
[19]LUO J,SHAN H,ZHANG G,et al.Exploiting Syntactic and Semantic Information for Textual Similarity Estimation[J].Mathematical Problems in Engineering,2021,2021:4186750.1-4186750.12.
[20]LAURIOLA I,LAVELLI A,AIOLLI F.An introduction todeep learning in natural language processing:models,techniques,and tools[J].Neurocomputing,2022,470:443-456.
[21]OTT M,CARDIE C,HANCOCK J T.Negative deceptive opi-nion spam[C]//Proceedings of the 2013 Conference of the North Smerican Chapter of the Association for Computational Linguistics:Human Language Technologies.2013:497-501.
[22]MUKHERJEE A,VENKATARAMAN V,LIU B,et al.Fake review detection:Classification and analysis of real and pseudo reviews:UIC-CS-03-2013[R].2013.
[1] FU Yan-ming, ZHU Jie-fu, JIANG Kan, HUANG Bao-hua, MENG Qing-wen, ZHOU Xing. Incentive Mechanism Based on Multi-constrained Worker Selection in Mobile Crowdsourcing [J]. Computer Science, 2022, 49(9): 275-282.
[2] CHEN Dan-hong, PENG Zhang-lin, WAN De-quan, YANG Shan-lin. Identification and Segmentation of User Value in Crowdsourcing Platforms:An Improved RFMModel [J]. Computer Science, 2022, 49(4): 37-42.
[3] SHEN Biao, SHEN Li-wei, LI Yi. Dynamic Task Scheduling Method for Space Crowdsourcing [J]. Computer Science, 2022, 49(2): 231-240.
[4] JIANG Jing, PING Yuan, WU Qiu-di, ZHANG Li. Developer Recommendation Method for Crowdsourcing Tasks in Open Source Community [J]. Computer Science, 2022, 49(12): 99-108.
[5] LI Yong-hong, WANG Ying, LI La-quan, ZHAO Zhi-qiang. Application of Improved Feature Selection Algorithm in Spam Filtering [J]. Computer Science, 2022, 49(11A): 211000028-5.
[6] ZHANG Shao-jie, LU Xu-dong, GUO Wei, WANG Shi-peng, HE Wei. Prevention of Dishonest Behavior in Supply-Demand Matching [J]. Computer Science, 2021, 48(4): 303-308.
[7] ZHAO Yang, NI Zhi-wei, ZHU Xu-hui, LIU Hao, RAN Jia-min. Multi-worker and Multi-task Path Planning Based on Improved Lion Evolutionary Algorithm forSpatial Crowdsourcing Platform [J]. Computer Science, 2021, 48(11A): 30-38.
[8] LI Yu, DUAN Hong-yue, YIN Yu-yu, GAO Hong-hao. Survey of Crowdsourcing Applications in Blockchain Systems [J]. Computer Science, 2021, 48(11): 12-27.
[9] YU Dun-hui, CHENG Tao, YUAN Xu. Software Crowdsourcing Task Recommendation Algorithm Based on Learning to Rank [J]. Computer Science, 2020, 47(12): 106-113.
[10] WANG Kuo, WANG Zhong-jie. Crowdsourcing Collaboration Process Recovery Method [J]. Computer Science, 2020, 47(10): 19-25.
[11] ZHANG Guang-yuan, WANG Ning. Truth Inference Based on Confidence Interval of Small Samples in Crowdsourcing [J]. Computer Science, 2020, 47(10): 26-31.
[12] HU Ying, WANG Ying-jie, TONG Xiang-rong. Task Recommendation Model Based on Crowd Worker’s Movement Trajectory [J]. Computer Science, 2020, 47(10): 32-40.
[13] CHEN Jin-yin, HUANG Guo-han, WU Yang-yang, JIA Cheng-yu. Double Cycle Graph Based Fraud Review Detection Algorithm [J]. Computer Science, 2019, 46(9): 229-236.
[14] XU Yao-li, LI Zhan-huai. Quality Control Agent Based on Probability Inference [J]. Computer Science, 2019, 46(4): 8-13.
[15] LV Jia-gao,LIANG Kui-yang,CAI Wei. Frontier Scientific Keyword Extraction Based on Bibliometric and Crowdsourcing [J]. Computer Science, 2019, 46(3): 275-282.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!