Computer Science ›› 2023, Vol. 50 ›› Issue (2): 57-68.doi: 10.11896/jsjkx.221100114

• Edge Intelligent Collaboration Technology and Frontier Applications • Previous Articles     Next Articles

UAV Frequency-based Crowdsensing Using Grouping Multi-agentDeep Reinforcement Learning

Cui ZHANG1, En WANG1, Funing YANG1, Yong jian YANG1 , Nan JIANG2   

  1. 1 College of Computer Science and Technology,Jilin University, Changchun 130012,China
    2 College of Information Engineering,East China Jiaotong University,Nanchang 330013,China
  • Received:2022-11-14 Revised:2023-01-03 Online:2023-02-15 Published:2023-02-22
  • Contact: Funing YANG(yfn@jlu.edu.cn)
  • About author:(zhangcui20@mails.jlu.edu.cn)
  • Supported by:
    Innovation Capacity Construction Project of Jilin Development and Reform Commission(2020C017-2) and Science and Technology Development Plan Project of Jilin Province(20210201082GX)

Abstract: Mobile CrowdSensing (MCS) is a promising sensing paradigm that recruits users to cooperatively perform sensing tasks.Recently,unmanned aerial vehicles (UAVs) as the powerful sensing devices are used to replace user participation and carry out some special tasks,such as epidemic monitoring and earthquakes rescue.In this paper,we focus on scheduling UAVs to sense the task Point-of-Interests (PoIs) with different frequency coverage requirements.To accomplish the sensing task,the scheduling strategy needs to consider the coverage requirement,geographic fairness and energy charging simultaneously.We consider the complex interaction among UAVs and propose a grouping multi-agent deep reinforcement learning approach (G-MADDPG) to schedule UAVs distributively.G-MADDPG groups all UAVs into some teams by a distance-based clustering algorithm (DCA),then it regards each team as an agent.In this way,G-MADDPG solves the problem that the training time of traditional MADDPG is too long to converge when the number of UAVs is large,and the trade-off between training time and result accuracy could be controlled flexibly by adjusting the number of teams.Extensive simulation results show that our scheduling strategy has better performance compared with three baselines and is flexible in balancing training time and result accuracy.

Key words: UAV Crowdsensing, Frequency coverage, Grouping multi-agent deep reinforcement learning

CLC Number: 

  • TP393
[1]YANG S,HAN K,ZHENG Z,et al.Towards personalized task matching in mobile crowdsensing via fine-grained user profiling[C]//IEEE INFOCOM.2018:2411-2419.
[2]WANG Z B,PANG X Y,CHEN Y H,et al.Privacy-Preserving Crowd-Sourced Statistical Data Publish-ing with An Untrusted Server[J].IEEE Transactions on Mobile Computing,2019,18(6):1356-1367.
[3]DUTTA P,AOKI P M,KUMAR N,et al.Common sense:Participatory urban sensing using a network of hand-held air quality monitors[C]//ACM Conference on Embedded Networked Sensor Systems.2009.
[4]GIL D S,D'OREY P M,AGUIAR A.On the challenges of mobile crowdsensing for traffic estimation[C]//ACM Conference on Embedded Networked Sensor Systems.2017.
[5]QIN Z,FANG Z,LIU Y,et al.A Measurement Framework for Explicit and Implicit Urban Traffic Sensing[J].ACM Transactions on Sensor Networks,2021,17(4):1-27.
[6]LIU C H,PIAO C,TANG J.Energy-efficient UAV crowdsen-sing with multiple charging stations by deep learning[C]//IEEE INFOCOM.2020:199-208.
[7]BARKA E,KERRACHE C A,LAGRAA N,et al.Behavior-aware UAV-assisted crowd sensing technique for urban vehicular environments[C]//IEEE Annual Consumer Communications Networking Conference (CCNC).2018:1-7.
[8]TAO C,ZHU K,CHEN B,et al.UAV-assisted ground signalmap construction based on 3-d spatial correlation[C]//IEEE Global Communications Conference.2020:1-5.
[9]ZHANG S,WU J,LU S.Collaborative mobile charging[J].IEEE Transactions on Computers,2015,64(3):654-667.
[10]LOWE R,WU Y,TAMAR A,et al.Multi-agent actor-critic for mixed cooperative-competitive environments[C]//NIPS.2017:6379-6390.
[11]LIU C H,DAI Z,ZHAO Y,et al.Distributed and Energy-efficient mobile crowdsensing with charging stations by deep reinforcement learning[J].IEEE Transactions on Mobile Computing,2021,21(1):130-146.
[12]LIU C H,CHEN Z,ZHAN Y.Energy-efficient distributed mobile crowd sensing:A deep learning approach[J].IEEE Journal on Selected Areas in Communications,2019,37(6):1262-1276.
[13]YANG Y,RUI L,LI M Z.MING,et al.Mean field multi-agent reinforcement learning[C]//The 35th International Conference on Machine Learning.2018:5571-5580.
[14]WANG Z B,HU J H,LV R Z,et al.Personalized Privacy-Preserving Task Allocation for Mobile Crowdsensing[J].IEEE Transactions on Mobile Computing,2018,18(6):1330-1341.
[15]WANG L,ZHIWEN Y U,GUO B,et al.Mobile crowd sensing task optimal allocation:a mobility pattern matching perspective[M]//Frontiers of Computer Science (print),2018:231-244.
[16]ZHANG B,LIU C H,TANG J,et al.Learning-based energy-efficient data collection by unmanned vehicles in smart cities[J].IEEE Transactions on Industrial Informatics,2018,14(4):1666-1676.
[17]MNIH V,KAVUKCUOGLU K,SILVER D,et al.Playing Atari with deep reinforcement learning[R].Computer Science,2013.
[18]MNIH V,BADIA A P,MIRZA M,et al.Asynchronous methods for deep reinforcement learning[C]//Proceedings of The 33rd International Conference on Machine Learning.2016:1928-1937.
[19]LILLICRAP T,HUNT J J,PRITZEL A,et al.Continuous control with deep reinforcement learning[C]//CoRR.2016.
[20]SONG M K,WANG Z B,ZHANG Z F,et al.Analyzing User-Level Privacy Attack Against Federated Learning[J].IEEE Journal on Selected Areas in Communications,2020,38(10):2430-2444.
[21]WEI Y,ZHENG R.Multi-robot path planning for mobile sensing through deep reinforcement learning[C]//IEEE INFOCOM.2021.
[22]DING R,YANG Z,WEI Y,et al.Multi-agent reinforcementlearning for urban crowd sensing with for-hire vehicles[C]//IEEE INFOCOM.2021.
[23]JAIN R.A quantitative measure of fairness and discrimination for resource allocation in shared computer systems[R].DEC Research Report,1984.
[24]VANSTEENWEGEN P,SOUFFRIAU W,OUDHEUSDEN D V.The orienteering problem:A survey[J].European Journal of Operational Research,2011,209(1):1-10.
[1] YANG Xin, LI Hui, QUE Jianming, MA Zhentai, LI Gengxin, YAO Yao, WANG Bin, JIANG Fuli. Efficiently Secure Architecture for Future Network [J]. Computer Science, 2023, 50(3): 360-370.
[2] Yifei ZOU, Senmao QI, Cong'an XU, Dongxiao YU. Distributed Weighted Data Aggregation Algorithm in End-to-Edge Communication Networks Based on Multi-armed Bandit [J]. Computer Science, 2023, 50(2): 13-22.
[3] CHEN Yipeng, YANG Zhe, GU Fei, ZHAO Lei. Resource Allocation Strategy Based on Game Theory in Mobile Edge Computing [J]. Computer Science, 2023, 50(2): 32-41.
[4] CHANG Sha, WU Yahui, DENG Su, MA Wubin, ZHOU Haohao. Online Task Allocation Strategy Based on Lyapunov Optimization in Mobile Crowdsensing [J]. Computer Science, 2023, 50(2): 50-56.
[5] ZHENG Hongqiang, ZHANG Jianshan, CHEN Xing. Deployment Optimization and Computing Offloading of Space-Air-Ground Integrated Mobile Edge Computing System [J]. Computer Science, 2023, 50(2): 69-79.
[6] SHANG Yuye, YUAN Jiabin. Task Offloading Method Based on Cloud-Edge-End Cooperation in Deep Space Environment [J]. Computer Science, 2023, 50(2): 80-88.
[7] YANG Xin, LI Gengxin, LI Hui. EHFM:An Efficient Hierarchical Filtering Method for Multi-source Network Malicious Alerts [J]. Computer Science, 2023, 50(2): 324-332.
[8] DAI Liang, WU Yibo, WANG Guiping. Review and Prospect of Connectivity Research on Cellular-V2X [J]. Computer Science, 2023, 50(1): 285-293.
[9] CHENG Wen-hui, ZHANG Qian-yuan, CHENG Liang-hua, XIANG Chao-can, YANG Zhen-dong, SHEN Xin, ZHANG Nai-fan. Review of Mobile Air-Ground Crowdsensing [J]. Computer Science, 2022, 49(11): 242-249.
[10] YU Hao-wen, LIU Bo, ZHOU Na-qin, LIN Wei-wei, LIU Peng. Survey of Multi-cloud Workflow Scheduling [J]. Computer Science, 2022, 49(11): 250-258.
[11] LI Xiao-bo, CHEN Peng, SHUAI Bin, XIA Yun-ni, LI Jian-qi. Novel Predictive Approach to Trajectory-aware Online Edge Service Allocation in Edge Environment [J]. Computer Science, 2022, 49(11): 277-283.
[12] BIAN Qing-rong, CHENG Bao-lei, FAN Jian-xi, PAN Zhi-yong. Construction Algorithm of Completely Independent Spanning Tree in Dragonfly Network [J]. Computer Science, 2022, 49(11): 284-292.
[13] LIU Pei-wen, SHU Hui, LYU Xiao-shao, ZHAO Yun-tian. Automatic Analysis Technology of Kernel Vulnerability Attack Based on Finite State Machine [J]. Computer Science, 2022, 49(11): 326-334.
[14] HE Yuan, XING Chang-you, ZHANG Guo-min, SONG Li-hua, YU Hang. Differential Privacy Based Fingerprinting Obfuscation Mechanism Towards NetworkReconnaissance Deception [J]. Computer Science, 2022, 49(11): 351-359.
[15] PAN Yu, WANG Shuai-hui, ZHANG Lei, HU Gu-yu, ZOU Jun-hua, WANG Tian-feng, PAN Zhi-song. Survey of Community Detection in Complex Network [J]. Computer Science, 2022, 49(11A): 210800144-11.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!