计算机科学 ›› 2023, Vol. 50 ›› Issue (2): 57-68.doi: 10.11896/jsjkx.221100114

• 边缘智能协同技术及前沿应用 • 上一篇    下一篇

UAV Frequency-based Crowdsensing Using Grouping Multi-agentDeep Reinforcement Learning

Cui ZHANG1, En WANG1, Funing YANG1, Yong jian YANG1 , Nan JIANG2   

  1. 1 College of Computer Science and Technology,Jilin University, Changchun 130012,China
    2 College of Information Engineering,East China Jiaotong University,Nanchang 330013,China
  • 收稿日期:2022-11-14 修回日期:2023-01-03 出版日期:2023-02-15 发布日期:2023-02-22

UAV Frequency-based Crowdsensing Using Grouping Multi-agentDeep Reinforcement Learning

Cui ZHANG1, En WANG1, Funing YANG1, Yong jian YANG1 , Nan JIANG2   

  1. 1 College of Computer Science and Technology,Jilin University, Changchun 130012,China
    2 College of Information Engineering,East China Jiaotong University,Nanchang 330013,China
  • Received:2022-11-14 Revised:2023-01-03 Online:2023-02-15 Published:2023-02-22
  • Contact: Funing YANG(yfn@jlu.edu.cn)
  • About author:(zhangcui20@mails.jlu.edu.cn)
  • Supported by:
    Innovation Capacity Construction Project of Jilin Development and Reform Commission(2020C017-2) and Science and Technology Development Plan Project of Jilin Province(20210201082GX)

Abstract: Mobile CrowdSensing (MCS) is a promising sensing paradigm that recruits users to cooperatively perform sensing tasks.Recently,unmanned aerial vehicles (UAVs) as the powerful sensing devices are used to replace user participation and carry out some special tasks,such as epidemic monitoring and earthquakes rescue.In this paper,we focus on scheduling UAVs to sense the task Point-of-Interests (PoIs) with different frequency coverage requirements.To accomplish the sensing task,the scheduling strategy needs to consider the coverage requirement,geographic fairness and energy charging simultaneously.We consider the complex interaction among UAVs and propose a grouping multi-agent deep reinforcement learning approach (G-MADDPG) to schedule UAVs distributively.G-MADDPG groups all UAVs into some teams by a distance-based clustering algorithm (DCA),then it regards each team as an agent.In this way,G-MADDPG solves the problem that the training time of traditional MADDPG is too long to converge when the number of UAVs is large,and the trade-off between training time and result accuracy could be controlled flexibly by adjusting the number of teams.Extensive simulation results show that our scheduling strategy has better performance compared with three baselines and is flexible in balancing training time and result accuracy.

Key words: UAV Crowdsensing, Frequency coverage, Grouping multi-agent deep reinforcement learning

中图分类号: 

  • TP393
[1]YANG S,HAN K,ZHENG Z,et al.Towards personalized task matching in mobile crowdsensing via fine-grained user profiling[C]//IEEE INFOCOM.2018:2411-2419.
[2]WANG Z B,PANG X Y,CHEN Y H,et al.Privacy-Preserving Crowd-Sourced Statistical Data Publish-ing with An Untrusted Server[J].IEEE Transactions on Mobile Computing,2019,18(6):1356-1367.
[3]DUTTA P,AOKI P M,KUMAR N,et al.Common sense:Participatory urban sensing using a network of hand-held air quality monitors[C]//ACM Conference on Embedded Networked Sensor Systems.2009.
[4]GIL D S,D'OREY P M,AGUIAR A.On the challenges of mobile crowdsensing for traffic estimation[C]//ACM Conference on Embedded Networked Sensor Systems.2017.
[5]QIN Z,FANG Z,LIU Y,et al.A Measurement Framework for Explicit and Implicit Urban Traffic Sensing[J].ACM Transactions on Sensor Networks,2021,17(4):1-27.
[6]LIU C H,PIAO C,TANG J.Energy-efficient UAV crowdsen-sing with multiple charging stations by deep learning[C]//IEEE INFOCOM.2020:199-208.
[7]BARKA E,KERRACHE C A,LAGRAA N,et al.Behavior-aware UAV-assisted crowd sensing technique for urban vehicular environments[C]//IEEE Annual Consumer Communications Networking Conference (CCNC).2018:1-7.
[8]TAO C,ZHU K,CHEN B,et al.UAV-assisted ground signalmap construction based on 3-d spatial correlation[C]//IEEE Global Communications Conference.2020:1-5.
[9]ZHANG S,WU J,LU S.Collaborative mobile charging[J].IEEE Transactions on Computers,2015,64(3):654-667.
[10]LOWE R,WU Y,TAMAR A,et al.Multi-agent actor-critic for mixed cooperative-competitive environments[C]//NIPS.2017:6379-6390.
[11]LIU C H,DAI Z,ZHAO Y,et al.Distributed and Energy-efficient mobile crowdsensing with charging stations by deep reinforcement learning[J].IEEE Transactions on Mobile Computing,2021,21(1):130-146.
[12]LIU C H,CHEN Z,ZHAN Y.Energy-efficient distributed mobile crowd sensing:A deep learning approach[J].IEEE Journal on Selected Areas in Communications,2019,37(6):1262-1276.
[13]YANG Y,RUI L,LI M Z.MING,et al.Mean field multi-agent reinforcement learning[C]//The 35th International Conference on Machine Learning.2018:5571-5580.
[14]WANG Z B,HU J H,LV R Z,et al.Personalized Privacy-Preserving Task Allocation for Mobile Crowdsensing[J].IEEE Transactions on Mobile Computing,2018,18(6):1330-1341.
[15]WANG L,ZHIWEN Y U,GUO B,et al.Mobile crowd sensing task optimal allocation:a mobility pattern matching perspective[M]//Frontiers of Computer Science (print),2018:231-244.
[16]ZHANG B,LIU C H,TANG J,et al.Learning-based energy-efficient data collection by unmanned vehicles in smart cities[J].IEEE Transactions on Industrial Informatics,2018,14(4):1666-1676.
[17]MNIH V,KAVUKCUOGLU K,SILVER D,et al.Playing Atari with deep reinforcement learning[R].Computer Science,2013.
[18]MNIH V,BADIA A P,MIRZA M,et al.Asynchronous methods for deep reinforcement learning[C]//Proceedings of The 33rd International Conference on Machine Learning.2016:1928-1937.
[19]LILLICRAP T,HUNT J J,PRITZEL A,et al.Continuous control with deep reinforcement learning[C]//CoRR.2016.
[20]SONG M K,WANG Z B,ZHANG Z F,et al.Analyzing User-Level Privacy Attack Against Federated Learning[J].IEEE Journal on Selected Areas in Communications,2020,38(10):2430-2444.
[21]WEI Y,ZHENG R.Multi-robot path planning for mobile sensing through deep reinforcement learning[C]//IEEE INFOCOM.2021.
[22]DING R,YANG Z,WEI Y,et al.Multi-agent reinforcementlearning for urban crowd sensing with for-hire vehicles[C]//IEEE INFOCOM.2021.
[23]JAIN R.A quantitative measure of fairness and discrimination for resource allocation in shared computer systems[R].DEC Research Report,1984.
[24]VANSTEENWEGEN P,SOUFFRIAU W,OUDHEUSDEN D V.The orienteering problem:A survey[J].European Journal of Operational Research,2011,209(1):1-10.
[1] 杨昕, 李挥, 阙建明, 马震太, 李更新, 姚尧, 王滨, 蒋傅礼.
面向未来网络的安全高效防护架构
Efficiently Secure Architecture for Future Network
计算机科学, 2023, 50(3): 360-370. https://doi.org/10.11896/jsjkx.220600265
[2] Yifei ZOU, Senmao QI, Cong'an XU, Dongxiao YU.
Distributed Weighted Data Aggregation Algorithm in End-to-Edge Communication Networks Based on Multi-armed Bandit
Distributed Weighted Data Aggregation Algorithm in End-to-Edge Communication Networks Based on Multi-armed Bandit
计算机科学, 2023, 50(2): 13-22. https://doi.org/10.11896/jsjkx.221100134
[3] 陈祎鹏, 杨哲, 谷飞, 赵雷.
一种基于博弈论的移动边缘计算资源分配策略
Resource Allocation Strategy Based on Game Theory in Mobile Edge Computing
计算机科学, 2023, 50(2): 32-41. https://doi.org/10.11896/jsjkx.220300198
[4] 常沙, 吴亚辉, 邓苏, 马武彬, 周浩浩.
基于李雅普诺夫优化的移动群智感知在线任务分配策略
Online Task Allocation Strategy Based on Lyapunov Optimization in Mobile Crowdsensing
计算机科学, 2023, 50(2): 50-56. https://doi.org/10.11896/jsjkx.221100179
[5] 郑鸿强, 张建山, 陈星.
空-天-地一体化移动边缘计算系统的部署优化和计算卸载
Deployment Optimization and Computing Offloading of Space-Air-Ground Integrated Mobile Edge Computing System
计算机科学, 2023, 50(2): 69-79. https://doi.org/10.11896/jsjkx.220600057
[6] 尚玉叶, 袁家斌.
深空环境中基于云边端协同的任务卸载方法
Task Offloading Method Based on Cloud-Edge-End Cooperation in Deep Space Environment
计算机科学, 2023, 50(2): 80-88. https://doi.org/10.11896/jsjkx.220800156
[7] 杨昕, 李更新, 李挥.
EHFM:一种面向多源网络攻击告警的高效层级化数据过滤方案
EHFM:An Efficient Hierarchical Filtering Method for Multi-source Network Malicious Alerts
计算机科学, 2023, 50(2): 324-332. https://doi.org/10.11896/jsjkx.220800049
[8] 代亮, 吴益钵, 汪贵平.
蜂窝车联网连通性研究综述与展望
Review and Prospect of Connectivity Research on Cellular-V2X
计算机科学, 2023, 50(1): 285-293. https://doi.org/10.11896/jsjkx.211000164
[9] 程文辉, 张乾元, 程梁华, 向朝参, 杨振东, 沈鑫, 张乃凡.
空地协同移动群智感知研究综述
Review of Mobile Air-Ground Crowdsensing
计算机科学, 2022, 49(11): 242-249. https://doi.org/10.11896/jsjkx.220400264
[10] 于浩雯, 刘波, 周娜琴, 林伟伟, 柳鹏.
多云工作流调度综述
Survey of Multi-cloud Workflow Scheduling
计算机科学, 2022, 49(11): 250-258. https://doi.org/10.11896/jsjkx.211200234
[11] 李晓波, 陈鹏, 帅彬, 夏云霓, 李建岐.
边缘环境下轨迹预测性感知的在线边缘服务分配
Novel Predictive Approach to Trajectory-aware Online Edge Service Allocation in Edge Environment
计算机科学, 2022, 49(11): 277-283. https://doi.org/10.11896/jsjkx.211100029
[12] 卞庆荣, 程宝雷, 樊建席, 潘志勇.
蜻蜓网络上完全独立生成树的构造算法
Construction Algorithm of Completely Independent Spanning Tree in Dragonfly Network
计算机科学, 2022, 49(11): 284-292. https://doi.org/10.11896/jsjkx.211000037
[13] 刘培文, 舒辉, 吕小少, 赵耘田.
基于有限状态机的内核漏洞攻击自动化分析技术
Automatic Analysis Technology of Kernel Vulnerability Attack Based on Finite State Machine
计算机科学, 2022, 49(11): 326-334. https://doi.org/10.11896/jsjkx.211200039
[14] 何源, 邢长友, 张国敏, 宋丽华, 余航.
面向网络侦察欺骗的差分隐私指纹混淆机制
Differential Privacy Based Fingerprinting Obfuscation Mechanism Towards NetworkReconnaissance Deception
计算机科学, 2022, 49(11): 351-359. https://doi.org/10.11896/jsjkx.220400285
[15] 潘雨, 王帅辉, 张磊, 胡谷雨, 邹军华, 王田丰, 潘志松.
复杂网络社团发现综述
Survey of Community Detection in Complex Network
计算机科学, 2022, 49(11A): 210800144-11. https://doi.org/10.11896/jsjkx.210800144
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!