Computer Science

Computer Science ›› 2023, Vol. 50 ›› Issue (1): 351-361.doi: 10.11896/jsjkx.220800269

• Information Security • Previous Articles     Next Articles

Backdoor Attack Against Deep Reinforcement Learning-based Spectrum Access Model

WEI Nan1, WEI Xianglin2, FAN Jianhua2, XUE Yu1, HU Yongyang2   

  1. 1 School of Computer and Software,Nanjing University of Information Science and Technology,Nanjing 210044,China
    2 The 63rd Research Institute,National University of Defense Technology,Nanjing 210007,China
  • Received:2022-08-31 Revised:2022-09-28 Online:2023-01-15 Published:2023-01-09
  • About author:WEI Nan,born in 1998,postgraduate.Her main research interests include deep reinforcement learning and spectrum intelligent computing.
    FAN Jianhua,born in 1971,Ph.D,research fellow.His main research intere-sts include software defined radio and spectrum intelligent computing.

PDF (PC)

1293

Abstract: Deep reinforcement learning(DRL) has attracted much attention in multi-user intelligent dynamic spectrum access due to its advantages in sensing and decision making.However,the weak interpretability of deep neural networks(DNNs) makes DRL models vulnerable to backdoor attacks.In this paper,a non-invasive backdoor attack method with low-cost is proposed against DSA-oriented DRL models in cognitive wireless networks.The attacker monitors the wireless channels to select backdoor triggers,and generates backdoor samples into the experience pool of a secondary user's DRL model.Then,the trigger can be implanted into the DRL model during the training phase.The attacker actively sends signals to activate the triggers in the DRL model during the inference phase,inducing secondary users to take the actions set by the attacker,thereby reducing their success rate of channel access.A series of simulation show that the proposed backdoor attack method can reduce the attack cost by 20%~30% while achieving an attack success rate over 90%,and is suitable for three different DRL models.

Key words: Dynamic spectrum access, Deep reinforcement learning, Backdoor attack, Trigger

CLC Number: 

  • TN925
[1]ALWARAFYA,ABDALLAHM,ÇIFTLERBS,et al.The Frontiers of Deep Reinforcement Learning for Resource Management in Future Wireless HetNets: Techniques,Challenges,and Research Directions[J].IEEE Open Journal of the Communications Society,2022(3): 322-365.
[2]HAN H,XU Y F,JIN Z,et al.Primary-User-Friendly Dynamic Spectrum Anti-Jamming Access: A GAN-Enhanced Deep Reinforcement Learning Approach[J].IEEE Wireless Communications Letters,2022,11(2): 258-262.
[3]ZHAO X Y,DING S F.Research on Deep Reinforcement Lear-ning[J].Computer Science,2018,45(7):1-6.
[4]CHEN J Y,ZHANG Y,WANG X K,et al.A Survey of Attack,Defense and Related Security Analysis for Deep Reinforcement Learning[J].Acta Automatica Sinica,2022,48(1):21-39.
[5]LIU J W,GAO F,LUO X L.Survey of Deep Reinforcement Learning Based on Value Function and Policy Gradient[J].Chinese Journal of Computers,2019,42(6):1406-1438.
[6]CHANG H H,SONG H,YI Y,et al.Distributive DynamicSpectrum Access through Deep Reinforcement Learning: A Reservoir Computing Based Approach[J].IEEE Internet of Things Journal,2019,6(2):1938-1948.
[7]ZHANG Y,CAI P,PAN C,et al.Multi-Agent Deep Reinforcement Learning-Based Cooperative Spectrum Sensing with Upper Confidence Bound Exploration[J].IEEE Access,2019(7): 118898-118906.
[8]TOMOVIC S,RADUSINOVIC I.A Novel Deep Q-learningMethod for Dynamic Spectrum Access[C]//2020 28th Telecommunications Forum(TELFOR).2020:1-4.
[9]XU Y,YU J,BUEHRER R M.The Application of Deep Reinforcement Learning to Distributed Spectrum Access in Dynamic Heterogeneous Environments with Partial Observations[J].IEEE Transactions on Wireless Communications,2020,19(7): 4494-4506.
[10]ZHONG C,LU Z,GURSOY M C,et al.A Deep Actor-Critic Reinforcement Learning Framework for Dynamic Multichannel Access[J].IEEE Transactions on Cognitive Communications and Networking,2019,5(4):1125-1139.
[11]GU T,LIU K,DOLAN-GAVITT B,et al.BadNets: Evaluating Backdooring Attacks on Deep Neural Networks[J].IEEE Access,2019(7): 47230-47244.
[12]GAO Y,DOAN B G,ZHANG Z,et al.Backdoor Attacks and Countermeasures on Deep Learning: A Comprehensive Review[J].arXiv: 2007.10760,2020.
[13]YANG Z,IYER N,REIMANN J,et al.Design of intentionalbackdoors in sequential models[J].arXiv: 1902.09972,2019.
[14]KIOURTI P,WARDEGA K,JHA S,et al.TrojDRL:Evalua-tion of Backdoor Attacks on Deep Reinforcement Learning[C]//2020 57th ACM/IEEE Design Automation Conference(DAC).ACM,2020:1-6.
[15]WANG Y,SARKER E,LI W Q,et al.Stop-and-Go: Exploring Backdoor Attacks on Deep Reinforcement Learning-based Traffic Congestion Control Systems[J].IEEE Transactions on Information Forensics and Security,2021(16): 4772-4787.
[16]ISLAM S,BADSHA S,KHALIL I,et al.A Triggerless Backdoor Attack and Defense Mechanism for Intelligent Task Offloading in Multi-UAV Systems[J].IEEE Internet of Things Journal(in press),doi: 10.1109/JIOT.2022.3172936,2022.
[17]SAGDUYU Y E,SHI Y,ERPEK T.Adversarial Deep Learning for Over-the-Air Spectrum Poisoning Attacks[J].IEEE Transac-tions on Mobile Computing,2021,20(2):306-319.
[18]KIM B,SHI Y,SAGDUYU Y E,et al.Adversarial Attacksagainst Deep Learning Based Power Control in Wireless Communications[C]//2021 IEEE Globecom Workshops(GC Wkshps).2021:1-6.
[19]LUO Z P,ZHAO S Q,LU Z,et al.When Attackers Meet AI: Learning-Empowered Attacks in Cooperative Spectrum Sensing[J].IEEE Transactions on Mobile Computing,2022,21(5):1892-1908.
[20]ZHONG C,WANG F,GURSOY M C,et al.Adversarial Jamming Attacks on Deep Reinforcement Learning Based Dynamic Multichannel Access[C]// 2020 IEEE Wireless Communications and Networking Conference(WCNC).IEEE,2020:1-6.
[21]PAN X N,CHEN Z,LI J Z,et al.A dynamic spectrum access algorithm based on prioritized experience replay deep Q-Learning[J].Telecommunication Engineering,2020,60(5):489-495.
[22]XING L,XU Y H,LI G Q,et al.Channel estimation algorithm for MIMO systems based on deep learning[J].Journal of Chongqing University of Posts and Telecommunications(Natural Science Edition),2022,34(4):685-693.
[23]PENG A J,WANG S X,PAN C H,et al.Multi-pair two-way massive MIMO DF relaying over Rician fading channels under imperfect CSI[J].IEEE Wireless Communication Letters,2022,11(2):225-229.
[24]ASHCRAFT C,KARRA K.Poisoning Deep ReinforcementLearning Agents with In-Distribution Triggers[J].arXiv: 2106.07798,2021.
[1] HUANG Yuzhou, WANG Lisong, QIN Xiaolin. Bi-level Path Planning Method for Unmanned Vehicle Based on Deep Reinforcement Learning [J]. Computer Science, 2023, 50(1): 194-204.
[2] ZHANG Qiyang, CHEN Xiliang, ZHANG Qiao. Sparse Reward Exploration Method Based on Trajectory Perception [J]. Computer Science, 2023, 50(1): 262-269.
[3] CHENG Yong, MAO Yingchi, WAN Xu, WANG Longbao, ZHU Min. Chinese Event Detection Without Triggers Based on Dual Attention [J]. Computer Science, 2023, 50(1): 276-284.
[4] YU Bin, LI Xue-hua, PAN Chun-yu, LI Na. Edge-Cloud Collaborative Resource Allocation Algorithm Based on Deep Reinforcement Learning [J]. Computer Science, 2022, 49(7): 248-253.
[5] LI Meng-fei, MAO Ying-chi, TU Zi-jian, WANG Xuan, XU Shu-fang. Server-reliability Task Offloading Strategy Based on Deep Deterministic Policy Gradient [J]. Computer Science, 2022, 49(7): 271-279.
[6] XIE Wan-cheng, LI Bin, DAI Yue-yue. PPO Based Task Offloading Scheme in Aerial Reconfigurable Intelligent Surface-assisted Edge Computing [J]. Computer Science, 2022, 49(6): 3-11.
[7] HONG Zhi-li, LAI Jun, CAO Lei, CHEN Xi-liang, XU Zhi-xiong. Study on Intelligent Recommendation Method of Dueling Network Reinforcement Learning Based on Regret Exploration [J]. Computer Science, 2022, 49(6): 149-157.
[8] LI Peng, YI Xiu-wen, QI De-kang, DUAN Zhe-wen, LI Tian-rui. Heating Strategy Optimization Method Based on Deep Learning [J]. Computer Science, 2022, 49(4): 263-268.
[9] OUYANG Zhuo, ZHOU Si-yuan, LYU Yong, TAN Guo-ping, ZHANG Yue, XIANG Liang-liang. DRL-based Vehicle Control Strategy for Signal-free Intersections [J]. Computer Science, 2022, 49(3): 46-51.
[10] CAI Yue, WANG En-liang, SUN Zhe, SUN Zhi-xin. Study on Dual Sequence Decision-making for Trucks and Cargo Matching Based on Dual Pointer Network [J]. Computer Science, 2022, 49(11A): 210800257-9.
[11] LIU Pei-wen, SHU Hui, LYU Xiao-shao, ZHAO Yun-tian. Automatic Analysis Technology of Kernel Vulnerability Attack Based on Finite State Machine [J]. Computer Science, 2022, 49(11): 326-334.
[12] DAI Shan-shan, LIU Quan. Action Constrained Deep Reinforcement Learning Based Safe Automatic Driving Method [J]. Computer Science, 2021, 48(9): 235-243.
[13] CHENG Zhao-wei, SHEN Hang, WANG Yue, WANG Min, BAI Guang-wei. Deep Reinforcement Learning Based UAV Assisted SVC Video Multicast [J]. Computer Science, 2021, 48(9): 271-277.
[14] ZHOU Shi-cheng, LIU Jing-ju, ZHONG Xiao-feng, LU Can-ju. Intelligent Penetration Testing Path Discovery Based on Deep Reinforcement Learning [J]. Computer Science, 2021, 48(7): 40-46.
[15] LI Bei-bei, SONG Jia-rui, DU Qing-yun, HE Jun-jiang. DRL-IDS:Deep Reinforcement Learning Based Intrusion Detection System for Industrial Internet of Things [J]. Computer Science, 2021, 48(7): 47-54.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.