Computer Science ›› 2020, Vol. 47 ›› Issue (7): 282-286.doi: 10.11896/jsjkx.200100135

• Information Security • Previous Articles     Next Articles

Reinforcement Learning Based Cache Scheduling Against Denial-of-Service Attacks in Embedded Systems

HUANG Jin-hao1, DING Yu-zhen1, XIAO Liang1, SHEN Zhi-rong1, ZHU Zhen-min2   

  1. 1 School of Informatics,Xiamen University,Xiamen 361005,China
    2 Institute of Computing technology,Chinese Academy of Sciences University,Beijing 100190,China
  • Received:2020-01-21 Online:2020-07-15 Published:2020-07-16
  • About author:HUANG Jin-hao,born in 1996,postgraduate.His main research interests include network security and wireless communication.
    XIAO Liang,born in 1980,Ph.D,professor,Ph.D supervisor,is a member of China Computer Federation.Her main research interests include network security and wireless communication.
  • Supported by:
    This work was supported by the National Natural Science Foundation of China (61971366,61671396)

Abstract: The sharing last level cache (LLC) scheduling of the central processor determines the instructions per cycle (IPC) of the user processes and the robustness of denial-of-service (DoS) attacks in the multicore embedded operating systems.However,existing scheduling schemes rely on the specific LLC scheduling model and DoS attack model,which makes it difficult for the processor to obtain the running information of the user processes in each scheduling cycle under different scheduling environments.Therefore,this paper proposes a reinforcement learning (RL) based LLC scheduling scheme to against DoS attacks in embedded systems,which optimizes the occupied position and the occupied space based on the measured occupied start and end positions,the previous IPC,load miss rate and store miss rate.The processor can jointly increase the IPC and reduce the success rate of the DoS attack from the malicious process without knowing the DoS attack model in the dynamic LLC scheduling environment.Simulations are implemented on the multicore embedded operating systems where multitenant virtual machines participate together,which show that the proposed scheme can significantly increase the IPC and reduce the success rate of the DoS attack.

Key words: DoS attack, Embedded systems, LLC scheduling, Reinforcement learning

CLC Number: 

  • TP316
[1]DETTI A,BRACCIALE L,LORETI P,et al.Modeling LRUcache with invalidation [J].Computer Networks,2018,134(7):55-65.
[2]UDIPI A N,MURALIMANOHAR N,BALSUBRAMONIANR,et al.Combining memory and a controller with photonics through 3D-stacking to enable scalable and energy-efficient systems[C]//International Symposium on Computer Architecture.ACM,2011:425-436.
[3]BROCK J,YE C,DING C,et al.Optimal cache partition-sharing[C]//International Conference on Parallel Processing.IEEE,2015:749-758.
[4]CHANG J,SOHI G S.Cooperative cache partitioning for chipmultiprocessors[C]//International Conference on Supercompu-ting.ACM,2014:402-412.
[5]HERDRICH A,VERPLANKE E,AUTEE P,et al.Cache QoS:From concept to reality in the Intel® Xeon® processor E5-2600 v3 product family[C]//International Symposium on High Performance Computer Architecture.IEEE,2016:657-668.
[6]OTOOM M,JALEEL A,TRANCOSO P.Using personalitymetrics to improve cache interference management in multicore processors[C]//ACM International Conference on Computing Frontiers.ACM,2017:251-254.
[7]KASTURE H,SANCHEZ D.Ubik:efficient cache sharing with strict qos for latency-critical workloads[C]// Architectural Support for Programming Languages and Operating Systems.ACM,2014:729-742.
[8]GRUSS D,MAURICE C,WAGNER K,et al.Flush+ Flush:a fast and stealthy cache attack[C]//Detection of Intrusions and Malware,and Vulnerability Assessment.Berlin:Springer,2016:279-299.
[9]BECHTEL M,YUN H.Denial-of-service attacks on sharedcache in multicore:Analysis and prevention[C]//Real-Time and Embedded Technology and Applications Symposium.IEEE,2019:357-367.
[10]XIANG Y,WANG X,HUANG Z,et al.DCAPS:dynamic cache allocation with partial sharing[C]//EuroSys.ACM,2018:13.
[11]XU C,RAJAMANI K,FERREIRA A,et al.dCat:dynamic cachemanagement for efficient,performance-sensitive infrastructure-as-a-service[C]//EuroSys.ACM,2018:14.
[12]MUTLU T M O.Memory performance attacks:Denial of memory service in multi-core systems[C]//USENIX Security Symposium.USENIX,2007:18.
[13]KIM Y,PAPAMICHAEL M,MUTLU O,et al.Thread cluster memory scheduling:Exploiting differences in memory access behavior[C]//IEEE/ACM International Symposium on Microarchitecture.ACM,2010:65-76.
[14]KERAMIDAS G,PETOUMENOS P,KAXIRAS S,et al.Pre-venting denial-of-service attacks in shared CMP caches[C]//International Workshop on Embedded Computer Systems.Berlin:Springer,2006:359-372.
[15]WATKINS C J C H,DAYAN P.Q-learning [J].Machine Lear-ning,1992,8(3-4):279-292.
[16]MNIH V,KAVUKCUOGLU K,SILVER D,et al.Human-level control through deep reinforcement learning [J].Nature,2015,518(7540):529.
[17]YE R,XU Q.Learning-based power management for multicore processors via idle period manipulation [J].IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems,2014,33(7):1043-1055.
[18]JAIN R,PANDA P R,SUBRAMONEY S.A coordinated multi-agent reinforcement learning approach to multi-level cache co-partitioning[C]//Design,Automation & Test in Europe Confe-rence & Exhibition.IEEE,2017:800-805.
[19]MIN M,WAN X,XIAO L,et al.Learning-based privacy-aware offloading for healthcare IoT with energy harvesting [J].IEEE Internet of Things Journal,2018,6(3):4307-4316.
[20]MIN M,XIAO L,XIE C,et al.Defense against advanced persistent threats in dynamic cloud storage:A colonel blotto game approach [J].IEEE Internet of Things Journal,2018,5(6):4250-4261.
[21]HE K,ZHANG X,REN S,et al.Delving deep into rectifiers:Surpassing human-level performance on imagenet classification[C]//IEEE International Conference on Computer Vision.IEEE,2015:1026-1034.
[1] LIU Xing-guang, ZHOU Li, LIU Yan, ZHANG Xiao-ying, TAN Xiang, WEI Ji-bo. Construction and Distribution Method of REM Based on Edge Intelligence [J]. Computer Science, 2022, 49(9): 236-241.
[2] SHI Dian-xi, ZHAO Chen-ran, ZHANG Yao-wen, YANG Shao-wu, ZHANG Yong-jun. Adaptive Reward Method for End-to-End Cooperation Based on Multi-agent Reinforcement Learning [J]. Computer Science, 2022, 49(8): 247-256.
[3] YUAN Wei-lin, LUO Jun-ren, LU Li-na, CHEN Jia-xing, ZHANG Wan-peng, CHEN Jing. Methods in Adversarial Intelligent Game:A Holistic Comparative Analysis from Perspective of Game Theory and Reinforcement Learning [J]. Computer Science, 2022, 49(8): 191-204.
[4] YU Bin, LI Xue-hua, PAN Chun-yu, LI Na. Edge-Cloud Collaborative Resource Allocation Algorithm Based on Deep Reinforcement Learning [J]. Computer Science, 2022, 49(7): 248-253.
[5] LI Meng-fei, MAO Ying-chi, TU Zi-jian, WANG Xuan, XU Shu-fang. Server-reliability Task Offloading Strategy Based on Deep Deterministic Policy Gradient [J]. Computer Science, 2022, 49(7): 271-279.
[6] GUO Yu-xin, CHEN Xiu-hong. Automatic Summarization Model Combining BERT Word Embedding Representation and Topic Information Enhancement [J]. Computer Science, 2022, 49(6): 313-318.
[7] FAN Jing-yu, LIU Quan. Off-policy Maximum Entropy Deep Reinforcement Learning Algorithm Based on RandomlyWeighted Triple Q -Learning [J]. Computer Science, 2022, 49(6): 335-341.
[8] XIE Wan-cheng, LI Bin, DAI Yue-yue. PPO Based Task Offloading Scheme in Aerial Reconfigurable Intelligent Surface-assisted Edge Computing [J]. Computer Science, 2022, 49(6): 3-11.
[9] HONG Zhi-li, LAI Jun, CAO Lei, CHEN Xi-liang, XU Zhi-xiong. Study on Intelligent Recommendation Method of Dueling Network Reinforcement Learning Based on Regret Exploration [J]. Computer Science, 2022, 49(6): 149-157.
[10] ZHANG Jia-neng, LI Hui, WU Hao-lin, WANG Zhuang. Exploration and Exploitation Balanced Experience Replay [J]. Computer Science, 2022, 49(5): 179-185.
[11] LI Peng, YI Xiu-wen, QI De-kang, DUAN Zhe-wen, LI Tian-rui. Heating Strategy Optimization Method Based on Deep Learning [J]. Computer Science, 2022, 49(4): 263-268.
[12] OUYANG Zhuo, ZHOU Si-yuan, LYU Yong, TAN Guo-ping, ZHANG Yue, XIANG Liang-liang. DRL-based Vehicle Control Strategy for Signal-free Intersections [J]. Computer Science, 2022, 49(3): 46-51.
[13] ZHOU Qin, LUO Fei, DING Wei-chao, GU Chun-hua, ZHENG Shuai. Double Speedy Q-Learning Based on Successive Over Relaxation [J]. Computer Science, 2022, 49(3): 239-245.
[14] LI Su, SONG Bao-yan, LI Dong, WANG Jun-lu. Composite Blockchain Associated Event Tracing Method for Financial Activities [J]. Computer Science, 2022, 49(3): 346-353.
[15] HUANG Xin-quan, LIU Ai-jun, LIANG Xiao-hu, WANG Heng. Load-balanced Geographic Routing Protocol in Aerial Sensor Network [J]. Computer Science, 2022, 49(2): 342-352.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!