Computer Science

Computer Science ›› 2022, Vol. 49 ›› Issue (12): 332-339.doi: 10.11896/jsjkx.210900042

• Computer Network • Previous Articles     Next Articles

Intelligent Routing Technology for Multi-terminal Access in Integrated Network

XU Yi-ming, MA Li, FU Ying-xun, LI Yang, MA Dong-chao   

  1. School of Information Science and Technology,North China University of Technology,Beijing 100144,China
  • Received:2021-09-06 Revised:2022-05-12 Published:2022-12-14
  • About author:XU Yi-ming,born in 1996,postgra-duate.His main research interests include software defined network and reinforcement learning.MA Li,born in 1968,Ph.D,professor,is a distinguished member of China Computer Federation.His main research interests include advanced computing technology,distributed computing,multi-agent system and IoT system.
  • Supported by:
    National Key Research and Development Program of China(2018YFB1800302),National Natural Science Foundation of China(62001007) and Natural Science Foundation of Beijing,China(KZ201810009011,4202020,19L2021).

PDF (PC)

294

Abstract: Aiming at the problem of network load balancing caused by the drastic fluctuation of network traffic caused by the access of a large number of terminal devices in the integrated heterogeneous network,an intelligent routing algorithm TDANRA based on reinforcement learning is proposed.Fine-grained and high-precision network traffic status parameters are obtained by software-defined network technology,TDANRA algorithm automatically generates real-time routing policies based on network traffic status and link bandwidth utilization threshold adjustment mechanism to guide the forwarding of network traffic,so as to solve the problem of drastic fluctuation of network traffic.Simulation results show that TDANRA algorithm can realize load ba-lancing of network traffic and reduce end-to-end transmission delay and data packet loss rate when a large number of terminal devices are connected to the network.

Key words: Software defined network, Reinforcement learning, Load balancing, Intelligent routing, Link bandwidth usage thres-hold adjustment mechanism

CLC Number: 

  • TP393
[1]FOOLADIVANDA D,ROSENBERG C.Joint Resource Allocation and User Association for Heterogeneous Wireless Cellular Networks[J].IEEE Transactions on Wireless Communications,2013,12(1):248-257.
[2]WANG G,ZHONG Y,HAN T,et al.On the Effect of Spatio-Temporal Fluctuation of Traffic in Wireless Networks[C]//IEEE Global Communications Conference,GLOBECOM 2018.Abu Dhabi,United Arab Emirates:IEEE,2018:1-7.
[3]TEFERA G,SHE K,CHEN M,et al.Congestion-aware adaptive decentralised computation offloading and caching for multi-access edge computing networks[J].IET Communications,2020,14(19):3410-3419.
[4]YU S,CHEN X,ZHOU Z,et al.When Deep Re-inforcementLearning Meets Federated Learning:Intelligent Multitimescale Resource Management for Multiaccess Edge Computing in 5G Ultradense Network[J].IEEE Internet of Things Journal,2021,8(4):2238-2251.
[5]ZORZI M,ZANELLA A,TESTOLIN A,et al.COBANETS:A new paradigm for cognitive communications systems[C]//2016 International Conference on Computing,Networking and Communications,ICNC 2016.Kauai,HI,USA:IEEE Computer So-ciety,2016:1-7.
[6]MCKEOWN N,ANDERSON T E,BALAKRISHNAN H,et al.OpenFlow:enabling innovation in campus networks[J].Computer Communication Review,2008,38(2):69-74.
[7]WANG X,DENG Q,REN J,et al.The Joint Optimization of Online Traffic Matrix Measurement and Traffic Engineering For Software-Defined Networks[J].IEEE/ACM Transactions on Networking,2020,28(1):234-247.
[8]MOSHREF M,YU M,GOVINDAN R,et al.DREAM:dynamic resource allocation for software-defined measurement[C]//Proceedings of the 2014 ACM Conference onSIGCOMM(SIG-COMM ’14).Association for Computing Machinery,2014:419-430.
[9]MALBOUBI M,WANG L,CHUAH C N,et al.Intelligent SDN based traffic(de)Aggregation and Measurement Paradigm(iSTAMP)[C]//IEEE Conference on Computer Communications.2014:934-942.
[10]AKYILDIZ I F,LEE A,WANG P,et al.Research challenges for traffic engineering in software defined networks[J].IEEE Network,2016,30(3):52-58.
[11]NAEEM F,TARIQ M,POOR H V.SDN-Enabled Energy-Efficient Routing Optimization Framework for Industrial Internet of Things[J].IEEE Transactions on Industrial Informatics,2021,17(8):5660-5667.
[12]DENG G,WANG K.An Application-aware QoS Routing Algorithm for SDN-based IoT Networking[C]//2018 IEEE Symposium on Computers and Communications,ISCC 2018.Natal,Brazil:IEEE,2018:186-191.
[13]TANG F,YANG L T,TANG C,et al.A Dynamical and Load-Balanced Flow Scheduling Approach for Big Data Centers in Clouds[J].IEEE Transactions on Cloud Computing,2018,6(4):915-928.
[14]GUO Y,WANG W,ZHANG H,et al.Traffic Engineering inhybrid Software Defined Network via Reinforcement Learning [J].Journal of Network and Computer Applications,2021,189:103116.
[15]LIU W X,CAI J,CHEN Q C,et al.DRL-R:Deep reinforcement learning approach for intelligent routing in software-defined data-center networks[J].Journal of Network and Computer Applications,2021,177:102865.
[16]YAO H,MAI T,XU X,et al.NetworkAI:An Intelligent Network Architecture for Self-Learning Control Strategies in Software Defined Networks[J].IEEE Internet of Things Journal,2018,5(6):4319-4327.
[17]SUN P,GUO Z,LAN J,et al.ScaleDRL:A Scalable Deep Reinforcement Learning Approach for Traffic Engineering in SDN with Pinning Control[J].Computer Networks,2021,190:107891.
[18]CHEN Y R,REZAPOUR A,TZENG W G,et al.RL-Routing:An SDN Routing Algorithm Based on Deep Reinforcement Learning[J].IEEE Transactions on Network Science and Engineering,2020,7(4):3185-3199.
[19]FUJIMOTO S,HOOF H V,MEGER D.Addressing Function Approximation Error in Actor-Critic Methods[C]//Proceedings of the 35th International Conference on Machine Learning,ICML 2018.2018:1582-1591.
[20]SUTTON R S.Learning to Predict by the Methods of Temporal Differences[J].Machine Learning,1988,3:9-44.
[21]SILVER D,LEVER G,HEESS N,et al.Deterministic Policy Gradient Algorithms[C]//Proceedings of the 31th International Conference on Machine Learning,ICML 2014.2014,32:387-395.
[22]ZHANG J,YE M,GUO Z,et al.CFR-RL:Traffic Engineering With Reinforcement Learning in SDN[J].IEEE Journal on Selected Areas in Communications,2020,38(10):2249-2259.
[1] LIU Xing-guang, ZHOU Li, LIU Yan, ZHANG Xiao-ying, TAN Xiang, WEI Ji-bo. Construction and Distribution Method of REM Based on Edge Intelligence [J]. Computer Science, 2022, 49(9): 236-241.
[2] YUAN Wei-lin, LUO Jun-ren, LU Li-na, CHEN Jia-xing, ZHANG Wan-peng, CHEN Jing. Methods in Adversarial Intelligent Game:A Holistic Comparative Analysis from Perspective of Game Theory and Reinforcement Learning [J]. Computer Science, 2022, 49(8): 191-204.
[3] SHI Dian-xi, ZHAO Chen-ran, ZHANG Yao-wen, YANG Shao-wu, ZHANG Yong-jun. Adaptive Reward Method for End-to-End Cooperation Based on Multi-agent Reinforcement Learning [J]. Computer Science, 2022, 49(8): 247-256.
[4] YU Bin, LI Xue-hua, PAN Chun-yu, LI Na. Edge-Cloud Collaborative Resource Allocation Algorithm Based on Deep Reinforcement Learning [J]. Computer Science, 2022, 49(7): 248-253.
[5] LI Meng-fei, MAO Ying-chi, TU Zi-jian, WANG Xuan, XU Shu-fang. Server-reliability Task Offloading Strategy Based on Deep Deterministic Policy Gradient [J]. Computer Science, 2022, 49(7): 271-279.
[6] TIAN Zhen-zhen, JIANG Wei, ZHENG Bing-xu, MENG Li-min. Load Balancing Optimization Scheduling Algorithm Based on Server Cluster [J]. Computer Science, 2022, 49(6A): 639-644.
[7] XIE Wan-cheng, LI Bin, DAI Yue-yue. PPO Based Task Offloading Scheme in Aerial Reconfigurable Intelligent Surface-assisted Edge Computing [J]. Computer Science, 2022, 49(6): 3-11.
[8] HONG Zhi-li, LAI Jun, CAO Lei, CHEN Xi-liang, XU Zhi-xiong. Study on Intelligent Recommendation Method of Dueling Network Reinforcement Learning Based on Regret Exploration [J]. Computer Science, 2022, 49(6): 149-157.
[9] GUO Yu-xin, CHEN Xiu-hong. Automatic Summarization Model Combining BERT Word Embedding Representation and Topic Information Enhancement [J]. Computer Science, 2022, 49(6): 313-318.
[10] FAN Jing-yu, LIU Quan. Off-policy Maximum Entropy Deep Reinforcement Learning Algorithm Based on RandomlyWeighted Triple Q -Learning [J]. Computer Science, 2022, 49(6): 335-341.
[11] ZHANG Jia-neng, LI Hui, WU Hao-lin, WANG Zhuang. Exploration and Exploitation Balanced Experience Replay [J]. Computer Science, 2022, 49(5): 179-185.
[12] GAO Jie, LIU Sha, HUANG Ze-qiang, ZHENG Tian-yu, LIU Xin, QI Feng-bin. Deep Neural Network Operator Acceleration Library Optimization Based on Domestic Many-core Processor [J]. Computer Science, 2022, 49(5): 355-362.
[13] LI Peng, YI Xiu-wen, QI De-kang, DUAN Zhe-wen, LI Tian-rui. Heating Strategy Optimization Method Based on Deep Learning [J]. Computer Science, 2022, 49(4): 263-268.
[14] OUYANG Zhuo, ZHOU Si-yuan, LYU Yong, TAN Guo-ping, ZHANG Yue, XIANG Liang-liang. DRL-based Vehicle Control Strategy for Signal-free Intersections [J]. Computer Science, 2022, 49(3): 46-51.
[15] ZHOU Qin, LUO Fei, DING Wei-chao, GU Chun-hua, ZHENG Shuai. Double Speedy Q-Learning Based on Successive Over Relaxation [J]. Computer Science, 2022, 49(3): 239-245.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.