计算机科学 ›› 2022, Vol. 49 ›› Issue (12): 332-339.doi: 10.11896/jsjkx.210900042
许逸铭, 马礼, 傅颖勋, 李阳, 马东超
XU Yi-ming, MA Li, FU Ying-xun, LI Yang, MA Dong-chao
摘要： 针对一体化异构网络中大量终端设备接入网络造成网络流量剧烈波动引发的网络负载均衡问题,提出了一种基于强化学习的智能路由算法TDANRA。TDANRA算法通过软件定义网络技术获取细粒度、高精度的网络流量状态参数,根据网络流量状态与链路带宽利用率阈值调整机制自动生成实时的路由策略,指导网络中流量的转发,从而解决网络流量剧烈波动的问题。仿真实验结果表明,TDANRA算法可以在大量终端设备接入网络的情况下实现网络流量的负载均衡,降低端到端传输时延与数据丢包率。
中图分类号:
| [1]FOOLADIVANDA D,ROSENBERG C.Joint Resource Allocation and User Association for Heterogeneous Wireless Cellular Networks[J].IEEE Transactions on Wireless Communications,2013,12(1):248-257. [2]WANG G,ZHONG Y,HAN T,et al.On the Effect of Spatio-Temporal Fluctuation of Traffic in Wireless Networks[C]//IEEE Global Communications Conference,GLOBECOM 2018.Abu Dhabi,United Arab Emirates:IEEE,2018:1-7. [3]TEFERA G,SHE K,CHEN M,et al.Congestion-aware adaptive decentralised computation offloading and caching for multi-access edge computing networks[J].IET Communications,2020,14(19):3410-3419. [4]YU S,CHEN X,ZHOU Z,et al.When Deep Re-inforcementLearning Meets Federated Learning:Intelligent Multitimescale Resource Management for Multiaccess Edge Computing in 5G Ultradense Network[J].IEEE Internet of Things Journal,2021,8(4):2238-2251. [5]ZORZI M,ZANELLA A,TESTOLIN A,et al.COBANETS:A new paradigm for cognitive communications systems[C]//2016 International Conference on Computing,Networking and Communications,ICNC 2016.Kauai,HI,USA:IEEE Computer So-ciety,2016:1-7. [6]MCKEOWN N,ANDERSON T E,BALAKRISHNAN H,et al.OpenFlow:enabling innovation in campus networks[J].Computer Communication Review,2008,38(2):69-74. [7]WANG X,DENG Q,REN J,et al.The Joint Optimization of Online Traffic Matrix Measurement and Traffic Engineering For Software-Defined Networks[J].IEEE/ACM Transactions on Networking,2020,28(1):234-247. [8]MOSHREF M,YU M,GOVINDAN R,et al.DREAM:dynamic resource allocation for software-defined measurement[C]//Proceedings of the 2014 ACM Conference onSIGCOMM(SIG-COMM ’14).Association for Computing Machinery,2014:419-430. [9]MALBOUBI M,WANG L,CHUAH C N,et al.Intelligent SDN based traffic(de)Aggregation and Measurement Paradigm(iSTAMP)[C]//IEEE Conference on Computer Communications.2014:934-942. [10]AKYILDIZ I F,LEE A,WANG P,et al.Research challenges for traffic engineering in software defined networks[J].IEEE Network,2016,30(3):52-58. [11]NAEEM F,TARIQ M,POOR H V.SDN-Enabled Energy-Efficient Routing Optimization Framework for Industrial Internet of Things[J].IEEE Transactions on Industrial Informatics,2021,17(8):5660-5667. [12]DENG G,WANG K.An Application-aware QoS Routing Algorithm for SDN-based IoT Networking[C]//2018 IEEE Symposium on Computers and Communications,ISCC 2018.Natal,Brazil:IEEE,2018:186-191. [13]TANG F,YANG L T,TANG C,et al.A Dynamical and Load-Balanced Flow Scheduling Approach for Big Data Centers in Clouds[J].IEEE Transactions on Cloud Computing,2018,6(4):915-928. [14]GUO Y,WANG W,ZHANG H,et al.Traffic Engineering inhybrid Software Defined Network via Reinforcement Learning [J].Journal of Network and Computer Applications,2021,189:103116. [15]LIU W X,CAI J,CHEN Q C,et al.DRL-R:Deep reinforcement learning approach for intelligent routing in software-defined data-center networks[J].Journal of Network and Computer Applications,2021,177:102865. [16]YAO H,MAI T,XU X,et al.NetworkAI:An Intelligent Network Architecture for Self-Learning Control Strategies in Software Defined Networks[J].IEEE Internet of Things Journal,2018,5(6):4319-4327. [17]SUN P,GUO Z,LAN J,et al.ScaleDRL:A Scalable Deep Reinforcement Learning Approach for Traffic Engineering in SDN with Pinning Control[J].Computer Networks,2021,190:107891. [18]CHEN Y R,REZAPOUR A,TZENG W G,et al.RL-Routing:An SDN Routing Algorithm Based on Deep Reinforcement Learning[J].IEEE Transactions on Network Science and Engineering,2020,7(4):3185-3199. [19]FUJIMOTO S,HOOF H V,MEGER D.Addressing Function Approximation Error in Actor-Critic Methods[C]//Proceedings of the 35th International Conference on Machine Learning,ICML 2018.2018:1582-1591. [20]SUTTON R S.Learning to Predict by the Methods of Temporal Differences[J].Machine Learning,1988,3:9-44. [21]SILVER D,LEVER G,HEESS N,et al.Deterministic Policy Gradient Algorithms[C]//Proceedings of the 31th International Conference on Machine Learning,ICML 2014.2014,32:387-395. [22]ZHANG J,YE M,GUO Z,et al.CFR-RL:Traffic Engineering With Reinforcement Learning in SDN[J].IEEE Journal on Selected Areas in Communications,2020,38(10):2249-2259. |
| [1] | 熊丽琴, 曹雷, 赖俊, 陈希亮. 基于值分解的多智能体深度强化学习综述 Overview of Multi-agent Deep Reinforcement Learning Based on Value Factorization 计算机科学, 2022, 49(9): 172-182. https://doi.org/10.11896/jsjkx.210800112 |
| [2] | 刘兴光, 周力, 刘琰, 张晓瀛, 谭翔, 魏急波. 基于边缘智能的频谱地图构建与分发方法 Construction and Distribution Method of REM Based on Edge Intelligence 计算机科学, 2022, 49(9): 236-241. https://doi.org/10.11896/jsjkx.220400148 |
| [3] | 袁唯淋, 罗俊仁, 陆丽娜, 陈佳星, 张万鹏, 陈璟. 智能博弈对抗方法:博弈论与强化学习综合视角对比分析 Methods in Adversarial Intelligent Game:A Holistic Comparative Analysis from Perspective of Game Theory and Reinforcement Learning 计算机科学, 2022, 49(8): 191-204. https://doi.org/10.11896/jsjkx.220200174 |
| [4] | 史殿习, 赵琛然, 张耀文, 杨绍武, 张拥军. 基于多智能体强化学习的端到端合作的自适应奖励方法 Adaptive Reward Method for End-to-End Cooperation Based on Multi-agent Reinforcement Learning 计算机科学, 2022, 49(8): 247-256. https://doi.org/10.11896/jsjkx.210700100 |
| [5] | 于滨, 李学华, 潘春雨, 李娜. 基于深度强化学习的边云协同资源分配算法 Edge-Cloud Collaborative Resource Allocation Algorithm Based on Deep Reinforcement Learning 计算机科学, 2022, 49(7): 248-253. https://doi.org/10.11896/jsjkx.210400219 |
| [6] | 李梦菲, 毛莺池, 屠子健, 王瑄, 徐淑芳. 基于深度确定性策略梯度的服务器可靠性任务卸载策略 Server-reliability Task Offloading Strategy Based on Deep Deterministic Policy Gradient 计算机科学, 2022, 49(7): 271-279. https://doi.org/10.11896/jsjkx.210600040 |
| [7] | 田真真, 蒋维, 郑炳旭, 孟利民. 基于服务器集群的负载均衡优化调度算法 Load Balancing Optimization Scheduling Algorithm Based on Server Cluster 计算机科学, 2022, 49(6A): 639-644. https://doi.org/10.11896/jsjkx.210800071 |
| [8] | 谢万城, 李斌, 代玥玥. 空中智能反射面辅助边缘计算中基于PPO的任务卸载方案 PPO Based Task Offloading Scheme in Aerial Reconfigurable Intelligent Surface-assisted Edge Computing 计算机科学, 2022, 49(6): 3-11. https://doi.org/10.11896/jsjkx.220100249 |
| [9] | 洪志理, 赖俊, 曹雷, 陈希亮, 徐志雄. 基于遗憾探索的竞争网络强化学习智能推荐方法研究 Study on Intelligent Recommendation Method of Dueling Network Reinforcement Learning Based on Regret Exploration 计算机科学, 2022, 49(6): 149-157. https://doi.org/10.11896/jsjkx.210600226 |
| [10] | 郭雨欣, 陈秀宏. 融合BERT词嵌入表示和主题信息增强的自动摘要模型 Automatic Summarization Model Combining BERT Word Embedding Representation and Topic Information Enhancement 计算机科学, 2022, 49(6): 313-318. https://doi.org/10.11896/jsjkx.210400101 |
| [11] | 范静宇, 刘全. 基于随机加权三重Q学习的异策略最大熵强化学习算法 Off-policy Maximum Entropy Deep Reinforcement Learning Algorithm Based on RandomlyWeighted Triple Q -Learning 计算机科学, 2022, 49(6): 335-341. https://doi.org/10.11896/jsjkx.210300081 |
| [12] | 张佳能, 李辉, 吴昊霖, 王壮. 一种平衡探索和利用的优先经验回放方法 Exploration and Exploitation Balanced Experience Replay 计算机科学, 2022, 49(5): 179-185. https://doi.org/10.11896/jsjkx.210300084 |
| [13] | 高捷, 刘沙, 黄则强, 郑天宇, 刘鑫, 漆锋滨. 基于国产众核处理器的深度神经网络算子加速库优化 Deep Neural Network Operator Acceleration Library Optimization Based on Domestic Many-core Processor 计算机科学, 2022, 49(5): 355-362. https://doi.org/10.11896/jsjkx.210500226 |
| [14] | 李鹏, 易修文, 齐德康, 段哲文, 李天瑞. 一种基于深度学习的供热策略优化方法 Heating Strategy Optimization Method Based on Deep Learning 计算机科学, 2022, 49(4): 263-268. https://doi.org/10.11896/jsjkx.210300155 |
| [15] | 欧阳卓, 周思源, 吕勇, 谭国平, 张悦, 项亮亮. 基于深度强化学习的无信号灯交叉路口车辆控制 DRL-based Vehicle Control Strategy for Signal-free Intersections 计算机科学, 2022, 49(3): 46-51. https://doi.org/10.11896/jsjkx.210700010 |
|
||
首页