Computer Science

Computer Science ›› 2021, Vol. 48 ›› Issue (12): 324-330.doi: 10.11896/jsjkx.201100159

• Artificial Intelligence • Previous Articles     Next Articles

Multi-objective Optimization Method Based on Reinforcement Learning in Multi-domain SFC Deployment

WANG Ke1, QU Hua1,2, ZHAO Ji-hong2,3   

  1. 1 School of Software Engineering,Xi'an Jiaotong University,Xi'an 710049,China
    2 School of Electronic and Information Engineering,Xi'an Jiaotong University,Xi'an 710049,China
    3 School of Communication and Information Engineering,Xi'an University of Posts & Telecommunications,Xi'an 710061,China
  • Received:2020-11-23 Revised:2021-04-30 Online:2021-12-15 Published:2021-11-26
  • About author:WANG Ke,born in 1992,Ph.D.His main research interests include software-defined network,network function virtualization and service function chain technology.
    QU Hua,born in 1982,Ph.D,professor,is a member of China Computer Federation.His main research interests include mobile internet,network protocol design,control strategies for supporting emerging applications in ubiquitous networks,and radio resource management in 5G radio communications systems.
  • Supported by:
    National Key R & D Program of China(2018YFB1800305).

PDF (PC)

909

Abstract: With the development of network virtualization technology,the deployment of service function chain in multi-domain network brings new challenges to the optimization of service function chain.The traditional deployment method usually optimizes a single target,which is not suitable for multi-objective optimization,and cannot measure and balance the weight among optimization targets.Therefore,in order to optimize the delay,network load balancing and acceptance rate of large-scale service function chain deployment requests synchronously,a data normalization processing scheme is proposed,and a two-step SFC deployment algorithm based on reinforcement learning is designed.The algorithm takes transmission delay and load balancing as feedback parameters and balances the weight relationship between them,and the SFC acceptance rate is optimized by using reinforcement learning framework simultaneously.The experimental results show that,the delay of the algorithm is reduced by 71.8% compared with LASP method,the acceptance rate is increased by 4.6% compared with MDSP method,and the average load balancing is increased by 39.1% compared with GREEDY method under the large-scale requests.The multi-objective optimization effect is guaranteed.

Key words: Data normalization, Multi-domain, Multi-objective optimization, Reinforcement learning, Service function chain

CLC Number: 

  • TP393
[1]YI B,WANG X W,LIK Q,et al.A comprehensive survey of Network Function Virtualization[J].Computer Networks,2018,133:212-262.
[2]JOSHI K,BENSON T.Network Function Virtualization[J]. IEEE Internet Computing,2016,20(6):7-9.
[3]HALPERN J,PIGNATARO C.Service Function Chaining Ar- chitecture,document RFC 7665 of the IETF Service Function Chaining Working Group[EB/OL].http://datatracker.ietf.org/doc/rfc7665/.
[4]LI Y,CHEN M.Software-defined network function virtualization:a survey[J].IEEE Access,2015,3:2542-2553.
[5]BERNINI G,GIARDINA P G,SPADARO S,et al.Multi-Do- main Orchestration of 5G Vertical Services and Network Slices[C]//2020 IEEE International Conference On Communications Workshops.Dublin,Ireland,2020:6.
[6]WIBOWO F X A,GREGORY M A,AHMED K,et al.Multi-domain Software Defined Networking:Research status and challenges[J].Journal of Network and Computer Applications,2017,87:32-45.
[7]CHEN W H,YIN X,WANG Z L,et al.Placement and Routing Optimization Problem for Service Function Chain:State of Art and Future Opportunities[J].arXiv:1910.02613.
[8]QU L,ASSI C,SHABAN K.Delay-Aware Scheduling and Resource Optimization With Network Function Virtualization[J].IEEE Transactions on Communications,2016,64(9):3746-3758.
[9]ALAMEDDINE H A,QU L,ASSI C.Scheduling Service Function Chains for Ultra-Low Latency Network Services[C]//13th International Conference on Network and Service Management.Tokyo,Japan,2017:9.
[10]SUN G,LI Y Y,LI Y,et al.Low-latency orchestration for workflow-oriented service function chain in edge computing[J].Future Generation Computer Systems-the International Journal of Science,2018,85:116-128.
[11]GOUAREB R,FRIDERIKOS V,AGHVAMI A H.Virtual Network Functions Routing and Placement for Edge Cloud Latency Minimization[J].IEEE Journal on Selected Areas in Communications,2018,36(10):2346-2357.
[12]YE Q,ZHUANG W H,LI X,et al.End-to-End Delay Modeling for Embedded VNF Chains in 5G Core Networks[J].IEEE Internet of Things Journal,2019,6(1):692-704.
[13]MIJUMBI R,SERRAT J,GORRICHO J L,et al.Design and evaluation of algorithms for mapping and scheduling of virtual network functions[C]//2015 1st IEEE Conference on Network Softwarization.London,UK,2015:9.
[14]ALLEG A,AHMED T,MOSBAH M,et al.Delay-aware VNF placement and chaining based on a flexible resource allocation approach[C]//2017 13th International Conference on Network and Service Management.Tokyo,Japan,2017:7.
[15]SHI Z,WU Z H,ZENG Y.A Method of Service Function Chain Arrangement for Load Balancing[C]//9th International Confe-rence on Computer Engineering and Networks.Changsha,China,2019:35-42.
[16]HAN H Y,MENG X R,YU Z H,et al.A Service Function Chain Deployment Method Based on Network Flow Theory for
Load Balance in Operator Networks[J].IEEE Access,2020,8:93187-93199.
[17]XIANG Y F,WU M,WU J,et al.A Load Balancing Method of Virtualization Service Function Chain Based on Time-varying Graphs Integration[J].Journal of Fujian Normal University(Natural Science Edition),2018,34(3):14-20.
[18]SUN G,LI Y,LIAO D,et al.Service Function Chain Orchestration Across Multiple Domains:A Full Mesh Aggregation Approach[J].IEEE Transactions on Network and Service Management,2018,15(3):1175-1191.
[19]XU Q,GAO D Y,LI TX,et al.Low Latency Security Function Chain Embedding Across Multiple Domains[J].IEEE Access,2018,6:14474-14484.
[20]LI G L,ZHOU H C,FENG B H,et al.Context-Aware Service Function Chaining and Its Cost-Effective Orchestration in Multi-Domain Networks[J].IEEE Access,2018,6:34976-34991.
[21]DIETRICH D,ABUJODA A,RIZK A,et al.Multi-Provider Service Chain Embedding With Nestor[J].IEEE Transactions on Network And Service Management,2017,14(1):91-105.
[22]ABUJODA A,PAPADIMITRIOU P.DistNSE:Distributed Network Service Embedding Across Multiple Providers[C]//8th International Conference on Communication Systems And Networks.Bangalore,India,2016:8.
[23]ZHANG C,WANG X W,LI F W,et al.Network Service Chains Deployment Across Multiple SDN Domains[J].International Journal of Communication Systems,2018,31(18):e3826.1-e3826.25.
[24]KAUR K,GARG S,KADDOUM G,et al.An Energy-driven Network Function Virtualization for Multi-domain Software Defined Networks[C]//IEEE Conference on Computer Communications.Paris,France,2019:121-126.
[25]ZHU G H,LI Q,LIANG S L.Cross-domain mapping algorithm of service function chain based on deep reinforcement learning[J].Application Research of Computers,2021,38(6):1834-1837,1842.
[1] LIU Xing-guang, ZHOU Li, LIU Yan, ZHANG Xiao-ying, TAN Xiang, WEI Ji-bo. Construction and Distribution Method of REM Based on Edge Intelligence [J]. Computer Science, 2022, 49(9): 236-241.
[2] YUAN Wei-lin, LUO Jun-ren, LU Li-na, CHEN Jia-xing, ZHANG Wan-peng, CHEN Jing. Methods in Adversarial Intelligent Game:A Holistic Comparative Analysis from Perspective of Game Theory and Reinforcement Learning [J]. Computer Science, 2022, 49(8): 191-204.
[3] SHI Dian-xi, ZHAO Chen-ran, ZHANG Yao-wen, YANG Shao-wu, ZHANG Yong-jun. Adaptive Reward Method for End-to-End Cooperation Based on Multi-agent Reinforcement Learning [J]. Computer Science, 2022, 49(8): 247-256.
[4] YU Bin, LI Xue-hua, PAN Chun-yu, LI Na. Edge-Cloud Collaborative Resource Allocation Algorithm Based on Deep Reinforcement Learning [J]. Computer Science, 2022, 49(7): 248-253.
[5] LI Meng-fei, MAO Ying-chi, TU Zi-jian, WANG Xuan, XU Shu-fang. Server-reliability Task Offloading Strategy Based on Deep Deterministic Policy Gradient [J]. Computer Science, 2022, 49(7): 271-279.
[6] XIE Wan-cheng, LI Bin, DAI Yue-yue. PPO Based Task Offloading Scheme in Aerial Reconfigurable Intelligent Surface-assisted Edge Computing [J]. Computer Science, 2022, 49(6): 3-11.
[7] HONG Zhi-li, LAI Jun, CAO Lei, CHEN Xi-liang, XU Zhi-xiong. Study on Intelligent Recommendation Method of Dueling Network Reinforcement Learning Based on Regret Exploration [J]. Computer Science, 2022, 49(6): 149-157.
[8] SUN Gang, WU Jiang-jiang, CHEN Hao, LI Jun, XU Shi-yuan. Hidden Preference-based Multi-objective Evolutionary Algorithm Based on Chebyshev Distance [J]. Computer Science, 2022, 49(6): 297-304.
[9] GUO Yu-xin, CHEN Xiu-hong. Automatic Summarization Model Combining BERT Word Embedding Representation and Topic Information Enhancement [J]. Computer Science, 2022, 49(6): 313-318.
[10] FAN Jing-yu, LIU Quan. Off-policy Maximum Entropy Deep Reinforcement Learning Algorithm Based on RandomlyWeighted Triple Q -Learning [J]. Computer Science, 2022, 49(6): 335-341.
[11] ZHANG Jia-neng, LI Hui, WU Hao-lin, WANG Zhuang. Exploration and Exploitation Balanced Experience Replay [J]. Computer Science, 2022, 49(5): 179-185.
[12] LI Hao-dong, HU Jie, FAN Qin-qin. Multimodal Multi-objective Optimization Based on Parallel Zoning Search and Its Application [J]. Computer Science, 2022, 49(5): 212-220.
[13] LI Peng, YI Xiu-wen, QI De-kang, DUAN Zhe-wen, LI Tian-rui. Heating Strategy Optimization Method Based on Deep Learning [J]. Computer Science, 2022, 49(4): 263-268.
[14] PENG Dong-yang, WANG Rui, HU Gu-yu, ZU Jia-chen, WANG Tian-feng. Fair Joint Optimization of QoE and Energy Efficiency in Caching Strategy for Videos [J]. Computer Science, 2022, 49(4): 312-320.
[15] OUYANG Zhuo, ZHOU Si-yuan, LYU Yong, TAN Guo-ping, ZHANG Yue, XIANG Liang-liang. DRL-based Vehicle Control Strategy for Signal-free Intersections [J]. Computer Science, 2022, 49(3): 46-51.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.