计算机科学 ›› 2021, Vol. 48 ›› Issue (12): 324-330.doi: 10.11896/jsjkx.201100159

• 人工智能 • 上一篇    下一篇

多域SFC部署中基于强化学习的多目标优化方法

王珂1, 曲桦1,2, 赵季红2,3   

  1. 1 西安交通大学软件学院 西安710049
    2 西安交通大学电子与信息工程学院 西安710049
    3 西安邮电大学通信与信息工程学院 西安710061
  • 收稿日期:2020-11-23 修回日期:2021-04-30 出版日期:2021-12-15 发布日期:2021-11-26
  • 通讯作者: 曲桦(qh@mail.xjtu.edu.cn)
  • 作者简介:215545230@qq.com
  • 基金资助:
    国家重点研发计划项目(2018YFB1800305)

Multi-objective Optimization Method Based on Reinforcement Learning in Multi-domain SFC Deployment

WANG Ke1, QU Hua1,2, ZHAO Ji-hong2,3   

  1. 1 School of Software Engineering,Xi'an Jiaotong University,Xi'an 710049,China
    2 School of Electronic and Information Engineering,Xi'an Jiaotong University,Xi'an 710049,China
    3 School of Communication and Information Engineering,Xi'an University of Posts & Telecommunications,Xi'an 710061,China
  • Received:2020-11-23 Revised:2021-04-30 Online:2021-12-15 Published:2021-11-26
  • About author:WANG Ke,born in 1992,Ph.D.His main research interests include software-defined network,network function virtualization and service function chain technology.
    QU Hua,born in 1982,Ph.D,professor,is a member of China Computer Federation.His main research interests include mobile internet,network protocol design,control strategies for supporting emerging applications in ubiquitous networks,and radio resource management in 5G radio communications systems.
  • Supported by:
    National Key R & D Program of China(2018YFB1800305).

摘要: 随着网络虚拟化技术的发展,多域网络中的服务功能链部署为服务功能链优化部署问题带来了新的挑战。传统的部署方法通常对单一目标进行优化,不适用于多目标优化问题,且无法对优化目标间权重进行衡量及平衡。因此,为了对大规模服务功能链部署请求下的时延、网络负载均衡性及接受率进行同步优化,提出了一种数据归一化处理方案,并设计了基于强化学习的两步SFC部署算法。该算法以传输时延与负载均衡性为反馈参数,平衡了两者的权重关系,并对其进行了同步优化,同时利用强化学习框架优化了SFC接受率。实验结果表明,所提算法在大规模请求数下,相比时延感知方法时延降低了71.8%,相比多域部署方法接受率提高了4.6%,相比贪心算法平均负载均衡性提高了39.1%,保证了多目标优化效果。

关键词: 多域, 服务功能链, 数据归一化, 强化学习, 多目标优化

Abstract: With the development of network virtualization technology,the deployment of service function chain in multi-domain network brings new challenges to the optimization of service function chain.The traditional deployment method usually optimizes a single target,which is not suitable for multi-objective optimization,and cannot measure and balance the weight among optimization targets.Therefore,in order to optimize the delay,network load balancing and acceptance rate of large-scale service function chain deployment requests synchronously,a data normalization processing scheme is proposed,and a two-step SFC deployment algorithm based on reinforcement learning is designed.The algorithm takes transmission delay and load balancing as feedback parameters and balances the weight relationship between them,and the SFC acceptance rate is optimized by using reinforcement learning framework simultaneously.The experimental results show that,the delay of the algorithm is reduced by 71.8% compared with LASP method,the acceptance rate is increased by 4.6% compared with MDSP method,and the average load balancing is increased by 39.1% compared with GREEDY method under the large-scale requests.The multi-objective optimization effect is guaranteed.

Key words: Multi-domain, Service function chain, Data normalization, Reinforcement learning, Multi-objective optimization

中图分类号: 

  • TP393
[1]YI B,WANG X W,LIK Q,et al.A comprehensive survey of Network Function Virtualization[J].Computer Networks,2018,133:212-262.
[2]JOSHI K,BENSON T.Network Function Virtualization[J]. IEEE Internet Computing,2016,20(6):7-9.
[3]HALPERN J,PIGNATARO C.Service Function Chaining Ar- chitecture,document RFC 7665 of the IETF Service Function Chaining Working Group[EB/OL].http://datatracker.ietf.org/doc/rfc7665/.
[4]LI Y,CHEN M.Software-defined network function virtualization:a survey[J].IEEE Access,2015,3:2542-2553.
[5]BERNINI G,GIARDINA P G,SPADARO S,et al.Multi-Do- main Orchestration of 5G Vertical Services and Network Slices[C]//2020 IEEE International Conference On Communications Workshops.Dublin,Ireland,2020:6.
[6]WIBOWO F X A,GREGORY M A,AHMED K,et al.Multi-domain Software Defined Networking:Research status and challenges[J].Journal of Network and Computer Applications,2017,87:32-45.
[7]CHEN W H,YIN X,WANG Z L,et al.Placement and Routing Optimization Problem for Service Function Chain:State of Art and Future Opportunities[J].arXiv:1910.02613.
[8]QU L,ASSI C,SHABAN K.Delay-Aware Scheduling and Resource Optimization With Network Function Virtualization[J].IEEE Transactions on Communications,2016,64(9):3746-3758.
[9]ALAMEDDINE H A,QU L,ASSI C.Scheduling Service Function Chains for Ultra-Low Latency Network Services[C]//13th International Conference on Network and Service Management.Tokyo,Japan,2017:9.
[10]SUN G,LI Y Y,LI Y,et al.Low-latency orchestration for workflow-oriented service function chain in edge computing[J].Future Generation Computer Systems-the International Journal of Science,2018,85:116-128.
[11]GOUAREB R,FRIDERIKOS V,AGHVAMI A H.Virtual Network Functions Routing and Placement for Edge Cloud Latency Minimization[J].IEEE Journal on Selected Areas in Communications,2018,36(10):2346-2357.
[12]YE Q,ZHUANG W H,LI X,et al.End-to-End Delay Modeling for Embedded VNF Chains in 5G Core Networks[J].IEEE Internet of Things Journal,2019,6(1):692-704.
[13]MIJUMBI R,SERRAT J,GORRICHO J L,et al.Design and evaluation of algorithms for mapping and scheduling of virtual network functions[C]//2015 1st IEEE Conference on Network Softwarization.London,UK,2015:9.
[14]ALLEG A,AHMED T,MOSBAH M,et al.Delay-aware VNF placement and chaining based on a flexible resource allocation approach[C]//2017 13th International Conference on Network and Service Management.Tokyo,Japan,2017:7.
[15]SHI Z,WU Z H,ZENG Y.A Method of Service Function Chain Arrangement for Load Balancing[C]//9th International Confe-rence on Computer Engineering and Networks.Changsha,China,2019:35-42.
[16]HAN H Y,MENG X R,YU Z H,et al.A Service Function Chain Deployment Method Based on Network Flow Theory for
Load Balance in Operator Networks[J].IEEE Access,2020,8:93187-93199.
[17]XIANG Y F,WU M,WU J,et al.A Load Balancing Method of Virtualization Service Function Chain Based on Time-varying Graphs Integration[J].Journal of Fujian Normal University(Natural Science Edition),2018,34(3):14-20.
[18]SUN G,LI Y,LIAO D,et al.Service Function Chain Orchestration Across Multiple Domains:A Full Mesh Aggregation Approach[J].IEEE Transactions on Network and Service Management,2018,15(3):1175-1191.
[19]XU Q,GAO D Y,LI TX,et al.Low Latency Security Function Chain Embedding Across Multiple Domains[J].IEEE Access,2018,6:14474-14484.
[20]LI G L,ZHOU H C,FENG B H,et al.Context-Aware Service Function Chaining and Its Cost-Effective Orchestration in Multi-Domain Networks[J].IEEE Access,2018,6:34976-34991.
[21]DIETRICH D,ABUJODA A,RIZK A,et al.Multi-Provider Service Chain Embedding With Nestor[J].IEEE Transactions on Network And Service Management,2017,14(1):91-105.
[22]ABUJODA A,PAPADIMITRIOU P.DistNSE:Distributed Network Service Embedding Across Multiple Providers[C]//8th International Conference on Communication Systems And Networks.Bangalore,India,2016:8.
[23]ZHANG C,WANG X W,LI F W,et al.Network Service Chains Deployment Across Multiple SDN Domains[J].International Journal of Communication Systems,2018,31(18):e3826.1-e3826.25.
[24]KAUR K,GARG S,KADDOUM G,et al.An Energy-driven Network Function Virtualization for Multi-domain Software Defined Networks[C]//IEEE Conference on Computer Communications.Paris,France,2019:121-126.
[25]ZHU G H,LI Q,LIANG S L.Cross-domain mapping algorithm of service function chain based on deep reinforcement learning[J].Application Research of Computers,2021,38(6):1834-1837,1842.
[1] 代珊珊, 刘全. 基于动作约束深度强化学习的安全自动驾驶方法[J]. 计算机科学, 2021, 48(9): 235-243.
[2] 吴少波, 傅启明, 陈建平, 吴宏杰, 陆悠. 基于相对熵的元逆强化学习方法[J]. 计算机科学, 2021, 48(9): 257-263.
[3] 成昭炜, 沈航, 汪悦, 王敏, 白光伟. 基于深度强化学习的无人机辅助弹性视频多播机制[J]. 计算机科学, 2021, 48(9): 271-277.
[4] 周仕承, 刘京菊, 钟晓峰, 卢灿举. 基于深度强化学习的智能化渗透测试路径发现[J]. 计算机科学, 2021, 48(7): 40-46.
[5] 李贝贝, 宋佳芮, 杜卿芸, 何俊江. DRL-IDS:基于深度强化学习的工业物联网入侵检测系统[J]. 计算机科学, 2021, 48(7): 47-54.
[6] 梁俊斌, 张海涵, 蒋婵, 王天舒. 移动边缘计算中基于深度强化学习的任务卸载研究进展[J]. 计算机科学, 2021, 48(7): 316-323.
[7] 王英恺, 王青山. 能量收集无线通信系统中基于强化学习的能量分配策略[J]. 计算机科学, 2021, 48(7): 333-339.
[8] 胡潇炜, 陈羽中. 一种结合自编码器与强化学习的查询推荐方法[J]. 计算机科学, 2021, 48(6A): 206-212.
[9] 陆嘉猷, 凌兴宏, 刘全, 朱斐. 基于自适应调节策略熵的元强化学习算法[J]. 计算机科学, 2021, 48(6): 168-174.
[10] 范家宽, 王皓月, 赵生宇, 周添一, 王伟. 数据驱动的开源贡献度量化评估与持续优化方法[J]. 计算机科学, 2021, 48(5): 45-50.
[11] 范艳芳, 袁爽, 蔡英, 陈若愚. 车载边缘计算中基于深度强化学习的协同计算卸载方案[J]. 计算机科学, 2021, 48(5): 270-276.
[12] 潘瑞杰, 王高才, 黄珩逸. 云计算下基于动态用户信任度的属性访问控制[J]. 计算机科学, 2021, 48(5): 313-319.
[13] 黄志勇, 吴昊霖, 王壮, 李辉. 基于平均神经网络参数的DQN算法[J]. 计算机科学, 2021, 48(4): 223-228.
[14] 李丽, 郑嘉利, 罗文聪, 全艺璇. 基于近端策略优化的RFID室内定位算法[J]. 计算机科学, 2021, 48(4): 274-281.
[15] 秦智慧, 李宁, 刘晓彤, 刘秀磊, 佟强, 刘旭红. 无模型强化学习研究综述[J]. 计算机科学, 2021, 48(3): 180-187.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] 王焕文,徐晓刚,徐冠雷,王孝通. 基于阴影不一致的简易人像篡改鉴别[J]. 计算机科学, 2014, 41(Z6): 129 -131 .
[2] 易军,许磊. 基于熵权的传感反应网络任务分派算法[J]. 计算机科学, 2011, 38(6): 106 -109 .
[3] . 目录[J]. 计算机科学, 2021, 48(5): 0 .
[4] 潘孝勤, 芦天亮, 杜彦辉, 仝鑫. 基于深度学习的语音合成与转换技术综述[J]. 计算机科学, 2021, 48(8): 200 -208 .
[5] 王俊, 王修来, 庞威, 赵鸿飞. 面向科技前瞻预测的大数据治理研究[J]. 计算机科学, 2021, 48(9): 36 -42 .
[6] 余力, 杜启翰, 岳博妍, 向君瑶, 徐冠宇, 冷友方. 基于强化学习的推荐研究综述[J]. 计算机科学, 2021, 48(10): 1 -18 .
[7] 王梓强, 胡晓光, 李晓筱, 杜卓群. 移动机器人全局路径规划算法综述[J]. 计算机科学, 2021, 48(10): 19 -29 .
[8] 高洪皓, 郑子彬, 殷昱煜, 丁勇. 区块链技术专题序言[J]. 计算机科学, 2021, 48(11): 1 -3 .
[9] 毛瀚宇, 聂铁铮, 申德荣, 于戈, 徐石成, 何光宇. 区块链即服务平台关键技术及发展综述[J]. 计算机科学, 2021, 48(11): 4 -11 .
[10] 杨青, 张亚文, 朱丽, 吴涛. 基于注意力机制和BiGRU融合的文本情感分析[J]. 计算机科学, 2021, 48(11): 307 -311 .