计算机科学

计算机科学 ›› 2025, Vol. 52 ›› Issue (10): 404-411.doi: 10.11896/jsjkx.240800015

• 信息安全 • 上一篇    下一篇

基于深度强化学习的安全感知服务功能链部署方法

朱子怡1, 张建辉1,2, 曾俊杰1, 张洪源1   

  1. 1 郑州大学网络空间安全学院 郑州 450000
    2 嵩山实验室 郑州 450000
  • 收稿日期:2024-08-02 修回日期:2024-11-10 出版日期:2025-10-15 发布日期:2025-10-14
  • 通讯作者: 张建辉(ndsczjh@163.com)
  • 作者简介:(zhuziyi@gs.zzu.edu.cn)
  • 基金资助:
    国家重点研发计划(2022YFB2901304);河南省重大科技专项(221100210900)

Security-aware Service Function Chain Deployment Method Based on Deep ReinforcementLearning

ZHU Ziyi1, ZHANG Jianhui1,2, ZENG Junjie1 and ZHANG Hongyuan1   

  1. 1 College of Cyberspace Security,Zhengzhou University,Zhengzhou 450000,China
    2 Songshan Laboratory,Zhengzhou 450000,China
  • Received:2024-08-02 Revised:2024-11-10 Online:2025-10-15 Published:2025-10-14
  • About author:ZHU Ziyi,born in 2001,postgraduate.Her main research interests include cyberspace security and service function chain orchestration.
    ZHANG Jianhui,born in 1977,Ph.D,associate researcher,master supervisor.His main research interests include new network architecture,network routing technology,network data analysis and security control.
  • Supported by:
    National Key Research and Development Program of China(2022YFB2901304) and Major Science and Technology Program of Henan Province(221100210900).

PDF (PC)

2

摘要： 服务功能链作为提升网络资源利用率的关键技术,其结合深度强化学习能够实现灵活且安全的部署。然而,如何有效地部署具有安全需求的服务功能链,同时最大化长期平均收益是服务功能链面临的一个重要的挑战。为此,提出了一种基于深度强化学习的安全感知服务功能链部署方法(DRL-SASFCD)。首先,提出了一种安全感知机制,用于评估物理网络节点的可信度,并引入安全需求指数感知SFC对安全性的需求。其次,利用图注意力网络和序列到序列模型提取底层物理网络信息以及服务功能链请求序列信息的相关特征,并依据这些特征生成服务功能链部署策略。最后,采用近端策略的优化方法来优化策略和训练网络参数,通过限制新旧策略之间的更新幅度,避免策略更新过程中的剧烈波动,从而提高安全策略优化效率。仿真实验结果表明,DRL-SASFCD在考虑服务功能链部署安全需求的同时,与现有方法相比,在部署接受率、长期平均收益以及长期平均收益成本比3个方面均有所提高。

关键词: 服务功能链, 虚拟网络功能, 深度强化学习, 安全, 部署收益

Abstract: As a key technology to improve the utilization of network resources,service function chain combined with deep reinforcement learning makes it possible to achieve flexible and secure deployment.However,how to effectively deploy service function chains with security requirements while maximizing long-term average revenue is an important challenge it faces.This paper proposes a deployment method for security-aware service function chain based on deep reinforcement learning(DRL-SASFCD).Firstly,a security-aware mechanism is proposed to evaluate the credibility of physical network nodes,and a security requirement index is introduced to perceive the security requirements of SFC.Secondly,this method utilizes graph attention network and sequence to sequence models to extract relevant features of underlying physical network information and service function chain request sequence information.It generates service function chain deployment strategies based on these features.Finally,the proximal policy optimization method is adopted to optimize the policy and training network parameters.By limiting the update amplitude between the new and old policies,the drastic fluctuations during the policy update process are avoided,thereby improving the efficiency of security policy optimization.The simulation results show that DRL-SASFCD can improve the deployment acceptance rate,long-term average revenue and long-term average revenue-cost ratio compared with the existing methods while considering the security requirements of service function chain deployment.

Key words: Service function chain,Virtual network function,Deep reinforcement learning,Security,Deployment revenue

中图分类号: 

  • TP393
[1]YANG S,LI F,TRAJANOVSKI S,et al.Recent advances of resource allocation in network function virtualization[J].IEEE Transactions on Parallel and Distributed Systems,2020,32(2):295-314.
[2]MATENCIO E A,WANG Q,CALERO J M A.SliceNetVS-witch:Definition,design and implementation of 5G multi-tenant network slicing in software data paths[J].IEEE Transactions on Network and Service Management,2020,17(4):2212-2225.
[3]TANG L,WANG K,ZHANG Y,et al.Service function chainanomaly detection based on distributed generative adversarial network in network slicing scenario[J] Journal of Electronics & Information Technology,2023,45(1):262-271.
[4]RUI L L,CHEN S Y,WANG S Y,et al.SFC Orchestration Method for Edge Cloud and Central Cloud Collaboration:QoS and Energy Consumption Joint Optimization Combined With Reputation Assessment[J].IEEE Transactions on Parallel and Distributed Systems,2023,34(10):2735-2748.
[5]COELHO R W,LEONARDO E J,MARTIMIANO L A F,et al.A survey of the characteristics of SDN,NFV and information security in IoT and 5G networks[J].Revista Brasileira de Computação Aplicada,2023,15(3):96-105.
[6]LI B,CHENG B,LIU X,et al.Joint Resource Optimization and Delay-Aware Virtual Network Function Migration in Data Center Networks [J].IEEE Transactions on Network and Service Management,2021,18(3):2960-2974.
[7]ZHANG Q X,XIAO Y K,LIU F,et al.Joint Optimization ofChain Placement and Request Scheduling for Network Function Virtualization[C]//2017 IEEE 37th International Conference on Distributed Computing Systems(ICDCS).IEEE,2017:731-741.
[8]BARI F,CHOWDHURY S R,AHMED R,et al.Orchestrating virtualized network functions[J].IEEE Transactions on Network and Service Management,2016,13(4):725-739.
[9]HUANG H,JIANG J,YANG Y K,et al.Online Service Function Chain Orchestration Method for Profit Maximization[J].Computer Science,2023,50(6):66-73.
[10]LIU H T,DING S D,WANG S Y,et al.Multi-objective optimization service function chain placement algorithm based on reinforcement learning[J].Journal of Network and Systems Ma-nagement,2022,30(4):58-83.
[11]HUANG Z W,ZHONG W J,LI D G,et al.Delay Constrained SFC Orchestration for Edge Intelligence-Enabled IIoT:A DRL Approach[J].Journal of Network and Systems Management,2023,31(3):53-79.
[12]XU H S,FAN G L,SUN L B,et al.Dynamic SFC placementscheme with parallelized SFCs and reuse of initialized VNFs:An A3C-based DRL approach[J].Journal of King Saud University-Computer and Information Sciences,2023,35(6):101577.
[13]JEONG E D,YOO J H,HONG J W K.SFC Consolidation:Energy-aware SFC Management using Deep Reinforcement Lear-ning[C]//2024 IEEE Network Operations and Management Symposium.IEEE,2024:1-5.
[14]RAN J,WANG W K,HU H F.Dynamic Service Function Chain Deployment and Readjustment Method Based on Deep Reinforcement Learning[J].Sensors,2023,23(6):3054.
[15]TANG L,HE L Q,LIAN Q Y,et al.Virtual Network FunctionPlacement Optimization Algorithm Based on Improve Deep Reinforcement Learning[J].Journal of Electronics & Information Technology,2021,43(6):1724-1732.
[16]LIU D H,WEI D E,XUAN H J,et al.Improved double deep Q network algorithm for service function chain deployment[J].Journal of Xidian University,2024,51(1):52-59.
[17]HUANG W W,LI S,WANG S N,et al.An Improved Adaptive Service Function Chain Mapping Method Based on Deep Reinforcement Learning[J].Electronics,2023,12(6):1307-1325.
[18]WANG T F,FAN Q L,LI X H,et al.Drl-sfcp:Adaptive service function chains placement with deep reinforcement learning[C]//IEEE International Conference on Communications(ICC 2021).2021:1-6.
[19]LIU Y C,LU Y,QIAO W X,et al.A dynamic compositionmechanism of security service chaining oriented to SDN/NFV-enabled networks[J].IEEE Access,2018,6:53918-53929.
[20]ALALUNA M,FERROLHO L,FIGUEIRA J R,et al.Secure multi-cloud virtual network embedding[C]//Computer Communications.2020:252-265.
[21]TORKZABAN N,BARAS J S.Trust-aware service functionchain embedding:A path-based approach[C]//2020 IEEE Conference on Network Function Virtualization and Software Defined Networks(NFV-SDN).IEEE,2020:31-36.
[22]ZHANG P Y,WANG C,JIANG C X,et al.Resource manage-ment and security scheme of ICPSs and IoT based on VNE algorithm[J].IEEE Internet of Things Journal,2021,9(22):22071-22080.
[23]VELICKOVIC P,CUCURULL G,CASANOVA A,et al.Graph attention networks[C]//IRLR 2018.2018.
[24]YAN Z X,GE J G,WU Y L, et al.Automatic virtual networkembedding:A deep reinforcement learning approach with graph convolutional networks[J].IEEE Journal on Selected Areas in Communications,2020,38(6):1040-1057.
[25]ZHANG P Y,LI H S,NI Y J,et al.Security aware virtual net-work embedding algorithm using information entropy TOPSIS[J].Journal of Network and Systems Management,2020,28(1):35-57.
[26]LIU X B,WANG B H,LIU S Q,et al.Heuristic algorithm for secure virtual network embedding [J].Systems Engineering and Electronic,2018,40(3):676-681.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发