Computer Science ›› 2025, Vol. 52 ›› Issue (2): 268-278.doi: 10.11896/jsjkx.240900059

• Computer Network • Previous Articles     Next Articles

Flow Cardinality Estimation Method Based on Distributed Sketch in SDN

WANG Yijie1, GAO Guoju1, SUN Yu'e2, HUANG He1   

  1. 1 School of Computer Science and Technology,Soochow University,Suzhou,Jiangsu 215006,China
    2 School of Rail Transportation,Soochow University,Suzhou,Jiangsu 215131,China
  • Received:2024-09-10 Revised:2024-11-18 Online:2025-02-15 Published:2025-02-17
  • About author:WANG Yijie,born in 2001,postgra-duate.Her main research interest is network traffic measurement.
    GAO Guoju,born in 1990.Ph.D,asso-ciate professor,is a member of CCF(No.C5973M).His main research interests include network traffic measurement,intelligent sensing and computing.
  • Supported by:
    National Natural Science Foundation of China(62472298,62332013,62102275,62072322,U20A20182,62202322),China Postdoctoral Science Foundation(2023M732537) and Natural Science Foundation of Jiangsu Province,China(BK20210704).

Abstract: In software-defined networks(SDN),statistical flow cardinality information plays a crucial role in various applications such as traffic engineering,traffic rerouting,and attack detection.Existing research primarily falls into two categories:measurement structures deployed on a single switch and collaborative measurements across multiple switches.However,neither of the two approaches achieves full network flow coverage,and collaborative measurements often employ independent measurements from each switch followed by merging,which can lead to duplicate counting.To address these issues,a flow cardinality estimation method based on distributed sketches is proposed.This method leverages the advantages of centralized control in the SDN control plane and collaboratively utilizes switches along the longest continuous common sub-path of each flow to construct a logical layer counting structure for that flow.Additionally,a mapping from the logical layer counting structure to the physical switch space for each flow is established,allowing participating switches to dynamically adjust the mapping intervals based on their own states and actual load conditions,thus achieving load balancing among all switches in the network.Taking the vHLL algorithm as an example,we implement a prototype of the distributed flow cardinality estimation method and conduct experimental evaluations using a real network traffic dataset on a four-layer Fat-Tree network topology.The experimental results demonstrate that the proposed method effectively achieves flow cardinality estimation across the entire network.In terms of accuracy,its average relative error(ARE) and average absolute error(AAE) values outperform the comparison experiments by up to 94.7% and 93.8%,respectively.Regarding load balancing,the method fully utilizes all switches in the network,achieving a normalized average packet load of 0.394,which is lower than that of the comparison methods,indicating its good performance.

Key words: Software-defined network, Flow cardinality measurement, Distributed network, Sketch, Load balancing

CLC Number: 

  • TP391
[1]HONG C Y,KANDULA S,MAHAJAN R,et al.Achievinghigh utilization with software-driven WAN[C]//Proceedings of the ACM SIGCOMM 2013 Conference on SIGCOMM.New York:Association for Computing Machinery,2013:15-26.
[2]JAIN S,KUMAR A,MANDAL S,et al.B4:Experience with a globally-deployed software defined WAN[J].ACM SIGCOMM Computer Communication Review,2013,43(4):3-14.
[3]XU H,YU Z,LI X Y,et al.Joint route selection and updatescheduling for low-latency update in SDNs[J].IEEE/ACM Transactions on Networking,2017,25(5):3073-3087.
[4]XU H,HUANG H,CHEN S,et al.Achieving high scalability through hybrid switching in software-defined networking[J].IEEE/ACM Transactions on Networking,2018,26(1):618-632.
[5]LIU Z,MANOUSIS A,VORSANGER G,et al.One sketch to rule them all:Rethinking network flow monitoring with univmon[C]//Proceedings of the 2016 ACM SIGCOMM Confe-rence.New York:Association for Computing Machinery,2016:101-114.
[6]YANG T,ZHANG H,LI J,et al.HeavyKeeper:an accurate algorithm for finding Top-$ k $ elephant flows[J].IEEE/ACM Transactions on Networking,2019,27(5):1845-1858.
[7]ZHAO B,LI X,TIAN B,et al.Dhs:Adaptive memory layout organization of sketch slots for fast and accurate data stream processing[C]//Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining.New York:Association for Computing Machinery,2021:2285-2293.
[8]YANG T,JIANG J,LIU P,et al.Elastic sketch:Adaptive and fast network-wide measurements[C]//Proceedings of the 2018 Conference of the ACM Special Interest Group on Data Communication.New York:Association for Computing Machinery,2018:561-575.
[9]LI T,CHEN S,LUO W,et al.Scan detection in high-speed networks based on optimal dynamic bit sharing[C]//2011 Procee-dings IEEE INFOCOM.Montreal:IEEE,2011:3200-3208.
[10]XIAO Q,CHEN S,ZHOU Y,et al.Cardinality estimation for elephant flows:A compact solution based on virtual register sharing[J].IEEE/ACM Transactions on Networking,2017,25(6):3738-3752.
[11]JIA P,WANG P,ZHANG Y,et al.Accurately estimating user cardinalities and detecting super spreaders over time[J].IEEE Transactions on Knowledge and Data Engineering,2020,34(1):92-106.
[12]ZHANG H,HUANG H,SUN Y E,et al.MIME:Fast and Accurate Flow Information Compression for Multi-Spread Estimation[C]//2023 IEEE 31st International Conference on Network Protocols(ICNP).Montreal:IEEE,2023:1-11.
[13]JING X,HAN H,YAN Z,et al.SuperSketch:A multi-dimen-sional reversible data structure for super host identification[J].IEEE Transactions on Dependable and Secure Computing,2021,19(4):2741-2754.
[14]BASAT R B,EINZIGER G,TAYH B.Cooperative network-wide flow selection[C]//2020 IEEE 28th International Confe-rence on Network Protocols(ICNP).Montreal:IEEE,2020:1-11.
[15]AL-FARES M,LOUKISSAS A,VAHDAT A.A scalable,commodity data center network architecture[J].ACM SIGCOMM Computer Communication Review,2008,38(4):63-74.
[16]LI Y,YU X,YANG Y,et al.Pyramid family:Generic frameworks for accurate and fast flow size measurement[J].IEEE/ACM Transactions on Networking,2021,30(2):586-600.
[17]ESTAN C,VARGHESE G,FISK M.Bitmap algorithms forcounting active flows on high speed links[C]//Proceedings of the 3rd ACM SIGCOMM Conference on Internet Measurement.New York:Association for Computing Machinery,2003:153-166.
[18]FLAJOLET P,MARTIN G N.Probabilistic counting algorithms for data base applications[J].Journal of Computer and System Sciences,1985,31(2):182-209.
[19]DURAND M,FLAJOLET P.Loglog counting of large cardinali-ties[C]//Algorithms-ESA 2003:11th Annual European Symposium,Budapest,Hungary,September 16-19,2003.Proceedings 11.Berlin Heidelberg:Springer,2003:605-617.
[20]FLAJOLET P,FUSY É,GANDOUET O,et al.Hyperloglog:the analysis of a near-optimal cardinality estimation algorithm[J] Discrete Mathematics & Theoretical Computer Science,2007:137-156.https://dmtcs.episciences.org/3545/pdf.
[21]DU Y,HUANG H,SUN Y E,et al.A Better Cardinality Estimator with Fewer Bits,Constant Update Time,and Mergeability[C]//IEEE INFOCOM 2023-IEEE Conference on Computer Communications.Montreal:IEEE,2023:1-10.
[22]TANG L,HUANG Q,LEE P P C.SpreadSketch:Toward in-vertible and network-wide detection of superspreaders[C]//IEEE INFOCOM 2020-IEEE Conference on Computer Communications.Montreal:IEEE,2020:1608-1617.
[23]JING X,YAN Z,HAN H,et al.ExtendedSketch:Fusing network traffic for super host identification with a memory efficient sketch[J].IEEE Transactions on Dependable and Secure Computing,2021,19(6):3913-3924.
[24]CAIDA.Anonymized internet traces 2019 [EB/OL].(2019-01)[2024-08-08].https://catalog.caida.org/details/dataset/passive_2019_pcap.
[1] WU Zongming, CAO Jijun, TANG Qiang. Online Parallel SDN Routing Optimization Algorithm Based on Deep Reinforcement Learning [J]. Computer Science, 2025, 52(6A): 240900018-9.
[2] ZHOU Kai, WANG Kai, ZHU Yuhang, PU Liming, LIU Shuxin, ZHOU Deqiang. Customized Container Scheduling Strategy Based on GMM [J]. Computer Science, 2025, 52(6): 346-354.
[3] HUANG Chenxi, LI Jiahui, YAN Hui, ZHONG Ying, LU Yutong. Investigation on Load Balancing Strategies for Lattice Boltzmann Method with Local Grid Refinement [J]. Computer Science, 2025, 52(5): 101-108.
[4] WANG Panxiang, CUI Yunhe, SHEN Guowei, GUO Chun, CHEN Yi, QIAN Qing. EvoTrace:A Lightweight In-band Network Telemetry Method Based on Nonlinear Embedding and Batch Processing [J]. Computer Science, 2025, 52(5): 291-298.
[5] ZHONG Yue, GU Jieming. 3D Reconstruction of Single-view Sketches Based on Attention Mechanism and Contrastive Loss [J]. Computer Science, 2025, 52(3): 77-85.
[6] ZHENG Longhai, XIAO Bohuai, YAO Zewei, CHEN Xing, MO Yuchang. Graph Reinforcement Learning Based Multi-edge Cooperative Load Balancing Method [J]. Computer Science, 2025, 52(3): 338-348.
[7] WANG Jiayu, YU Junqing, LI Dong, ZHAO Junyang. Network Microburst Traffic Measurement Method Based on Sketch Data Structure [J]. Computer Science, 2025, 52(1): 374-382.
[8] LIU Haohan, CHEN Zemao. Study on Malicious Access Detection in Industrial Control Networks Based on Dynamic BayesianGames [J]. Computer Science, 2025, 52(1): 383-392.
[9] GU Zhouchao, CHENG Guang, ZHAO Yuyu. Segmental Routing in Band Telemetry Method for Endogenous Secure Switches [J]. Computer Science, 2024, 51(5): 284-292.
[10] CHEN Xinyang, CHEN Hanze, ZHOU Jiasheng, HUANG Jiaqing, YU Jiashuo, ZHU Longlong, ZHANG Dong. IntervalSketch:Approximate Statistical Method for Interval Items in Data Stream [J]. Computer Science, 2024, 51(4): 4-10.
[11] MAO Chenyu, HUANG He, SUN Yu'e, DU Yang. Global Top-K Frequent Flow Measurement for Continuous Periods in Distributed Networks [J]. Computer Science, 2024, 51(4): 28-38.
[12] WU Yanni, ZHOU Zhengyan, CHEN Hanze, ZHANG Dong. RBFRadar:Detecting Remarkable Burst Flows with Programmable Data Plane [J]. Computer Science, 2024, 51(4): 48-55.
[13] LIAO Qihua, NIE Kai, HAN Lin, CHEN Mengyao, XIE Wenbing. Tile Selection Algorithm Based on Data Locality [J]. Computer Science, 2024, 51(12): 100-109.
[14] YANG Zheming, ZUO Lulu, JI Wen. Joint Optimization Method for Node Deployment and Resource Allocation Based on End-EdgeCollaboration [J]. Computer Science, 2024, 51(11A): 240200010-7.
[15] LI Chunjiang, YIN Shaoping, CHI Haotian, YANG Jing, GENG Haijun. DDoS Attack Detection Model Based on Statistics and Ensemble Autoencoders in SDN [J]. Computer Science, 2024, 51(11): 389-399.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!