Computer Science

Computer Science ›› 2026, Vol. 53 ›› Issue (10): 413-422.doi: 10.11896/jsjkx.250700146

• Information Security • Previous Articles     Next Articles

Survey of Container Anomaly Detection in Linux Environments

CHEN Hao, ZHOU Naqin, CHEN Zeyu, CHEN Haoqin   

  1. School of Cyberspace Security,Guangzhou University,Guangzhou 510006, China
  • Received:2025-07-22 Revised:2025-11-07 Published:2026-10-13
  • About author:CHEN Hao,born in 1999,postgraduate.His main research interest is container security.
    ZHOU Naqin,born in 1982,Ph.D,associate professor,is a member of CCF(No.Y7983M).Her main research interests include distributed computing,resource management scheduling,and container security.
  • Supported by:
    National Natural Science Foundation of China(62002078), Guangdong Provincial Science and Technology Program(2023A1111120005) and Guangzhou Basic Research Program(2025A03J3139).

PDF (PC)

53

Abstract: Container technology has gained widespread adoption in cloud and edge computing due to its lightweight nature,high performance,and scalability.However,the complex and dynamic behavior of containerized workloads in Linux environments introduces challenges such as resource abuse,privilege escalation,and container escape,which pose significant threats to system security and performance.Effective anomaly detection techniques are expected to meet essential requirements including real-time responsiveness,low overhead,adaptability,and high accuracy.This paper provides a systematic review of container anomaly detection technologies based on the Linux kernel mechanisms.Firstly,it presents a theoretical analysis of resource allocation logic and system call behavior patterns during container runtime.Secondly,it summarizes detection methods based on rules and learning by focusing on two key features:resource consumption and system calls.Finally,it discusses potential future research directions for container anomaly detection.

Key words: Linux, Container anomaly detection, System calls, Resource consumption, eBPF

CLC Number: 

  • TP309
[1] CHEN Y Y,WANG X N,LU S S,et al.A review of container technologies for high-performance computing systems[J].Computer Science,2023,50(2):353-363.
[2] JARAMILLO D,NGUYEN D V,SMART R.LeveragingMicroservices Architecture by Using Docker Technology[C]//SoutheastCon 2016.IEEE,2016:1-5.
[3] WU Y W,ZHANG Y,WANG T,et al.From Docker Containers to the Development of Container Technology:A Systematic Li-terature Review Perspective [J].Journal of Software,2023,34(12):5527-5551.
[4] Docker.Docker Official Website [EB/OL].[2025-08-27].ht-tp://www.docker.com.
[5] SPAHN N,HANKE N,HOLZ T,et al.Container Orchestration Honeypot:Observing Attacks in the Wild[C]//Proceedings of the 26th International Symposium on Research in Attacks,Intrusions and Defenses.IEEE Computer Society,2023:381-396.
[6] JADHAV Y C,SABLE A,SURESH M,et al.Securing Containers:Honeypots for Analysing Container Attacks[C]//2023 15th International Conference on Communication Systems & Networks.IEEE,2023:225-227.
[7] Tencent.White Paper on Container Security(2021) [R].Shen-zhen:Tencent Cloud,2021.
[8] Kubernetes.Kubernetes Official Website [EB/OL].[2025-08-27].https://kubernetes.io.
[9] Red Hat Inc.The State of Kubernetes Security Report 2024[R].Raleigh:Red Hat Inc.,2024.
[10] Namespace.Wikipedia [EB/OL].[2025-08-27].https://en.wikipedia.org/wiki/Namespace.
[11] Cgroups.Wikipedia [EB/OL].[2025-08-27].https://en.wikipedia.org/wiki/Cgroups.
[12] SHEN Z,SUN Z,SELA G E,et al.X-Containers:BreakingDown Barriers to Improve Performance and Isolation of Cloud-Native Containers[C]//Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems.IEEE,2019:121-135.
[13] YANG N Z,SHEN W B,LI J K,et al.Demons in the shared kernel:Abstract resource attacks against OS-level virtualization[C]//Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security.ACM,2021:764-778.
[14] YANG N Z,SHEN W B,LI J K,et al.Take over the wholecluster:Attacking Kubernetes via excessive permissions of third-party applications[C]//Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security.ACM,2023:3048-3062.
[15] XIAO J,YANG N,SHEN W,et al.Attacks Are Forwarded:Breaking the Isolation of MicroVM-Based Containers Through Operation Forwarding[C]//Proceedings of the 32nd USENIX Security Symposium.USENIX Association,2023:7517-7534.
[16] LIU D Q,HU H C,HUO S M.N-variant architecture for run-time security threats to containers[J].Computer Science,2024,51(6):399-408.
[17] YU J,XIE X,ZHANG C,et al.Bugs in Pods:UnderstandingBugs in Container Runtime Systems[C]//Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis.ACM,2024:1364-1376.
[18] HAQ M S,NGUYEN T D,TOSUN A Ş,et al.SoK:A Comprehensive Analysis and Evaluation of Docker Container Attack and Defense Mechanisms[C]//2024 IEEE Symposium on Security and Privacy(SP).IEEE,2024:4573-4590.
[19] PATCHA A,PARK J M.An overview of anomaly detectiontechniques:Existing solutions and latest technological trends[J].Computer Networks,2007,51(12):3448-3470.
[20] CHANDOLA V,BANERJEE A,KUMAR V.Anomaly detec-tion:A survey[J].ACM Computing Surveys,2009,41(3):1-58.
[21] FAN Y Y,ZHANG Z J,XU C H,et al.Research review onanomaly detection techniques for cloud container security [J].Science & Technology Innovation Herald,2021,18(13):118-125.
[22] YANG Y,SHEN W,RUAN B,et al.Security Challenges in the Container Cloud[C]//2021 Third IEEE International Confe-rence on Trust,Privacy and Security in Intelligent Systems and Applications.IEEE Computer Society,2021:137-145.
[23] LUO H X,WANG J S.Docker container security risks and defense review [J].Information Security and Communication Secrecy,2022(8):83-93.
[24] MIAO S T,LI X J.Research on Container Security Detection Technology in Cloud-Native [J].Journal of Secrecy Science and Technology,2022(7):14-19.
[25] DENG Q Q,SONG C,LU Z T, et al.A Survey on Threats and Countermeasures of Container Security [EB/OL].https://jcs.iie.ac.cn/xxaqxb/ch/reader/view_abstract.aspx?file_no=202210090000004.
[26] FENG Y,LI J,SISODIA D.CJ-Sniffer:Measurement and Content-Agnostic Detection of Cryptojacking Traffic[C]//Procee-dings of the 25th International Symposium on Research in Attacks,Intrusions and Defenses.ACM,2022:482-494.
[27] FRANCO J,ACAR A,ARIS A,et al.Forensic Analysis ofCryptojacking in Host-Based Docker Containers Using Honeypots[C]//ICC 2023－IEEE International Conference on Communications.IEEE,2023:4860-4865.
[28] HU X,SHU Z,SONG X,et al.Detecting Cryptojacking Traffic Based on Network Behavior Features[C]//2021 IEEE Global Communications Conference.IEEE,2021:1-6.
[29] NASEREDDIN M,AL-QASSAS R.A new approach for detecting process injection attacks using memory analysis[J].International Journal of Information Security,2024,23:2099-2121.
[30] CANZANESE R,MANCORIDIS S,KAM M.System call-based detection of malicious processes[C]//2015 IEEE International Conference on Software Quality,Reliability and Security.IEEE,2015:119-124.
[31] WIN T Y,TSO F P,MAIR Q,et al.PROTECT:Containerprocess isolation using system call interception[C]//2017 14th International Symposium on Pervasive Systems,Algorithms and Networks & 2017 11th International Conference on Frontier of Computer Science and Technology & 2017 Third International Symposium of Creative Computing.IEEE,2017:191-196.
[32] LEE W,NADIM M.Kernel-Level Rootkits Features to Train Learning Models Against Namespace Attacks on Containers[C]//2020 7th IEEE International Confe-rence on Cyber Secu-rity and Cloud Computing/2020 6th IEEE International Confe-rence on Edge Computing and Scalable Cloud.IEEE,2020:50-55.
[33] SIMONSSON J,ZHANG L,MORIN B,et al.Observability and chaos engineering on system calls for containerized applications in Docker[J].Future Generation Computer Systems,2021,122:117-129.
[34] YU G B,CHEN P F,LI P R,et al.LogReducer:Identify and Reduce Log Hotspots in Kernel on the Fly[C]//2023 IEEE/ACM 45th International Conference on Software Engineering.IEEE,2023:1763-1775.
[35] YAMAUCHI T,AKAO Y,YOSHITANI R,et al.Additionalkernel observer:privilege escalation attack prevention mechanism focusing on system call privilege changes[J].International Journal of Information Security,2021,20:461-473.
[36] KANDAN R,KHALID M F,ISMAIL B I,et al.A Generic Log Analyzer for Automated Troubleshooting in Container Orchestration System[C]//2020 8th International Conference on Information Technology and Multimedia.IEEE,2020:267-270.
[37] PANG G,SHEN C,CAO L,et al.Deep learning for anomaly detection:A review[J].ACM Computing Surveys,2021,54(2):Article 38.
[38] ZHANG Y,LUO S,PAN L,et al.Syscall-BSEM:Behavioral semantics enhancement method of system call sequence for high accurate and robust host intrusion detection[J].Future Generation Computer Systems,2021,125:112-126.
[39] HUANG S,LUAN Z.Semantic-Aware Log Understanding and Analysis[C]//Proceedings of the 33rd International Symposium on High-Performance Parallel and Distributed Computing.ACM,2024:413-416.
[40] AGYEMANG E F.Anomaly detection using unsupervised machine learning algorithms:A simulation study[J].Scientific African,2024,26:e02386.
[41] ZOU Z,XIE Y,HUANG K,et al.A Docker Container Anomaly Monitoring System Based on Optimized Isolation Forest[J].IEEE Transactions on Cloud Computing,2022,10(1):134-145.
[42] SAMIR A,PAHL C.Detecting and Localizing Anomalies inContainer Clusters Using Markov Models[J].Electronics,2020,9(1):64.
[43] LIU W,YI R,GU Y,et al.Escape attacks on Transwarp Data Cloud container[C]//2024 9th IEEE International Conference on Smart Cloud.IEEE,2024:97-102.
[44] GHAVAMNIA S,PALIT T,BENAMEUR A,et al.Confine:Automated System Call Policy Generation for Container Attack Surface Reduction [C]//Proceedings of the 23rd International Symposium on Research in Attacks,Intrusions and Defenses.USENIX Association,2020:443-458.
[45] KIM S,KIM B J,LEE D H.Prof-gen:Practical study on system call whitelist generation for container attack surface reduction[C]//2021 IEEE 14th International Conference on Cloud Computing.IEEE,2021:278-287.
[46] ROSTAMIPOOR M,GHAVAMNIA S,POLYCHRONAKISM.Confine:Fine-grained system call filtering for container attack surface reduction[J].Computers & Security,2023,132:103325.
[47] IIGUNI T,KAMEI H,SAISHO K.Sprofiler:Automatic Gene-rating System of Container-Native System Call Filtering Rules for Attack Surface Reduction[C]//2021 International Confe-rence on Computational Science and Computational Intelligence.IEEE,2021:704-709.
[48] WU S L,LIU W G,YAN M,et al.Real-time anomaly detection system for container clouds based on unsupervised system call rule generation[J].Information Network Security,2023,23(12):91-102.
[49] SONG S,SUNEJA S,LE M V,et al.On the Value of Sequence-Based System Call Filtering for Container Security[C]//2023 IEEE 16th International Conference on Cloud Computing.IEEE,2023:296-307.
[50] YANG S,KANG B B,NAM J.Optimus:association-based dynamic system call filtering for container attack surface reduction[J].Journal of Cloud Computing,2024,13:71.
[51] HER J,JO C H,PARK T,et al.KubeRosy:A dynamic systemcall filtering framework for containers[J].IEEE Access,2024,12:159889-159901.
[52] JIA J,ZHU Y,WILLIAMS D,et al.Programmable System Call Security with eBPF[J].arXiv:2302.10366,2023.
[53] Samir A Security.Falco:Cloud Native Runtime Security[EB/OL].(2016-03-01) [2024-11-28].https://github.com/falcosecurity/falco.
[54] Aqua Security.Tracee:Runtime Security and eBPF Tracing[EB/OL].(2020-03-24) [2024-11-28].https://github.com/aquasecurity/tracee.
[55] Cilium.Tetragon:eBPF-Based Security Observability and Run-time Enforcement[EB/OL].(2021-05-19) [2024-11-28].https://github.com/cilium/tetragon.
[56] FALCO.R6 Security Case Study [EB/OL].[2025-08-27].https://falco.org/about/case-studies/r6-security/.
[57] CAVALCANTI M,INACIO P,FREIRE M.Performance Evaluation of Container-Level Anomaly-Based Intrusion Detection Systems for Multi-Tenant Applications Using Machine Learning Algorithms[C]//Proceedings of the 16th International Confe-rence on Availability,Reliability and Security.ACM,2021:1-9.
[58] WANG Y,CHEN X,WANG Q,et al.Unsupervised Anomaly Detection for Container Cloud Via BILSTM-Based Variational Auto-Encoder[C]//ICASSP 2022－2022 IEEE International Conference on Acoustics,Speech and Signal Processing.IEEE,2022:3024-3028.
[59] TIEN C W,HUANG T Y,TIEN C,et al.KubAnomaly:Ano-maly detection for the Docker orchestration platform with neural network approaches[J].Engineering Reports,2019,1:e12080.
[60] ZUPPELLI M,GUARASCIO M,CAVIGLIONE L,et al.NoCountry for Leaking Containers:Detecting Exfiltration of Secrets Through AI and Syscalls[C]//Proceedings of the 19th International Conference on Availability,Reliability and Security.ACM,2024:1-8.
[61] TUNDE-ONADELE O,LIN Y,GU X,et al.Self-SupervisedMachine Learning Framework for Online Container Security Attack Detection[J].ACM Transactions on Autonomous and Adaptive Systems,2024,19(3):1-28.
[62] LI Y,HU H,LIU W,et al.An Optimal Active Defensive Security Framework for the Container-Based Cloud with Deep Reinforcement Learning[J].Electronics,2023,12:1598.
[63] GUO S,SIVANTHI T,SOMMER P,et al.A Zero-day Contai-ner Attack Detection based on Ensemble Machine Learning[C]//2023 IEEE 28th International Conference on Emerging Technologies and Factory Automation.IEEE,2023:1-8.
[64] WENG T,YANG W,YU G,et al.Kmon:An In-kernel Transparent Monitoring System for Microservice Systems with eBPF[C]//2021 IEEE/ACM International Workshop on Cloud Intelligence.IEEE,2021:25-30.
[65] HOU S,SAAS A,CHEN L,et al.Deep4MalDroid:A DeepLearning Framework for Android Malware Detection Based on Linux Kernel System Call Graphs[C]//2016 IEEE/WIC/ACM International Conference on Web Intelligence Workshops.IEEE,2016:104-111.
[66] EL KHAIRI A,CASELLI M,KNIERIM C,et al.Contextua-lizing System Calls in Containers for Anomaly-Based Intrusion Detection[C] // Proceedings of the 2022 Cloud Computing Security Workshop.ACM,2022:9-21.
[67] HUANG K,LI X,ZHOU Q F,et al.A trusted runtime monitoring scheme for containers based on eBPF[J].Information Network Security,2025,25(2):306-326.
[68] DONG F,LI S,JIANG P,et al.Are we there yet? An Industrial Viewpoint on Provenance-based Endpoint Detection and Response Tools [C]//Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security(CCS’23).ACM,2023:2396-2410.
[69] NAM J,LEE S,PORRAS P,et al.Secure Inter-Container Communications Using XDP/eBPF[J].IEEE/ACM Transactions on Networking,2023,31(2):934-947.
[70] SHEN J,ZHANG H,XIANG Y,et al.Network-Centric Distri-buted Tracing with DeepFlow:Troubleshooting Your Microservices in Zero Code[C]//Proceedings of the ACM SIGCOMM 2023 Conference.ACM,2023:420-437.
[71] LIM S Y,STELEA B,HAN X,et al.Secure Namespaced Kernel Audit for Containers[C] //Proceedings of the ACM Symposium on Cloud Computing.ACM,2021:518-532.
[72] HE Y,GUO R,XING Y,et al.Cross Container Attacks:TheBewildered eBPF on Clouds[C]//Proceedings of the 32nd USENIX Security Symposium.USENIX Association,2023:5971-5988.
[73] MAO J,DING H,ZHAI J,et al.Merlin:Multi-tier Optimization of eBPF Code for Performance and Compactness [C]//Procee-dings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems.ACM,2024:639-653.
[74] DIPTA D R,TIEMANN T,GULMEZOGLU B,et al.Dynamic Frequency-Based Fingerprinting Attacks against Modern Sandbox Environments[C]//2024 IEEE 9th European Symposium on Security and Privacy(EuroS&P).IEEE,2024:327-344.
[75] PATTEE J,LEE B K.Design Alternatives for Performance Monitoring Counter based Malware Detection[C]//2020 IEEE 39th International Performance Computing and Communications Conference(IPCCC).IEEE,2020:1-2.
[76] NIU W N,ZHAO C Y,ZHANG X S,et al.ROPDetector:Areal-time ROP attack detection method based on hardware performance counters[J].Chinese Journal of Computers,2021,44(4):761-772.
[1] FAN Yuhao, SUN Cong, ZHANG Linmao. Program Capabilities Reduction Based on Code Specialization [J]. Computer Science, 2026, 53(8): 446-454.
[2] SHI Heyuan, CHEN Shijun, ZHANG Qiang, SHEN Yuheng, JIANG Yu, SHI Ronghua. Configuration-guided Directed Kernel Fuzzing for Real-time Linux [J]. Computer Science, 2025, 52(6A): 240400161-8.
[3] LI Yuhang, TAN Ruixiong, CHAI Yunpeng. Study on Method for Collaborative Tuning Resources and Parameters of Cloud Database [J]. Computer Science, 2024, 51(6): 104-110.
[4] DENG Guanghong, ZHANG Qiheng. Container-based Scheduling Architecture for Mixed-Criticality Systems [J]. Computer Science, 2023, 50(6A): 220800215-5.
[5] WEI Hui, CHEN Ze-mao, ZHANG Li-qiang. Anomaly Detection Framework of System Call Trace Based on Sequence and Frequency Patterns [J]. Computer Science, 2022, 49(6): 350-355.
[6] ZHU Yu-jian, MA Jun-ming, AN Bo, CAO Dong-gang. Linux Container Cluster Networking Approach for Multiple Tenants [J]. Computer Science, 2018, 45(9): 46-51.
[7] CHEN Wei, YANG Qiu-hui and CHENG Xue-mei. Study on Security Enhancement Mechanism of Android System Kernel Based on Security Domain [J]. Computer Science, 2018, 45(2): 203-208.
[8] SHE Lei, ZHAO Xi-bin, CHEN Yu, SHI He-yuan and WEI Kang. Analysis and Optimization of Boot-up Performance for Railway Real-time Ethernet Switch [J]. Computer Science, 2017, 44(Z11): 276-280.
[9] SHU An, PENG Xin and ZHAO Wen-yun. Self-adaptive Approach for Container-based Cloud Resources Management [J]. Computer Science, 2017, 44(7): 120-127.
[10] WANG Huan, MAO Jun-jie, WANG Dan and CHEN Yu. Research on Linux Device Driver Reuse [J]. Computer Science, 2017, 44(4): 39-42.
[11] WEN Han-xiang, LI Yu-jun and HOU Meng-shu. Research on Privacy Protection Based on SEAndroid [J]. Computer Science, 2015, 42(Z11): 329-332.
[12] CUI Yong-jun and ZHANG Yong-hua. Linux System Dual Threshold Scheduling Algorithm Based on Characteristic Scale Equilibrium [J]. Computer Science, 2015, 42(6): 181-184.
[13] JIANG Meng-tao,PAN Peng-fei,SONG Yang and JING Qi. Method of Parsing Dependencies between the Linux Kernel Compiling Options,Source Files and Functions [J]. Computer Science, 2014, 41(Z6): 445-450.
[14] TU Xiao-qin,SHANG Wei and ZHOU Fan-fan. Experimental Design of Operating System in Independent College [J]. Computer Science, 2014, 41(Z11): 471-472.
[15] ZHANG Li-ping,LEI Da-jiang and ZENG Xian-hua. System Calls Based Intrusion Detection Method with Frequency Feature Vector [J]. Computer Science, 2013, 40(Z6): 330-333.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.