计算机科学 ›› 2017, Vol. 44 ›› Issue (5): 125-131.doi: 10.11896/j.issn.1002-137X.2017.05.023

• 信息安全 • 上一篇    下一篇

基于开源工具集的大数据网络安全态势感知及预警架构

琚安康,郭渊博,朱泰铭   

  1. 中国人民解放军信息工程大学 郑州450001数学工程与先进计算国家重点实验室 郑州450001,中国人民解放军信息工程大学 郑州450001数学工程与先进计算国家重点实验室 郑州450001,中国人民解放军信息工程大学 郑州450001数学工程与先进计算国家重点实验室 郑州450001
  • 出版日期:2018-11-13 发布日期:2018-11-13
  • 基金资助:
    本文受国家自然科学基金(61501515)资助

Framework for Big Data Network Security Situational Awareness and Threat Warning Based on Open Source Toolset

JU An-kang, GUO Yuan-bo and ZHU Tai-ming   

  • Online:2018-11-13 Published:2018-11-13

摘要: 对信息系统安全防护而言,大数据是一把双刃剑。信息量的巨增使得数据价值密度更小,给APT等攻击行为提供了更好的藏身环境;但大数据处理技术对海量数据的聚合、挖掘和分析又使得准确检测及预测攻击威胁成为可能。为增强信息系统的威胁感知与攻击预警能力,构建大数据威胁处理平台势在必行。基于最新的开源大数据组件集,构建了集数据收集整理、数据存储、离线分析发现、实时关联检测、威胁预警和态势呈现等功能于一体的、支持全流程安全事件处理过程的、完整的网络安全态势感知及预警架构,与现有同类平台架构相比,其具有高可用、可扩展、易部署等特点,且能较好地支持威胁情报的引入。

关键词: 开源工具,大数据,态势感知,威胁预警

Abstract: Big data is a double-edged sword for information system security protection.On the one hand,data value density decreased because of the dramatic increase in the amount of information,which provides a better shelter for attacks like APT.On the other hand,its processing technology in aggregation,mining and analysis of huge amounts of data makes it possible to identify security threats accurately.In order to strengthen the perceiving threat ability of information system,it is imperative to build a big data threat analyzing platform.Based on open source big data components,we proposed a situational awareness and threat warning platform for data collection and reduction,data storage,off-line analysis,real-time correlation,threat warning and situation awareness.Compared with existing platforms,this architecture has the advantages of high availability, scalability,and it is easy to deploy and is suitable for introducing threat intelligence.

Key words: Open source tools,Big data,Situational awareness,Threat warning

[1] LEE Y.Toward scalable internet traffic measurement and analysis with Hadoop[J].Acm Sigcomm Computer Communication Review,2013,43(1):5-13.
[2] CHEON J J,CHOE T Y.Distributed Processing of Snort Alert Log using Hadoop[J].International Journal of Engineering & Technology,2013,5(3):2685-2690.
[3] CHARISHMA P,VENKATESH K.Big Data Security Analytic Solution using Splunk[J].International Journal of Engineering Research & Applications,2015,5(4):50-53.
[4] LI B.Network Security Monitoring and Analysis Based On Big Data Technologies[D].Dissertations & Theses,2013.
[5] MARCHAL S,JIANG X,STATE R,et al.A Big Data Architecture for Large Scale Security Monitoring[C]∥Proceedings of the 2014 IEEE International Congress on Big Data.IEEE Computer Society,2014:56-63.
[6] SAURABH R.Big Data Analytics and Challenges:Network Security and Instruction Detection[J].International Research Journal of Computers and Electronics and Engineering,2015,3(1):290-295 .
[7] MA Z,SMITH P.Determining Risks from Advanced Multi-step Attacks to Critical Information Infrastructures[M]∥Critical Information Infrastructures Security.Sprin-ger International Publishing,2013:142-154.
[8] ALSERHANI F M.Knowledge-Based Model to Represent Security Information and Reason About Multi-stage Attacks[M]∥Advanced Information Systems Engineering Workshops.Sprin-ger International Publishing,2015:482-494.
[9] LIN S,LI Y,DU X.Study and research of APT detection technology based on big data processing architecture[C]∥International Conference on Electronics Information and Emergency Communication.IEEE,2015.
[10] Opensoc.http://opensoc.github.io/
[11] XU H.Research on the Tecom Fundamental Network Information Security Awareness Based on Big Data Analyzation[J].Journal of Information Security Research,2015(3):253-260.(in Chinese) 徐浩.基于大数据分析的电信基础网安全态势研究[J].信息安全研究,2015(3):253-260.
[12] LI M G,XIAO Y,CHEN J F,et al.Big Data-based Framework for Security Event Mining[J].Communications Technology,2015,48(3):346-350.(in Chinese) 李明桂,肖毅,陈剑锋,等.基于大数据的安全事件挖掘框架[J].通信技术,2015,48(3):346-350.
[13] FU Y,LI H C,WU X P,et al.Detecting APT attacks:a survey from theperspective of big data analysis[J].Journal on Communications,2015,36(11):1-14.(in Chinese) 付钰,李洪成,吴晓平,等.基于大数据分析的APT攻击检测研究综述[J].通信学报,2015,36(11):1-14.
[14] SUN D W,ZHANG G Y,ZHENG W M.Big data stream computing:Technologies and instances[J].Journal of Software,2014,5(4):839-862.(in Chinese) 孙大为,张广艳,郑纬民.大数据流式计算:关键技术及系统实例[J].软件学报,2014,25(4):839-862.
[15] Flume.http://flume.apache.org.
[16] Kafka.http://kafka.apache.org.
[17] Storm.http://storm.apache.org.
[18] Elastic Search.https://www.elastic.co/products/elasticsearch.

No related articles found!
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!