Computer Science ›› 2017, Vol. 44 ›› Issue (5): 125-131.doi: 10.11896/j.issn.1002-137X.2017.05.023

Previous Articles     Next Articles

Framework for Big Data Network Security Situational Awareness and Threat Warning Based on Open Source Toolset

JU An-kang, GUO Yuan-bo and ZHU Tai-ming   

  • Online:2018-11-13 Published:2018-11-13

Abstract: Big data is a double-edged sword for information system security protection.On the one hand,data value density decreased because of the dramatic increase in the amount of information,which provides a better shelter for attacks like APT.On the other hand,its processing technology in aggregation,mining and analysis of huge amounts of data makes it possible to identify security threats accurately.In order to strengthen the perceiving threat ability of information system,it is imperative to build a big data threat analyzing platform.Based on open source big data components,we proposed a situational awareness and threat warning platform for data collection and reduction,data storage,off-line analysis,real-time correlation,threat warning and situation awareness.Compared with existing platforms,this architecture has the advantages of high availability, scalability,and it is easy to deploy and is suitable for introducing threat intelligence.

Key words: Open source tools,Big data,Situational awareness,Threat warning

[1] LEE Y.Toward scalable internet traffic measurement and analysis with Hadoop[J].Acm Sigcomm Computer Communication Review,2013,43(1):5-13.
[2] CHEON J J,CHOE T Y.Distributed Processing of Snort Alert Log using Hadoop[J].International Journal of Engineering & Technology,2013,5(3):2685-2690.
[3] CHARISHMA P,VENKATESH K.Big Data Security Analytic Solution using Splunk[J].International Journal of Engineering Research & Applications,2015,5(4):50-53.
[4] LI B.Network Security Monitoring and Analysis Based On Big Data Technologies[D].Dissertations & Theses,2013.
[5] MARCHAL S,JIANG X,STATE R,et al.A Big Data Architecture for Large Scale Security Monitoring[C]∥Proceedings of the 2014 IEEE International Congress on Big Data.IEEE Computer Society,2014:56-63.
[6] SAURABH R.Big Data Analytics and Challenges:Network Security and Instruction Detection[J].International Research Journal of Computers and Electronics and Engineering,2015,3(1):290-295 .
[7] MA Z,SMITH P.Determining Risks from Advanced Multi-step Attacks to Critical Information Infrastructures[M]∥Critical Information Infrastructures Security.Sprin-ger International Publishing,2013:142-154.
[8] ALSERHANI F M.Knowledge-Based Model to Represent Security Information and Reason About Multi-stage Attacks[M]∥Advanced Information Systems Engineering Workshops.Sprin-ger International Publishing,2015:482-494.
[9] LIN S,LI Y,DU X.Study and research of APT detection technology based on big data processing architecture[C]∥International Conference on Electronics Information and Emergency Communication.IEEE,2015.
[10] Opensoc.http://opensoc.github.io/
[11] XU H.Research on the Tecom Fundamental Network Information Security Awareness Based on Big Data Analyzation[J].Journal of Information Security Research,2015(3):253-260.(in Chinese) 徐浩.基于大数据分析的电信基础网安全态势研究[J].信息安全研究,2015(3):253-260.
[12] LI M G,XIAO Y,CHEN J F,et al.Big Data-based Framework for Security Event Mining[J].Communications Technology,2015,48(3):346-350.(in Chinese) 李明桂,肖毅,陈剑锋,等.基于大数据的安全事件挖掘框架[J].通信技术,2015,48(3):346-350.
[13] FU Y,LI H C,WU X P,et al.Detecting APT attacks:a survey from theperspective of big data analysis[J].Journal on Communications,2015,36(11):1-14.(in Chinese) 付钰,李洪成,吴晓平,等.基于大数据分析的APT攻击检测研究综述[J].通信学报,2015,36(11):1-14.
[14] SUN D W,ZHANG G Y,ZHENG W M.Big data stream computing:Technologies and instances[J].Journal of Software,2014,5(4):839-862.(in Chinese) 孙大为,张广艳,郑纬民.大数据流式计算:关键技术及系统实例[J].软件学报,2014,25(4):839-862.
[15] Flume.http://flume.apache.org.
[16] Kafka.http://kafka.apache.org.
[17] Storm.http://storm.apache.org.
[18] Elastic Search.https://www.elastic.co/products/elasticsearch.

No related articles found!
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!