Computer Science ›› 2015, Vol. 42 ›› Issue (5): 221-224.doi: 10.11896/j.issn.1002-137X.2015.05.044

Previous Articles     Next Articles

Traffic Surveillance Video Storage in HDFS Based on Event Density

ZANG Ji-kun and YU Jian   

  • Online:2018-11-14 Published:2018-11-14

Abstract: Utilizing HDFS to store and process large scale traffic surveillance video is a reliable,highly efficient and scalable solution.However,the default rack awareness data placement strategy in HDFS may cause hotspots when placing data.To address this problem,this paper presented a traffic surveillance video data placement strategy based on traffic event density.The characteristic of traffic surveillance videos allows us to classify them according to traffic event types.When placing new data,the proposed strategy predicts the load of each datanode which is influenced by various of traffic events the datanode stores,then combines the instant load and disk capacity to evaluate each datanode,and chooses the most suitable datanode to store new data.Experiments show that the proposed strategy alleviates the hotspot problem and effectively improves the load balancing and throughput in comparison with the default strategy.

Key words: Traffic surveillance video,HDFS,Traffic event,Data placement,Throughput

[1] 王国锋,宋鹏飞,张蕴灵.智能交通系统发展与展望[J].公路,2012(5):217-222
[2] 张庆华.云存储技术在视频监控中的发展与应用[J].中国安防,2013(8):53-58
[3] Borthakur D.The hadoop distributed file system:Architecture and design [J].Hadoop Project Website,2007,11:21
[4] Shvachko K,Kuang H,Radia S,et al.The hadoop distributed file system [C]∥2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST).IEEE,2010:1-10
[5] Ghemawat S,Gobioff H,Leung S T.The Google file system [J].ACM SIGOPS Operating Systems Review.ACM,2003,37(5):29-43
[6] 蔡斌,陈湘萍.Hadoop技术内幕[M].北京:机械工业出版,2013:216-217
[7] 武文斌.视频监控云存储模型设计[J].山西科技,2012(3):35-37
[8] Xie J,Yin S,Ruan X,et al.Improving mapreduce performance through data placement in heterogeneous hadoop clusters [C]∥2010 IEEE International Symposium on Parallel & Distributed Processing,Workshops and Phd Forum (IPDPSW).IEEE,2010:1-9
[9] 林伟伟.一种改进的Hadoop数据放置策略[J].华南理工大学学报:自然科学版,2012,6(1):152-158
[10] 刘琨,钮文良.一种改进的Hadoop数据负载均衡算法[J].河南理工大学学报:自然科学版,2013,2(3):332-336
[11] 徐骁勇,潘郁,丁燕艳.基于灰色马尔可夫链预测模型的HDFS云存储副本选择策略[J].计算机应用,2012,1(A02):39-42
[12] Ananthanarayanan G,Agarwal S,Kandula S,et al.Scarlett:co-ping with skewed content popularity in mapreduce clusters [C]∥Proceedings of the sixth conference on Computer systems.ACM,2011:287-300
[13] Abad C L,Lu Y,Campbell R H.DARE:Adaptive data replication for efficient cluster scheduling [C]∥2011 IEEE International Conference on Cluster Computing (CLUSTER).IEEE,2011:159-168
[14] 吴萌.交通监控视频中的异常事件检测[D].北京:北京邮电大学,2010
[15] Massie M L,Chun B N,Culler D E.The ganglia distributed monitoring system:design,implementation,and experience[J].Paral-lel Computing,2004,30(7):817-840

No related articles found!
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!