Computer Science ›› 2016, Vol. 43 ›› Issue (6): 68-71.doi: 10.11896/j.issn.1002-137X.2016.06.014

Previous Articles     Next Articles

Mass Sensor Information Storage Infrastructure Based on Fusion Database

LEI Xing-bang and FANG Jun   

  • Online:2018-12-01 Published:2018-12-01

Abstract: In the internet of things,industrial control and other systems,large-scale sensor generates a large amount of data all the time.The Real-time database has an advantage in terms of time-sensive data processing,while it has a problem in storage capacity and scalability.On the contrary,the HBase has the advantage of high read and write perfor-mance,high scalability and high reliablity.Through the combination of real-time database and HBase,we designed and implemented the sensor information storage architecture based on fusion database.The architecture uses a multi-tenant mechanism to optimize HBase writes,the original sensor data are centrally stored,and the sensor metadata and historical data are stored separately,while maintaining the original real-time database queries,data structure characteristics.Our experiments verify that the system has high read and write performance and good scalability.And it effectively avoids the region write hot and achieves the objective of the cluster load balancing.

Key words: Real-time database,HBase,Sensor data,Centralized storage,Write-optimized

[1] Ding Zhi-ming,Gao Xu.A Database Cluster System Framework for Managing Massive Sensor Sampling Data in the Internet of Things[J].Chinese Journal of Computers,2012,5(6):1175-1191(in Chinese) 丁治明,高需.面向物联网海量传感器采样数据管理的数据库集群系统框架[J].计算机学报,2012,35(6):1175-1191
[2] Chen Qing-kui,Zhou Li-zhen.HBase-based storage system for large-scale data in wireless sensor network[J].Journal of Computer Application,2012,32(7):1920-1923,7(in Chinese) 陈庆奎,周利珍.基于HBase的大规模无线传感网络数据存储系统[J].计算机应用,2012,32(7):1920-1923,7
[3] Lu Ting,Fang Jun,Qiao Yan-ke.HBase-based Real-time Sto-rage System for Traffic Stream Data[J].Journal of Computer Application,2015,35(1):103-107,135(in Chinese) 陆婷,房俊,乔彦克.基于HBase的交通流数据实时存储系统[J].计算机应用,2015,35(1):103-107,135
[4] Lu Hui-ming,Zhou Zhao,Liao Chang-bin.Historical Data Processing Based On Real-time Database System [J].Electric PowerAutomation Equipment,2012,29(3):127-131(in Chinese) 陆会明,周钊,廖常斌.基于实时数据库系统的历史数据处理[J].电力自动化设备,2012,29(3):127-131
[5] George L.HBase:The Definitive Guide[M].2.Inc,USA:O’Reilly Media,2013:339-350
[6] Ku W Y,Chou T Y,Chung L K.The CloudBased Sensor Data Warehouse[C]∥International Symposium on Grids and Clouds and the Open Grid Forum.Taipei,Taiwan,2011:21-24
[7] Carstoiu D,Cernian A,Olteanu A.Hadoop Hbase-0.20.2 performance evaluation[C]∥2010 4th International Conference on New Trends in Information Science and Service Science (NISS).IEEE,2010:84-87
[8] The Apache Software Foundation.http://hadoop.apache.org
[9] Rabl T,Gómez-Villamor S,Sadoghi M,et al.Solving Big Data Challenges for Enterprise Application Performance Management[J].Proceedings of the Vldb Endowment,2012,5(12):1724-1735
[10] Kallman R,Kimura H,Natkins J,et al.H-store:a high-perfor-mance,distributed main memorytransaction processing system[J].PVLDB,2008,1(2):1496-1499
[11] Lakshman A,Malik P.Cassandra:a decentralized structured storage system[J].SIGOPS Operating Systems Review,2010,44(2):35-40
[12] http://planetcassandra.org/nosql-performance-benchmarks

No related articles found!
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!