基于边缘计算的数据无损压缩方法

doi:10.11896/jsjkx.210500195

计算机科学 ›› 2022, Vol. 49 ›› Issue (11A): 210500195-6.doi: 10.11896/jsjkx.210500195

基于边缘计算的数据无损压缩方法

张小梅^1,2, 曹蓥¹, 娄平^1,2, 江雪梅^1,2, 严俊伟^1,2, 李达^1,2

1 武汉理工大学信息工程学院武汉 430070
2 宽带无线通信和传感器网络湖北省重点实验室武汉 430070

出版日期:2022-11-10 发布日期:2022-11-21
通讯作者: 李达(frankli26@whut.edu.cn)
作者简介:(may125z@126.com)
基金资助:
武汉市科技局应用基础前沿专项(2020010601012176);国家自然科学基金(52075404)

Lossless Data Compression Method Based on Edge Computing

ZHANG Xiao-mei^1,2, CAO Ying¹, LOU Ping^1,2, JIANG Xue-mei^1,2, YAN Jun-wei^1,2, LI Da^1,2

1 School of Information Engineering,Wuhan University of Technology,Wuhan 430070,China
2 Hubei Key Laboratory of Broadband Wireless Communications and Sensor Networks,Wuhan 430070,China

Online:2022-11-10 Published:2022-11-21
About author:ZHANG Xiao-mei,born in 1977,Ph.D,professor,postgraduate supervisor.Her main research interests include intelligent information processing and information security
LI Da,born in 1980,Ph.D,associate professor.His main research interests include statistical signal processing,computer vision,machine learning, and pattern recognition.
Supported by:
Application Foundation Frontier Special Project of Wuhan Science and Technology Bureau(2020010601012176) and National Natural Science Foundation of China(52075404).

摘要/Abstract

摘要： 工业物联网(Industrial Internet of Things,IIoT)可以将各种工业设备、监测仪表以及传感器进行相互连接,设备的运行状态可通过监测仪表与传感器进行全面感知,并根据感知数据对设备状态进行分析与预测。然而要对海量的感知数据进行分析处理需要大量的存储空间与计算能力,将其传送到云平台势必将占用大量的带宽并产生较大的时延,很难满足对设备状态实时分析与诊断的需求。因此,针对工业设备状态感知的数据,提出了基于最优差分和线性拟合降熵变换的无损压缩方法,在数据采集的边缘侧对感知数据进行无损压缩,从而大大提高传输效率,使得感知数据能快速传送到云平台进行分析与处理。该方法根据数据差值序列方差大小和采集频率高低从最优差分和曲线拟合差值中选取有效的降熵变换进行压缩,并采用LZO(Lempel-Ziv-Oberhumer)压缩算法进行二次压缩。在两类不同的数据集上对所提方法进行了测试,实验结果表明,该方法的压缩率最低与最高分别可达77%和93%,同时验证了其无损重构的特性。

关键词: 工业物联网, 边缘计算, 感知数据, 降熵变换, LZO

Abstract: Through the Industrial Internet of Things(IIoT),various industrial equipments,monitoring instruments and sensors can be connected to each other.The operating status of equipment can be fully sensed through monitoring instruments and sensors,and the equipment status can be analyzed and predicted based on sensing data.However,the analysis and processing of massive amounts of sensing data requires a lot of storage space and computing power.Sending it to the cloud platform will inevitably take up a lot of bandwidth and cause a large delay.It is difficult to meet the requirements for real-time analysis and diagnosis of device status.Therefore,for the state-aware data of industrial equipment,a lossless compression method based on optimal diffe-rential and linear fitting entropy reduction transform is proposed,and the perceptual data is compressed losslessly at the edge of data collection,so that the transmission efficiency is greatly improved and the perceptual data can be quickly transmitted to the cloud platform for analysis and processing.This method selects an effective entropy reduction transform from the optimal diffe-rence and curve fitting difference according to the variance of data difference sequence and the acquisition frequency,and uses Lempel-Ziv-Oberhumer(LZO) compression algorithm for secondary compression.The new method is tested on two different data sets.Experimental results show that the lowest and highest compression rate of this method can reach 77% and 93%,respectively.At the same time,the characteristics of its lossless reconstruction are verified.

Key words: Industrial Internet of things, Edge computing, Perceptual data, Reduced entropy transformation, LZO

中图分类号:

TP391

张小梅, 曹蓥, 娄平, 江雪梅, 严俊伟, 李达. 基于边缘计算的数据无损压缩方法[J]. 计算机科学, 2022, 49(11A): 210500195-6. https://doi.org/10.11896/jsjkx.210500195

ZHANG Xiao-mei, CAO Ying, LOU Ping, JIANG Xue-mei, YAN Jun-wei, LI Da. Lossless Data Compression Method Based on Edge Computing[J]. Computer Science, 2022, 49(11A): 210500195-6. https://doi.org/10.11896/jsjkx.210500195

参考文献

[1]ZHANG J L,ZHAO Y C,CHEN B,et al.Survey of Edge Computing Data Security and Privacy Protection Research[J].Journal on Communications,2018,39(3):1-21.
[2]ZHAN X,GUO H,HE X Y,et al.Research on Security RiskAssessment Method of State Grid Edge Computing Information System [J].Computer Science,2019,46(S2):428-432.
[3]Cisco cloud index supplement.Cloud readiness regional details white paper[R/OL].https://blogs.cisco.com/tag/global-cloud-index.
[4]XU D,LI Q,ZHU H.Energy-Saving Computation Offloading by Joint Data Compression and Resource Allocation for Mobile-Edge Computing[J].IEEE Communications Letters,2019,23(4):704-707.
[5]WEI X.MVR:An Architecture for Computation Offloading in Mobile Edge Computing[C]//2017 IEEE International Confe-rence on Edge Computing(EDGE).Honolulu,HI,2017:232-235.
[6]FAN K,PAN Q,WANG J,et al.Cross-Domain Based Data Sharing Scheme in Cooperative Edge Computing[C]//2018 IEEE International Conference on Edge Computing(EDGE),San Francisco,CA,2018:87-92.
[7]CHEN B,WAN J,CELESTI A,et al.Edge Computing inIoT-Based Manufacturing[J].IEEE Communications Magazine,2018,56(9):103-109.
[8]LI S Q,YUE Y,LI Q Q,et al.A streaming data compression method based on Haar wavelet synopses[J].Geomatics and Information Science of Wuhan University,2021,46(8):1216-1223.
[9]LI C X,JIANG Q L,LU F,et al.Data compression method of subway sliding door based on segmented adaptive threshold wavelet[J].Journal of Harbin Institute of Technology,2020,52(9):101-106.
[10]ZHAO Z,LU J,DONG D,et al.Intelligent Terminal Data Compression Method Based on Edge Computing[C]//Proceedings of the 3rd International Conference on Information Technologies and Electrical Engineering.2020:579-584.
[11]LEE J,YOON S,HWANG E.Frequency Selective Auto-Encoder for Smart Meter Data Compression[J].Sensors,2021,21(4):1521.
[12]CHOWDHURY M R,TRIPATHI S,DE S.Adaptive multivariate data compression in smart metering Internet of Things[J].IEEE Transactions on Industrial Informatics,2020,17(2):1287-1297.
[13]AZAR J,DARAZI R,HABIB C,et al.Using dwt lifting scheme for lossless data compression in wireless body sensor networks[C]//2018 14th International Wireless Communications & Mobile Computing Conference(IWCMC).IEEE,2018:1465-1470.
[14]NI F T,ZHANG J,NOORI M N.Deep learning for data ano-maly detection and data compression of a long-span suspension bridge[J].Computer-Aided Civil and Infrastructure Enginee-ring,2020,35(7):685-700.
[15]UTHAYAKUMAR J,VENGATTARAMAN T,DHAVACH-ELVANP.A new lossless neighborhood indexing sequence(NIS) algorithm for data compression in wireless sensor networks[J].Ad Hoc Networks,2019,83(FEB.):149-157.
[16]PARK J,PARK H,CHOI Y.Data compression and prediction using machine learning for industrial IoT[C]//2018 International Conference on Information Networking(ICOIN).Chiang Mai,2018:818-820.
[17]YAO X Z,SHANG J F,CAO J J,et al.A data compressionmethod based on similarity matching for machine and pump monitoring analog data[J].Modular Machine Tool and Automatic Manufacturing Technology,2020(8):83-87,91.
[18]ZHENG Y L,ZHAO Y,ZHAO Y D,et al.Compressed sensing method of forest microclimate monitoring data based on switching dictionary[J].Transactions of the Chinese Society of Agricultural Machinery,2019,50(11):193-199.
[19]AZAR J,MAKHOUL A,BARHAMGI M,et al.An energy efficient IoT data compression approach for edge machine learning[J].Future Generation Computer Systems,2019,96(JUL.):168-175.
[20]FU Z Y.Information Theory-- basic Theory and Application(Second Edition) [M].Beijing:Electronic Industry Press,2007.
[21]REN X J.Research on lossless compression algorithm of sensing data in wireless sensor networks based on entropy drop transform [D].Iuinois:Northwestern University,2011.

相关文章 15

[1]	孙慧婷, 范艳芳, 马孟晓, 陈若愚, 蔡英. VEC中基于动态定价的车辆协同计算卸载方案 Dynamic Pricing-based Vehicle Collaborative Computation Offloading Scheme in VEC 计算机科学, 2022, 49(9): 242-248. https://doi.org/10.11896/jsjkx.210700166
[2]	于滨, 李学华, 潘春雨, 李娜. 基于深度强化学习的边云协同资源分配算法 Edge-Cloud Collaborative Resource Allocation Algorithm Based on Deep Reinforcement Learning 计算机科学, 2022, 49(7): 248-253. https://doi.org/10.11896/jsjkx.210400219
[3]	李梦菲, 毛莺池, 屠子健, 王瑄, 徐淑芳. 基于深度确定性策略梯度的服务器可靠性任务卸载策略 Server-reliability Task Offloading Strategy Based on Deep Deterministic Policy Gradient 计算机科学, 2022, 49(7): 271-279. https://doi.org/10.11896/jsjkx.210600040
[4]	袁昊男, 王瑞锦, 郑博文, 吴邦彦. 基于Fabric的电子病历跨链可信共享系统设计与实现 Design and Implementation of Cross-chain Trusted EMR Sharing System Based on Fabric 计算机科学, 2022, 49(6A): 490-495. https://doi.org/10.11896/jsjkx.210500063
[5]	方韬, 杨旸, 陈佳馨. D2D辅助移动边缘计算下的卸载策略优化 Optimization of Offloading Decisions in D2D-assisted MEC Networks 计算机科学, 2022, 49(6A): 601-605. https://doi.org/10.11896/jsjkx.210200114
[6]	刘漳辉, 郑鸿强, 张建山, 陈哲毅. 多无人机使能移动边缘计算系统中的计算卸载与部署优化 Computation Offloading and Deployment Optimization in Multi-UAV-Enabled Mobile Edge Computing Systems 计算机科学, 2022, 49(6A): 619-627. https://doi.org/10.11896/jsjkx.210600165
[7]	谢万城, 李斌, 代玥玥. 空中智能反射面辅助边缘计算中基于PPO的任务卸载方案 PPO Based Task Offloading Scheme in Aerial Reconfigurable Intelligent Surface-assisted Edge Computing 计算机科学, 2022, 49(6): 3-11. https://doi.org/10.11896/jsjkx.220100249
[8]	周天清, 岳亚莉. 超密集物联网络中多任务多步计算卸载算法研究 Multi-Task and Multi-Step Computation Offloading in Ultra-dense IoT Networks 计算机科学, 2022, 49(6): 12-18. https://doi.org/10.11896/jsjkx.211200147
[9]	彭冬阳, 王睿, 胡谷雨, 祖家琛, 王田丰. 视频缓存策略中QoE和能量效率的公平联合优化 Fair Joint Optimization of QoE and Energy Efficiency in Caching Strategy for Videos 计算机科学, 2022, 49(4): 312-320. https://doi.org/10.11896/jsjkx.210800027
[10]	张海波, 张益峰, 刘开健. 基于NOMA-MEC的车联网任务卸载、迁移与缓存策略 Task Offloading,Migration and Caching Strategy in Internet of Vehicles Based on NOMA-MEC 计算机科学, 2022, 49(2): 304-311. https://doi.org/10.11896/jsjkx.210100157
[11]	林潮伟, 林兵, 陈星. 边缘环境下基于模糊理论的科学工作流调度研究 Study on Scientific Workflow Scheduling Based on Fuzzy Theory Under Edge Environment 计算机科学, 2022, 49(2): 312-320. https://doi.org/10.11896/jsjkx.201000102
[12]	高月红, 陈露. 移动边缘计算中任务卸载研究综述 Survey of Research on Task Offloading in Mobile Edge Computing 计算机科学, 2022, 49(11A): 220400161-7. https://doi.org/10.11896/jsjkx.220400161
[13]	袁昕旺, 谢智东, 谭信. 无人机边缘计算中的资源管理优化研究综述 Survey of Resource Management Optimization of UAV Edge Computing 计算机科学, 2022, 49(11): 234-241. https://doi.org/10.11896/jsjkx.211100015
[14]	胡朝霞, 胡海周, 蒋从锋, 万健. 基于负载特征的边缘智能系统性能优化 Workload Characteristics Based Performance Optimization for Edge Intelligence 计算机科学, 2022, 49(11): 266-276. https://doi.org/10.11896/jsjkx.211000067
[15]	李晓波, 陈鹏, 帅彬, 夏云霓, 李建岐. 边缘环境下轨迹预测性感知的在线边缘服务分配 Novel Predictive Approach to Trajectory-aware Online Edge Service Allocation in Edge Environment 计算机科学, 2022, 49(11): 277-283. https://doi.org/10.11896/jsjkx.211100029

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

基于边缘计算的数据无损压缩方法

Lossless Data Compression Method Based on Edge Computing

PDF (PC)

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

Metrics

本文评价

推荐阅读 0