计算机科学 ›› 2021, Vol. 48 ›› Issue (12): 231-242.doi: 10.11896/jsjkx.201000055

• 计算机图形学&多媒体 • 上一篇    下一篇

图像去雨算法在云物联网应用中的研究综述

张育龙1, 王强1, 陈明康2, 孙静涛3   

  1. 1 武汉理工大学物流工程学院 武汉430070
    2 综合研究大学院大学信息学部 东京101-8430
    3 日本国立信息学研究所信息系统结构研究系 东京101-8430
  • 收稿日期:2020-10-11 修回日期:2021-03-19 出版日期:2021-12-15 发布日期:2021-11-26
  • 通讯作者: 王强(wangqiang@whut.edu.cn)
  • 作者简介:zyl27718842@whut.edu.cn
  • 基金资助:
    国家重点研发计划项目(2018YFC1407405);中央高校基本科研专项资金(WUT2019III103CG);国家自然科学基金面上项目(71672137)

Survey of Intelligent Rain Removal Algorithms for Cloud-IoT Systems

ZHANG Yu-long1, WANG Qiang1, CHEN Ming-kang2, SUN Jing-tao3   

  1. 1 School of Logistics Engineering,Wuhan University of Technology,Wuhan 430070,China
    2 Department of Informatics,the Graduate University for Advanced Studies (SOKENDAI),Tokyo 101-8430,Japan
    3 Information Systems Architecture Research Division,National Institute of Informatics,Tokyo 101-8430,Japan
  • Received:2020-10-11 Revised:2021-03-19 Online:2021-12-15 Published:2021-11-26
  • About author:ZHANG Yu-long,born in 2000,M.S.candidate.His main research interests include computer network architecture,image processing and Internet of Things.
    WANG Qiang,born in 1984,associate professor.His main research interests include multi-agent collaboration and IoT-collaboration.
  • Supported by:
    National Key R&D Program of China(2018YFC1407405), Fundamental Research Funds for the Central Universities(WUT:2019III103CG) and National Natural Science Foundation of China(71672137).

摘要: 《2020 年中国智能物联网(AIoT)白皮书》显示,随着我国5G网络的迅猛发展,大容量低价格的IoT(Internet of Things)传感器设备快速普及,数据呈爆发性增长,图像处理在物联网的诸多领域(如智慧城市、智慧交通、智慧医疗等)得到了广泛应用。在这些领域研究中,科研人员往往相对轻视数据收集过程中的实际问题,如天气变化、季节迁移、昼夜交替等时间变化带来的图像数据退化,以及随着物体移动、叠加、模糊、部分遮挡等诸多空间变化带来的噪声问题。其中,以雨天为代表的复杂天气下的图像模糊问题非常常见,也最具挑战。因此,文中对数据收集过程中的上述实际问题进行了系统性的调查,归类和总结了复杂天气下的图像去雨算法。与此同时,鉴于此类算法的执行需要消耗大量GPU计算资源,文中通过利用Amazon EC2云服务器中G4和P3系列的GPU实例对综述的各种去雨算法的处理时长和去雨效果进行了定量化评估,并阐述了各类去雨算法的特点和在云物联网应用中的最新趋势。

关键词: 图像处理, 去雨算法, 深度学习, 大数据, 云物联网

Abstract: According to the “White Paper on China's Intelligent Internet of Things (AIoT) 2020”,with the prompt development of China's 5G network,the rapid popularization of large-capacity with low-price IoT sensor devices and the explosive growth of data,image processing is widely used in various fields of Internet of Things,such as smart city,smart transportation,smart healthcare,and other industry,etc.In these research areas,researchers usually ignore the actual problems in the data collection process,for instance,data degradation caused by time changes:seasonal shifting,diurnal variation,weather changes,and noise problems caused by spatial changes:object superposition,blur,and partial occlusion.Among those problems,the weather pro-blems represented by rainy days are the most challenging and common.Therefore,this paper systematically investigates the actual problems in the data collection process above,classifies and summarizes the image rain-removal algorithms under complex weather conditions.At the same time,regarding the compute-intensive execution of such algorithms,we utilize the Amazon EC2 cloud instance G4 and P3 series to quantitatively evaluate the processing time and effect of various reviewed rain removal algorithms.Finally,we illustrate the characteristics of various rain removal algorithms and the latest trends in Cloud-IoT applications.

Key words: Image processing, Rain removal algorithm, Deep learning, Big data, Cloud-IoT

中图分类号: 

  • TP391
[1]ABBAS N,ZHANG Y,TAHERKORDI A,et al.Mobile Edge Computing:A Survey[J].IEEE Internet of Things Journal,2018,5(1):450-465.
[2]HONG K,LILLETHUN D,RAMACHANDRAN U,et al.Mobile fog:A programming model for large-scale applications on the internet of things[C]//MCC 2013 - Proceedings of the 2nd,2013 ACM SIGCOMM Workshop on Mobile Cloud Computing.New York,NY,USA:Association for Computing Machinery,2013:15-20.
[3]FARRIS I,MILITANO L,NITTI M,et al.MIFaaS:A Mobile-IoT-Federation-as-a-Service Model for dynamic cooperation of IoT Cloud Providers[J].Future Generation Computer Systems,2017,70(December):126-137.
[4]BAI Y,CHEUNG G,LIU X,et al.Graph-based blind image deblurring from a single photograph[J].IEEE Transactions on Image Processing,2019,28(3):1404-1418.
[5]FU X,HUANG J,DING X,et al.Clearing the skies:A deep net- work architecture for single-image rain removal[J].IEEE Transactions on Image Processing,2017,26(6):2944-2956.
[6]ZHENG Z,WU L,MA H,et al.Going Clear from Misty Rain in Dark Channel Guided Network[C]//IJCAI Workshop on AI for Internet of Things.2017.
[7]WANG J,GUO Y,YING Y,et al.Fast non-local algorithm for image denoising[J].Proceedings - International Conference on Image Processing,ICIP,2006:1429-1432.
[8]LI R,CHEONG L F,TAN R T.Heavy rain image restoration:Integrating physics model and conditional adversarial learning[C]//Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition,2019.2019(1):1633-1642.
[9]EIGEN D,KRISHNAN D,FERGUS R.Restoring an image taken through a window covered with dirt or rain[J].Procee-dings of the IEEE International Conference on Computer Vision,2013:633-640.
[10]YANG W,LIU J,FENG J.Frame-consistent recurrent video deraining with dual-level flow[C]//Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Re-cognition,2019.2019:1661-1670.
[11]WANG Y,ZHANG H,LIU Y,et al.Gradient Information Guided Deraining with A Novel Network and Adversarial Trai-ning[J].arXiv:1910.03839,2019.
[12]PRUSTY P,NAYAK J.Detection and removal of rain from static background videos[J].International Journal of Applied Engineering Research,2015,10(9):22791-22797.
[13]MARSHALL J S,PALMER W M K.the Distribution of Raindrops With Size[J].Journal of Meteorology,1948,5(4):165-166.
[14]FOOTE G B,DU TOIT P S.Terminal Velocity of Raindrops Aloft[J].Journal of Applied Meteorology,1969,8(2):249-253.
[15]BARNUM P,KANADE T,NARASIMHAN S.Spatio-Temporal Frequency Analysis for Removing Rain and Snow from Vi-deos[J].the First International Workshop on Photometric Analysis for Computer Vision,2007:8.
[16]KANG L W,LIN C W,FU Y H.Automatic single-image-based rain streaks removal via image decomposition[J].IEEE Tran-sactions on Image Processing,2012,21(4):1742-1755.
[17]LUO Y,XU Y,JI H.Removing rain from a single image via discriminative sparse coding[C]//Proceedings of the IEEE International Conference on Computer Vision,2015.2015:3397-3405.
[18]HU X,FU C W,ZHU L,et al.Depth-attentional features for single-image rain removal[C]//Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Re-cognition,2019.2019:8014-8023.
[19]YANG W,TAN R T,WANG S,et al.Single Image Deraining:From Model-Based to Data-Driven and Beyond[J].IEEE Tran-sactions on Pattern Analysis and Machine Intelligence,2020(2012):1-1.
[20]KIM J H,LEE C,SIM J Y,et al.Single-image deraining using an adaptive nonlocal means filter[C]//2013 IEEE International Conference on Image Processing(ICIP 2013).IEEE,2013(1):914-917.
[21]BOSSU J,HAUTIÈRE N,TAREL J P.Rain or snow detection in image sequences through use of a histogram of orientation of streaks[J].International Journal of Computer Vision,2011,93(3):348-367.
[22]KANG L W,LIN C W,LIN C T,et al.Self-learning-based rain streak removal for image/video[C]//2012 IEEE International Symposium on Circuits and Systems(ISCAS 2012).2012:1871-1874.
[23]RAMYA C,SUBHA RANI S.Rain removal in image sequence using sparse coding[J].Communications in Computer and Information Science,2012,330 CCIS:361-370.
[24]ZHENG X,LIAO Y,GUO W,et al.Single-image-based rain and snow removal using multi-guided filter[C]//Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics).2013,8228 LNCS(PART 3):258-265.
[25]CHEN D Y,CHEN C C,KANG L W.Visual depth guided color image rain streaks removal using sparse coding[J].IEEETran-sactions on Circuits and Systems for Video Technology,2014,24(8):1430-1455.
[26]XU J,ZHAO W,LIU P,et al.An Improved Guidance Image Based Method to Remove Rain and Snow in a Single Image[J].Computer and Information Science,2012,5(3):49-55.
[27]TALLEY N J.Update on the role of drug therapy in non-ulcer dyspepsia[J].Reviews in Gastroenterological Disorders,2003,3(1):25-30.
[28]SHIH Y,KRISHNAN D,DURAND F,et al.Reflection removal using ghosting cues[C]//Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition.2015:3193-3201.
[29]LI Y,TAN R T,GUO X,et al.Rain Streak Removal Using La- yer Priors[J].Proceedings of the IEEE Computer Society Confe-rence on Computer Vision and Pattern Recognition,2016:2736-2744.
[30]LIU Y F,JAW D W,HUANG S C,et al.DesnowNet:Context-Aware Deep Network for Snow Removal[J].IEEE Transactions on Image Processing,IEEE,2018,27(6):3064-3073.
[31]HUANG D A,KANG L W,WANG Y C F,et al.Self-learning based image decomposition with applications to single image denoising[J].IEEE Transactions on Multimedia,IEEE,2014,16(1):83-93.
[32]WANG Y,CHEN C,ZHU S,et al.A framework of single-image deraining method based on analysis of rain characteristics[C]//Proceedings - International Conference on Image Processing.IEEE,2016:4087-4091.
[33]GHOSH S,MANDAL A K,CHAUDHURY K N.Pruned non-local means[J].IET Image Processing,2017,11(5):317-323.
[34]DONG C,LOY C C,HE K,et al.Image Super-Resolution Using Deep Convolutional Networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2016,38(2):295-307.
[35]XIE J,XU L,CHEN E.Image denoising and inpainting with deep neural networks[J].Advances in Neural Information Processing Systems,2012,1:341-349.
[36]HUANG D A,KANG L W,YANG M C,et al.Context-aware single image rain removal[C]//Proceedings-IEEE InternationalConference on Multimedia and Expo.2012(July):164-169.
[37]YANG W,TAN R T,FENG J,et al.Deep joint rain detection and removal from a single image[C]//Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition(CVPR 2017).2017:1685-1694.
[38]ZHANG H,SINDAGI V,PATEL V M.Image De-raining Using a Conditional Generative Adversarial Network[C]//IEEE Transactions on Circuits and Systems for Video Technology.2019:1-1.
[39]QIAN R,TAN R T,YANG W,et al.Attentive Generative Adversarial Network for Raindrop Removal from A Single Image[C]//Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition.2018:2482-2491.
[40]FU X,HUANG J,ZENG D,et al.Removing rain from single images via a deep detail network[C]//Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition(CVPR 2017).2017:1715-1723.
[41]FU X,LIANG B,HUANG Y,et al.Lightweight Pyramid Networks for Image Deraining[J].IEEE Transactions on Neural Networks and Learning Systems,2020,31(6):1794-1807.
[42]REN D,ZUO W,HU Q,et al.Progressive image deraining networks:A better and simpler baseline[C]//Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition.2019:3932-3941.
[43]YASARLA R,SINDAGI V A,PATEL V M.Syn2Real Transfer Learning for Image Deraining Using Gaussian Processes [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:2726-2736.
[44]LIU L,LIU B,HUANG H,et al.No-reference image quality assessment based on spatial and spectral entropies[J].Signal Pro-cessing:Image Communication,Elsevier,2014,29(8):856-863.
[45]WANG Z M.Review of no-reference image quality assessment[J].Zidonghua Xuebao/Acta Automatica Sinica,2015,41(6):1062-1079.
[46]WU Y,LING H,YU J,et al.Blurred target tracking by blur-driven tracker[C]//Proceedings of the IEEE International Conference on Computer Vision.2011:1100-1107.
[47]ZHAO S,ZHANG S,ZHANG L.Towards occlusion handling:Object tracking with background estimation[J].IEEE Transactions on Cybernetics,2018,48(7):2086-2100.
[48]HENRIQUES J F,CASEIRO R,MARTINS P,et al.High- speed tracking with kernelized correlation filters[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2015,37(3):583-596.
[49]LI M,LIU J,YANG W,et al.Structure-Revealing Low-Light Image Enhancement Via Robust Retinex Model[J].IEEE Transactions on Image Processing,2018,27(6):2828-2841.
[50]SUN J,YANG C,TANJO T,et al.Implementation of self-adaptive middleware for mobile vehicle tracking applications on edge computing[C]//International Conference on Internet and Distributed Computing Systems.Springer,Cham,2018:1-15.
[1] 董晓梅, 王蕊, 邹欣开. 面向推荐应用的差分隐私方案综述[J]. 计算机科学, 2021, 48(9): 21-35.
[2] 王俊, 王修来, 庞威, 赵鸿飞. 面向科技前瞻预测的大数据治理研究[J]. 计算机科学, 2021, 48(9): 36-42.
[3] 周新民, 胡宜桂, 刘文洁, 孙荣俊. 基于多模态多层级数据融合方法的城市功能识别研究[J]. 计算机科学, 2021, 48(9): 50-58.
[4] 钱梦薇, 过弋. 融合偏置深度学习的距离分解Top-N推荐算法[J]. 计算机科学, 2021, 48(9): 103-109.
[5] 余乐章, 夏天宇, 荆一楠, 何震瀛, 王晓阳. 面向大数据分析的智能交互向导系统[J]. 计算机科学, 2021, 48(9): 110-117.
[6] 徐涛, 田崇阳, 刘才华. 基于深度学习的人群异常行为检测综述[J]. 计算机科学, 2021, 48(9): 125-134.
[7] 张新峰, 宋博. 一种基于改进三元组损失和特征融合的行人重识别方法[J]. 计算机科学, 2021, 48(9): 146-152.
[8] 林椹尠, 张梦凯, 吴成茂, 郑兴宁. 利用生成对抗网络的人脸图像分步补全法[J]. 计算机科学, 2021, 48(9): 174-180.
[9] 黄晓生, 徐静. 基于PCANet的非下采样剪切波域多聚焦图像融合[J]. 计算机科学, 2021, 48(9): 181-186.
[10] 田野, 陈宏巍, 王法胜, 陈兴文. 室内移动机器人的SLAM算法综述[J]. 计算机科学, 2021, 48(9): 223-234.
[11] 谢良旭, 李峰, 谢建平, 许晓军. 基于融合神经网络模型的药物分子性质预测[J]. 计算机科学, 2021, 48(9): 251-256.
[12] 冯霞, 胡志毅, 刘才华. 跨模态检索研究进展综述[J]. 计算机科学, 2021, 48(8): 13-23.
[13] 王立梅, 朱旭光, 汪德嘉, 张勇, 邢春晓. 基于深度学习的民事案件判决结果分类方法研究[J]. 计算机科学, 2021, 48(8): 80-85.
[14] 郭琳, 李晨, 陈晨, 赵睿, 范仕霖, 徐星雨. 基于通道注意递归残差网络的图像超分辨率重建[J]. 计算机科学, 2021, 48(8): 139-144.
[15] 刘帅, 芮挺, 胡育成, 杨成松, 王东. 基于深度学习SuperGlue算法的单目视觉里程计[J]. 计算机科学, 2021, 48(8): 157-161.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] 蒋苏蓉,蓝江桥,杨玉海. Hadoop框架下的情报分析大数据调度超时预测方法[J]. 计算机科学, 2014, 41(Z6): 409 -413 .
[2] 管涛,牛晓太. α稳定分布及其在斑点抑制中的应用研究[J]. 计算机科学, 2014, 41(Z6): 196 -202 .
[3] 王定成, 陆一祎, 邹勇杰. 多输出直觉模糊最小二乘支持向量回归算法[J]. 计算机科学, 2019, 46(5): 163 -168 .
[4] 周凯, 任怡, 汪哲, 管剑波, 张芳, 赵言亢. 基于主题模型的Ubuntu操作系统缺陷报告的分类及分析[J]. 计算机科学, 2020, 47(12): 35 -41 .
[5] 潘孝勤, 芦天亮, 杜彦辉, 仝鑫. 基于深度学习的语音合成与转换技术综述[J]. 计算机科学, 2021, 48(8): 200 -208 .
[6] 王俊, 王修来, 庞威, 赵鸿飞. 面向科技前瞻预测的大数据治理研究[J]. 计算机科学, 2021, 48(9): 36 -42 .
[7] 余力, 杜启翰, 岳博妍, 向君瑶, 徐冠宇, 冷友方. 基于强化学习的推荐研究综述[J]. 计算机科学, 2021, 48(10): 1 -18 .
[8] 王梓强, 胡晓光, 李晓筱, 杜卓群. 移动机器人全局路径规划算法综述[J]. 计算机科学, 2021, 48(10): 19 -29 .
[9] 高洪皓, 郑子彬, 殷昱煜, 丁勇. 区块链技术专题序言[J]. 计算机科学, 2021, 48(11): 1 -3 .
[10] 毛瀚宇, 聂铁铮, 申德荣, 于戈, 徐石成, 何光宇. 区块链即服务平台关键技术及发展综述[J]. 计算机科学, 2021, 48(11): 4 -11 .