计算机科学 ›› 2020, Vol. 47 ›› Issue (9): 150-156.doi: 10.11896/jsjkx.190700213

• 计算机图形学&多媒体 • 上一篇    下一篇

基于视差信息的无参考立体图像质量评价

朱玲莹, 桑庆兵, 顾婷婷   

  1. 江南大学物联网工程学院 江苏 无锡214122
  • 收稿日期:2019-07-31 发布日期:2020-09-10
  • 通讯作者: 桑庆兵(sangqb@163.com)
  • 作者简介:zlingyingzly@163.com
  • 基金资助:
    江苏省自然科学基金面上项目(BK20171142)

No-reference Stereo Image Quality Assessment Based on Disparity Information

ZHU Ling-ying, SANG Qing-bing, GU Ting-ting   

  1. School of Internet of Things Engineering,Jiangnan University,Wuxi,Jiangsu 214122,China
  • Received:2019-07-31 Published:2020-09-10
  • About author:ZHU Ling-ying,master student.Her main research interests include image quality assessment and so on.
    SANG Qing-bing,Ph.D,professor.His main research interests include image processing,quality assessment and machine learning.
  • Supported by:
    Natural Science Foundation of Jiangsu Province,China (BK20171142).

摘要: 近年来,随着深度学习在图像质量评价领域的快速发展,平面图像质量评价得到了有效的改善,但是立体图像质量评价还有待提高。为此,文中结合三分支卷积神经网络,提出了基于视差信息的无参考立体图像质量评价方法,并分析了不同视差图对模型性能的影响。该方法将左右视图以及视差图小块作为输入,自动提取特征,通过训练得到回归模型,从而实现对立体图像的预测。文中使用了5种不同立体匹配算法来生成视差图,实验结果表明使用SAD算法得到的效果最好。在立体图像库LIVE3D和MCL3D上的实验结果表明,该方法不仅适用于评估对称失真图像,还适用于非对称失真的立体图像评价。该方法在总体失真上的结果优于其他对比算法,尤其是在MCL3D图像库上,所提方法的PLCC和SROCC比其他方法高出1%和4%。实验数据表明,所提模型提高了立体图像质量评价的性能,与人类主观感知高度一致。

关键词: 视差信息, 立体匹配算法, 卷积神经网络, 立体图像质量评价, 非对称失真图像

Abstract: In recent years,with the rapid development of deep learning in the field of image quality assessment (IQA),2D-IQA has been improved,but 3D-IQA still needs to be improved.Therefore,combining the three-branch convolutional neural network,the paper proposes a no-reference stereo image quality assessment based on disparity information and analyzes the influence of different disparity maps on the performance of the model.The algorithm takes the left/right view patches and the disparity map patches as input,automatically extracts features,and obtains the regression model through training to realize the prediction of the stereo images.In this paper,5 different stereo matching algorithms are used to generate disparity maps.The experimental results show that the SAD algorithm is the best.The experimental results on stereo image databases LIVE3D and MCL3D show that the method is not only suitable for evaluating symmetric distortion images,but also for evaluating asymmetric distortion stereo images.The overall distortion results of this method are superior to other comparison algorithms.Especially on the MCL3D image database,the evaluation method PLCC and SROCC of the proposed method are 1% and 4% higher than other methods.The Experimental data shows that the proposed model improves the performance of stereo image quality assessment,which is highly consistent with human subjective perception.

Key words: Disparity information, Stereo matching algorithm, Convolutional neural network, Stereo image quality assessment, Asymmetric distortion image

中图分类号: 

  • TP183
[1] CHEN M J,LAWRENCE K,CORMACK,et al.No-Refe-rence Quality Assessment of Natural Stereopairs [J].IEEE Trans on Image Processing,2013,22(9):3379-3391.
[2] BENOIT A,CALLET P L,CAMPISI P,et al.Quality Assessment of Stereoscopic Images [J].Eurasip Journal on Image & Video Processing,2008,2008(1):1-13.
[3] ZHANG Y,CHANDLER D M.Learning Natural Statistics of Binocular Contrast for No Reference Quality Assessment of Stereoscopic Images [C]//IEEE International Conference on Image Processing.Washington,USA:IEEE,2018:186-190.
[4] ZHOU W,YU L,QIU W,et al.Utilizing Binocular Vision to Facilitate Completely Blind 3D Image Quality Measurement [J].Signal Processing,2016,129(C):130-136.
[5] APPINA B,KHAN S,CHANNAPPAYYA S.No-referenceStereoscopic Image Quality Assessment Using Natural Scene Statistics [J].Signal Processing Image Communication,2016,43:1-14.
[6] ZHANG W,QU C,MA L,et al.Learning structure of stereoscopic image for no-reference quality assessment with convolutional neural network[J].Pattern Recognition,2016,59(C):176-187.
[7] SHEN L,LEI J,HOU C.No-reference stereoscopic 3D imagequality assessment via combined model[J].Multimedia Tools and Applications,2017,77(7):8195-8212.
[8] LI Y,HU X.No-Reference Stereoscopic Image Quality Assessment Using Natural Scene Statistics [C]//InternationalConfe-rence on Multimedia and Image Processing.Washington,USA:IEEE Computer Society,2017:123-127.
[9] LIN C T,LIU T J,LIU K H.Visual Quality Prediction on Distorted Stereoscopic Images [C]//IEEE International Conference on Image Processing.Washington,USA:IEEE,2018:3480-3484.
[10] DING Y,DENG R,XIE X,et al.No-Reference Stereoscopic Image Quality Assessment Using Convolutional Neural Network for Adaptive Feature Extraction[J].IEEE Access,2018,6:37595-37603.
[11] BOYKOV Y,KOLMOGOROV V.An Experimental Comparison of Min-cut/Max-flow Algorithms for Energy Minimization in Vision [J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2004,26(9):1124-1137.
[12] HIRSCHMULLER H.Stereo Processing by Semiglobal Matching and Mutual Information [J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2008,30(2):328-341.
[13] WEIS D,LAI S H.Fast Template Matching Based on Norma-lized Cross Correlation with Adaptive Multilevel Winner Update [J].IEEE Transactions on Image Processing,2008,17(11):2227-2235.
[14] LI JIAN J Q,LI Y C.Research Progress on Calculation Method of Disparity Map in Stereo Matching [J].Remote Sensing Information,2017,32(2):7-14.
[15] SIMONYAN K,ZISSERMAN A.Very Deep Convolutional Networks for Large-Scale Image Recognition [J].Computer Science,2014:1409-1556.
[16] HARA K,SAITO D,SHOUNO H.Analysis of Function of Rectified Linear Unit Used in Deep Learning [C]//International Joint Conference on Neural Networks.Washington,USA:IEEE,2015:1-8.
[17] ALAIN H,ZIOU D.Image Quality Metrics:PSNR vs. SSIM[C]//2010 International Conference on Pattern Recognition.Washington,USA:IEEE Computer Society,2010:2366-2369.
[18] WANG Z,BOVIK A C,SHEIKH H R,et al.Image Quality Assessment:From Error Visibility to Structural Similarity [J].IEEE Trans on Image Processing,2004,13(4):600-612.
[19] WANG X,MA L,KWONG S,et al.Quaternion representation based visual saliency for stereoscopic image quality assessment[J].Signal Processing,2018,145:202-213.
[20] BENOITA.LE CALLET P,CAMPISI P,et al.Quality Assessment of StereoscopicImages [J].EURASIP Journal on Image and Video Processing,2007,2008(1):1-13.
[21] RYU S,KIM D H,SOHN K.Stereoscopic Image Quality Metric Based on Binocular Perception Model [C]//IEEE International Conference on Image Processing.Washington,USA:IEEE,2013:609-612.
[22] TIAN S,ZHANG L,MORIN L,et al.NIQSV:ANo Reference Image Quality Assessment Metric for 3D Synthesized Views [C]//IEEE International Conference on Acoustics.Washington,USA:IEEE,2017:1248-1252.
[23] FARID M S,LUCENTEFORTE M,GRANGETTO M.Evaluating Virtual Image Quality using the Side-Views Information Fusion and Depth Maps [J].Information Fusion,2018,43:47-56.
[24] YANG J C,SIM K,JIANG B,et al.No-reference stereoscopic image quality assessment based on hue summation-difference mapping image and binocular joint mutual filtering[J].Applied Optics,2018,57(14):3915-3926.
[1] 单美静, 秦龙飞, 张会兵. L-YOLO:适用于车载边缘计算的实时交通标识检测模型[J]. 计算机科学, 2021, 48(1): 89-95.
[2] 何彦辉, 吴桂兴, 吴志强. 基于域适应的X光图像的目标检测[J]. 计算机科学, 2021, 48(1): 175-181.
[3] 李亚男, 胡宇佳, 甘伟, 朱敏. 基于深度学习的miRNA靶位点预测研究综述[J]. 计算机科学, 2021, 48(1): 209-216.
[4] 张佳嘉, 张小洪. 多分支卷积神经网络肺结节分类方法及其可解释性[J]. 计算机科学, 2020, 47(9): 129-134.
[5] 崔彤彤, 王桂玲, 高晶. 基于1DCNN-LSTM的船舶轨迹分类方法[J]. 计算机科学, 2020, 47(9): 175-184.
[6] 刘海潮, 王莉. 基于深度图卷积胶囊网络的图分类模型[J]. 计算机科学, 2020, 47(9): 219-225.
[7] 梁正友, 何景琳, 孙宇. 一种用于微表情自动识别的三维卷积神经网络进化方法[J]. 计算机科学, 2020, 47(8): 227-232.
[8] 孙彦丽, 叶炯耀. 基于剪枝与量化的卷积神经网络压缩方法[J]. 计算机科学, 2020, 47(8): 261-266.
[9] 程哲, 白茜, 张浩, 王世普, 梁宇. 使用深层卷积神经网络提高Hi-C 数据分辨率[J]. 计算机科学, 2020, 47(6A): 70-74.
[10] 赫磊, 邵展鹏, 张剑华, 周小龙. 基于深度学习的行为识别算法综述[J]. 计算机科学, 2020, 47(6A): 139-147.
[11] 孙正, 王新宇. 深度学习在光声成像中的应用现状[J]. 计算机科学, 2020, 47(6A): 148-152.
[12] 马海江. 基于卷积神经网络与约束概率矩阵分解的推荐算法[J]. 计算机科学, 2020, 47(6A): 540-545.
[13] 王燕, 王丽. 面向高光谱图像分类的局部Gabor卷积神经网络[J]. 计算机科学, 2020, 47(6): 151-156.
[14] 汪航, 陈晓, 田晟兆, 陈端兵. 基于小样本学习的SAR图像识别[J]. 计算机科学, 2020, 47(5): 124-128.
[15] 蔡强, 邓毅彪, 李海生, 余乐, 明少锋. 基于深度学习的人体行为识别方法综述[J]. 计算机科学, 2020, 47(4): 85-93.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] 伍建辉,黄中祥,李武,吴健辉,彭鑫,张生. 城市道路建设时序决策的鲁棒优化[J]. 计算机科学, 2018, 45(4): 89 -93 .
[2] 史雯隽,武继刚,罗裕春. 针对移动云计算任务迁移的快速高效调度算法[J]. 计算机科学, 2018, 45(4): 94 -99 .
[3] 周燕萍,业巧林. 基于L1-范数距离的最小二乘对支持向量机[J]. 计算机科学, 2018, 45(4): 100 -105 .
[4] 王振朝,侯欢欢,连蕊. 抑制CMT中乱序程度的路径优化方案[J]. 计算机科学, 2018, 45(4): 122 -125 .
[5] 杨羽琦,章国安,金喜龙. 车载自组织网络中基于车辆密度的双簇头路由协议[J]. 计算机科学, 2018, 45(4): 126 -130 .
[6] 战芸娇,魏欧,胡军. 面向DO-178C的襟缝翼控制系统需求的形式化描述[J]. 计算机科学, 2018, 45(4): 196 -202 .
[7] 侯彦娥,孔云峰,党兰学. 求解多车型校车路径问题的混合集合划分的GRASP算法[J]. 计算机科学, 2018, 45(4): 240 -246 .
[8] 李昊阳,符云清. 基于标签聚类与项目主题的协同过滤推荐算法[J]. 计算机科学, 2018, 45(4): 247 -251 .
[9] 崔建京,龙军,闵尔学,于洋,殷建平. 同态加密在加密机器学习中的应用研究综述[J]. 计算机科学, 2018, 45(4): 46 -52 .
[10] 杜秀丽,顾斌斌,胡兴,邱少明,陈波. 用于图像重构的基于行间支撑集相似度的CoSaMP算法[J]. 计算机科学, 2018, 45(4): 306 -311 .