计算机科学 ›› 2020, Vol. 47 ›› Issue (9): 150-156.doi: 10.11896/jsjkx.190700213

• 计算机图形学&多媒体 • 上一篇    下一篇

基于视差信息的无参考立体图像质量评价

朱玲莹, 桑庆兵, 顾婷婷   

  1. 江南大学物联网工程学院 江苏 无锡214122
  • 收稿日期:2019-07-31 发布日期:2020-09-10
  • 通讯作者: 桑庆兵(sangqb@163.com)
  • 作者简介:zlingyingzly@163.com
  • 基金资助:
    江苏省自然科学基金面上项目(BK20171142)

No-reference Stereo Image Quality Assessment Based on Disparity Information

ZHU Ling-ying, SANG Qing-bing, GU Ting-ting   

  1. School of Internet of Things Engineering,Jiangnan University,Wuxi,Jiangsu 214122,China
  • Received:2019-07-31 Published:2020-09-10
  • About author:ZHU Ling-ying,master student.Her main research interests include image quality assessment and so on.
    SANG Qing-bing,Ph.D,professor.His main research interests include image processing,quality assessment and machine learning.
  • Supported by:
    Natural Science Foundation of Jiangsu Province,China (BK20171142).

摘要: 近年来,随着深度学习在图像质量评价领域的快速发展,平面图像质量评价得到了有效的改善,但是立体图像质量评价还有待提高。为此,文中结合三分支卷积神经网络,提出了基于视差信息的无参考立体图像质量评价方法,并分析了不同视差图对模型性能的影响。该方法将左右视图以及视差图小块作为输入,自动提取特征,通过训练得到回归模型,从而实现对立体图像的预测。文中使用了5种不同立体匹配算法来生成视差图,实验结果表明使用SAD算法得到的效果最好。在立体图像库LIVE3D和MCL3D上的实验结果表明,该方法不仅适用于评估对称失真图像,还适用于非对称失真的立体图像评价。该方法在总体失真上的结果优于其他对比算法,尤其是在MCL3D图像库上,所提方法的PLCC和SROCC比其他方法高出1%和4%。实验数据表明,所提模型提高了立体图像质量评价的性能,与人类主观感知高度一致。

关键词: 非对称失真图像, 卷积神经网络, 立体匹配算法, 立体图像质量评价, 视差信息

Abstract: In recent years,with the rapid development of deep learning in the field of image quality assessment (IQA),2D-IQA has been improved,but 3D-IQA still needs to be improved.Therefore,combining the three-branch convolutional neural network,the paper proposes a no-reference stereo image quality assessment based on disparity information and analyzes the influence of different disparity maps on the performance of the model.The algorithm takes the left/right view patches and the disparity map patches as input,automatically extracts features,and obtains the regression model through training to realize the prediction of the stereo images.In this paper,5 different stereo matching algorithms are used to generate disparity maps.The experimental results show that the SAD algorithm is the best.The experimental results on stereo image databases LIVE3D and MCL3D show that the method is not only suitable for evaluating symmetric distortion images,but also for evaluating asymmetric distortion stereo images.The overall distortion results of this method are superior to other comparison algorithms.Especially on the MCL3D image database,the evaluation method PLCC and SROCC of the proposed method are 1% and 4% higher than other methods.The Experimental data shows that the proposed model improves the performance of stereo image quality assessment,which is highly consistent with human subjective perception.

Key words: Asymmetric distortion image, Convolutional neural network, Disparity information, Stereo image quality assessment, Stereo matching algorithm

中图分类号: 

  • TP183
[1] CHEN M J,LAWRENCE K,CORMACK,et al.No-Refe-rence Quality Assessment of Natural Stereopairs [J].IEEE Trans on Image Processing,2013,22(9):3379-3391.
[2] BENOIT A,CALLET P L,CAMPISI P,et al.Quality Assessment of Stereoscopic Images [J].Eurasip Journal on Image & Video Processing,2008,2008(1):1-13.
[3] ZHANG Y,CHANDLER D M.Learning Natural Statistics of Binocular Contrast for No Reference Quality Assessment of Stereoscopic Images [C]//IEEE International Conference on Image Processing.Washington,USA:IEEE,2018:186-190.
[4] ZHOU W,YU L,QIU W,et al.Utilizing Binocular Vision to Facilitate Completely Blind 3D Image Quality Measurement [J].Signal Processing,2016,129(C):130-136.
[5] APPINA B,KHAN S,CHANNAPPAYYA S.No-referenceStereoscopic Image Quality Assessment Using Natural Scene Statistics [J].Signal Processing Image Communication,2016,43:1-14.
[6] ZHANG W,QU C,MA L,et al.Learning structure of stereoscopic image for no-reference quality assessment with convolutional neural network[J].Pattern Recognition,2016,59(C):176-187.
[7] SHEN L,LEI J,HOU C.No-reference stereoscopic 3D imagequality assessment via combined model[J].Multimedia Tools and Applications,2017,77(7):8195-8212.
[8] LI Y,HU X.No-Reference Stereoscopic Image Quality Assessment Using Natural Scene Statistics [C]//InternationalConfe-rence on Multimedia and Image Processing.Washington,USA:IEEE Computer Society,2017:123-127.
[9] LIN C T,LIU T J,LIU K H.Visual Quality Prediction on Distorted Stereoscopic Images [C]//IEEE International Conference on Image Processing.Washington,USA:IEEE,2018:3480-3484.
[10] DING Y,DENG R,XIE X,et al.No-Reference Stereoscopic Image Quality Assessment Using Convolutional Neural Network for Adaptive Feature Extraction[J].IEEE Access,2018,6:37595-37603.
[11] BOYKOV Y,KOLMOGOROV V.An Experimental Comparison of Min-cut/Max-flow Algorithms for Energy Minimization in Vision [J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2004,26(9):1124-1137.
[12] HIRSCHMULLER H.Stereo Processing by Semiglobal Matching and Mutual Information [J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2008,30(2):328-341.
[13] WEIS D,LAI S H.Fast Template Matching Based on Norma-lized Cross Correlation with Adaptive Multilevel Winner Update [J].IEEE Transactions on Image Processing,2008,17(11):2227-2235.
[14] LI JIAN J Q,LI Y C.Research Progress on Calculation Method of Disparity Map in Stereo Matching [J].Remote Sensing Information,2017,32(2):7-14.
[15] SIMONYAN K,ZISSERMAN A.Very Deep Convolutional Networks for Large-Scale Image Recognition [J].Computer Science,2014:1409-1556.
[16] HARA K,SAITO D,SHOUNO H.Analysis of Function of Rectified Linear Unit Used in Deep Learning [C]//International Joint Conference on Neural Networks.Washington,USA:IEEE,2015:1-8.
[17] ALAIN H,ZIOU D.Image Quality Metrics:PSNR vs. SSIM[C]//2010 International Conference on Pattern Recognition.Washington,USA:IEEE Computer Society,2010:2366-2369.
[18] WANG Z,BOVIK A C,SHEIKH H R,et al.Image Quality Assessment:From Error Visibility to Structural Similarity [J].IEEE Trans on Image Processing,2004,13(4):600-612.
[19] WANG X,MA L,KWONG S,et al.Quaternion representation based visual saliency for stereoscopic image quality assessment[J].Signal Processing,2018,145:202-213.
[20] BENOITA.LE CALLET P,CAMPISI P,et al.Quality Assessment of StereoscopicImages [J].EURASIP Journal on Image and Video Processing,2007,2008(1):1-13.
[21] RYU S,KIM D H,SOHN K.Stereoscopic Image Quality Metric Based on Binocular Perception Model [C]//IEEE International Conference on Image Processing.Washington,USA:IEEE,2013:609-612.
[22] TIAN S,ZHANG L,MORIN L,et al.NIQSV:ANo Reference Image Quality Assessment Metric for 3D Synthesized Views [C]//IEEE International Conference on Acoustics.Washington,USA:IEEE,2017:1248-1252.
[23] FARID M S,LUCENTEFORTE M,GRANGETTO M.Evaluating Virtual Image Quality using the Side-Views Information Fusion and Depth Maps [J].Information Fusion,2018,43:47-56.
[24] YANG J C,SIM K,JIANG B,et al.No-reference stereoscopic image quality assessment based on hue summation-difference mapping image and binocular joint mutual filtering[J].Applied Optics,2018,57(14):3915-3926.
[1] 周乐员, 张剑华, 袁甜甜, 陈胜勇.
多层注意力机制融合的序列到序列中国连续手语识别和翻译
Sequence-to-Sequence Chinese Continuous Sign Language Recognition and Translation with Multi- layer Attention Mechanism Fusion
计算机科学, 2022, 49(9): 155-161. https://doi.org/10.11896/jsjkx.210800026
[2] 李宗民, 张玉鹏, 刘玉杰, 李华.
基于可变形图卷积的点云表征学习
Deformable Graph Convolutional Networks Based Point Cloud Representation Learning
计算机科学, 2022, 49(8): 273-278. https://doi.org/10.11896/jsjkx.210900023
[3] 陈泳全, 姜瑛.
基于卷积神经网络的APP用户行为分析方法
Analysis Method of APP User Behavior Based on Convolutional Neural Network
计算机科学, 2022, 49(8): 78-85. https://doi.org/10.11896/jsjkx.210700121
[4] 朱承璋, 黄嘉儿, 肖亚龙, 王晗, 邹北骥.
基于注意力机制的医学影像深度哈希检索算法
Deep Hash Retrieval Algorithm for Medical Images Based on Attention Mechanism
计算机科学, 2022, 49(8): 113-119. https://doi.org/10.11896/jsjkx.210700153
[5] 檀莹莹, 王俊丽, 张超波.
基于图卷积神经网络的文本分类方法研究综述
Review of Text Classification Methods Based on Graph Convolutional Network
计算机科学, 2022, 49(8): 205-216. https://doi.org/10.11896/jsjkx.210800064
[6] 张颖涛, 张杰, 张睿, 张文强.
全局信息引导的真实图像风格迁移
Photorealistic Style Transfer Guided by Global Information
计算机科学, 2022, 49(7): 100-105. https://doi.org/10.11896/jsjkx.210600036
[7] 戴朝霞, 李锦欣, 张向东, 徐旭, 梅林, 张亮.
基于DNGAN的磁共振图像超分辨率重建算法
Super-resolution Reconstruction of MRI Based on DNGAN
计算机科学, 2022, 49(7): 113-119. https://doi.org/10.11896/jsjkx.210600105
[8] 刘月红, 牛少华, 神显豪.
基于卷积神经网络的虚拟现实视频帧内预测编码
Virtual Reality Video Intraframe Prediction Coding Based on Convolutional Neural Network
计算机科学, 2022, 49(7): 127-131. https://doi.org/10.11896/jsjkx.211100179
[9] 徐鸣珂, 张帆.
Head Fusion:一种提高语音情绪识别的准确性和鲁棒性的方法
Head Fusion:A Method to Improve Accuracy and Robustness of Speech Emotion Recognition
计算机科学, 2022, 49(7): 132-141. https://doi.org/10.11896/jsjkx.210100085
[10] 金方焱, 王秀利.
融合RACNN和BiLSTM的金融领域事件隐式因果关系抽取
Implicit Causality Extraction of Financial Events Integrating RACNN and BiLSTM
计算机科学, 2022, 49(7): 179-186. https://doi.org/10.11896/jsjkx.210500190
[11] 杨玥, 冯涛, 梁虹, 杨扬.
融合交叉注意力机制的图像任意风格迁移
Image Arbitrary Style Transfer via Criss-cross Attention
计算机科学, 2022, 49(6A): 345-352. https://doi.org/10.11896/jsjkx.210700236
[12] 杨健楠, 张帆.
一种结合双注意力机制和层次网络结构的细碎农作物分类方法
Classification Method for Small Crops Combining Dual Attention Mechanisms and Hierarchical Network Structure
计算机科学, 2022, 49(6A): 353-357. https://doi.org/10.11896/jsjkx.210200169
[13] 王杉, 徐楚怡, 师春香, 张瑛.
基于CNN-LSTM的卫星云图云分类方法研究
Study on Cloud Classification Method of Satellite Cloud Images Based on CNN-LSTM
计算机科学, 2022, 49(6A): 675-679. https://doi.org/10.11896/jsjkx.210300177
[14] 孙福权, 崔志清, 邹彭, 张琨.
基于多尺度特征的脑肿瘤分割算法
Brain Tumor Segmentation Algorithm Based on Multi-scale Features
计算机科学, 2022, 49(6A): 12-16. https://doi.org/10.11896/jsjkx.210700217
[15] 吴子斌, 闫巧.
基于动量的映射式梯度下降算法
Projected Gradient Descent Algorithm with Momentum
计算机科学, 2022, 49(6A): 178-183. https://doi.org/10.11896/jsjkx.210500039
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!