Computer Science ›› 2020, Vol. 47 ›› Issue (9): 150-156.doi: 10.11896/jsjkx.190700213

• Computer Graphics & Multimedia • Previous Articles     Next Articles

No-reference Stereo Image Quality Assessment Based on Disparity Information

ZHU Ling-ying, SANG Qing-bing, GU Ting-ting   

  1. School of Internet of Things Engineering,Jiangnan University,Wuxi,Jiangsu 214122,China
  • Received:2019-07-31 Published:2020-09-10
  • About author:ZHU Ling-ying,master student.Her main research interests include image quality assessment and so on.
    SANG Qing-bing,Ph.D,professor.His main research interests include image processing,quality assessment and machine learning.
  • Supported by:
    Natural Science Foundation of Jiangsu Province,China (BK20171142).

Abstract: In recent years,with the rapid development of deep learning in the field of image quality assessment (IQA),2D-IQA has been improved,but 3D-IQA still needs to be improved.Therefore,combining the three-branch convolutional neural network,the paper proposes a no-reference stereo image quality assessment based on disparity information and analyzes the influence of different disparity maps on the performance of the model.The algorithm takes the left/right view patches and the disparity map patches as input,automatically extracts features,and obtains the regression model through training to realize the prediction of the stereo images.In this paper,5 different stereo matching algorithms are used to generate disparity maps.The experimental results show that the SAD algorithm is the best.The experimental results on stereo image databases LIVE3D and MCL3D show that the method is not only suitable for evaluating symmetric distortion images,but also for evaluating asymmetric distortion stereo images.The overall distortion results of this method are superior to other comparison algorithms.Especially on the MCL3D image database,the evaluation method PLCC and SROCC of the proposed method are 1% and 4% higher than other methods.The Experimental data shows that the proposed model improves the performance of stereo image quality assessment,which is highly consistent with human subjective perception.

Key words: Asymmetric distortion image, Convolutional neural network, Disparity information, Stereo image quality assessment, Stereo matching algorithm

CLC Number: 

  • TP183
[1] CHEN M J,LAWRENCE K,CORMACK,et al.No-Refe-rence Quality Assessment of Natural Stereopairs [J].IEEE Trans on Image Processing,2013,22(9):3379-3391.
[2] BENOIT A,CALLET P L,CAMPISI P,et al.Quality Assessment of Stereoscopic Images [J].Eurasip Journal on Image & Video Processing,2008,2008(1):1-13.
[3] ZHANG Y,CHANDLER D M.Learning Natural Statistics of Binocular Contrast for No Reference Quality Assessment of Stereoscopic Images [C]//IEEE International Conference on Image Processing.Washington,USA:IEEE,2018:186-190.
[4] ZHOU W,YU L,QIU W,et al.Utilizing Binocular Vision to Facilitate Completely Blind 3D Image Quality Measurement [J].Signal Processing,2016,129(C):130-136.
[5] APPINA B,KHAN S,CHANNAPPAYYA S.No-referenceStereoscopic Image Quality Assessment Using Natural Scene Statistics [J].Signal Processing Image Communication,2016,43:1-14.
[6] ZHANG W,QU C,MA L,et al.Learning structure of stereoscopic image for no-reference quality assessment with convolutional neural network[J].Pattern Recognition,2016,59(C):176-187.
[7] SHEN L,LEI J,HOU C.No-reference stereoscopic 3D imagequality assessment via combined model[J].Multimedia Tools and Applications,2017,77(7):8195-8212.
[8] LI Y,HU X.No-Reference Stereoscopic Image Quality Assessment Using Natural Scene Statistics [C]//InternationalConfe-rence on Multimedia and Image Processing.Washington,USA:IEEE Computer Society,2017:123-127.
[9] LIN C T,LIU T J,LIU K H.Visual Quality Prediction on Distorted Stereoscopic Images [C]//IEEE International Conference on Image Processing.Washington,USA:IEEE,2018:3480-3484.
[10] DING Y,DENG R,XIE X,et al.No-Reference Stereoscopic Image Quality Assessment Using Convolutional Neural Network for Adaptive Feature Extraction[J].IEEE Access,2018,6:37595-37603.
[11] BOYKOV Y,KOLMOGOROV V.An Experimental Comparison of Min-cut/Max-flow Algorithms for Energy Minimization in Vision [J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2004,26(9):1124-1137.
[12] HIRSCHMULLER H.Stereo Processing by Semiglobal Matching and Mutual Information [J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2008,30(2):328-341.
[13] WEIS D,LAI S H.Fast Template Matching Based on Norma-lized Cross Correlation with Adaptive Multilevel Winner Update [J].IEEE Transactions on Image Processing,2008,17(11):2227-2235.
[14] LI JIAN J Q,LI Y C.Research Progress on Calculation Method of Disparity Map in Stereo Matching [J].Remote Sensing Information,2017,32(2):7-14.
[15] SIMONYAN K,ZISSERMAN A.Very Deep Convolutional Networks for Large-Scale Image Recognition [J].Computer Science,2014:1409-1556.
[16] HARA K,SAITO D,SHOUNO H.Analysis of Function of Rectified Linear Unit Used in Deep Learning [C]//International Joint Conference on Neural Networks.Washington,USA:IEEE,2015:1-8.
[17] ALAIN H,ZIOU D.Image Quality Metrics:PSNR vs. SSIM[C]//2010 International Conference on Pattern Recognition.Washington,USA:IEEE Computer Society,2010:2366-2369.
[18] WANG Z,BOVIK A C,SHEIKH H R,et al.Image Quality Assessment:From Error Visibility to Structural Similarity [J].IEEE Trans on Image Processing,2004,13(4):600-612.
[19] WANG X,MA L,KWONG S,et al.Quaternion representation based visual saliency for stereoscopic image quality assessment[J].Signal Processing,2018,145:202-213.
[20] BENOITA.LE CALLET P,CAMPISI P,et al.Quality Assessment of StereoscopicImages [J].EURASIP Journal on Image and Video Processing,2007,2008(1):1-13.
[21] RYU S,KIM D H,SOHN K.Stereoscopic Image Quality Metric Based on Binocular Perception Model [C]//IEEE International Conference on Image Processing.Washington,USA:IEEE,2013:609-612.
[22] TIAN S,ZHANG L,MORIN L,et al.NIQSV:ANo Reference Image Quality Assessment Metric for 3D Synthesized Views [C]//IEEE International Conference on Acoustics.Washington,USA:IEEE,2017:1248-1252.
[23] FARID M S,LUCENTEFORTE M,GRANGETTO M.Evaluating Virtual Image Quality using the Side-Views Information Fusion and Depth Maps [J].Information Fusion,2018,43:47-56.
[24] YANG J C,SIM K,JIANG B,et al.No-reference stereoscopic image quality assessment based on hue summation-difference mapping image and binocular joint mutual filtering[J].Applied Optics,2018,57(14):3915-3926.
[1] ZHOU Le-yuan, ZHANG Jian-hua, YUAN Tian-tian, CHEN Sheng-yong. Sequence-to-Sequence Chinese Continuous Sign Language Recognition and Translation with Multi- layer Attention Mechanism Fusion [J]. Computer Science, 2022, 49(9): 155-161.
[2] CHEN Yong-quan, JIANG Ying. Analysis Method of APP User Behavior Based on Convolutional Neural Network [J]. Computer Science, 2022, 49(8): 78-85.
[3] ZHU Cheng-zhang, HUANG Jia-er, XIAO Ya-long, WANG Han, ZOU Bei-ji. Deep Hash Retrieval Algorithm for Medical Images Based on Attention Mechanism [J]. Computer Science, 2022, 49(8): 113-119.
[4] DAI Zhao-xia, LI Jin-xin, ZHANG Xiang-dong, XU Xu, MEI Lin, ZHANG Liang. Super-resolution Reconstruction of MRI Based on DNGAN [J]. Computer Science, 2022, 49(7): 113-119.
[5] LIU Yue-hong, NIU Shao-hua, SHEN Xian-hao. Virtual Reality Video Intraframe Prediction Coding Based on Convolutional Neural Network [J]. Computer Science, 2022, 49(7): 127-131.
[6] XU Ming-ke, ZHANG Fan. Head Fusion:A Method to Improve Accuracy and Robustness of Speech Emotion Recognition [J]. Computer Science, 2022, 49(7): 132-141.
[7] WU Zi-bin, YAN Qiao. Projected Gradient Descent Algorithm with Momentum [J]. Computer Science, 2022, 49(6A): 178-183.
[8] ZHANG Jia-hao, LIU Feng, QI Jia-yin. Lightweight Micro-expression Recognition Architecture Based on Bottleneck Transformer [J]. Computer Science, 2022, 49(6A): 370-377.
[9] WANG Jian-ming, CHEN Xiang-yu, YANG Zi-zhong, SHI Chen-yang, ZHANG Yu-hang, QIAN Zheng-kun. Influence of Different Data Augmentation Methods on Model Recognition Accuracy [J]. Computer Science, 2022, 49(6A): 418-423.
[10] SUN Jie-qi, LI Ya-feng, ZHANG Wen-bo, LIU Peng-hui. Dual-field Feature Fusion Deep Convolutional Neural Network Based on Discrete Wavelet Transformation [J]. Computer Science, 2022, 49(6A): 434-440.
[11] YANG Yue, FENG Tao, LIANG Hong, YANG Yang. Image Arbitrary Style Transfer via Criss-cross Attention [J]. Computer Science, 2022, 49(6A): 345-352.
[12] YANG Jian-nan, ZHANG Fan. Classification Method for Small Crops Combining Dual Attention Mechanisms and Hierarchical Network Structure [J]. Computer Science, 2022, 49(6A): 353-357.
[13] ZHAO Zheng-peng, LI Jun-gang, PU Yuan-yuan. Low-light Image Enhancement Based on Retinex Theory by Convolutional Neural Network [J]. Computer Science, 2022, 49(6): 199-209.
[14] HU Fu-yuan, WAN Xin-jun, SHEN Ming-fei, XU Jiang-lang, YAO Rui, TAO Zhong-ben. Survey Progress on Image Instance Segmentation Methods of Deep Convolutional Neural Network [J]. Computer Science, 2022, 49(5): 10-24.
[15] XU Hua-chi, SHI Dian-xi, CUI Yu-ning, JING Luo-xi, LIU Cong. Time Information Integration Network for Event Cameras [J]. Computer Science, 2022, 49(5): 43-49.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!