Computer Science ›› 2020, Vol. 47 ›› Issue (9): 150-156.doi: 10.11896/jsjkx.190700213

• Computer Graphics & Multimedia • Previous Articles     Next Articles

No-reference Stereo Image Quality Assessment Based on Disparity Information

ZHU Ling-ying, SANG Qing-bing, GU Ting-ting   

  1. School of Internet of Things Engineering,Jiangnan University,Wuxi,Jiangsu 214122,China
  • Received:2019-07-31 Published:2020-09-10
  • About author:ZHU Ling-ying,master student.Her main research interests include image quality assessment and so on.
    SANG Qing-bing,Ph.D,professor.His main research interests include image processing,quality assessment and machine learning.
  • Supported by:
    Natural Science Foundation of Jiangsu Province,China (BK20171142).

Abstract: In recent years,with the rapid development of deep learning in the field of image quality assessment (IQA),2D-IQA has been improved,but 3D-IQA still needs to be improved.Therefore,combining the three-branch convolutional neural network,the paper proposes a no-reference stereo image quality assessment based on disparity information and analyzes the influence of different disparity maps on the performance of the model.The algorithm takes the left/right view patches and the disparity map patches as input,automatically extracts features,and obtains the regression model through training to realize the prediction of the stereo images.In this paper,5 different stereo matching algorithms are used to generate disparity maps.The experimental results show that the SAD algorithm is the best.The experimental results on stereo image databases LIVE3D and MCL3D show that the method is not only suitable for evaluating symmetric distortion images,but also for evaluating asymmetric distortion stereo images.The overall distortion results of this method are superior to other comparison algorithms.Especially on the MCL3D image database,the evaluation method PLCC and SROCC of the proposed method are 1% and 4% higher than other methods.The Experimental data shows that the proposed model improves the performance of stereo image quality assessment,which is highly consistent with human subjective perception.

Key words: Disparity information, Stereo matching algorithm, Convolutional neural network, Stereo image quality assessment, Asymmetric distortion image

CLC Number: 

  • TP183
[1] CHEN M J,LAWRENCE K,CORMACK,et al.No-Refe-rence Quality Assessment of Natural Stereopairs [J].IEEE Trans on Image Processing,2013,22(9):3379-3391.
[2] BENOIT A,CALLET P L,CAMPISI P,et al.Quality Assessment of Stereoscopic Images [J].Eurasip Journal on Image & Video Processing,2008,2008(1):1-13.
[3] ZHANG Y,CHANDLER D M.Learning Natural Statistics of Binocular Contrast for No Reference Quality Assessment of Stereoscopic Images [C]//IEEE International Conference on Image Processing.Washington,USA:IEEE,2018:186-190.
[4] ZHOU W,YU L,QIU W,et al.Utilizing Binocular Vision to Facilitate Completely Blind 3D Image Quality Measurement [J].Signal Processing,2016,129(C):130-136.
[5] APPINA B,KHAN S,CHANNAPPAYYA S.No-referenceStereoscopic Image Quality Assessment Using Natural Scene Statistics [J].Signal Processing Image Communication,2016,43:1-14.
[6] ZHANG W,QU C,MA L,et al.Learning structure of stereoscopic image for no-reference quality assessment with convolutional neural network[J].Pattern Recognition,2016,59(C):176-187.
[7] SHEN L,LEI J,HOU C.No-reference stereoscopic 3D imagequality assessment via combined model[J].Multimedia Tools and Applications,2017,77(7):8195-8212.
[8] LI Y,HU X.No-Reference Stereoscopic Image Quality Assessment Using Natural Scene Statistics [C]//InternationalConfe-rence on Multimedia and Image Processing.Washington,USA:IEEE Computer Society,2017:123-127.
[9] LIN C T,LIU T J,LIU K H.Visual Quality Prediction on Distorted Stereoscopic Images [C]//IEEE International Conference on Image Processing.Washington,USA:IEEE,2018:3480-3484.
[10] DING Y,DENG R,XIE X,et al.No-Reference Stereoscopic Image Quality Assessment Using Convolutional Neural Network for Adaptive Feature Extraction[J].IEEE Access,2018,6:37595-37603.
[11] BOYKOV Y,KOLMOGOROV V.An Experimental Comparison of Min-cut/Max-flow Algorithms for Energy Minimization in Vision [J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2004,26(9):1124-1137.
[12] HIRSCHMULLER H.Stereo Processing by Semiglobal Matching and Mutual Information [J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2008,30(2):328-341.
[13] WEIS D,LAI S H.Fast Template Matching Based on Norma-lized Cross Correlation with Adaptive Multilevel Winner Update [J].IEEE Transactions on Image Processing,2008,17(11):2227-2235.
[14] LI JIAN J Q,LI Y C.Research Progress on Calculation Method of Disparity Map in Stereo Matching [J].Remote Sensing Information,2017,32(2):7-14.
[15] SIMONYAN K,ZISSERMAN A.Very Deep Convolutional Networks for Large-Scale Image Recognition [J].Computer Science,2014:1409-1556.
[16] HARA K,SAITO D,SHOUNO H.Analysis of Function of Rectified Linear Unit Used in Deep Learning [C]//International Joint Conference on Neural Networks.Washington,USA:IEEE,2015:1-8.
[17] ALAIN H,ZIOU D.Image Quality Metrics:PSNR vs. SSIM[C]//2010 International Conference on Pattern Recognition.Washington,USA:IEEE Computer Society,2010:2366-2369.
[18] WANG Z,BOVIK A C,SHEIKH H R,et al.Image Quality Assessment:From Error Visibility to Structural Similarity [J].IEEE Trans on Image Processing,2004,13(4):600-612.
[19] WANG X,MA L,KWONG S,et al.Quaternion representation based visual saliency for stereoscopic image quality assessment[J].Signal Processing,2018,145:202-213.
[20] BENOITA.LE CALLET P,CAMPISI P,et al.Quality Assessment of StereoscopicImages [J].EURASIP Journal on Image and Video Processing,2007,2008(1):1-13.
[21] RYU S,KIM D H,SOHN K.Stereoscopic Image Quality Metric Based on Binocular Perception Model [C]//IEEE International Conference on Image Processing.Washington,USA:IEEE,2013:609-612.
[22] TIAN S,ZHANG L,MORIN L,et al.NIQSV:ANo Reference Image Quality Assessment Metric for 3D Synthesized Views [C]//IEEE International Conference on Acoustics.Washington,USA:IEEE,2017:1248-1252.
[23] FARID M S,LUCENTEFORTE M,GRANGETTO M.Evaluating Virtual Image Quality using the Side-Views Information Fusion and Depth Maps [J].Information Fusion,2018,43:47-56.
[24] YANG J C,SIM K,JIANG B,et al.No-reference stereoscopic image quality assessment based on hue summation-difference mapping image and binocular joint mutual filtering[J].Applied Optics,2018,57(14):3915-3926.
[1] SHAN Mei-jing, QIN Long-fei, ZHANG Hui-bing. L-YOLO:Real Time Traffic Sign Detection Model for Vehicle Edge Computing [J]. Computer Science, 2021, 48(1): 89-95.
[2] HE Yan-hui, WU Gui-xing, WU Zhi-qiang. Domain Alignment Based Object Detection of X-ray Images [J]. Computer Science, 2021, 48(1): 175-181.
[3] LI Ya-nan, HU Yu-jia, GAN Wei, ZHU Min. Survey on Target Site Prediction of Human miRNA Based on Deep Learning [J]. Computer Science, 2021, 48(1): 209-216.
[4] ZHANG Jia-jia, ZHANG Xiao-hong. Multi-branch Convolutional Neural Network for Lung Nodule Classification and Its Interpretability [J]. Computer Science, 2020, 47(9): 129-134.
[5] CUI Tong-tong, WANG Gui-ling, GAO Jing. Ship Trajectory Classification Method Based on 1DCNN-LSTM [J]. Computer Science, 2020, 47(9): 175-184.
[6] LIU Hai-chao, WANG Li. Graph Classification Model Based on Capsule Deep Graph Convolutional Neural Network [J]. Computer Science, 2020, 47(9): 219-225.
[7] LIANG Zheng-you, HE Jing-lin, SUN Yu. Three-dimensional Convolutional Neural Network Evolution Method for Facial Micro-expression Auto-recognition [J]. Computer Science, 2020, 47(8): 227-232.
[8] SUN Yan-li, YE Jiong-yao. Convolutional Neural Networks Compression Based on Pruning and Quantization [J]. Computer Science, 2020, 47(8): 261-266.
[9] LIU Xiao, YUAN Guan, ZHANG Yan-mei, YAN Qiu-yan, WANG Zhi-xiao. Hand Gesture Recognition Based on Self-adaptive Multi-classifiers Fusion [J]. Computer Science, 2020, 47(7): 103-110.
[10] CHENG Zhe, BAI Qian, ZHANG Hao, WANG Shi-pu and LIANG Yu. Improving Hi-C Data Resolution with Deep Convolutional Neural Networks [J]. Computer Science, 2020, 47(6A): 70-74.
[11] HE Lei, SHAO Zhan-peng, ZHANG Jian-hua and ZHOU Xiao-long. Review of Deep Learning-based Action Recognition Algorithms [J]. Computer Science, 2020, 47(6A): 139-147.
[12] SUN Zheng and WANG Xin-yu. Application of Deep Learning in Photoacoustic Imaging [J]. Computer Science, 2020, 47(6A): 148-152.
[13] MA Hai-Jiang. Recommendation Algorithm Based on Convolutional Neural Network and Constrained Probability Matrix Factorization [J]. Computer Science, 2020, 47(6A): 540-545.
[14] WANG Yan, WANG Li. Local Gabor Convolutional Neural Network for Hyperspectral Image Classification [J]. Computer Science, 2020, 47(6): 151-156.
[15] WANG Hang, CHEN Xiao, TIAN Sheng-zhao, CHEN Duan-bing. SAR Image Recognition Based on Few-shot Learning [J]. Computer Science, 2020, 47(5): 124-128.
Full text



[1] WU Jian-hui, HUANG Zhong-xiang, LI Wu, WU Jian-hui, PENG Xin and ZHANG Sheng. Robustness Optimization of Sequence Decision in Urban Road Construction[J]. Computer Science, 2018, 45(4): 89 -93 .
[2] SHI Wen-jun, WU Ji-gang and LUO Yu-chun. Fast and Efficient Scheduling Algorithms for Mobile Cloud Offloading[J]. Computer Science, 2018, 45(4): 94 -99 .
[3] ZHOU Yan-ping and YE Qiao-lin. L1-norm Distance Based Least Squares Twin Support Vector Machine[J]. Computer Science, 2018, 45(4): 100 -105 .
[4] WANG Zhen-chao, HOU Huan-huan and LIAN Rui. Path Optimization Scheme for Restraining Degree of Disorder in CMT[J]. Computer Science, 2018, 45(4): 122 -125 .
[5] YANG Yu-qi, ZHANG Guo-an and JIN Xi-long. Dual-cluster-head Routing Protocol Based on Vehicle Density in VANETs[J]. Computer Science, 2018, 45(4): 126 -130 .
[6] ZHAN Yun-jiao, WEI Ou and HU Jun. Formal Description of Requirement of Slats and Flaps Control System for DO-178C Case[J]. Computer Science, 2018, 45(4): 196 -202 .
[7] HOU Yan-e, KONG Yun-feng and DANG Lan-xue. Greedy Randomized Adaptive Search Procedure Algorithm Combining Set Partitioning for Heterogeneous School Bus Routing Problems[J]. Computer Science, 2018, 45(4): 240 -246 .
[8] LI Hao-yang and FU Yun-qing. Collaborative Filtering Recommendation Algorithm Based on Tag Clustering and Item Topic[J]. Computer Science, 2018, 45(4): 247 -251 .
[9] CUI Jian-jing, LONG Jun, MIN Er-xue, YU Yang and YIN Jian-ping. Survey on Application of Homomorphic Encryption in Encrypted Machine Learning[J]. Computer Science, 2018, 45(4): 46 -52 .
[10] DU Xiu-li, GU Bin-bin, HU Xing, QIU Shao-ming and CHEN Bo. Support Similarity between Lines Based CoSaMP Algorithm for Image Reconstruction[J]. Computer Science, 2018, 45(4): 306 -311 .