基于CPNet的相对图像质量评估

doi:10.11896/jsjkx.190900052

摘要/Abstract

摘要： 对于两幅不同质量的图像,人眼视觉系统(Human Visual System,HVS)能够比较容易地区分两幅图像间的质量差异,因此通过模拟HVS来判断两幅图像的相对质量比给出图像的绝对质量分数更加准确。文中提出了一种用于评估图像间相对质量的CPNet(Compare-net)模型,该模型是一种分数无关类型的算法,利用图像组合的形式解决数据量的限制,相比绝对质量分数标签,提出的相对质量标签以及相对质量顺序标签具有更广阔的应用场景,并且获取方式更加方便、准确。首先,通过分析卷积神经网络结构相关参数对网络性能的影响,来构建合理的网络基础结构;其次,以双通道输入网络和设计特征求差的方式得到两幅图像的质量差异特征,并结合图像对相对质量标签来完成分类学习;最后,通过在公共数据库上的实验证明了该算法的精度优于其他算法。所提算法在相同参考图像类型实验中分别取得了0.971和0.947的最优精度;在不同参考图像类型实验上也取得了很有竞争力的精度,分别为0.926和0.860。另外,设计了三通道网络并进行实验来探究将所提算法扩展到多通道的可能性。

关键词: CPNet, 卷积网络, 图像质量评估, 相对质量顺序, 质量差异

Abstract: For two images with different quality,the human visual system (HVS) can easily distinguish their quality difference.Thus,it is more accurate to judge the relative quality of two images by simulating HVS than to give the absolute quality score of images.A CPNet (Compare-net) model for evaluating the relative quality between images is proposed in this paper.It is a score-independent algorithm that uses the form of image combination to solve the limitation of data volume.Compared with the absolute quality score label,the proposed relative quality label and relative quality order label have a broader application scenario than the absolute quality score label and are more convenient and accurate to obtain.Firstly,by analyzing the influence of convolutional neural network structure related parameters on network performance,a reasonable network infrastructure is constructed.Secondly,the quality difference characteristics of two images are obtained by the methods of two-channel input network and the feature differentiation,and the classification learning is completed by combining the relative quality labels of the image pairs.Finally,experiments on public database show that the accuracy of the proposed algorithm is better than that of other algorithms.CPNet achieved the best accuracy of 0.971 and 0.947 in the same reference image experiment,and also achieved a very competitive accuracy in different reference image experiments,0.926 and 0.860 respectively.In addition,a three-channel network is designed and experiments are carried out to explore the possibility of extending the proposed algorithm to multiple channels.

Key words: Convolutional network, CPNet, Image quality assessment, Quality difference, Relative quality order

中图分类号:

TP391.4

李凯文, 徐琳, 陈强. 基于CPNet的相对图像质量评估[J]. 计算机科学, 2020, 47(11): 159-167. https://doi.org/10.11896/jsjkx.190900052

LI Kai-wen, XU Lin, CHEN Qiang. Relative Image Quality Assessment Based on CPNet[J]. Computer Science, 2020, 47(11): 159-167. https://doi.org/10.11896/jsjkx.190900052

参考文献

[1] ALAKUIJALA J,OBRYK R,STOLIARCHUK O,et al.Guetzli:Perceptually guided JPEG encoder[EB/OL].arXiv:arxiv/abs/1703.04421.
[2] CHANNAPPAYYA S S,BOVIK A C,CARAMANIS C,et al.Design of linear equalizers optimized for the structural similarity index [J].IEEE Transactions on Image Processing,2008,17(6):857-872.
[3] GU K,TAO D C,QIAO J F,et al.Learning a no-reference quality assessment model of enhanced images with big data [J].IEEE Transactions on Neural Networks and Learning Systems,2018,29(4):1301-1313.
[4] ZHOU W,BOVIK A C.Mean squared error:Love it or leave it?A new look at Signal Fidelity Measures [J].IEEE Signal Processing Magazine,2009,26(1):98-117.
[5] ZHANG L,ZHANG L,MOU X Q,et al.FSIM:A feature similarity index for image quality assessment [J].IEEE Transactions on Image Processing,2011,20(8):2378.
[6] VENKATESH B R,PERKIS A.An HVS-based no-referenceperceptual quality assessment of JPEG coded images using neural networks [C]//Proceedings of the IEEE International Conference on Image Processing.Genova:IEEE Press,2005,1:I-433.
[7] BRANDÃO T,QUELUZ M P.No-reference image quality as-sessment based on DCT domain statistics [J].Signal Processing,2008,88(4):822-833.
[8] PAN F,LIN X,RAHARDJA S,et al.Measuring blocking artifacts using edge direction information [C]//Proceedings of the IEEE International Conference on Multimedia and Expo.Taipei:IEEE Press,2004:1491-1494.
[9] VLACHOS T.Detection of blocking artifacts in compressed video [J].Electronics Letters,2000,36(13):1106-1108.
[10] CHEN M J,BOVIK A C.No-reference image blur assessment using multiscale gradient [C]//2009 International Workshop on Quality of Multimedia Experience.IEEE,2011.
[11] CIANCIO A,COSTA A L,SILVA E A,et al.No-reference blur assessment of digital pictures based on multifeature classifiers [J].IEEE Transactions on Image Processing,2010,20(1):64-75.
[12] LI C F,BOVIK A C,WU X J.Blind image quality assessmentusing a general regression neural network [J].IEEE Transactions on Neural Networks,2011,22(5):793-799.
[13] YE P,DOERMANN D.No-reference image quality assessment using visual codebooks [J].IEEE Transactions on Image Processing,2012,21(7):3129-3138.
[14] ZHANG Y,WANG C,MOU X Q.SPCA:A no-reference image quality assessment based on the statistic property of the PCA on nature images [J].Digital Photography IX,2013,8660(1):311-316.
[15] ZHOU L Y,ZHANG Z B.No-reference image quality assessment based on noise,blurring and blocking effect [J].Optik-International Journal for Light and Electron Optics,2014,125(19):5677-5680.
[16] LIU L X,LIU B,HUANG H,et al.Noreference image quality assessment based on spatial and spectral entropies [J].Signal Processing:Image Communication,2014,29(8):856-863.
[17] SAAD M A,BOVIK A C,CORMACK L.A DCT statistics-based blind image quality index [J].IEEE Signal Processing Letter,2010,17(6):583-586.
[18] SAAD M A,BOVIK A C,CHARRIER C.DCT statistics model-based blind image quality assessment [C]//IEEE International Conference on Image Processing.Brussels:IEEE Press,2011:3093-3096.
[19] SAAD M A,BOVIK A C,CHARRIER C.Blind image quality assessment:a natural scene statistics approach in the DCT domain [J].IEEE Transactions on Image Processing,2012,21(8):3339-3352.
[20] MOORTH A K,BOVIK A C.A two-step framework for constructing blind image quality Indice [J].IEEE Signal Processing Letter,2012,17(5):513-516.
[21] MOORTH A K,BOVIK A C.Blind image quality assessment:From natural scene statistics to perceptual quality [J].IEEE Transactions on Image Processing,2011,20(12):3350-3364.
[22] TANG H X,JOSHI N,KAPOOR A.Learning a blind measure of perceptual image quality [C]//Proceedings of the IEEE International Conference on Image Processing.Brussels:IEEE Press,2011:305-312.
[23] MITTAL A,MOORTHY A K,BOVIK A C.No-reference image quality assessment in the spatial domain [J].IEEE Transactions on Image Processing,2012,21(12):4695-4708.
[24] MITTAL A,SOUNDARARAJAN R,BOVIK A C.Making a‘completely blind' image quality analyzer [J].IEEE Signal Processing Letters,2013,20(3):209-212.
[25] MITTAL A,SOUNDARARAJAN R,MURALIDHAR G S,et al.Blind image quality assessment without training on human opinion scores [C]//Proceedings of the SPIE Human Vision and Electronic Imaging XVIII.Florida:IEEE Press,2013.
[26] KANG L,YE P,LI Y,et al.Convolutional neural networks for no-reference image quality assessment [C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Columbus,USA,2014:1733-1740.
[27] KIM J,LEE S.Fully Deep Blind image quality predictor [J].IEEE Journal of Selected Topics in Signal Processing,2017,11(1):206-220.
[28] LI J,ZOU L,YAN J,et al.No-reference image quality assessment using Prewitt magnitude based on convolutional neural networks [J].Signal Image and Video Processing,2016,10(4):609-616.
[29] LI Y M,PO L M,FENG L T,et al.No-reference image quality assessment with deep convolutional neural networks [C]//Proceedings of the IEEE International Conference on Digital Signal Processing.London:IEEE Press,2017:685-689.
[30] BOSSE S,MANIRY D,MÜLLER K R,et al.Deep Neural Networks for No-Reference and Full-Reference Image Quality Assessment [J].IEEE Transactions on Image Processing,2016,PP(99):1-1.
[31] GU J,MENG G F,XIANG S M,et al.Blind image quality assessment via learnable attention-based pooling [J].Pattern Recognition,2019,91:332-334.
[32] GAO F,TAO D C,GAO X B,et al.Learning to rank for blind image quality assessment [J].IEEE Transactions on Neural Networks and Learning Systems,2015,26(10):2275-2290.
[33] LIU X L,WEIJER J V D,BAGDANOV A D.RankIQA:Learning from Rankings for No-Reference Image Quality Assessment [C]//Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV).Venice:IEEE Press,2017:1040-1049.

相关文章 15

[1]	汪鸣, 彭舰, 黄飞虎. 基于多时间尺度时空图网络的交通流量预测模型 Multi-time Scale Spatial-Temporal Graph Neural Network for Traffic Flow Prediction 计算机科学, 2022, 49(8): 40-48. https://doi.org/10.11896/jsjkx.220100188
[2]	李健智, 王红玲, 王中卿. 基于图卷积网络的专利摘要自动生成研究 Automatic Generation of Patent Summarization Based on Graph Convolution Network 计算机科学, 2022, 49(6A): 172-177. https://doi.org/10.11896/jsjkx.210400117
[3]	赵小虎, 叶圣, 李晓. 多算法融合的骨骼重建信息动作分类方法 Multi-algorithm Fusion Behavior Classification Method for Body Bone Information Reconstruction 计算机科学, 2022, 49(6): 269-275. https://doi.org/10.11896/jsjkx.210500070
[4]	周海榆, 张道强. 面向多中心数据的超图卷积神经网络及应用 Multi-site Hyper-graph Convolutional Neural Networks and Application 计算机科学, 2022, 49(3): 129-133. https://doi.org/10.11896/jsjkx.201100152
[5]	潘志豪, 曾碧, 廖文雄, 魏鹏飞, 文松. 基于交互注意力图卷积网络的方面情感分类 Interactive Attention Graph Convolutional Networks for Aspect-based Sentiment Classification 计算机科学, 2022, 49(3): 294-300. https://doi.org/10.11896/jsjkx.210100180
[6]	解宇, 杨瑞玲, 刘公绪, 李德玉, 王文剑. 基于动态拓扑图的人体骨架动作识别算法 Human Skeleton Action Recognition Algorithm Based on Dynamic Topological Graph 计算机科学, 2022, 49(2): 62-68. https://doi.org/10.11896/jsjkx.210900059
[7]	龚浩田, 张萌. 基于关键点检测的无锚框轻量级目标检测算法 Lightweight Anchor-free Object Detection Algorithm Based on Keypoint Detection 计算机科学, 2021, 48(8): 106-110. https://doi.org/10.11896/jsjkx.200700161
[8]	邢豪, 李明. 基于3D CNNS的深度伪造视频篡改检测 Deepfake Video Detection Based on 3D Convolutional Neural Networks 计算机科学, 2021, 48(7): 86-92. https://doi.org/10.11896/jsjkx.210200127
[9]	宋龙泽, 万怀宇, 郭晟楠, 林友芳. 面向出租车空载时间预测的多任务时空图卷积网络 Multi-task Spatial-Temporal Graph Convolutional Network for Taxi Idle Time Prediction 计算机科学, 2021, 48(7): 112-117. https://doi.org/10.11896/jsjkx.201000089
[10]	卿来云, 张建功, 苗军. 在线异常事件检测的时序建模 Temporal Modeling for Online Anomaly Detection 计算机科学, 2021, 48(7): 206-212. https://doi.org/10.11896/jsjkx.200900093
[11]	程思伟, 葛唯益, 王羽, 徐建. BGCN:基于BERT和图卷积网络的触发词检测 BGCN:Trigger Detection Based on BERT and Graph Convolution Network 计算机科学, 2021, 48(7): 292-298. https://doi.org/10.11896/jsjkx.200500133
[12]	宋元隆, 吕光宏, 王桂芝, 贾吾财. 基于图卷积神经网络的SDN网络流量预测 SDN Traffic Prediction Based on Graph Convolutional Network 计算机科学, 2021, 48(6A): 392-397. https://doi.org/10.11896/jsjkx.200800090
[13]	吕明琪, 洪照雄, 陈铁明. 一种融合时空关联与社会事件的交通流预测方法 Traffic Flow Forecasting Method Combining Spatio-Temporal Correlations and Social Events 计算机科学, 2021, 48(2): 264-270. https://doi.org/10.11896/jsjkx.200300098
[14]	王丽芳, 王蕊芳, 蔺素珍, 秦品乐, 高媛, 张晋. 基于双残差超密集网络的多模态医学图像融合 Multimodal Medical Image Fusion Based on Dual Residual Hyper Densely Networks 计算机科学, 2021, 48(2): 160-166. https://doi.org/10.11896/jsjkx.200400095
[15]	赵冬梅, 宋会倩, 张红斌. 基于时间因子和复合CNN结构的网络安全态势评估 Network Security Situation Based on Time Factor and Composite CNN Structure 计算机科学, 2021, 48(12): 349-356. https://doi.org/10.11896/jsjkx.210400227

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed