计算机科学

计算机科学 ›› 2023, Vol. 50 ›› Issue (6A): 220700016-8.doi: 10.11896/jsjkx.220700016

• 图像处理&多媒体技术 • 上一篇    下一篇

基于稀疏编码非局部注意力对偶网络的病理图像超分辨率重建

梁美彦1, 张宇1, 梁建安1, 陈庆辉1, 王茹1, 王琳2,3   

  1. 1 山西大学物理电子工程学院 太原 030006;
    2 山西白求恩医院,山西医科大学第三医院 太原 030032;
    3 华中科技大学同济医学院附属同济医院 武汉 430030
  • 出版日期:2023-06-10 发布日期:2023-06-12
  • 通讯作者: 梁美彦(meiyanliang@sxu.edu.cn)
  • 基金资助:
    国家自然科学基金(11804209);山西省面上青年基金(201901D211173);山西省高校科技创新计划(2019L0064);山西省应用基础研究计划项目(201901D211172);山西省中央引导地方科技发展资金项目(YDZX20201400001547);山西省高等学校科技创新项目(2020L0048)

Pathological Image Super-resolution Reconstruction Based on Sparse Coding Non-local AttentionDual Network

LIANG Meiyan1, ZHANG Yu1, LIANG Jianan1, CHEN Qinghui1, WANG Ru1, WANG Lin2,3   

  1. 1 College of Physics and Electronics Engineering,Shanxi University,Taiyuan 030006,China;
    2 Shanxi Bethune Hospital,Third Hospital of Shanxi Medical University,Taiyuan 030032,China;
    3 Tongji Hospital Tongji Medical College,Huazhong University of Science and Technology,Wuhan 430030,China
  • Online:2023-06-10 Published:2023-06-12
  • About author:LIANG Meiyan,born in 1984,Ph.D,associate professor.Her main research interests include machine learning,deep learning and medical image processing.
  • Supported by:
    National Natural Science Foundation of China(11804209),Shanxi Provincial Youth Fund(201901D211173),Shanxi Provincial University Science and Technology Innovation Program(2019L0064),Shanxi Province Applied Basic Research Program(201901D211172),Central Government of Shanxi Province Guides Local Science and Technology Development Fund Projects(YDZX20201400001547) and Science and Technology Innovation Project of Shanxi Provincial Colleges and Universities(2020L0048).

PDF (PC)

272

摘要： 高分辨率的病理学图像是疾病高精度诊断的客观依据,在精准医学领域具有重要意义。然而,受硬件设备分辨率和扫描时长的限制,实时获取高分辨率病理图像存在困难。经典的图像超分辨率重建算法由于模型的参数较难估计,导致重建后图像细节模糊且不够真实,不适用于病理学图像。为此,文中提出稀疏编码非局部注意力对偶网络,通过上采样和降采样对偶分支中的稀疏编码非局部注意力机制、高斯约束以及参数共享策略来实现病理学图像的超分辨率重建。重建后的病理图像峰值信噪比和结构相似性分别达到了30.84dB和0.914。研究结果表明,所提方法不但能够实现病理学图像中高频细节的精确重建,轻量化的稀疏编码非局部注意力机制也有效地提高了建模的效率,是病理学图像超分辨率重建的一种有效方法。

关键词: 稀疏编码, 非局部注意力, 对偶网络, 病理图像, 超分辨率

Abstract: High-resolution pathological images are the objective criteria for high-precision disease diagnosis,which have great significance in the field of precision medicine.However,it is difficult to obtain high-resolution pathological images in real time,due to the limited resolution and constrained scanning time of hardware devices.Classical image super-resolution reconstruction algorithm is not suitable for pathological images because the parameters of the model are difficult to estimate,resulting in blurred and unrealistic image details after super-resolution reconstruction.Therefore,sparse-coding non-local attention dual network(SNADN) is proposed,which uses Gaussian constraints,hash coding and parameter sharing strategy in the dual branches,to achieve high accuracy and high efficiency super-resolution reconstruction of pathological images.The PSNR and SSIM of the reconstructed pathological images can reach 30.84dB and 0.914,respectively.The results show that SNADN can not only achieve accurate reconstruction of high-frequency details in pathological images,but also the lightweight sparse coding non-local attention mechanism can effectively improve the modeling efficiency.It is an effective method for super-resolution reconstruction of pathological images.

Key words: Sparse coding, Non-local attention, Dual network, Pathological image, Super resolution

中图分类号: 

  • TP391
[1]TSAI R,HUANG T.Multiframe image restoration and registration[J].Advances in Computer Vision and Image Processing,1984,1(2):317-339.
[2]YUE L,SHEN H,LI J,et al.Image super-resolution:The techniques,applications,and future[J].Signal Processing,2016,128:389-408.
[3]TSAI R,HUANG T.Moving image restoration and registration[C]//IEEE International Conference on Acoustics,Speech,and Signal Processing.IEEE,1980:418-421.
[4]LIU L,LUO T.Application of Interpolation in Image Processing[J].Silicon Valley,2009,9:9-10.
[5]IRANI M,PELEG S.Improving resolution by image registration[J].CVGIP:Graphical Models an Image Processing,1991,53(3):231-239.
[6]PATTI A J,SEZAN M, TEKALP A.High resolution standards conversion of low resolution video[C]//1995 International Conference on Acoustics,Speech,and Signal Processing.IEEE,1995,4:2197-2200.
[7]LIU C,SUN D.On Bayesian adaptive video super resolution[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2013,36(2):346-360.
[8]SEUNG H,LEE D.The manifold ways of perception[J].Science,2000,290(5500):2268-2269.
[9]YANG J,WRIGHT J,HUANG T,et al.Image super-resolution via sparse representation[J].IEEE Transactions on Image Processing,2010,19(11):2861-2873.
[10]HINTON G,SALAKHUTDINOV R.Reducing the dimensionality of data with neural networks[J].Science,2006,313(5786):504-507.
[11]DONG C,LOY C,HE K,et al.Learning a deep convolutional network for image super-resolution[C]//European Conference on Computer Vision.Cham:Springer,2014:184-199.
[12]KIM J,LEE J,LEE K.Accurate image super-resolution using very deep convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:1646-1654.
[13]DONG C,LOY C,TANG X.Accelerating the super-resolution convolutional neural network[C]//European Conference on Computer Vision.Cham:Springer,2016:391-407.
[14]SHI W,CABALLERO J,HUSZÁR F,et al.Real-time singleimage and video super-resolution using an efficient sub-pixel convolutional neural network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:1874-1883.
[15]GOODFELLOW I,POUGET-ABADIE J,MIRZA M,et al.Ge-nerative adversarial nets[J].Communications of the ACM,2020,63(11):139-144.
[16]LEDIG C,THEIS L,HUSZAR F,et al.Photo-realistic singleimage super-resolution using a generative adversarial network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:4681-4690.
[17]KIM J,LEE J,LEE K.Deeply-recursive convolutional network for image super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:1637-1645.
[18]IANDOLA,MOSKEWICZ M,KARAYEV S,et al.Densenet:Implementing efficient convnet descriptor pyramids[J].arXiv:1404.1869,2014.
[19]TONG T,LI G,LIU X,et al.Image super-resolution usingdense skip connections[C]//Proceedings of the IEEE International Conference on Computer Vision.2017:4799-4807.
[20]ZHANG K,SUN M,HAN T,et al.Residual networks of resi-dual networks:Multilevel residual networks[J].IEEE Transactions on Circuits and Systems for Video Technology,2017,28(6):1303-1314.
[21]TAI Y,YANG J,LIU X,et al.Memnet:A persistent memory network for image restoration[C]//Proceedings of the IEEE International Conference on Computer Vision.2017:4539-4547.
[22]LIM B,SON S,KIM H,et al.Enhanced deep residual networks for single image super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops.2017:136-144.
[23]YU J,FAN Y,YANG J,et al.Wide activation for efficient and accurate image super-resolution[J].arXiv:1808.08718,2018.
[24]ZHANG Y,TIAN Y,KONG Y,et al.Residual dense network for image super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:2472-2481.
[25]ZHANG Y,LI K,LI K,et al.Image super-resolution usingvery deep residual channel attention networks[C]//Proceedings of the European Conference on Computer vision(ECCV).2018:286-301.
[26]LAI W,HUANG J,AHUJA N,et al.Deep laplacian pyramidnetworks for fast and accurate super-resolution[C]//Procee-dings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:624-632.
[27]CHEN R,ZHANG H,LIU J.Multi-attention augmented network for single image super-resolution[J].Pattern Recognition,2022,122:108349.
[28]WANG X T,YU K,DONG C,et al.Recovering realistic texture in image super-resolution by deep spatial feature transform[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:606-615.
[29]LIU J,ZHANG W,TANG Y,et al.Residual feature aggregation network for image super-resolution[C]//Proceedings of the IEEE/CVFCconference on Computer Vision and Pattern Recognition.2020:2359-2368.
[30]LIU Y,ZHANG S,XU J,et al.An accurate and lightweight method for human body image super-resolution[J]. IEEE Transactions on Image Processing,2021,30:2888-2897.
[31]AN Z,ZHANG J,SHENG Z,et al.RBDN:Residual Bottleneck Dense Network for Image Super-Resolution[J].IEEE Access,2021,9:103440-103451.
[32]ANDONI A,INDYK P,LAARHOVEN T,et al.Practical and optimal LSH for angular distance[J].Advances in Neural Information Processing Systems,2015,28:1225-1233.
[33]TERASAWA K,TANAKA Y.Spherical LSH for approximate nearest neighbor search on unit hypersphere[C]//Workshop on Algorithms and Data Structures.Springer,Berlin,Heidelberg,2007:27-38.
[34]TIAN C,ZHANG X,LIN J,et al.Generative Adversarial Networks for Image Super-Resolution:A Survey[J].arXiv:2204.13620,2022.
[35]WANG X,YU K,WU S,et al.Esrgan:Enhanced super-resolution generative adversarial networks[C]//Proceedings of the European Conference on Computer Vision(ECCV) workshops.2018:8-14.
[36]JIANG D,ZHANG S,DAI L,et al.Multi-scale generative adversarial network for image super-resolution[J].Soft Computing,2022,26(8):3631-3641.
[37]PARK S,MOON Y,CHO N.Flexible Style Image Super-Resolution using Conditional Objective[J].IEEE Access,2022,10:9774-9792.
[38]SONG J,YI H,XU W,et al.Dual Perceptual Loss for SingleImage Super-Resolution Using ESRGAN[J].arXiv:2201.06383,2022.
[39]MEI Y,FAN Y,ZHOU Y,et al.Image super-resolution with cross-scale non-local attention and exhaustive self-exemplars mining[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:5690-5699.
[40]PANG S,CHEN Z,YIN F.Lightweight multi-scale aggregated residual attention networks for image super-resolution[J].MultimediaTtools and Applications,2022,81(4):4797-4819.
[41]MEI Y,FAN Y,ZHOU Y.Image super-resolution with non-local sparse attention[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:3517-3526.
[42]RONNEBERGER O,FISCHER P,BROX T.U-net:Convolu-tional networks for biomedical image segmentation[C]//International Conference on Medical Image Computing and Compu-ter-assisted Intervention.Cham:Springer,2015:234-241.
[43]HU X,NAIEL M,WONG A,et al.RUNet:A robust UNet architecture for image super-resolution[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops.2019:505-507.
[44]XIAO M,ZHENG S,LIU C,et al.Invertible image rescaling[C]//European Conference on Computer Vision.Cham:Sprin-ger,2020:126-144.
[45]GUO Y,CHEN J,WANG J,et al.Closed-loop matters:Dual regression networks for single image super-resolution[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:5407-5416.
[46]DENG X,DRAGOTTI P.Deep convolutional neural network for multi-modal image restoration and fusion[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2020,43(10):3333-3348.
[47]YANG F,YANG H,FU J,et al.Learning texture transformer network for image super-resolution[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:5791-5800.
[48]DINH L,KRUEGER D,BENGIO Y.NICE:Non-linear inde-pendent components estimation[C]//Workshop ofthe International Conference on Learning Representations.2015.
[49]DINH L,SOHI-DICKSTEIN J,BENGIO S.Density estimation using real nvp[J].arXiv:1605.08803,2016.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发