Computer Science

Computer Science ›› 2023, Vol. 50 ›› Issue (6A): 220700016-8.doi: 10.11896/jsjkx.220700016

• Image Processing & Multimedia Technology • Previous Articles     Next Articles

Pathological Image Super-resolution Reconstruction Based on Sparse Coding Non-local AttentionDual Network

LIANG Meiyan1, ZHANG Yu1, LIANG Jianan1, CHEN Qinghui1, WANG Ru1, WANG Lin2,3   

  1. 1 College of Physics and Electronics Engineering,Shanxi University,Taiyuan 030006,China;
    2 Shanxi Bethune Hospital,Third Hospital of Shanxi Medical University,Taiyuan 030032,China;
    3 Tongji Hospital Tongji Medical College,Huazhong University of Science and Technology,Wuhan 430030,China
  • Online:2023-06-10 Published:2023-06-12
  • About author:LIANG Meiyan,born in 1984,Ph.D,associate professor.Her main research interests include machine learning,deep learning and medical image processing.
  • Supported by:
    National Natural Science Foundation of China(11804209),Shanxi Provincial Youth Fund(201901D211173),Shanxi Provincial University Science and Technology Innovation Program(2019L0064),Shanxi Province Applied Basic Research Program(201901D211172),Central Government of Shanxi Province Guides Local Science and Technology Development Fund Projects(YDZX20201400001547) and Science and Technology Innovation Project of Shanxi Provincial Colleges and Universities(2020L0048).

PDF (PC)

267

Abstract: High-resolution pathological images are the objective criteria for high-precision disease diagnosis,which have great significance in the field of precision medicine.However,it is difficult to obtain high-resolution pathological images in real time,due to the limited resolution and constrained scanning time of hardware devices.Classical image super-resolution reconstruction algorithm is not suitable for pathological images because the parameters of the model are difficult to estimate,resulting in blurred and unrealistic image details after super-resolution reconstruction.Therefore,sparse-coding non-local attention dual network(SNADN) is proposed,which uses Gaussian constraints,hash coding and parameter sharing strategy in the dual branches,to achieve high accuracy and high efficiency super-resolution reconstruction of pathological images.The PSNR and SSIM of the reconstructed pathological images can reach 30.84dB and 0.914,respectively.The results show that SNADN can not only achieve accurate reconstruction of high-frequency details in pathological images,but also the lightweight sparse coding non-local attention mechanism can effectively improve the modeling efficiency.It is an effective method for super-resolution reconstruction of pathological images.

Key words: Sparse coding, Non-local attention, Dual network, Pathological image, Super resolution

CLC Number: 

  • TP391
[1]TSAI R,HUANG T.Multiframe image restoration and registration[J].Advances in Computer Vision and Image Processing,1984,1(2):317-339.
[2]YUE L,SHEN H,LI J,et al.Image super-resolution:The techniques,applications,and future[J].Signal Processing,2016,128:389-408.
[3]TSAI R,HUANG T.Moving image restoration and registration[C]//IEEE International Conference on Acoustics,Speech,and Signal Processing.IEEE,1980:418-421.
[4]LIU L,LUO T.Application of Interpolation in Image Processing[J].Silicon Valley,2009,9:9-10.
[5]IRANI M,PELEG S.Improving resolution by image registration[J].CVGIP:Graphical Models an Image Processing,1991,53(3):231-239.
[6]PATTI A J,SEZAN M, TEKALP A.High resolution standards conversion of low resolution video[C]//1995 International Conference on Acoustics,Speech,and Signal Processing.IEEE,1995,4:2197-2200.
[7]LIU C,SUN D.On Bayesian adaptive video super resolution[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2013,36(2):346-360.
[8]SEUNG H,LEE D.The manifold ways of perception[J].Science,2000,290(5500):2268-2269.
[9]YANG J,WRIGHT J,HUANG T,et al.Image super-resolution via sparse representation[J].IEEE Transactions on Image Processing,2010,19(11):2861-2873.
[10]HINTON G,SALAKHUTDINOV R.Reducing the dimensionality of data with neural networks[J].Science,2006,313(5786):504-507.
[11]DONG C,LOY C,HE K,et al.Learning a deep convolutional network for image super-resolution[C]//European Conference on Computer Vision.Cham:Springer,2014:184-199.
[12]KIM J,LEE J,LEE K.Accurate image super-resolution using very deep convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:1646-1654.
[13]DONG C,LOY C,TANG X.Accelerating the super-resolution convolutional neural network[C]//European Conference on Computer Vision.Cham:Springer,2016:391-407.
[14]SHI W,CABALLERO J,HUSZÁR F,et al.Real-time singleimage and video super-resolution using an efficient sub-pixel convolutional neural network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:1874-1883.
[15]GOODFELLOW I,POUGET-ABADIE J,MIRZA M,et al.Ge-nerative adversarial nets[J].Communications of the ACM,2020,63(11):139-144.
[16]LEDIG C,THEIS L,HUSZAR F,et al.Photo-realistic singleimage super-resolution using a generative adversarial network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:4681-4690.
[17]KIM J,LEE J,LEE K.Deeply-recursive convolutional network for image super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:1637-1645.
[18]IANDOLA,MOSKEWICZ M,KARAYEV S,et al.Densenet:Implementing efficient convnet descriptor pyramids[J].arXiv:1404.1869,2014.
[19]TONG T,LI G,LIU X,et al.Image super-resolution usingdense skip connections[C]//Proceedings of the IEEE International Conference on Computer Vision.2017:4799-4807.
[20]ZHANG K,SUN M,HAN T,et al.Residual networks of resi-dual networks:Multilevel residual networks[J].IEEE Transactions on Circuits and Systems for Video Technology,2017,28(6):1303-1314.
[21]TAI Y,YANG J,LIU X,et al.Memnet:A persistent memory network for image restoration[C]//Proceedings of the IEEE International Conference on Computer Vision.2017:4539-4547.
[22]LIM B,SON S,KIM H,et al.Enhanced deep residual networks for single image super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops.2017:136-144.
[23]YU J,FAN Y,YANG J,et al.Wide activation for efficient and accurate image super-resolution[J].arXiv:1808.08718,2018.
[24]ZHANG Y,TIAN Y,KONG Y,et al.Residual dense network for image super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:2472-2481.
[25]ZHANG Y,LI K,LI K,et al.Image super-resolution usingvery deep residual channel attention networks[C]//Proceedings of the European Conference on Computer vision(ECCV).2018:286-301.
[26]LAI W,HUANG J,AHUJA N,et al.Deep laplacian pyramidnetworks for fast and accurate super-resolution[C]//Procee-dings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:624-632.
[27]CHEN R,ZHANG H,LIU J.Multi-attention augmented network for single image super-resolution[J].Pattern Recognition,2022,122:108349.
[28]WANG X T,YU K,DONG C,et al.Recovering realistic texture in image super-resolution by deep spatial feature transform[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:606-615.
[29]LIU J,ZHANG W,TANG Y,et al.Residual feature aggregation network for image super-resolution[C]//Proceedings of the IEEE/CVFCconference on Computer Vision and Pattern Recognition.2020:2359-2368.
[30]LIU Y,ZHANG S,XU J,et al.An accurate and lightweight method for human body image super-resolution[J]. IEEE Transactions on Image Processing,2021,30:2888-2897.
[31]AN Z,ZHANG J,SHENG Z,et al.RBDN:Residual Bottleneck Dense Network for Image Super-Resolution[J].IEEE Access,2021,9:103440-103451.
[32]ANDONI A,INDYK P,LAARHOVEN T,et al.Practical and optimal LSH for angular distance[J].Advances in Neural Information Processing Systems,2015,28:1225-1233.
[33]TERASAWA K,TANAKA Y.Spherical LSH for approximate nearest neighbor search on unit hypersphere[C]//Workshop on Algorithms and Data Structures.Springer,Berlin,Heidelberg,2007:27-38.
[34]TIAN C,ZHANG X,LIN J,et al.Generative Adversarial Networks for Image Super-Resolution:A Survey[J].arXiv:2204.13620,2022.
[35]WANG X,YU K,WU S,et al.Esrgan:Enhanced super-resolution generative adversarial networks[C]//Proceedings of the European Conference on Computer Vision(ECCV) workshops.2018:8-14.
[36]JIANG D,ZHANG S,DAI L,et al.Multi-scale generative adversarial network for image super-resolution[J].Soft Computing,2022,26(8):3631-3641.
[37]PARK S,MOON Y,CHO N.Flexible Style Image Super-Resolution using Conditional Objective[J].IEEE Access,2022,10:9774-9792.
[38]SONG J,YI H,XU W,et al.Dual Perceptual Loss for SingleImage Super-Resolution Using ESRGAN[J].arXiv:2201.06383,2022.
[39]MEI Y,FAN Y,ZHOU Y,et al.Image super-resolution with cross-scale non-local attention and exhaustive self-exemplars mining[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:5690-5699.
[40]PANG S,CHEN Z,YIN F.Lightweight multi-scale aggregated residual attention networks for image super-resolution[J].MultimediaTtools and Applications,2022,81(4):4797-4819.
[41]MEI Y,FAN Y,ZHOU Y.Image super-resolution with non-local sparse attention[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:3517-3526.
[42]RONNEBERGER O,FISCHER P,BROX T.U-net:Convolu-tional networks for biomedical image segmentation[C]//International Conference on Medical Image Computing and Compu-ter-assisted Intervention.Cham:Springer,2015:234-241.
[43]HU X,NAIEL M,WONG A,et al.RUNet:A robust UNet architecture for image super-resolution[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops.2019:505-507.
[44]XIAO M,ZHENG S,LIU C,et al.Invertible image rescaling[C]//European Conference on Computer Vision.Cham:Sprin-ger,2020:126-144.
[45]GUO Y,CHEN J,WANG J,et al.Closed-loop matters:Dual regression networks for single image super-resolution[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:5407-5416.
[46]DENG X,DRAGOTTI P.Deep convolutional neural network for multi-modal image restoration and fusion[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2020,43(10):3333-3348.
[47]YANG F,YANG H,FU J,et al.Learning texture transformer network for image super-resolution[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:5791-5800.
[48]DINH L,KRUEGER D,BENGIO Y.NICE:Non-linear inde-pendent components estimation[C]//Workshop ofthe International Conference on Learning Representations.2015.
[49]DINH L,SOHI-DICKSTEIN J,BENGIO S.Density estimation using real nvp[J].arXiv:1605.08803,2016.
[1] LI Fan, JIA Dongli, YAO Yumin, TU Jun. Graph Neural Network Few Shot Image Classification Network Based on Residual and Self-attention Mechanism [J]. Computer Science, 2023, 50(6A): 220500104-5.
[2] YIN Haitao, WANG Tianyou. Image Denoising Algorithm Based on Deep Multi-scale Convolution Sparse Coding [J]. Computer Science, 2023, 50(4): 133-140.
[3] ZHAO Qian, ZHOU Dongming, YANG Hao, WANG Changchen. Image Deblurring Based on Residual Attention and Multi-feature Fusion [J]. Computer Science, 2023, 50(1): 147-155.
[4] SUN Qi, JI Gen-lin, ZHANG Jie. Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection [J]. Computer Science, 2022, 49(8): 172-177.
[5] WANG Xin-tong, WANG Xuan, SUN Zhi-xin. Network Traffic Anomaly Detection Method Based on Multi-scale Memory Residual Network [J]. Computer Science, 2022, 49(8): 314-322.
[6] GAO Rong-hua, BAI Qiang, WANG Rong, WU Hua-rui, SUN Xiang. Multi-tree Network Multi-crop Early Disease Recognition Method Based on Improved Attention Mechanism [J]. Computer Science, 2022, 49(6A): 363-369.
[7] HAN Hong-qi, RAN Ya-xin, ZHANG Yun-liang, GUI Jie, GAO Xiong, YI Meng-lin. Study on Cross-media Information Retrieval Based on Common Subspace Classification Learning [J]. Computer Science, 2022, 49(5): 33-42.
[8] ZHAO Ren-xing, XU Pin-jie, LIU Yao. ECG-based Atrial Fibrillation Detection Based on Deep Convolutional Residual Neural Network [J]. Computer Science, 2022, 49(5): 186-193.
[9] QU Zhong, CHEN Wen. Concrete Pavement Crack Detection Based on Dilated Convolution and Multi-features Fusion [J]. Computer Science, 2022, 49(3): 192-196.
[10] YAN Rui, LIANG Zhi-yong, LI Jin-tao, REN Fei. Predicting Tumor-related Indicators Based on Deep Learning and H&E Stained Pathological Images:A Survey [J]. Computer Science, 2022, 49(2): 69-82.
[11] ZHANG Xi-ke, MA Zhi-qing, ZHAO Wen-hua, CUI Dong-mei. Review on Classification of Breast Cancer Histopathological Images Based on Convolutional Neural Network [J]. Computer Science, 2022, 49(11A): 210800232-9.
[12] HUANG Xiao-ling, ZHANG De-ping. Fault Diagnosis Based on Channel Splitting CLAHE and Adaptive Threshold Residual NetworkUnder Variable Operating Conditions [J]. Computer Science, 2022, 49(11A): 211100122-7.
[13] WU Ji-sheng, HONG Zheng, MA Tian-tian, LIN Pei-hong. Application Layer Protocol Recognition Based on Residual Network and Recurrent Neural Network [J]. Computer Science, 2022, 49(11): 293-301.
[14] GUO Lin, LI Chen, CHEN Chen, ZHAO Rui, FAN Shi-lin, XU Xing-yu. Image Super-resolution Reconstruction Using Recursive ResidualNetwork Based on ChannelAttention [J]. Computer Science, 2021, 48(8): 139-144.
[15] XU Hua-jie, ZHANG Chen-qiang, SU Guo-shao. Accurate Segmentation Method of Aerial Photography Buildings Based on Deep Convolutional Residual Network [J]. Computer Science, 2021, 48(8): 169-174.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.