基于特征相似性分析的轻量级图像超分辨率重建

doi:10.11896/jsjkx.250100057

摘要/Abstract

摘要： 基于Transformer的轻量级图像超分辨率网络已经取得了显著成果,然而大多数研究工作专注于设计轻量级网络结构,却忽视了对网络架构冗余性的分析。因此,提出了一种基于特征相似性的超分网络设计方法,通过压缩网络中具有较高特征相似性的注意力组,并保留具有较低相似性的注意力组,有效减少了模型冗余。进一步,设计了一种结合频域和空间域的特征提取模块,通过在频域和空间域上分别进行局部频域特征提取和局部空间特征提取,使模型能够利用更广泛且具有积极影响的输入像素,从而有效提高了对细节纹理的修复能力。将上述方法应用在基线模型上,在多个数据集上的对比结果表明,所提模型具有低复杂度且实现了较好的视觉感知质量和重建性能。

关键词: 图像超分辨率, 轻量级网络, 中心内核对齐, 特征相似性, 频域

Abstract: Lightweight image super-resolution(SR) networks based on Transformer have achieved promising results.However,most research efforts have focused on designing lightweight architectures while neglecting the analysis of structural redundancy within the SR networks.To address this,the feature similarity-based model design approach is proposed,which compresses attention groups with high feature similarity while retaining those with low similarity,effectively reducing redundancy within SR network.Furthermore,a novel feature extraction module integrating the frequency and spatial domains is proposed.By separately performing localized frequency domain and spatial domain feature extraction,the model can leverage a broader range of input pixels with positive influences,effectively enhancing its capability to restore fine texture details.By applying the proposed method to baseline models,comparative results on multiple benchmark datasets demonstrate that the proposed approach achieves superior visual perceptual quality and reconstruction performance while maintaining low computational complexity.

Key words: Image super-resolution, Lightweight model, Centered kernel alignment, Feature similarity, Frequency domain

中图分类号:

TP391

刘兴鹏, 薛一鸣, 林钰扬, 李岩, 彭万里. 基于特征相似性分析的轻量级图像超分辨率重建[J]. 计算机科学, 2025, 52(11A): 250100057-8. https://doi.org/10.11896/jsjkx.250100057

LIU Xingpeng, XUE Yiming, LIN Yuyang, LI Yan, PENG Wanli. Lightweight Image Super-resolution Reconstruction Based on Feature Similarity Analysis[J]. Computer Science, 2025, 52(11A): 250100057-8. https://doi.org/10.11896/jsjkx.250100057

参考文献

[1]HUANG Y,SHAO L,FRANGI A F.Simultaneous super-resolution and cross-modality synthesis of 3D medical im-ages using weakly-supervised joint convolutional sparse coding[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Honolulu,HI,USA:IEEE Computer Society Press,2017:6070-6079.
[2]ZHANG J,LEI J,XIE W,et al.SuperYOLO:Super resolutionassisted object detection in multimodal remote sensing imagery[J].IEEE Transactions on Geoscience and Remote Sensing,2023,61:1-15.
[3]RASTI P,UIBOUPIN T,ESCALERA S,et al.ConvolutionalNeural Network Super Resolution for Face Recognition in Surveil-lance Monitoring[C]//Proceedings of the 9th International Conference on Articulated Motion and Deformable Objects.Palma de Mallorca,Spain:Springer,2016:175-184.
[4]WANG Z H,CHEN J,HOI S C H.Deep Learning for Image Super-Resolution:A Survey[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2021,43(10):3365-3387.
[5]KIM J,LEE J K,LEE K M.Accurate image super-resolutionusing very deep convolutional networks[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas,USA:IEEE Computer Society Press,2016:1646-1654.
[6]TONG T,LI G,LIU X,et al.Image super-resolution usingdense skip connections[C]//Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition.Venice,Italy:IEEE Computer Society Press,2017:4809-4817.
[7]LEDIG C,THEIS L,HUSZÁR F,et al.Photo-realistic singleimage super-resolution using a generative adversarial network[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition.Honolulu,HI,USA:IEEE Computer Society Press,2017:105-114.
[8]LIM B,SON S,KIM H,et al.Enhanced deep residual networks for single image super-resolution[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops.Honolulu,HI,USA:IEEE Computer Society Press,2017:1132-1140.
[9]ZHANG Y L,LI K P,LI K,et al.Image super-resolution using very deep residual channel attention networks[C]//Proceedings of the 15th European Conference on Computer Vision.Munich,Germany:Springer,2018:294-310.
[10]WANG X,YU K,WU S,et al.ESRGAN:Enhanced Super-Resolution Generative Adversarial Networks[C]//Proceedings of the European Conference on Computer Vision Workshops.Munich,Germany:Springer,2018:63-79.
[11]LIANG J,CAO J,SUN G,et al.SwinIR:Image RestorationUsing Swin Transformer[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops.Montreal,QC,Canada:IEEE Computer Society Press,2021:1833-1844.
[12]CHEN X Y,WANG X T,ZHOU J T,et al.Activating MorePixels in Image Super-Resolution Transformer[C]//Procee-dings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Vancouver,BC,Canada:IEEE Computer Society Press,2023:22367-22377.
[13]CHEN Z,ZHANG Y,GU J,et al.Recursive generalizationtransformer for image super-resolution[EB/OL].https://doi.org/10.48550/arXiv.2303.06373.
[14]CHEN Z,ZHANG Y,GU J,et al.Cross Aggregation Transformer for Image Restoration[EB/OL].[2024-07-09].https://arxiv.org/pdf/2211.13654.pdf.
[15]LI Y W,ZHANG Y L,TIMOFTE R,et al.NTIRE 2023 Challenge on Efficient Super- Resolution:Methods and Results[C]//Proceedings of 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops.Vancouver,BC,Canada:IEEE Computer Society Press,2023:1922-1960.
[16]CORINNA C,MEHRYAR M,AFSHIN R.Algorithms forlearning kernels based on centered alignment[J].Journal of Machine Learning Research,2012,13:795-828.
[17]SIMON K,MOHAMMAD N,HONGLAK L,et al.Similarity of neural network representations revisited[C]//Proceedings of the 36th International Conference on Machine Learning.Long Beach,California,USA:PMLR,2019:3519-3529.
[18]RAGHU M,UNTERTHINER T,KORNBLITH S,et al.Dovision transformers see like convolutional neural networks?[C]//Proceedings of Advances in Neural Information Proces-sing Systems.virtual:NeurIPS,2021:12116-12128.
[19]SUVOROV R,LOGACHEVA E,MASHIKHIN A,et al.Resolution-robust Large Mask Inpainting with Fourier Convolutions[C]//Proceedings of the Winter Conference on Applications of Computer Vision.Waikoloa,HI,USA:IEEE,2022:3172-3182.
[20]ZHANG D F,HUANG F Y,LIU S Z,et al.SwinFIR:Revisiting the swinir with fast fourier convolution and improved training for image super-resolution[J].arXiv:2208.11247,2023.
[21]KONG L S,DONG J X,GE J J,et al.Efficient Frequency Domain-based Transformers for High-Quality Image Deblurring[C]//Proceedings of the Conference on Computer Vision and Pattern Recognition.Vancouver,BC,Canada:IEEE,2023:5886-5895.
[22]CUI Y,KNOLL A.Dual-domain strip attention for image restoration[J].Neural Networks,2024,171:429-439.
[23]HUI Z,WANG X,GAO X.Fast and accurate single image super-resolution via information distillation network[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Salt Lake City,UT,USA:IEEE,2018:723-731.
[24]HUI Z,GAO X B,YANG Y C,et al.Lightweight image super-resolution with information multi-distillation network[C]//Proceedings of the 27th ACM International Conference on Multimedia.Nice,France:ACM,2019:2024-2032.
[25]LIU J,TANG J,WU G.Residual Feature Distillation Network for Lightweight Image Super-Resolution[C]//Proceedings of the European Conference on Computer Vision.Glasgow,UK:Springer,2020:41-55.
[26]LI Z,LIU Y,CHEN X,et al.Blueprint Separable Residual Network for Efficient Image Super-Resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.New Orleans,LA,USA:IEEE Computer Society Press,2022:833-843.
[27]ZHOU Y P,LI Z,GUO C L,et al.SRFormer:Permuted Self-Attention for Single Image Super-Resolution[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision and Pattern Recognition.Paris,France:IEEE,2023:12734-12745.
[28]WANG Y,LIU Y,ZHAO S,et al.CAMixerSR:Only DetailsNeed More “Attention”[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Seattle,USA:IEEE,2024:25837-25846.
[29]ZHANG A,REN W,LIU Y,et al.Lightweight image super-resolution with superpixel token interaction[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision and Pattern Recognition.Paris,France:IEEE,2023:12682-12691.
[30]CHEN J,KAO S H,HE H,et al.Run,Don’t Walk:ChasingHigher FLOPS for Faster Neural Networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Vancouver,BC,Canada:IEEE,2023:12021-12031.
[31]GAO G,WANG Z,LI J,et al.Lightweight Bimodal Network for Single-Image Super-Resolution via Symmetric CNN and Recursive Transformer[C]//Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence.Vienna,Aus-tria:IJCAI,2022:913-919.
[32]ZHANG X D,ZENG H,GUO S,et al.Efficient Long-Range Attention Network for Image Super-resolution[C]//Proceedings of the 17th European Conference on Computer Vision.Tel Aviv,Israel:Springer,2022:649-667.
[33]GU J J,DONG C.Interpreting Super-Resolution Networks with Local Attribution Maps[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Nashville,TN,USA:IEEE Computer Society Press,2021:9195-9204.
[34]ZHAO H Y,KONG X T,HE J G,et al.Efficient Image Super-Resolution Using Pixel Attention[C]//Proceedings of the European Conference on Computer Vision.Glasgow,UK:Springer,2020:56-72.
[35]HU J F,GU JJ,YU S Y,et al.Interpreting Low-level Vision Models with Causal Effect Maps[EB/OL].(2024-09-18)[2025-01-05].https://doi.org/10.48550/arXiv.2407.19789.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed