计算机科学 ›› 2025, Vol. 52 ›› Issue (12): 158-165.doi: 10.11896/jsjkx.241000124

• 计算机图形学&多媒体 • 上一篇    下一篇

利用Involution级联注意力机制的古代壁画图像修复网络

周啟雪, 余映, 胡家绿   

  1. 云南大学信息学院 昆明 650504
  • 收稿日期:2024-10-22 修回日期:2025-02-08 出版日期:2025-12-15 发布日期:2025-12-09
  • 通讯作者: 余映(yuying.mail@163.com)
  • 作者简介:(zhouqixue217@126.com)
  • 基金资助:
    国家自然科学基金(62166048)

Ancient Mural Image Restoration Network Using Involution Cascaded Attention Mechanism

ZHOU Qixue, YU Ying, HU Jialv   

  1. School of Information Science & Engineering, Yunnan University, Kunming 650504, China
  • Received:2024-10-22 Revised:2025-02-08 Published:2025-12-15 Online:2025-12-09
  • About author:ZHOU Qixue,born in 2000,postgra-duate.Her main research interests include image processing,digital protection and restoration of mural cultural heritage.
    YU Ying,born in 1977,Ph.D,associate professor.His main research interests include image and vision,artificial neural network.
  • Supported by:
    This work was supported by the National Natural Science Foundationof China(62166048).

摘要: 中国古代壁画是珍贵的人类文化遗产,记录了中国历代各地区人们的社会、宗教、文化、艺术活动等方面的特征。由于长时间暴露在自然环境中,很多壁画出现了裂隙、划痕、腐蚀、甚至大面积脱落等病害现象,因此,壁画的保护和修复工作非常迫切。破损壁画数字修复技术通过重新构建壁画图像的结构和纹理,对其破损区域进行虚拟填充,成为解决这一问题的重要手段。大多现有的壁画图像修复方法难以较好地修复结构复杂、色彩丰富变化的缺失壁画内容。针对该问题,提出利用Involution级联注意力机制的古代壁画图像修复网络。该网络首先利用对合(Involution)操作代替传统卷积,以提高破损壁画纹理和颜色修复的质量。其次,提出一个级联注意力模块,可以捕捉不同尺度的上下文信息,更好地修复不同大小的壁画破损区域。此外,引入FFC残差块来捕捉全局结构信息,以提升网络对壁画破损区域的色彩修复能力。在模拟和真实破损壁画数据集上进行实验,将修复结果与其他4种经典方法进行比较。实验结果表明,提出的模型在修复壁画纹理清晰度、颜色一致性和结构连续性方面均优于其他对比方法。

关键词: 古代壁画图像修复, 级联注意力模块, 深度学习, FFC, Involution

Abstract: Chinese ancient murals are precious cultural heritage of humanity,recording the social,religious,cultural,and artistic activities of people in various regions of China throughout history.Due to prolonged exposure to the natural environment,many murals have developed defects such as cracks,scratches,corrosion,and even large-scale peeling.Therefore,the protection and restoration of murals are urgently needed.The digital restoration technology for damaged murals has become an important means to solve this problem by reconstructing the structure and texture of the mural images and virtually filling the damaged areas.Most existing mural image restoration methods are hard to effectively restore missing mural content with complex structures and rich color variations.In response to this issue,this paper proposes an ancient mural image restoration network using the involution cascade attention mechanism.The network firstly uses involution operations instead of traditional convolutions to improve the quality of repairing damaged mural textures and colors.Secondly,a cascaded attention module is proposed to capture contextual information at different scales,which can better repair damaged areas of murals of different sizes.Thirdly,FFC residual blocks are introduced to capture global structural information to enhance the network’s color restoration ability for damaged areas of murals.This article conducts experiments on simulated and real damaged mural datasets,comparing the restoration results with four other classic methods.The experimental results show that the proposed model outperforms other comparative methods in restoring the clarity,color consistency,and structural continuity of mural textures.

Key words: Ancient mural digital restoration, Cascading attention module, Deep learning, Fast Fourier Convolutional, Involution

中图分类号: 

  • TP391
[1]LIANG Y,GUO D.Research on the color representation ofDunhuang mural art[C]//Proceedings of the 2017 2nd International Conference on Education,Sports,Arts and Management Engineering.2017.
[2]DENG X C,YU Y,YANG L X,et al.Automatic calibration algorithm for the diseased areas of Ming Dynasty murals in Jianshui Zhilin Temple [J].Journal of Computer Aided Design and Graphics,2023,35(10):1554-1565.
[3]MARCELO B,GUILLERMO S,VINCENT C,et al.Image inpainting[C]//Proceedings of the 27th Annual Conference on Computer Gra-phics and Interactive Techniques.2020:417-424.
[4]JAIDILERT S,FAROOQUE G.Crack Detection and Images Inpainting Method for Thai Mural Painting Images[C]//2018 IEEE 3rd International Conference on Image,Vision and Computing.2018:143-148.
[5]ZHAO C X,LIU J N,LI B Q,et al.Multiscale construction of bifunctional electrocatalysts for long-lifespan rechargeable zinc-air batteries[J].Advanced Functional Materials,2020,30(36):2003619.
[6]JIAO L J,WANG W J,LI B J,et al.Wutai mountain mural inpainting based on improved block matching algorithm[J].Journal of Computer Aided Design and Graphics,2019,31(1):118-125.
[7]CAO J,LI Y,ZHANG Q,et al.Restoration of an ancient temple mural by a local search algorithm of an adaptive sample block[J].Heritage Science,2019,7(1):39.
[8]XU H,KANG J M,ZHANG J W.Digital Mural Restoration Method Based on Feature Perception [J].Computer Science,2022,49(6):217-223.
[9]ZHANG L,YU Y,GE H.Mural Restoration Based on FastFourier Convolution and Feature Trimming Coordinate Attention [J].Computer Science,2024,51(S1):338-346.
[10]CAO J,ZHANG Z,ZHAO A,et al.Ancient mural restoration based on a modified generative adversarial network[J].Heritage Science,2020,8:1-14.
[11]WANG N,WANG W,HU W,et al.Thanka Mural Inpainting Based on Multi-Scale Adaptive Partial Convolution and Stroke-Like Mask[J].IEEE Transactions on Image Processing,2021,30:3720-3733.
[12]LYU C,LI Z,SHEN Y,et al.SeparaFill:Two generators connected mural image restoration based on generative adversarial network with skip connect[J].Heritage Science,2022,10(1):135.
[13]CHEN Y,CHEN J,TAO M F.Ural inpainting progressive ge-nerative adversarial networks based on structure guided[J].Journal of Beijing University of Aeronautics and Astronautics,2021,49(6):1247-1259.
[14]LI J,WANG H,DENG Z,et al.Restoration of non-structural damaged murals in Shenzhen Bao’an based on a generator-discriminator network[J].Heritage Science,2021,9:1-14.
[15]张茼茼,刘恒.基于潜在特征重构和注意力机制的人脸图像修复[J].重庆工商大学学报(自然科学版),2023,40(2):73-78.
[16]YU T X,LIN C,ZHANG S J,et al.End-to-End Partial Convolutions Neural Networks for Dunhuang Grottoes Wall-Painting Restoration[C]//2019 IEEE/CVF International Conference on Computer Vision Workshop.2019:1447-1455.
[17]SCHMIDT A,MADHU P,MAIER A,et al.ARIN:Adaptive Resampling and Instance Normalization for Robust Blind Inpainting of Dunhuang Cave Paintings[C]//2022 Eleventh International Conference on Image Processing Theory,Tools and Applications(IPTA).2022.
[18]CIORTAN I M,GEORGE S,HARDEBERG J Y.Colour-balanced edge-guided digital inpainting:Applications on artworks[J].Sensors,2021,21(6):2091.
[19]CHI L,JIANG B,MU Y.Fast fourier convolution[J].Advances in Neural Information Processing Systems,2020,33:4479-4488.
[20]BRIGHAM E O,MORROW R E.The fast Fourier transform[J].IEEE Spectrum,1967,4(12):63-70.
[21]NAZERI K,NG E,JOSEPH T,et al.EdgeConnect:Structure Guided Image Inpainting using Edge Prediction[C]//2019 IEEE/CVF International Conference on Computer Vision Workshop(ICCVW).2019:3265-3274.
[22]LI J,HE F,ZHANG L,et al.Progressive Reconstruction of Vi-sual Structure for Image Inpainting[C]//2019 IEEE/CVF International Conference on Computer Vision(ICCV).2019:5961-5970.
[23]ANG N,ZHANG Y,ZHANG L.Dynamic Selection Network for Image Inpainting[J].IEEE Transactions on Image Processing,2021,30:1784-1798.
[24]DENG J,DONG W,SOCHER R,et al.ImageNet:A large-scale hierarchical image database[C]//2009 IEEE Conference on Computer Vision and Pattern Recognition.2009:248-255.
[25]ZENG Y,FU J,CHAO H,et al.Aggregated contextual transformations for high-resolution image inpainting[J].IEEE Transactions on Visualization and Computer Graphics,2022,29(7):3266-3280.
[26]WANG N,ZHANG Y,ZHANG L.Dynamic selection networkfor image inpainting[J].IEEE Transactions on Image Proces-sing,2021,30:1784-1798.
[27]NAZERI K,NG E,JOSEPH T,et al.Edgeconnect:Structureguided image inpainting using edge prediction[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops.2019.
[28]XIA B,ZHANG Y,WANG S,et al.Diffir:Efficient diffusionmodel for image restoration[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2023:13095-13105.
[29]LIU G,REDA F A,SHIH K J,et al.Image inpainting for irre-gular holes using partial convolutions[C]//Proceedings of the European Conference on Computer Vision(ECCV).2018:85-100.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!