Computer Science ›› 2024, Vol. 51 ›› Issue (11A): 231200162-8.doi: 10.11896/jsjkx.231200162

• Image Processing & Multimedia Technology • Previous Articles     Next Articles

Integration of Multi-scale and Attention Mechanism for Ancient Mural Detachment Area Localization

WANG Xinchao, YU Ying, CHEN An, ZHAO Huirong   

  1. School of Information Science and Engineering,Yunnan University,Kunming 650500,China
  • Online:2024-11-16 Published:2024-11-13
  • About author:WANG Xinchao,born in 1998,postgraduate,is a member of CCF(No.R9235G).His main research interests include computer vision and deep lear-ning.
    YU Ying,born in 1977,Ph.D,associate professor.His main research interests include image and vision,artificial neural network.
  • Supported by:
    National Natural Science Foundation of China(62166048,61263048)and Yunnan Provincial Applied Research Program(2018FB102).

Abstract: In response to the challenging problem of accurately automating the localization of peeling areas in ancient murals,this paper proposes a lightweight network model based on a multi-scale fusion attention network.Firstly,a multi-scale fusion attention module is introduced to enable the network to learn features at different scales,with a focus on the most critical features,thus improving the accuracy of mural missing area localization.Deep separable convolutions are employed in the proposed multi-scale fusion attention module to make the network model more lightweight.Secondly,a combination of cross-entropy loss and Dice score is used as the loss function,and the Adam optimizer is applied to further enhance the accuracy of mural missing area localization.Additionally,datasets of Dunhuang Mogao Grottoes murals and Yunnan Shiping Luose Temple murals are constructed,and their peeling areas are manually annotated.Experimental results demonstrate that the proposed network model accurately localizes peeling regions in ancient murals.In comparison with existing deep learning methods,this model significantly reduces the number of parameters and exhibits better performance in terms of subjective visual quality,objective evaluation metrics,and generalization capabilities.

Key words: Mural damage, U-Net, Multi-scale, Attention mechanism, Deep learning, Lightweight

CLC Number: 

  • TP391
[1]ROSA DE A,BONACEHI A M,CAPPELLINI V,et al.Image segmentation and region filling for virtual restoration of artworks [C]//Proceedings 2001 International Conference on Image Processing.Thessaloniki:IEEE,2001:562-565.
[2]LIU J M.Research on the protection and intelligent restoration technology of ancient mural images [D].Hangzhou:Zhejiang University,2010.
[3]TURAKHIA N,SHAH R,JOSHI M.Automatic crack detec-tion in heritage site images for image inpainting [C]//Proceedings of the Eighth Indian Conference on Computer Vision,Graphics and Image Processing.Mumbai:ACM,2012:1-8.
[4]WU M,WANG H Q,LI W Y.Research on multi-scale detection and image inpainting of Tang dynastytomb murals [J].Computer Engineering and Applications,2016,52(11):169-174.
[5]LI C Y,WANG H Q,WU M,et al.Automatic recognition and virtual restoration of mud spot disease of Tang dynasty tomb murals image [J].Computer Engineering and Applications,2016,52(15):233-236.
[6]CAO J F,LI Y F,CUI H Y,et al.The Application of Improved Region Growing Algorithm for the Automatic Calibration of Shedding Disease [J].Journal of Xinjiang University(Natural Science Edition),2018,35(4):429-436.
[7]DENG X,YU Y.Automatic calibration of crack and flaking dis-eases in ancient temple murals [J].Heritage Science,2022,10(1):163.
[8]HUANG R,FENG W,FAN M,et al.Learning multi-path CNNfor mural deterioration detection [J].Journal of Ambient Intelligence and Humanized Computing,2020,11:3101-3108.
[9]LIN Y,XU C,LYU S.Disease Regions Recognition on MuralHyperspectral Images Combined by MNF and BP Neural Network [C]//Journal of Physics Conference Series.Qingdao:IOP,2019:104-110.
[10]LÜ S Q,WANG S H,HOU M L,et al.Extraction of mural paint loss diseases based on improved U-Net [J].Geomatics World,2022,29(1):69-74.
[11]SHELHAMER E,LONG J,DARRELL T.Fully convolutional networks for semantic segmentation [J].IEEE Trans Pattern Anal Mach Intell,2017,39(4):640-651.
[12]RONNEBERGER O,FISCHER P,BROX T.U-Net:convolu-tional networks for biomedical image segmentation [C]//Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Inter-vention.Munich:Springer,2015:234-241.
[13]ALOM M Z,HASAN M,YAKOPCIC C,et al.Recurrent residual convolutional neural network based on u-net(r2u-net)for medical image segmentation [EB/OL].(2018-02-20)[2018-05-29].https://arxiv.org/abs/1802.06955.
[14]OKTAY O,SCHLEMPER J,FOLGOC L L,et al.Attention u-net:Learning where to look for the pancreas [EB/OL].(2018-04-11)[2018-05-20].https://arxiv.org/abs/1804.03999.
[15]ZHOU Z,RAHMAN SIDDIQUEE M M,TAJBAKHSH N,et al.Unet++:A nested u-net architecture for medical image segmentation [C]//Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support Cham.Springer,2018:3-11.
[16]CHEN J,LU Y,YU Q,et al.Transunet:Transformers makestrong encoders for medical image segmentation [EB/OL].(2021-02-18)[2021-02-18].https://arxiv.org/abs/2102.04306.
[17]CAO H,WANG Y,CHEN J,et al.Swin-unet:Unet-like puretransformer for medical image segmentation [C]//European Conference on Computer Vision.Cham:Springer,2022:205-218.
[18]KIM N,KIM D,LAN C,et al.Restr:Convolution-free referringimage segmentation using transformers [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.New Orleans:IEEE,2022:18145-18154.
[19]RU L,ZHAN Y,YU B,et al.Learning affinity from attention:End-to-end weakly-supervised semantic segmentation with transformers [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.New Orleans:IEEE,2022:16846-16855.
[20]ZHANG J,YANG K,MA C,et al.Bending reality:Distortion-aware transformers for adapting to panoramic semantic segmentation [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.New Orleans:IEEE,2022:16917-16927.
[21]ZHANG H,LI F,XU H,et al.MP-Former:Mask-piloted transformer for image segmentation [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Vancouver:IEEE,2023:18074-18083.
[22]ZHANG C,BENGIO S,HARDT M,et al.Understanding deep learning(still)requires rethinking generalization[J].Communications of the ACM,2021,64(3):107-115.
[23]WOO S,PARK J,LEE J Y,et al.Cbam:Convolutional block attention module [C]//Proceedings of the European Conference on Computer Vision.Cham:Springer,2018:3-19.
[24]CHOLLET F.Xception:Deep learning with depthwise separable convolutions [C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Honolulu:IEEE,2017:1251-1258.
[25]BADRINARAYANAN V,KENDALL A,CIPOLLA R.Segnet:A deep convolutional encoder-decoder architecture for image segmentation [J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,39(12):2481-2495.
[26]CHEN G P,LI L,DAI Y,et al.NU-net:An unpretentious nested U-net for breast tumor segmentation [EB/OL].(2022-09-15)[2022-12-13].https://arxiv.org/abs/2209.07193
[27]PASZKE A,CHAURASIA A,KIM S,et al.Enet:A deep neural network architecture for real-time semantic segmentation[J].arXiv:1606.02147,2016.
[28]DINH B D,NGUYEN T T,TRAN T T,et al.1M parameters are enough? A lightweight CNN-based model for medical image segmentation[C]//2023 Asia Pacific Signal and Information Processing Association Annual Summit and Conference(APSIPA ASC).IEEE,2023:1279-1284.
[1] DU Yu, YU Zishu, PENG Xiaohui, XU Zhiwei. Padding Load:Load Reducing Cluster Resource Waste and Deep Learning Training Costs [J]. Computer Science, 2024, 51(9): 71-79.
[2] XU Jinlong, GUI Zhonghua, LI Jia'nan, LI Yingying, HAN Lin. FP8 Quantization and Inference Memory Optimization Based on MLIR [J]. Computer Science, 2024, 51(9): 112-120.
[3] LI Yunchen, ZHANG Rui, WANG Jiabao, LI Yang, WANG Ziqi, CHEN Yao. Re-parameterization Enhanced Dual-modal Realtime Object Detection Model [J]. Computer Science, 2024, 51(9): 162-172.
[4] HU Pengfei, WANG Youguo, ZHAI Qiqing, YAN Jun, BAI Quan. Night Vehicle Detection Algorithm Based on YOLOv5s and Bistable Stochastic Resonance [J]. Computer Science, 2024, 51(9): 173-181.
[5] LIU Qian, BAI Zhihao, CHENG Chunling, GUI Yaocheng. Image-Text Sentiment Classification Model Based on Multi-scale Cross-modal Feature Fusion [J]. Computer Science, 2024, 51(9): 258-264.
[6] LI Zhe, LIU Yiyang, WANG Ke, YANG Jie, LI Yafei, XU Mingliang. Real-time Prediction Model of Carrier Aircraft Landing Trajectory Based on Stagewise Autoencoders and Attention Mechanism [J]. Computer Science, 2024, 51(9): 273-282.
[7] LIU Qilong, LI Bicheng, HUANG Zhiyong. CCSD:Topic-oriented Sarcasm Detection [J]. Computer Science, 2024, 51(9): 310-318.
[8] YAO Yao, YANG Jibin, ZHANG Xiongwei, LI Yihao, SONG Gongkunkun. CLU-Net Speech Enhancement Network for Radio Communication [J]. Computer Science, 2024, 51(9): 338-345.
[9] SUN Yumo, LI Xinhang, ZHAO Wenjie, ZHU Li, LIANG Ya’nan. Driving Towards Intelligent Future:The Application of Deep Learning in Rail Transit Innovation [J]. Computer Science, 2024, 51(8): 1-10.
[10] KONG Lingchao, LIU Guozhu. Review of Outlier Detection Algorithms [J]. Computer Science, 2024, 51(8): 20-33.
[11] LIU Sichun, WANG Xiaoping, PEI Xilong, LUO Hangyu. Scene Segmentation Model Based on Dual Learning [J]. Computer Science, 2024, 51(8): 133-142.
[12] TANG Ruiqi, XIAO Ting, CHI Ziqiu, WANG Zhe. Few-shot Image Classification Based on Pseudo-label Dependence Enhancement and NoiseInterferenceReduction [J]. Computer Science, 2024, 51(8): 152-159.
[13] ZHANG Rui, WANG Ziqi, LI Yang, WANG Jiabao, CHEN Yao. Task-aware Few-shot SAR Image Classification Method Based on Multi-scale Attention Mechanism [J]. Computer Science, 2024, 51(8): 160-167.
[14] WANG Qian, HE Lang, WANG Zhanqing, HUANG Kun. Road Extraction Algorithm for Remote Sensing Images Based on Improved DeepLabv3+ [J]. Computer Science, 2024, 51(8): 168-175.
[15] XIAO Xiao, BAI Zhengyao, LI Zekai, LIU Xuheng, DU Jiajin. Parallel Multi-scale with Attention Mechanism for Point Cloud Upsampling [J]. Computer Science, 2024, 51(8): 183-191.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!