Computer Science ›› 2025, Vol. 52 ›› Issue (6A): 240800014-7.doi: 10.11896/jsjkx.240800014

• Image Processing & Multimedia Technology • Previous Articles     Next Articles

Material SEM Image Retrieval Method Based on Multi-scale Features and Enhanced HybridAttention Mechanism

ZENG Fanyun, LIAN Hechun, FENG Shanshan, WANG Qingmei   

  1. National Center for Materials Service Safety,University of Science and Technology Beijing,Beijing 100083,China
  • Online:2025-06-16 Published:2025-06-12
  • About author:ZENG Fanyun,born in 2000,postgra-duate.His main research interests include data mining and image retrieval.
    WANG Qingmei,born in 1975,Ph.D,associate researcher,master supervisor.Her main research interests include deep learning and computer vision.
  • Supported by:
    National Major Science and Technology Infrastructure Operation Program(GJFG2024001).

Abstract: Material SEM images are rich in content,and traditional retrieval methods and general-domain retrieval methods are easily affected byvarious factors such as image distortion and complex textures in image feature extraction,resulting in suboptimal extraction of key features.Aiming at the shortcomimgs of conventional methods in feature extraction and efficient retrieval of material SEM images,this paper proposes an image retrieval method based on multi-scale feature information,integratingAtrous Spatial Pyramid Pooling(ASPP) and an enhanced convolutional block attention module(ECBAM).This method employs the ConvNeXt network for feature extraction,leveraging the advantages of dilated convolutions with large receptive fields and residual networks to capture more details and complex textures,effectively extracting both local and global features.Additionally,by incorporating the latest Mamba module and modifying it into a bidirectional architecture to integrate CBAM,the enhanced mixed attention mechanism ECBAM is proposed.The combination of ASPP and ECBAM ensures stable and efficient feature fusion and enhancement.Experimental results demonstrate that this method achieves superior retrieval performance on material SEM image datasets,with an average retrieval accuracy improvement of 1.5% compared to mainstream retrieval methods.

Key words: Micro image, Image retrieval, ASPP, Hybrid attention mechanism, Mamba

CLC Number: 

  • TP391
[1]MA C,MU X,SHA D.Multi-layers feature fusion of convolutional neural network for scene classification of remote sensing[J].IEEE Access,2019,7:121685-121694.
[2]LIU Z,MAO H,WU C Y,et al.A convnet for the 2020s[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:11976-11986.
[3]XIE S,GIRSHICK R,DOLLÁR P,et al.Aggregated residual transformations for deep neural networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:1492-1500.
[4]WOO S,PARK J,LEE J Y,et al.Cbam:Convolutional block attention module[C]//Proceedings of the European Conference on Computer Vision(ECCV).2018:3-19.
[5]HE K,ZHANG X,REN S,et al.Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:770-778.
[6]VASWANI A,SHAZEER N,PARMAR N,et al.Attention isall you need[J].arXiv:1706.03762,2017.
[7]HAN K,WANG Y,CHEN H,et al.A survey on vision transformer[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2022,45(1):87-110.
[8]YANG P,WANG F,WEI W.Research on ConvNeXt FeatureExtraction for Image Data [J].Computer Science,2024,51(S1):295-301.
[9]CHEN L C,PAPANDREOU G,SCHROFF F,et al.Rethinking atrous convolution for semantic image segmentation[J].arXiv:1706.05587,2017.
[10]WANG Q,HE L,WANG Z Q,et al.Road Extraction Algorithm for Remote Sensing Images Based on Improved DeepLabv3+ [J].Computer Science,2024,51(8):168-175.
[11]HU J,SHEN L,SUN G.Squeeze-and-excitation networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:7132-7141.
[12]JIANG B,WAN Y,XIE X.Lightweight Steel Surface Defect Detection Model Based on Improved YOLOv5s [J].Computer Science,2023,50(S2):271-277.
[13]ZHANG S,LIH,ZHANG Y,et al.Image Retrieval Algorithm Based on Independent Attention Mechanism [J].Computer Science,2023,50(S1):328-333.
[14]ZHU L,LIAO B,ZHANG Q,et al.Vision mamba:Efficient visual representation learning with bidirectional state space model[J].arXiv:2401.09417,2024.
[15]LI K,LI X,WANG Y,et al.Videomamba:State space model for efficient video understanding[J].arXiv:2403.06977,2024.
[16]GUO T,WANG Y,MENG C,et al.Mambamorph:a mamba-based backbone with contrastive feature learning for deformable mr-ct registration[J].arXiv:2401.13934,2024.
[17]TEICHMANN M,ARAUJO A,ZHU M,et al.Detect-to-re-trieve:Efficient regional aggregation for image search[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:5109-5118.
[18]LINDENBERGER P,SARLIN P E,POLLEFEYS M.Light-glue:Local feature matching at light speed[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2023:17627-17638.
[19]GLEIZE P,WANG W,FEISZLI M.Silk:Simple learned keypoints[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2023:22499-22508.
[20]SUN J,SHEN Z,WANG Y,et al.LoFTR:Detector-free local feature matching with transformers[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:8922-8931.
[21]JIANG W,TRULLS E,HOSANG J,et al.Cotr:Correspondence transformer for matching across images[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2021:6207-6217.
[22]TIAN Y,BALNTAS V,NG T,et al.D2D:Keypoint extraction with describe to detect approach[C]//Proceedings of the Asian Conference on Computer Vision.2020.
[23]HOWARD A,SANDLER M,CHU G,et al.Searching for mobilenetv3[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2019:1314-1324.
[24]HUANG G,LIU Z,VANDERMAATEN L,et al.Densely connected convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:4700-4708.
[25]TAN M,LE Q.Efficientnet:Rethinking model scaling for convolutional neural networks[C]//International Conference on Machine Learning.PMLR,2019:6105-6114.
[1] LU Yuhan, CHEN Liquan, WANG Yu, HU Zhiyuan. Efficient Encrypted Image Content Retrieval System Based on SecureCNN [J]. Computer Science, 2023, 50(9): 26-34.
[2] ZHANG Shunyao, LI Huawang, ZHANG Yonghe, WANG Xinyu, DING Guopeng. Image Retrieval Based on Independent Attention Mechanism [J]. Computer Science, 2023, 50(6A): 220300092-6.
[3] ZHU Cheng-zhang, HUANG Jia-er, XIAO Ya-long, WANG Han, ZOU Bei-ji. Deep Hash Retrieval Algorithm for Medical Images Based on Attention Mechanism [J]. Computer Science, 2022, 49(8): 113-119.
[4] ZHANG Jian-xin, WU Yue, ZHANG Qiang, WEI Xiao-peng. Supervised Similarity Preserving Deep Second-order Hashing [J]. Computer Science, 2022, 49(11A): 210900021-8.
[5] MIAO Zhuang, WANG Ya-peng, LI Yang, WANG Jia-bao, ZHANG Rui, ZHAO Xin-xin. Robust Hash Learning Method Based on Dual-teacher Self-supervised Distillation [J]. Computer Science, 2022, 49(10): 159-168.
[6] WANG Chun-jing, LIU Li, TAN Yan-yan, ZHANG Hua-xiang. Image Retrieval Method Based on Fuzzy Color Features and Fuzzy Smiliarity [J]. Computer Science, 2021, 48(8): 191-199.
[7] WANG Xiao-fei, ZHOU Chao, LIU Li-gang. Lightweight Image Retrieval System Based on Feature Clustering [J]. Computer Science, 2021, 48(2): 148-152.
[8] XU Xing, SUN Jia-liang, WANG Zheng, YANG Yang. Feature Transformation for Defending Adversarial Attack on Image Retrieval [J]. Computer Science, 2021, 48(10): 258-265.
[9] JIAO Yang, YANG Chuan-ying, SHI Bao. Relevance Feedback Method Based on SVM in Shoeprint Images Retrieval [J]. Computer Science, 2020, 47(11A): 244-247.
[10] LI Zong-min, LI Si-yuan, LIU Yu-jie, LI Hua. Sketch-based Image Retrieval Based on Attention Model [J]. Computer Science, 2020, 47(11): 199-204.
[11] SUN Wei, ZHAO Yu-pu. Enhanced Rotation Invariant LBP Algorithm and Its Application in Image Retrieval [J]. Computer Science, 2019, 46(7): 263-267.
[12] HOU Yuan-yuan, HE Ru-han, LI Min, CHEN Jia. Clothing Image Retrieval Method Combining Convolutional Neural Network Multi-layerFeature Fusion and K-Means Clustering [J]. Computer Science, 2019, 46(6A): 215-221.
[13] HE Xia, TANG Yi-ping, WANG Li-ran, CHEN Peng, YUAN Gong-ping. Multitask Hierarchical Image Retrieval Technology Based on Faster RCNNH [J]. Computer Science, 2019, 46(3): 303-313.
[14] LI Xiao-yu, NIE Xiu-shan, CUI Chao-ran, JIAN Mu-wei, YIN Yi-long. Image Retrieval Algorithm Based on Transfer Learning [J]. Computer Science, 2019, 46(1): 73-77.
[15] LIU Ying, ZHANG Shuai, GE Yu-xiang, WANG Fu-ping, LI Da-xiang. Survey of Tire Pattern Image Retrieval Techniques [J]. Computer Science, 2018, 45(12): 52-60.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!