Computer Science ›› 2024, Vol. 51 ›› Issue (4): 209-216.doi: 10.11896/jsjkx.230100141

• Computer Graphics & Multimedia • Previous Articles     Next Articles

Metal Surface Defect Detection Method Based on Dual-stream YOLOv4

XU Hao, LI Fengrun, LU Lu   

  1. School of Computer Science and Engineering,South China University of Technology,Guangzhou 510640,China
  • Received:2023-01-31 Revised:2023-05-18 Online:2024-04-15 Published:2024-04-10
  • Supported by:
    General Project of Natural Science Foundation of Guangdong Province(2022B0101070001) and Major Program of the Zhongshan Industry-Academia-Research Fund(201602103890051).

Abstract: Currently,many researchers use deep learning for surface defect detection.However,most of these studies follow the mainstream object detection algorithm and focus on high-level semantic features while neglecting the importance of low-level semantic information(color,shape) for surface defect detection,resulting in unsatisfactory defect detection effect.To address this issue,a metal surface defect detection network called the dual-stream YOLOv4 network is proposed.The backbone network is split into two branches,with inputs consisting of high-resolution and low-resolution images.The shallow branch is responsible for extracting low-level features from the high-resolution image,while the deep branch is responsible for extracting high-level features from the low-resolution image.The model's total parameter volume is reduced by cutting down the number of layers and channels in both branches.To enhance the low-level semantic features,a tree-structured multi-scale feature fusion method(TMFF) is proposed,and a feature fusion module with a polarized self-attention mechanism and spatial pyramid pooling(FFM-PSASPP) is designed and applied to the TMFF.The algorithm's map@50 results on the test sets of the Northeastern University hot-rolled strip surface defect dataset(NEU-DET),the metal surface defect dataset(GC10-DET),and the enaiter rice cooker inner pot defect dataset are 0.80,0.66,and 0.57,respectively.Compared to most mainstream object detection algorithms used for defect detection,there is an improvement,and the model's parameter volume is only half that of the original YOLOv4,with a speed close to YOLOv4,making it suitable for practical use.

Key words: Metal surface defect detection, Object detection, YOLOv4, Dual-stream backbone network, Multi-scale feature enhancement

CLC Number: 

  • TP391
[1]LIU M,LIU Y,HU H,et al.Genetic algorithm and mathematical morphology based binarization method for strip steel defect image with non-uniform illumination[J].Journal of Visual Communication and Image Representation,2016,37:70-77.
[2]ZHAO X,HE Z,ZHANG S,et al.A sparse-representation-based robust inspection system for hidden defects classification in casting components[J].Neurocomputing,2015,153:1-10.
[3]MA J,WANG Y,SHI C,et al.Fast surface defect detectionusing improved Gabor filters[C]//2018 25th IEEE International Conference on Image Processing(ICIP).Athens:IEEE,2018:1508-1512.
[4]XIANG S,SHUN'ICHI K,LIANG D.Robust Method for Detecting Defect in Images Printed on 3D Micro-textured Surfaces:Modified Multiple Paired Pixel Consistency[C]//VISIGRAPP(5:VISAPP).2020:260-267.
[5]YU Z,WU X,GU X.Fully convolutional networks for surface defect inspection in industrial environment[C]//International Conference on Computer Vision Systems.Shenzhen:Springer,2017:417-426.
[6]LONG J,SHELHAMER E,DARRELL T.Fully convolutional networks for semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Boston:IEEE,2015:3431-3440.
[7]LIU E,CHEN K,XIANG Z,et al.Conductive particle detection via deep learning for ACF bonding in TFT-LCD manufacturing[J].Journal of Intelligent Manufacturing,2020,31(4):1037-1049.
[8]RONNEBERGER O,FISCHER P,BROX T.U-net:Convolu-tional networks for biomedical image segmentation[C]//International Conference on Medicalimage Computing and Computer-assisted Intervention.Munich:Springer,2015:234-241.
[9]XU Y,LI D,XIE Q,et al.Automatic defect detection and segmentation of tunnel surface using modified Mask R-CNN[J].Measurement,2021,178:109316.
[10]HE K,GKIOXARI G,DOLLÁR P,et al.Mask r-cnn[C]//Proceedings of the IEEE International Conference on Computer Vision.Venice:IEEE,2017:2961-2969.
[11]CHEN X,LV J,FANG Y,et al.Online Detection of Surface Defects Based on Improved YOLOV3[J].Sensors,2022,22(3):817.
[12]SANDLER M,HOWARD A,ZHU M,et al.Mobilenetv2:In-verted residuals and linear bottlenecks[C]//Proceedings of the IEEE Conference on Computer Visionand Pattern Recognition.Salt Lake City:IEEE,2018:4510-4520.
[13]ZHANG H,SONG Y,CHEN Y,et al.MRSDI-CNN:Multi-model rail surface defect inspection system based on convolutional neural networks[J].IEEE Transactions on Intelligent Transportation Systems,2021,23:11162-11177.
[14]LIU W,ANGUELOV D,ERHAN D,et al.Ssd:Single shotmultibox detector[C]//European Conference on Computer Vision.Amsterdam:Springer,2016:21-37.
[15]REDMON J,FARHADI A.Yolov3:An incremental improve-ment [EB/OL].(2018-04-08)[2022-11-20].https://arxiv.org/abs/1804.02767.
[16]FANG J,TAN X,WANG Y.ACRM:Attention Cascade R-CNN with Mix-NMS for Metallic Surface Defect Detection[C]//2020 25th International Conference on Pattern Recognition(ICPR).Milan:IEEE,2021:423-430.
[17]CAI Z,VASCONCELOS N.Cascade r-cnn:Delving into highquality object detection[C]//Proceedings of the IEEE Confe-rence on Computer Vision and Pattern Recognition.Salt Lake City:IEEE,2018:6154-6162.
[18]TANG M,LI Y,YAO W,et al.A strip steel surface defect detection method based on attention mechanism and multi-scale maxpooling[J].Measurement Science and Technology,2021,32(11):115401.
[19]REN S,HE K,GIRSHICK R,et al.Faster r-cnn:Towards real-time object detection with region proposal networks[J].Advances in Neural Information Processing Systems,2015,28:91-99.
[20]XU Y,ZHANG K,WANG L.Metal Surface Defect Detection Using Modified YOLO[J].Algorithms,2021,14(9):257.
[21]HAO R,LU B,CHENG Y,et al.A steel surface defect inspection approach towards smart industrial monitoring[J].Journal of Intelligent Manufacturing,2021,32(7):1833-1843.
[22]ZHAO W,CHEN F,HUANG H,et al.A new steel defect detection algorithm based on deep learning[J].Computational Intelligence and Neuroscience,2021,2021:1-13.
[23]DAI J,QI H,XIONG Y,et al.Deformable convolutional net-works[C]//Proceedings of the IEEE International Conference on Computer Vision.Venice:IEEE,2017:764-773.
[24]LIN T Y,DOLLÁR P,GIRSHICK R,et al.Feature pyramidnetworks for object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Ho-nolulu:IEEE,2017:2117-2125.
[25]HE X,QIAN X.A real-time surface defect detectionsystem for industrial products with long-tailed distribution[C]//2021 IEEE Industrial Electronics and Applications Conference(IEACon).Penang:IEEE,2021:313-317.
[26]VASWANI A,SHAZEER N,PARMAR N,et al.Attention isall you need[J].Advances in Neural Information Processing Systems,2017,30:5998-6008.
[27]BOCHKOVSKIV A,WANG C Y,LIAO H Y M.Yolov4:Optimal speed and accuracy of object detection[EB/OL].(2020-04-23)[2022-11-20].https://arxiv.org/abs/2004.10934.
[28]MA J H,YANG D,WANG J S,ET AL.Research on improved Debevec-YOLOv5 for surfaced fect detection methods of high-reflective metals[J].Journal of Chongqing University of Technology(Natural Science),2023,37(7):169-176.
[29]ZHANG C,CHANG C,JAMSHIDI M.Concrete bridge surface damage detection using a single stage detector[J].Computer Aided Civil and Infrastructure Engineering,2020,35(4):389-409.
[30]LIN T Y,GOYAL P,GIRSHICK R,et al.Focal loss for dense object detection[C]//Proceedings of the IEEE International Conference on Computer Vision.Venice:IEEE,2017:2980-2988.
[31]LIU Z,GAO G,SUN L,et al.HRDNet:High-resolution detection network for small objects[C]//2021 IEEE International Conference on Multimedia and Expo(ICME).Shenzhen:IEEE,2021:1-6.
[32]ZHENG C,SHI D,YAN X,et al.GlassNet:Label Decoupling-based Three-stream Neural Network for Robust Image Glass Detection[J].Computer Graphics Forum,2022,41(1):377-388.
[33]ZHU H,LI P,XIE H,et al.I can find you! Boundary-guided separated attention network for camouflaged object detection[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2022:3608-3616.
[34]HE K,ZHANG X,REN S,et al.Spatial pyramid pooling in deepconvolutional networks for visual recognition[J].IEEE transactions on pattern analysis and machine intelligence,2015,37(9):1904-1916.
[35]LIU H,LIU F,FAN X,et al.Polarized self-attention:Towardshigh-quality pixel-wise mapping[J].Neurocomputing,2022,506:158-167.
[36]HE Y,SONG K,MENG Q,et al.An end-to-end steel surfacedefect detection approach via fusing multiple hierarchical features[J].IEEE Transactions on Instrumentation and Measurement,2019,69(4):1493-1504.
[37]LV X,DUAN F,JIANG J,et al.Deep metallic surface defect detection:The new benchmark and detectionnetwork[J].Sensors,2020,20(6):1562.
[38]SIMONYAN K,ZISSERMAN A.Very deep convolutional networks for large-scale image recognition[EB/OL].(2014-09-04)[2020-11-20].https://arxiv.org/abs/1409.1556.
[39]HE K,ZHANG X,REN S,et al.Deep residual learning forimage recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas:IEEE,2016:770-778.
[1] LIU Zeyu, LIU Jianwei. Video and Image Salient Object Detection Based on Multi-task Learning [J]. Computer Science, 2024, 51(4): 217-228.
[2] HAO Ran, WANG Hongjun, LI Tianrui. Deep Neural Network Model for Transmission Line Defect Detection Based on Dual-branch Sequential Mixed Attention [J]. Computer Science, 2024, 51(3): 135-140.
[3] ZHANG Yang, XIA Ying. Object Detection Method with Multi-scale Feature Fusion for Remote Sensing Images [J]. Computer Science, 2024, 51(3): 165-173.
[4] WANG Weijia, XIONG Wenzhuo, ZHU Shengjie, SONG Ce, SUN He, SONG Yulong. Method of Infrared Small Target Detection Based on Multi-depth Feature Connection [J]. Computer Science, 2024, 51(1): 175-183.
[5] SHI Dianxi, LIU Yangyang, SONG Linna, TAN Jiefu, ZHOU Chenlei, ZHANG Yi. FeaEM:Feature Enhancement-based Method for Weakly Supervised Salient Object Detection via Multiple Pseudo Labels [J]. Computer Science, 2024, 51(1): 233-242.
[6] YANG Yi, SHEN Sheng, DOU Zhiyang, LI Yuan, HAN Zhenjun. Tiny Person Detection for Intelligent Video Surveillance [J]. Computer Science, 2023, 50(9): 75-81.
[7] ZHU Ye, HAO Yingguang, WANG Hongyu. Deep Learning Based Salient Object Detection in Infrared Video [J]. Computer Science, 2023, 50(9): 227-234.
[8] LIU Yubo, GUO Bin, MA Ke, QIU Chen, LIU Sicong. Design of Visual Context-driven Interactive Bot System [J]. Computer Science, 2023, 50(9): 260-268.
[9] WANG Xu, WU Yanxia, ZHANG Xue, HONG Ruize, LI Guangsheng. Survey of Rotating Object Detection Research in Computer Vision [J]. Computer Science, 2023, 50(8): 79-92.
[10] HUO Weile, JING Tao, REN Shuang. Review of 3D Object Detection for Autonomous Driving [J]. Computer Science, 2023, 50(7): 107-118.
[11] QI Xuanlong, CHEN Hongyang, ZHAO Wenbing, ZHAO Di, GAO Jingyang. Study on BGA Packaging Void Rate Detection Based on Active Learning and U-Net++ Segmentation [J]. Computer Science, 2023, 50(6A): 220200092-6.
[12] WANG Guogang, WU Yan, LIU Yibo. Target Detection Algorithm Based on Compound Scaling Deep Iterative CNN by RegressionConverging and Scaling Mixture [J]. Computer Science, 2023, 50(6A): 220500230-9.
[13] WU Liuchen, ZHANG Hui, LIU Jiaxuan, ZHAO Chenyang. Defect Detection of Transmission Line Bolt Based on Region Attention Mechanism andMulti-scale Feature Fusion [J]. Computer Science, 2023, 50(6A): 220200096-7.
[14] DOU Zhi, HU Chenguang, LIANG Jingyi, ZHENG Liming, LIU Guoqi. Lightweight Target Detection Algorithm Based on Improved Yolov4-tiny [J]. Computer Science, 2023, 50(6A): 220700006-7.
[15] JIA Tianhao, PENG Li. SSD Object Detection Algorithm with Residual Learning and Cyclic Attention [J]. Computer Science, 2023, 50(5): 170-176.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!