计算机科学

计算机科学 ›› 2024, Vol. 51 ›› Issue (4): 209-216.doi: 10.11896/jsjkx.230100141

• 计算机图形学&多媒体 • 上一篇    下一篇

基于双流YOLOv4的金属表面缺陷检测方法

徐浩, 李丰润, 陆璐   

  1. 华南理工大学计算机科学与工程学院 广州510640
  • 收稿日期:2023-01-31 修回日期:2023-05-18 出版日期:2024-04-15 发布日期:2024-04-10
  • 通讯作者: 徐浩(13928828885@139.com)
  • 基金资助:
    广东省重点领域研发计划项目(2022B0101070001);中山市产学研重大项目(201602103890051)

Metal Surface Defect Detection Method Based on Dual-stream YOLOv4

XU Hao, LI Fengrun, LU Lu   

  1. School of Computer Science and Engineering,South China University of Technology,Guangzhou 510640,China
  • Received:2023-01-31 Revised:2023-05-18 Online:2024-04-15 Published:2024-04-10
  • Supported by:
    General Project of Natural Science Foundation of Guangdong Province(2022B0101070001) and Major Program of the Zhongshan Industry-Academia-Research Fund(201602103890051).

PDF (PC)

125

摘要： 目前有许多学者使用深度学习进行表面缺陷检测研究,由于这些研究大都沿用主流目标检测算法的思路,注重高级语义特征,而忽视了低级语义信息(色彩、形状)对表面缺陷检测的重要性,因此导致缺陷检测效果不够理想。为解决上述问题,提出了一种金属表面缺陷检测网络——双流YOLOv4网络,骨干网络分成两个分支,输入分为高分辨率图像和低分辨率图像,浅分支负责从高分辨率图像中提取低级特征,深分支负责从低分辨率图像中提取高级特征,通过削减两分支的层数和通道数来减少模型总参数量;为了强化低级语义特征,提出了一种树形多尺度融合方法(Tree-structured Multi-scale Feature Fusion Me-thod,TMFF),并设计了一个结合极化自注意力机制和空间金字塔池化的特征融合模块(Feature Fusion Module with Polarized Self-Attention Mechanism and Spatial Pyramid Pooling,FFM-PSASPP)应用到TMFF中。在东北大学热轧带表面缺陷数据集NEU-DET、金属表面缺陷数据集GC10-DET和伊莱特电饭煲内胆缺陷数据集Enaiter的测试集上对所提算法进行了测试,测得的map@50结果分别为0.80,0.66和0.57,相比大部分主流的用于缺陷检测的目标检测算法均有提升,且模型参数量仅为原YOLOv4的一半,速度与YOLOv4接近,可满足实际使用需求。

关键词: 金属表面缺陷检测, 目标检测, YOLOv4, 双流骨干网络, 多尺度特征强化

Abstract: Currently,many researchers use deep learning for surface defect detection.However,most of these studies follow the mainstream object detection algorithm and focus on high-level semantic features while neglecting the importance of low-level semantic information(color,shape) for surface defect detection,resulting in unsatisfactory defect detection effect.To address this issue,a metal surface defect detection network called the dual-stream YOLOv4 network is proposed.The backbone network is split into two branches,with inputs consisting of high-resolution and low-resolution images.The shallow branch is responsible for extracting low-level features from the high-resolution image,while the deep branch is responsible for extracting high-level features from the low-resolution image.The model's total parameter volume is reduced by cutting down the number of layers and channels in both branches.To enhance the low-level semantic features,a tree-structured multi-scale feature fusion method(TMFF) is proposed,and a feature fusion module with a polarized self-attention mechanism and spatial pyramid pooling(FFM-PSASPP) is designed and applied to the TMFF.The algorithm's map@50 results on the test sets of the Northeastern University hot-rolled strip surface defect dataset(NEU-DET),the metal surface defect dataset(GC10-DET),and the enaiter rice cooker inner pot defect dataset are 0.80,0.66,and 0.57,respectively.Compared to most mainstream object detection algorithms used for defect detection,there is an improvement,and the model's parameter volume is only half that of the original YOLOv4,with a speed close to YOLOv4,making it suitable for practical use.

Key words: Metal surface defect detection, Object detection, YOLOv4, Dual-stream backbone network, Multi-scale feature enhancement

中图分类号: 

  • TP391
[1]LIU M,LIU Y,HU H,et al.Genetic algorithm and mathematical morphology based binarization method for strip steel defect image with non-uniform illumination[J].Journal of Visual Communication and Image Representation,2016,37:70-77.
[2]ZHAO X,HE Z,ZHANG S,et al.A sparse-representation-based robust inspection system for hidden defects classification in casting components[J].Neurocomputing,2015,153:1-10.
[3]MA J,WANG Y,SHI C,et al.Fast surface defect detectionusing improved Gabor filters[C]//2018 25th IEEE International Conference on Image Processing(ICIP).Athens:IEEE,2018:1508-1512.
[4]XIANG S,SHUN'ICHI K,LIANG D.Robust Method for Detecting Defect in Images Printed on 3D Micro-textured Surfaces:Modified Multiple Paired Pixel Consistency[C]//VISIGRAPP(5:VISAPP).2020:260-267.
[5]YU Z,WU X,GU X.Fully convolutional networks for surface defect inspection in industrial environment[C]//International Conference on Computer Vision Systems.Shenzhen:Springer,2017:417-426.
[6]LONG J,SHELHAMER E,DARRELL T.Fully convolutional networks for semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Boston:IEEE,2015:3431-3440.
[7]LIU E,CHEN K,XIANG Z,et al.Conductive particle detection via deep learning for ACF bonding in TFT-LCD manufacturing[J].Journal of Intelligent Manufacturing,2020,31(4):1037-1049.
[8]RONNEBERGER O,FISCHER P,BROX T.U-net:Convolu-tional networks for biomedical image segmentation[C]//International Conference on Medicalimage Computing and Computer-assisted Intervention.Munich:Springer,2015:234-241.
[9]XU Y,LI D,XIE Q,et al.Automatic defect detection and segmentation of tunnel surface using modified Mask R-CNN[J].Measurement,2021,178:109316.
[10]HE K,GKIOXARI G,DOLLÁR P,et al.Mask r-cnn[C]//Proceedings of the IEEE International Conference on Computer Vision.Venice:IEEE,2017:2961-2969.
[11]CHEN X,LV J,FANG Y,et al.Online Detection of Surface Defects Based on Improved YOLOV3[J].Sensors,2022,22(3):817.
[12]SANDLER M,HOWARD A,ZHU M,et al.Mobilenetv2:In-verted residuals and linear bottlenecks[C]//Proceedings of the IEEE Conference on Computer Visionand Pattern Recognition.Salt Lake City:IEEE,2018:4510-4520.
[13]ZHANG H,SONG Y,CHEN Y,et al.MRSDI-CNN:Multi-model rail surface defect inspection system based on convolutional neural networks[J].IEEE Transactions on Intelligent Transportation Systems,2021,23:11162-11177.
[14]LIU W,ANGUELOV D,ERHAN D,et al.Ssd:Single shotmultibox detector[C]//European Conference on Computer Vision.Amsterdam:Springer,2016:21-37.
[15]REDMON J,FARHADI A.Yolov3:An incremental improve-ment [EB/OL].(2018-04-08)[2022-11-20].https://arxiv.org/abs/1804.02767.
[16]FANG J,TAN X,WANG Y.ACRM:Attention Cascade R-CNN with Mix-NMS for Metallic Surface Defect Detection[C]//2020 25th International Conference on Pattern Recognition(ICPR).Milan:IEEE,2021:423-430.
[17]CAI Z,VASCONCELOS N.Cascade r-cnn:Delving into highquality object detection[C]//Proceedings of the IEEE Confe-rence on Computer Vision and Pattern Recognition.Salt Lake City:IEEE,2018:6154-6162.
[18]TANG M,LI Y,YAO W,et al.A strip steel surface defect detection method based on attention mechanism and multi-scale maxpooling[J].Measurement Science and Technology,2021,32(11):115401.
[19]REN S,HE K,GIRSHICK R,et al.Faster r-cnn:Towards real-time object detection with region proposal networks[J].Advances in Neural Information Processing Systems,2015,28:91-99.
[20]XU Y,ZHANG K,WANG L.Metal Surface Defect Detection Using Modified YOLO[J].Algorithms,2021,14(9):257.
[21]HAO R,LU B,CHENG Y,et al.A steel surface defect inspection approach towards smart industrial monitoring[J].Journal of Intelligent Manufacturing,2021,32(7):1833-1843.
[22]ZHAO W,CHEN F,HUANG H,et al.A new steel defect detection algorithm based on deep learning[J].Computational Intelligence and Neuroscience,2021,2021:1-13.
[23]DAI J,QI H,XIONG Y,et al.Deformable convolutional net-works[C]//Proceedings of the IEEE International Conference on Computer Vision.Venice:IEEE,2017:764-773.
[24]LIN T Y,DOLLÁR P,GIRSHICK R,et al.Feature pyramidnetworks for object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Ho-nolulu:IEEE,2017:2117-2125.
[25]HE X,QIAN X.A real-time surface defect detectionsystem for industrial products with long-tailed distribution[C]//2021 IEEE Industrial Electronics and Applications Conference(IEACon).Penang:IEEE,2021:313-317.
[26]VASWANI A,SHAZEER N,PARMAR N,et al.Attention isall you need[J].Advances in Neural Information Processing Systems,2017,30:5998-6008.
[27]BOCHKOVSKIV A,WANG C Y,LIAO H Y M.Yolov4:Optimal speed and accuracy of object detection[EB/OL].(2020-04-23)[2022-11-20].https://arxiv.org/abs/2004.10934.
[28]MA J H,YANG D,WANG J S,ET AL.Research on improved Debevec-YOLOv5 for surfaced fect detection methods of high-reflective metals[J].Journal of Chongqing University of Technology(Natural Science),2023,37(7):169-176.
[29]ZHANG C,CHANG C,JAMSHIDI M.Concrete bridge surface damage detection using a single stage detector[J].Computer Aided Civil and Infrastructure Engineering,2020,35(4):389-409.
[30]LIN T Y,GOYAL P,GIRSHICK R,et al.Focal loss for dense object detection[C]//Proceedings of the IEEE International Conference on Computer Vision.Venice:IEEE,2017:2980-2988.
[31]LIU Z,GAO G,SUN L,et al.HRDNet:High-resolution detection network for small objects[C]//2021 IEEE International Conference on Multimedia and Expo(ICME).Shenzhen:IEEE,2021:1-6.
[32]ZHENG C,SHI D,YAN X,et al.GlassNet:Label Decoupling-based Three-stream Neural Network for Robust Image Glass Detection[J].Computer Graphics Forum,2022,41(1):377-388.
[33]ZHU H,LI P,XIE H,et al.I can find you! Boundary-guided separated attention network for camouflaged object detection[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2022:3608-3616.
[34]HE K,ZHANG X,REN S,et al.Spatial pyramid pooling in deepconvolutional networks for visual recognition[J].IEEE transactions on pattern analysis and machine intelligence,2015,37(9):1904-1916.
[35]LIU H,LIU F,FAN X,et al.Polarized self-attention:Towardshigh-quality pixel-wise mapping[J].Neurocomputing,2022,506:158-167.
[36]HE Y,SONG K,MENG Q,et al.An end-to-end steel surfacedefect detection approach via fusing multiple hierarchical features[J].IEEE Transactions on Instrumentation and Measurement,2019,69(4):1493-1504.
[37]LV X,DUAN F,JIANG J,et al.Deep metallic surface defect detection:The new benchmark and detectionnetwork[J].Sensors,2020,20(6):1562.
[38]SIMONYAN K,ZISSERMAN A.Very deep convolutional networks for large-scale image recognition[EB/OL].(2014-09-04)[2020-11-20].https://arxiv.org/abs/1409.1556.
[39]HE K,ZHANG X,REN S,et al.Deep residual learning forimage recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas:IEEE,2016:770-778.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发