计算机科学 ›› 2023, Vol. 50 ›› Issue (2): 209-213.doi: 10.11896/jsjkx.220500153

• 计算机图形学&多媒体 • 上一篇    下一篇

基于特征融合的小样本目标检测

华杰, 刘学亮, 赵烨   

  1. 合肥工业大学计算机与信息学院 合肥 230601
  • 收稿日期:2022-05-17 修回日期:2022-11-08 出版日期:2023-02-15 发布日期:2023-02-22
  • 通讯作者: 刘学亮(Liuxueliang1982@gmail.com)
  • 作者简介:(jiehua@mail.hfut.edu.cn)
  • 基金资助:
    国家重点研发计划(2018AAA0102002);国家自然科学基金(61976076,61632007)

Few-shot Object Detection Based on Feature Fusion

HUA Jie, LIU Xueliang, ZHAO Ye   

  1. School of Computer Science and Information Engineering,Hefei University of Technology,Hefei 230601,China
  • Received:2022-05-17 Revised:2022-11-08 Online:2023-02-15 Published:2023-02-22
  • Supported by:
    National Key R & D Program of China(2018AAA0102002) and National Natural Science Foundation of China(61976076,61632007)

摘要: 小样本目标检测旨在通过少量的样本学习来训练目标检测模型,现有的小样本目标检测方法大多基于经典的目标检测算法。在二阶段的检测方法中,由于新类别样本数量少,产生了许多无关的边界框,导致候选区域的准确率较低。为了解决这个问题,提出了一种基于特征融合的小样本目标检测算法FF-FSOD。该方法采用特征融合的方法进行数据增强,对新类别样本进行补充,扩大样本的覆盖范围,同时引入FPN网络进行多尺度特征提取,再对RPN网络进行改进,引入支持集图像分支,计算支持集图像特征与查询集图像特征的深度互相关性,得到注意力特征图,进而获得更精确的候选框。所提模型的有效性在MS COCO和FSOD数据集上得到了验证,实验结果表明,该方法获得了更精准的候选框,进而提升了检测精度。

关键词: 小样本学习, 目标检测, 深度学习, 特征融合, 特征金字塔

Abstract: Few-shot object detection aims to train target detection model through a small amount of sample learning.At present,most of the existing few-shot object detection methods are based on classical target detection algorithms.In the two-stage detection method,due to the small number of new class samples,many irrelevant border boxes are generated,resulting in low accuracy of candidate regions.To solve this problem,this paper proposes a few-shot object detection algorithm FF-FSOD based on feature fusion.It uses the feature fusion method to enhance the data,supplements the new category samples,increases the coverage range of the sample,and introduces the FPN network to extract multi-scale feature.Then,the RPN network is improved,and the support set image branch is introduced.The depth correlation between the support set image feature and the query set image feature is calculated,and the attention feature map is obtained,and the more accurate candidate box is obtained.The effectiveness of the proposed model is verified on MS COCO and FSOD datasets.Experimental results show that the proposed method obtains more accurate candidate boxes and improves the detection accuracy.

Key words: Few-shot learning, Object detection, Deep learning, Feature fusion, Feature pyramid

中图分类号: 

  • TP391
[1]CAO J,QU X,LI X X.Few-shot image classification methodbased on sliding feature vectors[J].Journal of Jilin University(Engineering and Technology Edition),2021,51(5):1785-1791.
[2]CHEN H,WANG Y,WANG G,et al.LSTD:a low-shot transfer detector for object detection[C]//Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence and Thirtieth Innovative Applications of Artificial Intelligence Conference and Eighth AAAI Symposium on Educational Advances in Artificial Intelligence.2018:2836-2843.
[3]DONAHUE J,JIA Y,VINYALS O,et al.Decaf:A deep convolutional activation feature for generic visual recognition[C]//International Conference on Machine Learning.PMLR,2014:647-655.
[4]WU J,LIU S,HUANG D,et al.Multi-scale positive sample refinement for few-shot object detection[C]//European Confe-rence on Computer Vision.Cham:Springer,2020:456-472.
[5]YAN X,CHEN Z,XU A,et al.Meta r-cnn:Towards general solver for instance-level low-shot learning[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2019:9577-9586.
[6]XIAO Z,ZHONG P,QUAN Y,et al.Few-shot object detection with feature attention highlight module in remote sensing images[C]//2020 International Conference on Image,Video Processing and Artificial Intelligence.SPIE,2020:217-223.
[7]FAN Q,ZHUO W,TANG C K,et al.Few-shot object detection with attention-RPN and multi-relation detector[C]//Procee-dings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:4013-4022.
[8]WANG X,HUANG T,GONZALEZ J,et al.Frustratingly simple few-shot object detection[C]//International Conference on Machine Learning.PMLR,2020:9919-9928.
[9]LI X,DENG J,FANG Y.Few-shot object detection on remote sensing images[J].IEEE Transactions on Geoscience and Remote Sensing,2021,60:1-14.
[10]KANG B,LIU Z,WANG X,et al.Few-shot object detection via feature reweighting[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2019:8420-8429.
[11]ZHANG W,WANG Y X.Hallucination improves few-shot object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:13008-13017.
[12]RIOU K,ZHU J,LING S,et al.Few-Shot Object Detection in Real Life:Case Study on Auto-Harvest[C]//2020 IEEE 22nd International Workshop on Multimedia Signal Processing(MMSP).IEEE,2020:1-6.
[13]ZHU C,CHEN F,AHMED U,et al.Semantic relation reasoning for shot-stable few-shot object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:8782-8791.
[14]RAHMAN S,KHAN S,BARNES N,et al.Any-shot object detection[C]//Proceedings of the Asian Conference on Computer Vision.2020.
[15]CHEN Z,FU Y,ZHANG Y,et al.Semantic feature augmentation in few-shot learning[J].arXiv:1804.05298,2018.
[16]KARLINSKY L,SHTOK J,HARARY S,et al.Repmet:Representative-based metric learning for classification and few-shot object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:5197-5206.
[17]ZHANG T,ZHANG Y,SUN X,et al.Comparison network for one-shot conditional object detection[J].arXiv:1904.02317,2019.
[18]XU P B,SANG J T,LU D Y.Few shot image recognition based on class semantic similarity supervision[J].Journal of Image and Graphics,2021,26(7):1594-1603.
[19]XIAO Y,MARLET R.Few-shot object detection and viewpoint estimation for objects in the wild[C]//European Conference on Computer Vision.Cham:Springer,2020:192-210.
[20]SUN B,LI B,CAI S,et al.Fsce:Few-shot object detection via contrastive proposal encod-ing[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:7352-7362.
[21]DONG X,ZHENG L,MA F,et al.Few-example object detection with model communication[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2018,41(7):1641-1654.
[22]CHEN X,JIANG M,ZHAO Q.Leveraging bottom-up and top-down attention for few-shot object detection[J].arXiv:2007.12104,2020.
[23]REN S,HE K,GIRSHICK R,et al.Faster R-CNN:Towardsreal-time object detection with region proposal networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2016,39(6):1137-1149.
[24]GIRSHICK R.Fast r-cnn[C]//Proceedings of the IEEE International Conference on Computer Vision.2015:1440-1448.
[25]HARRIS E,MARCU A,PAINTER M,et al.Fmix:Enhancing mixed sample data augmentation[J].arXiv:2002.12047,2020.
[26]LIN T Y,DOLLÁR P,GIRSHICK R,et al.Feature pyramidnetworks for object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:2117-2125.
[27]TSUNGYL,PRIYA G,ROSS G,et al.Focal loss for dense object detection[C]//ICCV.2017.
[28]REDMON J,FARHADI A.Yolov3:An incremental improvement[J].arXiv:1804.02767,2018.
[29]WU A,HAN Y,ZHU L,et al.Universal-prototype enhancing for few-shot object detection[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2021:9567-9576.
[1] 董永峰, 黄港, 薛婉若, 李林昊.
融合IRT的图注意力深度知识追踪模型
Graph Attention Deep Knowledge Tracing Model Integrated with IRT
计算机科学, 2023, 50(3): 173-180. https://doi.org/10.11896/jsjkx.211200134
[2] 华晓凤, 冯娜, 于俊清, 何云峰.
基于规则推理的足球视频任意球射门事件检测
Shooting Event Detection of Free Kick in Soccer Video Based on Rule Reasoning
计算机科学, 2023, 50(3): 181-190. https://doi.org/10.11896/jsjkx.220300062
[3] 梅鹏程, 杨吉斌, 张强, 黄翔.
一种基于三维卷积的声学事件联合估计方法
Sound Event Joint Estimation Method Based on Three-dimension Convolution
计算机科学, 2023, 50(3): 191-198. https://doi.org/10.11896/jsjkx.220500259
[4] 白雪飞, 马亚楠, 王文剑.
基于特征融合的边缘引导乳腺超声图像分割方法
Segmentation Method of Edge-guided Breast Ultrasound Images Based on Feature Fusion
计算机科学, 2023, 50(3): 199-207. https://doi.org/10.11896/jsjkx.211200294
[5] 刘航, 普园媛, 吕大华, 赵征鹏, 徐丹, 钱文华.
极化自注意力约束颜色溢出的图像自动上色
Polarized Self-attention Constrains Color Overflow in Automatic Coloring of Image
计算机科学, 2023, 50(3): 208-215. https://doi.org/10.11896/jsjkx.220100149
[6] 谢秦秦, 何朗, 徐汝利.
基于多特征融合的油画艺术风格分类
Classification of Oil Painting Art Style Based on Multi-feature Fusion
计算机科学, 2023, 50(3): 223-230. https://doi.org/10.11896/jsjkx.211200110
[7] 张卫良, 陈秀宏.
跨层融合和感受野扩增的SSD目标检测算法
SSD Object Detection Algorithm with Cross-layer Fusion and Receptive Field Amplification
计算机科学, 2023, 50(3): 231-237. https://doi.org/10.11896/jsjkx.211100281
[8] 陈亮, 王璐, 李生春, 刘昌宏.
基于深度学习的可视化仪表板生成技术研究
Study on Visual Dashboard Generation Technology Based on Deep Learning
计算机科学, 2023, 50(3): 238-245. https://doi.org/10.11896/jsjkx.230100064
[9] 张译, 吴秦.
特征增强损失与前景注意力人群计数网络
Crowd Counting Network Based on Feature Enhancement Loss and Foreground Attention
计算机科学, 2023, 50(3): 246-253. https://doi.org/10.11896/jsjkx.220100219
[10] 应宗浩, 吴槟.
深度学习模型的后门攻击研究综述
Backdoor Attack on Deep Learning Models:A Survey
计算机科学, 2023, 50(3): 333-350. https://doi.org/10.11896/jsjkx.220600031
[11] 刘泽京, 邬楠, 黄抚群, 宋友.
基于知识图谱与协同过滤混合策略的在线编程评测系统题目推荐模型
Hybrid Programming Task Recommendation Model Based on Knowledge Graph and Collaborative Filtering for Online Judge
计算机科学, 2023, 50(2): 106-114. https://doi.org/10.11896/jsjkx.211200105
[12] 邹芸竹, 杜圣东, 滕飞, 李天瑞.
一种基于多模态深度特征融合的视觉问答模型
Visual Question Answering Model Based on Multi-modal Deep Feature Fusion
计算机科学, 2023, 50(2): 123-129. https://doi.org/10.11896/jsjkx.211200303
[13] 王鹏宇, 台文鑫, 刘芳, 钟婷, 罗绪成, 周帆.
基于数据增强的自监督飞行航迹预测
Self-supervised Flight Trajectory Prediction Based on Data Augmentation
计算机科学, 2023, 50(2): 130-137. https://doi.org/10.11896/jsjkx.211200016
[14] 郭楠, 李婧源, 任曦.
基于深度学习的刚体位姿估计方法综述
Survey of Rigid Object Pose Estimation Algorithms Based on Deep Learning
计算机科学, 2023, 50(2): 178-189. https://doi.org/10.11896/jsjkx.211200164
[15] 李俊林, 欧阳智, 杜逆索.
基于改进区域候选网络的场景文本检测
Scene Text Detection with Improved Region Proposal Network
计算机科学, 2023, 50(2): 201-208. https://doi.org/10.11896/jsjkx.211000191
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!