计算机科学 ›› 2023, Vol. 50 ›› Issue (2): 209-213.doi: 10.11896/jsjkx.220500153

• 计算机图形学&多媒体 • 上一篇    下一篇


华杰, 刘学亮, 赵烨   

  1. 合肥工业大学计算机与信息学院 合肥 230601
  • 收稿日期:2022-05-17 修回日期:2022-11-08 出版日期:2023-02-15 发布日期:2023-02-22
  • 通讯作者: 刘学亮(Liuxueliang1982@gmail.com)
  • 作者简介:(jiehua@mail.hfut.edu.cn)
  • 基金资助:

Few-shot Object Detection Based on Feature Fusion

HUA Jie, LIU Xueliang, ZHAO Ye   

  1. School of Computer Science and Information Engineering,Hefei University of Technology,Hefei 230601,China
  • Received:2022-05-17 Revised:2022-11-08 Online:2023-02-15 Published:2023-02-22
  • Supported by:
    National Key R & D Program of China(2018AAA0102002) and National Natural Science Foundation of China(61976076,61632007)

摘要: 小样本目标检测旨在通过少量的样本学习来训练目标检测模型,现有的小样本目标检测方法大多基于经典的目标检测算法。在二阶段的检测方法中,由于新类别样本数量少,产生了许多无关的边界框,导致候选区域的准确率较低。为了解决这个问题,提出了一种基于特征融合的小样本目标检测算法FF-FSOD。该方法采用特征融合的方法进行数据增强,对新类别样本进行补充,扩大样本的覆盖范围,同时引入FPN网络进行多尺度特征提取,再对RPN网络进行改进,引入支持集图像分支,计算支持集图像特征与查询集图像特征的深度互相关性,得到注意力特征图,进而获得更精确的候选框。所提模型的有效性在MS COCO和FSOD数据集上得到了验证,实验结果表明,该方法获得了更精准的候选框,进而提升了检测精度。

关键词: 小样本学习, 目标检测, 深度学习, 特征融合, 特征金字塔

Abstract: Few-shot object detection aims to train target detection model through a small amount of sample learning.At present,most of the existing few-shot object detection methods are based on classical target detection algorithms.In the two-stage detection method,due to the small number of new class samples,many irrelevant border boxes are generated,resulting in low accuracy of candidate regions.To solve this problem,this paper proposes a few-shot object detection algorithm FF-FSOD based on feature fusion.It uses the feature fusion method to enhance the data,supplements the new category samples,increases the coverage range of the sample,and introduces the FPN network to extract multi-scale feature.Then,the RPN network is improved,and the support set image branch is introduced.The depth correlation between the support set image feature and the query set image feature is calculated,and the attention feature map is obtained,and the more accurate candidate box is obtained.The effectiveness of the proposed model is verified on MS COCO and FSOD datasets.Experimental results show that the proposed method obtains more accurate candidate boxes and improves the detection accuracy.

Key words: Few-shot learning, Object detection, Deep learning, Feature fusion, Feature pyramid


  • TP391
[1]CAO J,QU X,LI X X.Few-shot image classification methodbased on sliding feature vectors[J].Journal of Jilin University(Engineering and Technology Edition),2021,51(5):1785-1791.
[2]CHEN H,WANG Y,WANG G,et al.LSTD:a low-shot transfer detector for object detection[C]//Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence and Thirtieth Innovative Applications of Artificial Intelligence Conference and Eighth AAAI Symposium on Educational Advances in Artificial Intelligence.2018:2836-2843.
[3]DONAHUE J,JIA Y,VINYALS O,et al.Decaf:A deep convolutional activation feature for generic visual recognition[C]//International Conference on Machine Learning.PMLR,2014:647-655.
[4]WU J,LIU S,HUANG D,et al.Multi-scale positive sample refinement for few-shot object detection[C]//European Confe-rence on Computer Vision.Cham:Springer,2020:456-472.
[5]YAN X,CHEN Z,XU A,et al.Meta r-cnn:Towards general solver for instance-level low-shot learning[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2019:9577-9586.
[6]XIAO Z,ZHONG P,QUAN Y,et al.Few-shot object detection with feature attention highlight module in remote sensing images[C]//2020 International Conference on Image,Video Processing and Artificial Intelligence.SPIE,2020:217-223.
[7]FAN Q,ZHUO W,TANG C K,et al.Few-shot object detection with attention-RPN and multi-relation detector[C]//Procee-dings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:4013-4022.
[8]WANG X,HUANG T,GONZALEZ J,et al.Frustratingly simple few-shot object detection[C]//International Conference on Machine Learning.PMLR,2020:9919-9928.
[9]LI X,DENG J,FANG Y.Few-shot object detection on remote sensing images[J].IEEE Transactions on Geoscience and Remote Sensing,2021,60:1-14.
[10]KANG B,LIU Z,WANG X,et al.Few-shot object detection via feature reweighting[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2019:8420-8429.
[11]ZHANG W,WANG Y X.Hallucination improves few-shot object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:13008-13017.
[12]RIOU K,ZHU J,LING S,et al.Few-Shot Object Detection in Real Life:Case Study on Auto-Harvest[C]//2020 IEEE 22nd International Workshop on Multimedia Signal Processing(MMSP).IEEE,2020:1-6.
[13]ZHU C,CHEN F,AHMED U,et al.Semantic relation reasoning for shot-stable few-shot object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:8782-8791.
[14]RAHMAN S,KHAN S,BARNES N,et al.Any-shot object detection[C]//Proceedings of the Asian Conference on Computer Vision.2020.
[15]CHEN Z,FU Y,ZHANG Y,et al.Semantic feature augmentation in few-shot learning[J].arXiv:1804.05298,2018.
[16]KARLINSKY L,SHTOK J,HARARY S,et al.Repmet:Representative-based metric learning for classification and few-shot object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:5197-5206.
[17]ZHANG T,ZHANG Y,SUN X,et al.Comparison network for one-shot conditional object detection[J].arXiv:1904.02317,2019.
[18]XU P B,SANG J T,LU D Y.Few shot image recognition based on class semantic similarity supervision[J].Journal of Image and Graphics,2021,26(7):1594-1603.
[19]XIAO Y,MARLET R.Few-shot object detection and viewpoint estimation for objects in the wild[C]//European Conference on Computer Vision.Cham:Springer,2020:192-210.
[20]SUN B,LI B,CAI S,et al.Fsce:Few-shot object detection via contrastive proposal encod-ing[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:7352-7362.
[21]DONG X,ZHENG L,MA F,et al.Few-example object detection with model communication[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2018,41(7):1641-1654.
[22]CHEN X,JIANG M,ZHAO Q.Leveraging bottom-up and top-down attention for few-shot object detection[J].arXiv:2007.12104,2020.
[23]REN S,HE K,GIRSHICK R,et al.Faster R-CNN:Towardsreal-time object detection with region proposal networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2016,39(6):1137-1149.
[24]GIRSHICK R.Fast r-cnn[C]//Proceedings of the IEEE International Conference on Computer Vision.2015:1440-1448.
[25]HARRIS E,MARCU A,PAINTER M,et al.Fmix:Enhancing mixed sample data augmentation[J].arXiv:2002.12047,2020.
[26]LIN T Y,DOLLÁR P,GIRSHICK R,et al.Feature pyramidnetworks for object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:2117-2125.
[27]TSUNGYL,PRIYA G,ROSS G,et al.Focal loss for dense object detection[C]//ICCV.2017.
[28]REDMON J,FARHADI A.Yolov3:An incremental improvement[J].arXiv:1804.02767,2018.
[29]WU A,HAN Y,ZHU L,et al.Universal-prototype enhancing for few-shot object detection[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2021:9567-9576.
[1] 董永峰, 黄港, 薛婉若, 李林昊.
Graph Attention Deep Knowledge Tracing Model Integrated with IRT
计算机科学, 2023, 50(3): 173-180. https://doi.org/10.11896/jsjkx.211200134
[2] 华晓凤, 冯娜, 于俊清, 何云峰.
Shooting Event Detection of Free Kick in Soccer Video Based on Rule Reasoning
计算机科学, 2023, 50(3): 181-190. https://doi.org/10.11896/jsjkx.220300062
[3] 梅鹏程, 杨吉斌, 张强, 黄翔.
Sound Event Joint Estimation Method Based on Three-dimension Convolution
计算机科学, 2023, 50(3): 191-198. https://doi.org/10.11896/jsjkx.220500259
[4] 白雪飞, 马亚楠, 王文剑.
Segmentation Method of Edge-guided Breast Ultrasound Images Based on Feature Fusion
计算机科学, 2023, 50(3): 199-207. https://doi.org/10.11896/jsjkx.211200294
[5] 刘航, 普园媛, 吕大华, 赵征鹏, 徐丹, 钱文华.
Polarized Self-attention Constrains Color Overflow in Automatic Coloring of Image
计算机科学, 2023, 50(3): 208-215. https://doi.org/10.11896/jsjkx.220100149
[6] 谢秦秦, 何朗, 徐汝利.
Classification of Oil Painting Art Style Based on Multi-feature Fusion
计算机科学, 2023, 50(3): 223-230. https://doi.org/10.11896/jsjkx.211200110
[7] 张卫良, 陈秀宏.
SSD Object Detection Algorithm with Cross-layer Fusion and Receptive Field Amplification
计算机科学, 2023, 50(3): 231-237. https://doi.org/10.11896/jsjkx.211100281
[8] 陈亮, 王璐, 李生春, 刘昌宏.
Study on Visual Dashboard Generation Technology Based on Deep Learning
计算机科学, 2023, 50(3): 238-245. https://doi.org/10.11896/jsjkx.230100064
[9] 张译, 吴秦.
Crowd Counting Network Based on Feature Enhancement Loss and Foreground Attention
计算机科学, 2023, 50(3): 246-253. https://doi.org/10.11896/jsjkx.220100219
[10] 应宗浩, 吴槟.
Backdoor Attack on Deep Learning Models:A Survey
计算机科学, 2023, 50(3): 333-350. https://doi.org/10.11896/jsjkx.220600031
[11] 刘泽京, 邬楠, 黄抚群, 宋友.
Hybrid Programming Task Recommendation Model Based on Knowledge Graph and Collaborative Filtering for Online Judge
计算机科学, 2023, 50(2): 106-114. https://doi.org/10.11896/jsjkx.211200105
[12] 邹芸竹, 杜圣东, 滕飞, 李天瑞.
Visual Question Answering Model Based on Multi-modal Deep Feature Fusion
计算机科学, 2023, 50(2): 123-129. https://doi.org/10.11896/jsjkx.211200303
[13] 王鹏宇, 台文鑫, 刘芳, 钟婷, 罗绪成, 周帆.
Self-supervised Flight Trajectory Prediction Based on Data Augmentation
计算机科学, 2023, 50(2): 130-137. https://doi.org/10.11896/jsjkx.211200016
[14] 郭楠, 李婧源, 任曦.
Survey of Rigid Object Pose Estimation Algorithms Based on Deep Learning
计算机科学, 2023, 50(2): 178-189. https://doi.org/10.11896/jsjkx.211200164
[15] 李俊林, 欧阳智, 杜逆索.
Scene Text Detection with Improved Region Proposal Network
计算机科学, 2023, 50(2): 201-208. https://doi.org/10.11896/jsjkx.211000191
Full text



No Suggested Reading articles found!