Computer Science ›› 2023, Vol. 50 ›› Issue (2): 267-274.doi: 10.11896/jsjkx.220900212

• Artificial Intelligence • Previous Articles     Next Articles

Incremental Object Detection Inspired by Memory Mechanisms in Brain

SHANG Di, LYU Yanfeng, QIAO Hong   

  1. State Key Laboratory of Multimodal Artificial Intelligence System,Institute of Automation,Chinese Academy of Sciences,Beijing 100190,China
    School of Artificial Intelligence,University of Chinese Academy of Sciences,Beijing 100049,China
    State Key Laboratory of Complex System Management and Control,Institute of Automation,Chinese Academy of Sciences,Beijing 100190,China
  • Received:2022-09-22 Revised:2022-10-28 Online:2023-02-15 Published:2023-02-22
  • Supported by:
    Natural Science Foundation of Beijing(L211023),National Key Research and Development Plan of China(2020AAA0105900) and National Natural Science Foundation of China(91948303)

Abstract: Incremental learning is key to bridging the enormous gap between artificial intelligence and human intelligence,mea-ning that agents can learn several tasks sequentially from a continuous stream of correlated data without forgetting,just as humans do.Object detection is one of the core tasks in the field of computer vision and the cornerstone of computer images understanding.Therefore,the incremental object detection has important research and practical significance.Although incremental learning has achieved good results in image classification,the research on incremental learning based on object detection is still in its infancy.This is because object detection is more complex than image classification,which needs to solve both classification and bounding box regression problems.Many researchers have made great efforts to solve this problem,but most of the work only focuses on how to retain previous learning,ignoring fast adaptability to new tasks,which is a critical requirement for incremental learning.Based on the memory mechanism of the brain,humans can constantly extract knowledge during learning,so as to learn new tasks better and faster without forgetting.Inspired by this,an incremental meta-learning method that integrates the codec memory replay mechanism is proposed.This method encodes,stores,decodes and replays the feature vectors of learned samples,so as to approximate the dynamic learning environment as a local stationary environment and avoid catastrophic forgetting.Besides,a double-loop online meta-learning strategy is designed,which can help model to extract common structures of tasks and improve generalization performance on new tasks encountered during learning.The model is respectively updated by SGD with multiple batches of old and new mixed data in the inner loop,and is meta-updated in the outer loop.We evaluate the proposed approach on three incremental object detection settings defined on PASCAL VOC and MS COCO datasets,where the proposed algorithm performs favorably well against state-of-the-art methods.It proves that it can help the model to resist forgetting better and have better generalization performance on new tasks.The proposed algorithm is gradient-based and model-agnostic,so it has strongadaptability and can be applied on more complex detection frameworks.

Key words: Incremental learning, Object detection, Brain inspiration, Meta-learning, Resistance to forgetting, Generalization

CLC Number: 

  • TP391
[1]MICHAEL M,NEAL J C.Catastrophic interference in connectionist networks:The sequential learning problem[M]//Psychology of Learning and Motivation.Academic Press,1989:109-165.
[2]RAIA H,DUSHYANT R,ANDREI A R,et al.Embracingchange:Continual learning in deep neural networks [J].Trends in Cognitive Sciences,2020,24(12):1028-1040.
[3]KONSTANTIN S,CORDELIA S,KARTEEK A.Incrementallearning of object detectors without catastrophic forgetting[C]// IEEE International Conference on Computer Vision.2017:3420-3429.
[4]HAO Y,FU Y W,JIANG Y G,et al.An end-to end architecture for class-incremental object detection with knowledge distillation[C]// IEEE International Conference on Multimedia and Expo.2019:1-6.
[5]CHEN L,YU C Y,CHEN L C.A new knowledge distillation for incremental object detection[C]// International Joint Confe-rence on Neural Networks.2019:1-7.
[6]MANO A,TYLER L H,CHRISTOPHER K.RODEO:replayfor online object detection[C]// British Machine Vision Confe-rence.2020.
[7]ALAN B.Working memory [J].Science,1992,255:556-559.
[8]ALAN B.The episodic buffer:a new component of workingmemory? [J].Trends in Cognitive Sciences,2000,4(11):417-423.
[9]PATRICIA S,GOLDMAN R.Regional and cellular fractiona-tion of working memory [J].Proceedings of the National Aca-demy of Sciences of the United States of America,1996,93(24):13473-13480.
[10]CHARLE H,SARAH M,GORDON D B.Memory for familiar and unfamiliar words:Evidence for a long-term memory contribution to short-term memory span [J].Journal of Memory and Language,1991,30(6):685-701.
[11]PHILIP G,VINCENT F C,SAMUEL S,et al.The long and the short of long-term memory-a molecular framework [J].Nature,1986,322(6078):419-422.
[12]ROSS G.Fast r-cnn[C]// IEEE International Conference on Computer Vision.2015:1440-1448.
[13]MARK E,AE S M,LUC V G,et al.The pascal visual object classes challenge:A retrospective [J].International Journal of Computer Vision,2015,111(1):98-136.
[14]TSUN Y L,MICHAEL M,SERGE B,et al.Microsoft COCO:Common Objects in Context[C]// European Conference on Computer Vision.Berlin:Springer,2014:740-755.
[15]DAVUD L,MARC A R.Gradient episodic memory for continual learning[C]// Advances in Neural Information Processing Systems.Curran Associates,Inc,2017:6470-6479.
[16]SYLVESTRE A R,ALEXANDER K,GEORG S,et al.icarl:Incremental classifier and representation learning[C]//Procee-dings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:5533-5542.
[17]DUSHYANT R,FRANCESCO V,ANDREI A R,et al.Conti-nual unsupervised representation learning[C]//Advances in Neural Information Processing Systems.Curran Associates,Inc,2019:7645-7655.
[18]ARSLAN C,ALBERT G,PUNEET K D,et al.Using hindsight to anchor past knowledge in continual learning[C]// Procee-dings of the AAAI Conference on Artificial Intelligence.2021:6993-7001.
[19]JAMES K,RAZYAN P,NEIL R,et al.Overcoming catastrophicforgetting in neural networks [J].Proceedings of the National Academy of Sciences of the United States of America,2017,114(13):3521-3526.
[20]PRAVENDRA S,VINARY K V,PRATIK M,et al.Calibrating cnns for lifelong learning[C]// Advances in Neural Information Processing Systems.Curran Associates,Inc,2020:15579-15590.
[21]JAEHONG Y,SAEHOON K,EUNHO Y,et al.Scalable andorder-robust continual learning with additive parameter decomposition[C]// International Conference on Learning Representations.2020.
[22]EKIN D C,BARRET Z,DANDELION M,et al.AutoAugment:Learning Augmentation Policies from Data[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:113-123.
[23]PENG C,ZHAO K,BRIAN C L.Faster ilod:Incremental lear-ning for object detectors based on faster rcnn [J].PatternRe-cognition Letters,2020,140:109-115.
[24]JOSEPH K J J,SALMAN K,FAHAD S K,et al.Towards open world object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:5830-5840.
[25]ZHANG J T,ZHANG T,SHALINI G,et al.Class-incremental learning via deep model consolidation [C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision.2020.
[26]JOSEPH K,JATHUSHAN R,SALMAN K,et al.Incremental object detection via meta-learning [J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2021,44(22):9209-9216.
[27]CHELSEA F,PIETER A,SERGEY L.Model agnosticmeta-learning for fast adaptation of deep networks[C]//Proceedings of the 34rd International Conference on Machine Learning.2017:1126-1135.
[28]ALEX N,JOSHUA A,JOHN S.On first-order meta-learning algorithms [J].arXiv:1803.02999,2018.
[29]ANDREI A R,DUSHYANT R,JAKUB S,et al.Meta-learning with latent embedding optimization[C]//International Confe-rence on Learning Representations.2019.
[30]ADAM S,SERGEY B,MATTHEW B,et al.Meta-learning with memory-augmented neural networks[C]//Proceedings of The 33rd International Conference on Machine Learning.2016:20-22.
[31]MISHRA N,ROHANINEJAD M,CHEN X,et al.Meta-lear-ning with temporal convolutions [J].arXiv:1707.03141,2017.
[32]JAKE S,KEVIN S,RICHARD Z.Prototypical networks forfew-shot learning[C]//Advances in Neural Information Processing Systems.Curran Associates,Inc,2017.
[33]FLOOD S,YONGXIN Y,LI Z,et al.Learning to compare:Relation network for few-shot learning[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018.
[34]TIAN Y L,WANG Y,DILIP K,et al.Rethinking few-shotimage classification:A good embedding is all you need?[C]// European Conference on Computer Vision.Berlin:Springer,2020:266-282.
[35]WANG T C,RAO M A,HISHAM C,et al.Learning rich features at high-speed for single-shot object detection[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2019.
[36]NIE J,RAO M A,HISHAM C,et al.Enriched feature guidedrefinement network for object detection[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2019.
[37]PANG Y W,WANG T C,Rao M A,et al.Efficient featurized image pyramid network for single shot detector[C]//Procee-dings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019.
[38]FAHAD S K,XU J L,JOOST V D W,et al.Recognizing actions through action-specific person detection [J].IEEE Transactions on Image Processing,2015,24(11):4422-4432.
[39]JIALE C,HISHAM C,RAO M A,et al.D2det:Towards high quality object detection and instance segmentation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020.
[40]HISHAM C,SUN G L,FAHAD S K,et al.Object counting and instance segmentation with image-level supervision[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019.
[1] ZHANG Weiliang, CHEN Xiuhong. SSD Object Detection Algorithm with Cross-layer Fusion and Receptive Field Amplification [J]. Computer Science, 2023, 50(3): 231-237.
[2] CHEN Liang, WANG Lu, LI Shengchun, LIU Changhong. Study on Visual Dashboard Generation Technology Based on Deep Learning [J]. Computer Science, 2023, 50(3): 238-245.
[3] HUA Jie, LIU Xueliang, ZHAO Ye. Few-shot Object Detection Based on Feature Fusion [J]. Computer Science, 2023, 50(2): 209-213.
[4] RONG Huan, QIAN Minfeng, MA Tinghuai, SUN Shengjie. Novel Class Reasoning Model Towards Covered Area in Given Image Based on InformedKnowledge Graph Reasoning and Multi-agent Collaboration [J]. Computer Science, 2023, 50(1): 243-252.
[5] CAI Xiao, CEHN Zhihua, SHENG Bin. SPT:Swin Pyramid Transformer for Object Detection of Remote Sensing [J]. Computer Science, 2023, 50(1): 105-113.
[6] LIU Dong-mei, XU Yang, WU Ze-bin, LIU Qian, SONG Bin, WEI Zhi-hui. Incremental Object Detection Method Based on Border Distance Measurement [J]. Computer Science, 2022, 49(8): 136-142.
[7] WANG Can, LIU Yong-jian, XIE Qing, MA Yan-chun. Anchor Free Object Detection Algorithm Based on Soft Label and Sample Weight Optimization [J]. Computer Science, 2022, 49(8): 157-164.
[8] QI Xiu-xiu, WANG Jia-hao, LI Wen-xiong, ZHOU Fan. Fusion Algorithm for Matrix Completion Prediction Based on Probabilistic Meta-learning [J]. Computer Science, 2022, 49(7): 18-24.
[9] CHEN Yong-ping, ZHU Jian-qing, XIE Yi, WU Han-xiao, ZENG Huan-qiang. Real-time Helmet Detection Algorithm Based on Circumcircle Radius Difference Loss [J]. Computer Science, 2022, 49(6A): 424-428.
[10] CHEN Jia-zhou, ZHAO Yi-bo, XU Yang-hui, MA Ji, JIN Ling-feng, QIN Xu-jia. Small Object Detection in 3D Urban Scenes [J]. Computer Science, 2022, 49(6): 238-244.
[11] HU Fu-yuan, WAN Xin-jun, SHEN Ming-fei, XU Jiang-lang, YAO Rui, TAO Zhong-ben. Survey Progress on Image Instance Segmentation Methods of Deep Convolutional Neural Network [J]. Computer Science, 2022, 49(5): 10-24.
[12] SHEN Shao-peng, MA Hong-jiang, ZHANG Zhi-heng, ZHOU Xiang-bing, ZHU Chun-man, WEN Zuo-cheng. Three-way Drift Detection for State Transition Pattern on Multivariate Time Series [J]. Computer Science, 2022, 49(4): 144-151.
[13] XU Tao, CHEN Yi-ren, LYU Zong-lei. Study on Reflective Vest Detection for Apron Workers Based on Improved YOLOv3 Algorithm [J]. Computer Science, 2022, 49(4): 239-246.
[14] ZHOU Ying, CHANG Ming-xin, YE Hong, ZHANG Yan. Super Resolution Reconstruction Method of Solar Panel Defect Images Based on Meta-transfer [J]. Computer Science, 2022, 49(3): 185-191.
[15] DONG Lin, HUANG Li-qing, YE Feng, HUANG Tian-qiang, WENG Bin, XU Chao. Survey on Generalization Methods of Face Forgery Detection [J]. Computer Science, 2022, 49(2): 12-30.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!