基于深度学习的红外视频显著性目标检测

doi:10.11896/jsjkx.220700204

Abstract

Abstract: In the face of massive infrared video images with more and more complex background,the performance of the tradi-tional methods for salient object detection decreases significantly.In order to improve the performance of salient object detection in infrared images,this paper proposes a deep learning-based salient object detection model for infrared video,which mainly consists of a spatial feature extraction module,a temporal feature extraction module,a residual skip connection module and a pixel-wise classifier.First,the spatial feature extraction module is used to extract spatial saliency features from raw input video frames.Secondly,the temporal feature extraction module is used to obtain temporal saliency features and spatio-temporal coherence mo-deling.Finally,the spatial-temporal feature information and the spatial low-level feature information obtained by connecting the spatial module with the residual skip connection layer are sent into the pixel-wise classifier to generate the final salient object detection results.To improve the stability of the model,BCEloss and DICEloss are combined to train the network.The test is carried out on infrared video dataset OTCBVS and infrared video sequences with complex background.The proposed model can obtain accurate salient object detection results,and has robustness and good generalization ability.

Key words: Infrared video, Salient object detection, Deep learning, Convolutional neural network, Loss function

CLC Number:

TP751

ZHU Ye, HAO Yingguang, WANG Hongyu. Deep Learning Based Salient Object Detection in Infrared Video[J].Computer Science, 2023, 50(9): 227-234.

References

[1]ZHANG B H,JIAO D D,PEI H Q,et al.Infrared moving object detection based on local saliency and sparserepresentation[J].Infrared Physics & Technology,2017,86(12):187-193.
[2]ZHAO J,FENG C,SHAO F Q,et al.Moving object detection and segmentation based on adaptive frame difference and level set [J].Information and Control,2012,41(2):153-158.
[3]LEE M,CHO S,LEE S,et al.Unsupervised Video Object Segmentation via Prototype Memory Network[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision.2023:5924-5934.
[4]AHN D,KIM S,HONG H,et al.STAR-Transformer:A Spatio-temporal Cross Attention Transformer for Human Action Re-cognition[C]//Proceedings of the IEEE/CVF Winter Confe-rence on Applications of Computer Vision.2023:3330-3339.
[5]ZHOU F,KANG S B,COHEN M F.Time-Mapping UsingSpace-Time Saliency[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2014:3358-3365.
[6]HOU X D,ZHANG L Q.Saliency detection:A spectral residual approach[C]//Proceedings of the IEEE Conference on Compu-ter Vision and Pattern Recognition.2007:1-8.
[7]ACHANTA R,HEMAMI S,ESTRADA F,et al.Frequency-tuned salient region detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2009:1597-1604.
[8]CHENG M M,MITRA N J,HUANG X L,et al.Global Contrast Based Salient Region Detection[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2011,37(3):569-572.
[9]RAHTU E,HEIKKILA J.A simple and efficient saliency detector for background subtraction [C]//Proceedings Eighth IEEE International Conferenceon Computer Vision(ICCV 2009).IEEE,2009:1137-1144.
[10]HAN J H,MA Y,ZHOU B,et al.A robust infrared small target detection algorithm based on human visual system[J].IEEE Geo-science and Remote Sensing Letters,2014,11(12):2168-2172.
[11]WANG W,SHEN J,SHAO L.Video salient object detection via fully convolutional networks[J].IEEE Transactions on Image Processing,2018,27(1):38-49.
[12]SIMONYAN K,ZISSERMAN A.Two-Stream ConvolutionalNetworks for Action Recognition in Videos[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems.2014:568-576.
[13]LI H F,CHEN G Q,LI G B,et al.Motion guided attention for video salient object detection[C]//Proceedings Eighth IEEE International Conference on Computer Vision(ICCV 2019).IEEE,2019:7273-7282.
[14]FAN D P,WANG W,CHENG M M,et al.Shifting More Atten-tion to Video Salient Object Detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2019:8546-8556.
[15]LI G B,XIE Y,WEI T H,et al.Flow Guided Recurrent Neural Encoder for Video Salient Object Detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:3243-3252.
[16]HE K M,ZHANG X Y,REN S H,et al.Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:770-778.
[17]WOO S,PARK J,LEE J Y,et al.CBAM:convolutional block at-tention module[C]//Proceedings of the European Conference on Computer Vision(ECCV).2018:3-19.
[18]CHEN L C,PAPANDREOU G,KOKKINOS I,et al.DeepLab:Semantic Image Segmentation with Deep Convolutional Nets,Atrous Convolution,and Fully Connected CRFs[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2018,40(4):834-848.
[19]WANG X,GIRSHICK R,GUPTA A,et al.Non-local neural networks[C]//Proceedings of the IEEE Conference on Compu-ter Vision and Pattern Recognition.2018:7794-7803.
[20]BALLAS N,YAO L,PAL C,et al.Delving deeper into convolutional networks for learning video representations[J].arXiv:2016.06432,2022.
[21]KYUNGHYUN C,BART V,CAGLAR G,et al.Learningphrase representations usingrnn encoder-decoder for statistical machine translation[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing.2014:1724-1734.
[22]HOCHREITER S,SCHMIDHUBER J.Long short-term memory[J].Neural Computation,1997,9(8):1735-1780.
[23]SONG H M,WANG W G,ZHAO S Y,et al.Pyramid dilated deeper convlstm for video salient object detection[C]//Procee-dings of the European Conference on Computer Vision(ECCV).2018:715-731.
[24]BUADES A,COLL B,MOREL M.A non-local algorithm for image denoising[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2005:60-65.
[25]HE K M,ZHANG XY,REN S H,et al.Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:770-780.
[26]FEDERICO P,PONT-TUSET J,MCWILLIAMS B,et al.Abenchmark dataset and evaluation methodology for video object segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:724-732.
[27]LI J,CHEN X.A benchmark dataset and saliency-guidedstacked autoencoders for video based salient object detection[J].IEEE Transactions on Image Processing,2018,27(1):349-364.
[28]PERAZZI F,KRÄHENBÜHL P,PRITCH Y,et al.Saliency filters:Contrast based filtering for salient region detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2012:733-740.
[29]MARGOLIN R,ZELNIK-MANOR L,TAL A.How to evaluate foreground maps[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2014:248-255.
[30]FAN D P,CHENG M M,CHENG G,et al.Enhanced-alignment measure for binary foreground map evaluation[C]//Interna-tional Joint Conferences on Artificial Intelligence.2018:698-704.
[31]FAN D P,CHENG M M,LIU Y,et al.Structure-measure:A new way to evaluate foreground maps[C]//Proceedings Eighth IEEE International Conference on Computer Vision(ICCV 2017).IEEE,2017:4558-4567.
[32]WU Z,SU L,HUANG Q.Cascaded partial decoder for fast and accurate salient object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2019:3902-3911.
[33]YAN P X,LI G B,XIE Y,et al.Semi-supervised video salient object detection using pseudo-labels[C]//Proceedings Eighth IEEE International Conference on Computer Vision(ICCV 2019).IEEE,2019:7283-7292.

Related Articles 15

[1]	ZHAO Mingmin, YANG Qiuhui, HONG Mei, CAI Chuang. Smart Contract Fuzzing Based on Deep Learning and Information Feedback [J]. Computer Science, 2023, 50(9): 117-122.
[2]	LI Haiming, ZHU Zhiheng, LIU Lei, GUO Chenkai. Multi-task Graph-embedding Deep Prediction Model for Mobile App Rating Recommendation [J]. Computer Science, 2023, 50(9): 160-167.
[3]	HUANG Hanqiang, XING Yunbing, SHEN Jianfei, FAN Feiyi. Sign Language Animation Splicing Model Based on LpTransformer Network [J]. Computer Science, 2023, 50(9): 184-191.
[4]	YI Liu, GENG Xinyu, BAI Jing. Hierarchical Multi-label Text Classification Algorithm Based on Parallel Convolutional Network Information Fusion [J]. Computer Science, 2023, 50(9): 278-286.
[5]	WANG Yu, WANG Zuchao, PAN Rui. Survey of DGA Domain Name Detection Based on Character Feature [J]. Computer Science, 2023, 50(8): 251-259.
[6]	ZHANG Yian, YANG Ying, REN Gang, WANG Gang. Study on Multimodal Online Reviews Helpfulness Prediction Based on Attention Mechanism [J]. Computer Science, 2023, 50(8): 37-44.
[7]	SONG Xinyang, YAN Zhiyuan, SUN Muyi, DAI Linlin, LI Qi, SUN Zhenan. Review of Talking Face Generation [J]. Computer Science, 2023, 50(8): 68-78.
[8]	WANG Xu, WU Yanxia, ZHANG Xue, HONG Ruize, LI Guangsheng. Survey of Rotating Object Detection Research in Computer Vision [J]. Computer Science, 2023, 50(8): 79-92.
[9]	ZHOU Ziyi, XIONG Hailing. Image Captioning Optimization Strategy Based on Deep Learning [J]. Computer Science, 2023, 50(8): 99-110.
[10]	ZHANG Xiao, DONG Hongbin. Lightweight Multi-view Stereo Integrating Coarse Cost Volume and Bilateral Grid [J]. Computer Science, 2023, 50(8): 125-132.
[11]	LI Kun, GUO Wei, ZHANG Fan, DU Jiayu, YANG Meiyue. Adversarial Malware Generation Method Based on Genetic Algorithm [J]. Computer Science, 2023, 50(7): 325-331.
[12]	WANG Mingxia, XIONG Yun. Disease Diagnosis Prediction Algorithm Based on Contrastive Learning [J]. Computer Science, 2023, 50(7): 46-52.
[13]	SHEN Zhehui, WANG Kailai, KONG Xiangjie. Exploring Station Spatio-Temporal Mobility Pattern:A Short and Long-term Traffic Prediction Framework [J]. Computer Science, 2023, 50(7): 98-106.
[14]	HUO Weile, JING Tao, REN Shuang. Review of 3D Object Detection for Autonomous Driving [J]. Computer Science, 2023, 50(7): 107-118.
[15]	ZHOU Bo, JIANG Peifeng, DUAN Chang, LUO Yuetong. Study on Single Background Object Detection Oriented Improved-RetinaNet Model and Its Application [J]. Computer Science, 2023, 50(7): 137-142.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 0

No Suggested Reading articles found!

Deep Learning Based Salient Object Detection in Infrared Video

PDF (PC)

Abstract

Cite this article

share this article

References

Related Articles 15

Metrics

Comments

Recommended 0