Computer Science ›› 2024, Vol. 51 ›› Issue (1): 233-242.doi: 10.11896/jsjkx.230500035

• Computer Graphics & Multimedia • Previous Articles     Next Articles

FeaEM:Feature Enhancement-based Method for Weakly Supervised Salient Object Detection via Multiple Pseudo Labels

SHI Dianxi1,2, LIU Yangyang1,3, SONG Linna1,3, TAN Jiefu1, ZHOU Chenlei1, ZHANG Yi2   

  1. 1 Tianjin Artificial Intelligence Innovation Center,Tianjin 300450,China
    2 Intelligent Game and Decision Lab(IGDL),Beijing 100091,China
    3 College of Computer,National University of Defense Technology,Changsha 410073,China
  • Received:2023-05-08 Revised:2023-10-10 Online:2024-01-15 Published:2024-01-12
  • About author:SHI Dianxi,born in 1966,Ph.D,professor,Ph.D supervisor.His main research interests include artificial intelligence,robot operating system,distributed computing, and cloud computing.
    ZHANG Yi,born in 1987,Ph.D.His main research interests include AI security and information security.
  • Supported by:
    Science and Technology Commission of Tianjin Binhai New Area(BHXQKJXM-PT-RGZNJMZX-2019001) and National Natural Science Foundation of China(91948303).

Abstract: Salient object detection is designed to detect the most obvious areas of an image.The traditional method based on single label is inevitably affected by the refinement algorithm and shows bias characteristics,which further affects the detection perfor-mance of saliency network.To solve this problem,based on the structure of multi-instruction filter,this paper proposes a feature enhancement-based method for weakly supervised salient object detection via multiple pseudo labels(FeaEM),which integrates more comprehensive and accurate saliency cues from multiple labels to effectively improve the performance of object detection.The core of FeaEM method is to introduce a new multi-instruction filter structure and use multiple pseudo-labels to avoid the negative effects of a single label.By introducing the feature selection mechanism into the instruction filter,more accurate significance clues are extracted and filtered from the noise false label,so as to learn more effective representative features.At the same time,the existing weak supervised object detection methods are very sensitive to the scale of the input image,and the prediction structure of the input of different sizes of the same image has a large deviation.The scale feature fusion mechanism is introduced to ensure that the output of the same image of different sizes is consistent,so as to effectively improve the scale generalization ability of the model.A large number of experiments on multiple data sets show that the FeaEM method proposed in this paper is superior to the most representative methods.

Key words: Deep learning, Object detection, Salient, Pseudo labels, Attention mechanism

CLC Number: 

  • TP391.41
[1]WANG T,ZHANG L,WANG S,et al.Detect globally,refine locally:A novel approach to saliency detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:3127-3135.
[2]WU Z,SU L,HUANG Q.Cascaded partial decoder for fast and accurate salient object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:3907-3916.
[3]ZHANG Q,CONG R,LI C,et al.Dense attention fluid network for salient object detection in optical remote sensing images[J].IEEE Transactions on Image Processing,2020,30:1305-1317.
[4]LIU J J,HOU Q,LIU Z A,et al.Poolnet+:Exploring the potential of pooling for salient object detection[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2022,45(1):887-904.
[5]ZHAO L,LIU G,GUO D,et al.Boosting Few-shot visual recognition via saliency-guided complementary attention[J].Neurocomputing,2022,507:412-427.
[6]WU H,ZHANG L,MA J.Remote sensing image super-resolution via saliency-guided feedback GANs[J].IEEE Transactions on Geoscience and Remote Sensing,2020,60:1-16.
[7]YANG K,ZHANG P,QIAO P,et al.Objectness consistent representation for weakly supervised object detection[C]//Proceedings of the 28th ACM International Conference on Multimedia.2020:1688-1696.
[8]HU Y T,HUANG J B,SCHWING A G.Unsupervised videoobject segmentation using motion saliency-guided spatio-temporal propagation[C]//Proceedings of the European Conference on Computer Vision(ECCV).2018:786-802.
[9]ZHANG W,ZHENG L,WANG H,et al.Saliency HierarchyModeling via Generative Kernels for Salient Object Detection[C]//Computer Vision-ECCV 2022:17th European Confe-rence,Tel Aviv,ISRAEL,Part XXVIII.Cham:Springer Nature Switzerland,2022:570-587.
[10]ZHANG M,LI J,WEI J,et al.Memory-oriented decoder forlight field salient object detection[C]//Neural Information Processing Systems.2019.
[11]ZENG Y,ZHUGE Y Z,LU H C,et al.Multi-source weak supervision for saliency detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:6074-6083.
[12]DENG J,DONG W,SOCHER R,et al.Imagenet:A large-scale hierarchical image database[C]//2009 IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2009:248-255.
[13]LIN T Y,MAIRE M,BELONGIE S,et al.Microsoft coco:Common objects in context[C]//Computer Vision-ECCV 2014:13th European Conference,Zurich,Switzerland,Part V 13.Springer International Publishing,2014:740-755.
[14]CHENG M M,MITRA N J,HUANG X,et al.Global contrast based salient region detection[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2014,37(3):569-582.
[15]LEE G,TAI Y W,KIM J.Deep saliency with encoded low level distance map and high level features[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:660-668.
[16]DENG Z,HU X,LEI Z,et al.R3 Net:Recurrent ResidualRefinement Network for Saliency Detection[C]//International Joint Conference on Artificial Intelligence(IJCAI).2018.
[17]HU X,ZHU L,QIN J,et al.Recurrently aggregating deep features for salient object detection[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2018.
[18]PANG Y,ZHAO X,ZHANG L,et al.Multi-scale interactivenetwork for salient object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:9413-9422.
[19]ZHANG M,LIU T,PIAO Y,et al.Auto-msfnet:Search multi-scale fusion network for salient object detection[C]//Procee-dings of the 29th ACM International Conference on Multimedia.2021:667-676.
[20]ZHANG J,LIANG Q,SHI Y.Kd-scfnet:Towards more accurate and efficient salient object detection via knowledge distillation[J].arXiv:2208.02178,2022.
[21]WANG L,LU H,WANG Y,et al.Learning to detect salient objects with image-level supervision[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:136-145.
[22]LI G B,XIE Y,LIN L.Weakly supervised salient object detection using image labels[C]//Proceedings of the AAAI Confe-rence on Artificial Intelligence.2018.
[23]ZHANG J,YU X,LI A,et al.Weakly-supervised salient object detection via scribble annotations[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:12546-12555.
[24]PIAO Y,WU W,ZHANG M,et al.Noise-sensitive adversarial learning for weakly supervised salient object detection[J].IEEE Transactions on Multimedia,2022,25:2888-2897.
[25]LIN T Y,DOLLÁR P,GIRSHICK R,et al.Feature pyramidnetworks for object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:2117-2125.
[26]LIU S,QI L,QIN H,et al.Path aggregation network for instance segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:8759-8768.
[27]GHIASI G,LIN T Y,LE Q V.Nas-fpn:Learning scalable feature pyramid architecture for object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:7036-7045.
[28]TAN M,PANG R,LE Q V.Efficientdet:Scalable and efficient object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:10781-10790.
[29]CHEN Z,XU Q,CONG R,et al.Global context-aware progressive aggregation network for salient object detection[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2020:10599-10606.
[30]HUANG G,LIU Z,VAN DER MAATEN L,et al.Densely con-nected convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:4700-4708.
[31]ARASLANOV N,ROTH S.Single-stage semantic segmentationfrom image labels[C]//Proceedings of the IEEE/CVF Confe-rence on Computer Vision and Pattern Recognition.2020:4253-4262.
[32]ACHANTA R,SHAJI A,SMITH K,et al.SLIC superpixelscompared to state-of-the-art superpixel methods[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2012,34(11):2274-2282.
[33]ZHOU B,KHOSLA A,LAPEDRIZA A,et al.Learning deep features for discriminative localization[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:2921-2929.
[34]KRÄHENBÜHL P,KOLTUN V.Efficient inference in fullyconnected crfs with gaussian edge potentials[J].arXiv:1210.5644,2011.
[35]LI G,YU Y.Visual saliency based on multiscale deep features[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2015:5455-5463.
[36]YANG C,ZHANG L,LU H,et al.Saliency detection via graph-based manifold ranking[C]//Proceedings of the IEEE Confe-rence on Computer Vision and Pattern Recognition.2013:3166-3173.
[37]LI Y,HOU X,KOCH C,et al.The secrets of salient object segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2014:280-287.
[38]FAN D P,CHENG M M,LIU Y,et al.Structure-measure:Anew way to evaluate foreground maps[C]//Proceedings of the IEEE International Conference on Computer Vision.2017:4548-4557.
[39]ACHANTA R,HEMAMI S,ESTRADA F,et al.Frequency-tuned salient region detection[C]//2009 IEEE conference on Computer Vision and Pattern Recognition.IEEE,2009:1597-1604.
[40]FAN D P,GONG C,CAO Y,et al.Enhanced-alignment measure for binary foreground map evaluation[C]//Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligenc.2018:698-704.
[41]PIAO Y,WANG J,ZHANG M,et al.Mfnet:Multi-filter directive network for weakly supervised salient object detection[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2021:4136-4145.
[1] XU Jinlong, GUI Zhonghua, LI Jia'nan, LI Yingying, HAN Lin. FP8 Quantization and Inference Memory Optimization Based on MLIR [J]. Computer Science, 2024, 51(9): 112-120.
[2] WANG Jiahui, PENG Guangling, DUAN Liang, YUAN Guowu, YUE Kun. Few-shot Shadow Removal Method for Text Recognition [J]. Computer Science, 2024, 51(9): 147-154.
[3] LI Yunchen, ZHANG Rui, WANG Jiabao, LI Yang, WANG Ziqi, CHEN Yao. Re-parameterization Enhanced Dual-modal Realtime Object Detection Model [J]. Computer Science, 2024, 51(9): 162-172.
[4] HU Pengfei, WANG Youguo, ZHAI Qiqing, YAN Jun, BAI Quan. Night Vehicle Detection Algorithm Based on YOLOv5s and Bistable Stochastic Resonance [J]. Computer Science, 2024, 51(9): 173-181.
[5] LIU Qian, BAI Zhihao, CHENG Chunling, GUI Yaocheng. Image-Text Sentiment Classification Model Based on Multi-scale Cross-modal Feature Fusion [J]. Computer Science, 2024, 51(9): 258-264.
[6] LI Zhe, LIU Yiyang, WANG Ke, YANG Jie, LI Yafei, XU Mingliang. Real-time Prediction Model of Carrier Aircraft Landing Trajectory Based on Stagewise Autoencoders and Attention Mechanism [J]. Computer Science, 2024, 51(9): 273-282.
[7] LIU Qilong, LI Bicheng, HUANG Zhiyong. CCSD:Topic-oriented Sarcasm Detection [J]. Computer Science, 2024, 51(9): 310-318.
[8] YAO Yao, YANG Jibin, ZHANG Xiongwei, LI Yihao, SONG Gongkunkun. CLU-Net Speech Enhancement Network for Radio Communication [J]. Computer Science, 2024, 51(9): 338-345.
[9] DU Yu, YU Zishu, PENG Xiaohui, XU Zhiwei. Padding Load:Load Reducing Cluster Resource Waste and Deep Learning Training Costs [J]. Computer Science, 2024, 51(9): 71-79.
[10] ZHANG Lu, DUAN Youxiang, LIU Juan, LU Yuxi. Chinese Geological Entity Relation Extraction Based on RoBERTa and Weighted Graph Convolutional Networks [J]. Computer Science, 2024, 51(8): 297-303.
[11] CHEN Shanshan, YAO Subin. Study on Recommendation Algorithms Based on Knowledge Graph and Neighbor PerceptionAttention Mechanism [J]. Computer Science, 2024, 51(8): 313-323.
[12] CHEN Siyu, MA Hailong, ZHANG Jianhui. Encrypted Traffic Classification of CNN and BiGRU Based on Self-attention [J]. Computer Science, 2024, 51(8): 396-402.
[13] SUN Yumo, LI Xinhang, ZHAO Wenjie, ZHU Li, LIANG Ya’nan. Driving Towards Intelligent Future:The Application of Deep Learning in Rail Transit Innovation [J]. Computer Science, 2024, 51(8): 1-10.
[14] KONG Lingchao, LIU Guozhu. Review of Outlier Detection Algorithms [J]. Computer Science, 2024, 51(8): 20-33.
[15] LIU Sichun, WANG Xiaoping, PEI Xilong, LUO Hangyu. Scene Segmentation Model Based on Dual Learning [J]. Computer Science, 2024, 51(8): 133-142.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!