计算机科学 ›› 2022, Vol. 49 ›› Issue (6): 210-216.doi: 10.11896/jsjkx.210300267

• 计算机图形学&多媒体 • 上一篇    下一篇

基于样本分布损失的图像多标签分类研究

朱旭东, 熊贇   

  1. 复旦大学计算机科学与技术学院 上海 200433
    上海市数据科学重点实验室(复旦大学) 上海 200433
  • 收稿日期:2021-03-26 修回日期:2021-08-15 出版日期:2022-06-15 发布日期:2022-06-08
  • 通讯作者: 熊贇(yunx@fudan.edu.cn)
  • 作者简介:(18212010052@fudan.edu.cn)
  • 基金资助:
    国家自然科学基金(U1636207)

Study on Multi-label Image Classification Based on Sample Distribution Loss

ZHU Xu-dong, XIONG Yun   

  1. School of Computer Science and Technology,Fudan University,Shanghai 200433,China
    Research Center of Dataology and Data Science,Fudan University,Shanghai 200433,China
  • Received:2021-03-26 Revised:2021-08-15 Online:2022-06-15 Published:2022-06-08
  • About author:ZHU Xu-dong,born in 1995,postgra-duate.His main research interests include computer vision and graph neural network.
    XIONG Yun,born in 1980,Ph.D,professor,Ph.D supervisor,is a member of China Computer Federation.Her main research interests include data mining and graph neural network.
  • Supported by:
    National Natural Science Foundation of China(U1636207).

摘要: 与一般图像分类场景下的数据分布情况不同,在图像多标签分类问题的场景下,不同标签类别之间存在样本数量分布不均衡,少量头部类别通常占据大多数样本数量的情况。而由于多个标签间同时标记的相关性,再加上多标签下困难样本的分布还与数据分布和类别分布相关,使得单标签问题中解决数据不平衡的重采样等方法在多标签场景下无法有效适用。文中提出了一种基于图像多标签场景下样本分布损失和深度学习的分类方法。首先对多标签数据不均衡分布设置类别相关重采用损失,并通过动态学习方式防止分布过度异化,然后设计非对称样本学习损失,设置对正负样本和困难样本的不同学习能力,同时通过软化样本学习权重减少信息丢失。相关数据集的实验显示,所提算法在解决多标签数据分布不均衡场景下的样本学习问题时取得了很好的效果。

关键词: 标签关系, 多标签, 深度学习, 图像分类, 重采样

Abstract: Different from the data distribution in general image classification scenarios,in the scenario of multi label image classification,the sample number distribution among different label categories is unbalanced,and a small number of head categories often account for the majority of sample size.However,due to the correlation between multiple labels,and the distribution of diffi-cult samples under multiple labels is also related to the data distribution and category distribution,the re-sampling and other methods for solving the data imbalance in the single label problem cannot be effectively applied in the multi label scenario.This paper proposes a classification method based on the loss of sample distribution in multi label image scene and deep learning.Firs-tly,the unbalanced distribution of multi label data is set with category correlation,and the loss is re-used,and the dynamic lear-ning method is used to prevent the excessive alienation of distribution.Then,the asymmetric sample learning loss is designed,and different learning abilities for positive and negative samples and difficult samples are set.At the same time,the information loss is reduced by softening the sample learning weight.Experiments on related data sets show that the algorithm has achieved good results in solving the sample learning problem in the scene of uneven distribution of multi-label data.

Key words: Deep learning, Image classification, Label relation, Multi-Label, Re-sample

中图分类号: 

  • TP391
[1] WANG J,YI Y,MAO J H,et al.Cnn-rnn:A unified framework for multi-label image classification[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.IEEE Press,2016:2285-2294.
[2] CHEN Z M,WEI X S,WANG P,et al.Multi-Label Image Reco-gnition With Graph Convolutional Networks.[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.IEEE Press,2016:5177-5186.
[3] ZHOU B,CUI Q,WEI X S,et al.Bbn:Bilateral-branch network with cumulative learning for long-tailed visual recognition[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:9719-9728.
[4] WU T,HUANG Q,LIU Z,et al.Distribution-balanced loss for multi-label classification in long-tailed datasets[C]//European Conference on Computer Vision.Cham:Springer,2020:162-178.
[5] LIN T Y,GOYAL P,GIRSHICK R,et al.Focal loss for dense object detection[C]//Proceedings of the IEEE International Conference on Computer Vision.2017:2980-2988.
[6] BEN-BARUCH E,RIDNIK T,ZAMIR N,et al.AsymmetricLoss For Multi-Label Classification[EB/OL].(2020-09-29)[2021-05-18].https://arxiv.org/abs/2009.14119.
[7] GONG Y,JIA Y,LEUNG T,et al.Deep convolutional ranking for multilabel image annotation[EB/OL].(2013-12-17)[2014-04-14].https://arxiv.org/abs/1312.4894.
[8] SHENG L,MA J F,YANG R X.Research on CNN Image Clas-sification Algorithm Based on Feature Exchange[J].Computer Engineering,2016,29(6):927-933.
[9] WANG Z,CHEN T,LI G,et al.Multi-label image recognition by recurrently discovering attentional regions[C]//Proceedings of the IEEE International Conference on Computer Vision.2017:464-472.
[10] WANG M,LUO C,HONG R,et al.Beyond object proposals:Random crop pooling for multi-label image recognition[J].IEEE Transactions on Image Processing,2016,25(12):5678-5688.
[11] WANG Y B,ZHENG W J,CHEN Y S.Multi label classification algorithm based on PLSA learning probability distribution semantic information[J].Journal of Nanjing University(Natural Science),2016,29(6):927-933.
[12] YOU R,GUO Z,CUI L,et al.Cross-modality attention with semantic graph embedding for multi-label classification[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2020,34(7):12709-12716.
[13] CHEN T,XU M,HUI X,et al.Learning semantic-specific graph representation for multi-label image recognition[C]//Procee-dings of the IEEE/CVF International Conference on Computer Vision.2019:522-531.
[14] GU G H,CAO Y Y,LI G.Image Hierarchical Classification Based on Semantic Label Generation and Partial Order Structure[J].Journal of Software,2016,29(6):927-933.
[15] WANG Y,XIE Y,LIU Y,et al.Fast Graph Convolution Network Based Multi-label Image Recognition via Cross-modal Fusion[C]//Proceedings of the 29th ACM International Confe-rence on Information & Knowledge Management.2020:1575-1584.
[16] LIU Z,MIAO Z,ZHAN X,et al.Large-scale long-tailed recognition in an open world[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:2537-2546.
[17] CUI Y,JIA M,LIN T Y,et al.Class-balanced loss based on effe-ctive number of samples[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:9268-9277.
[18] HAYAT M,KHAN S,ZAMIR W,et al.Max-margin class imbalanced learning with gaussian affinity[J].arXiv:1901.07711,2019.
[19] FELZENSZWALB P F,GIRSHICK R B,MCALLESTER D,et al.Object detection with discriminatively trained part-based models[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2009,32(9):1627-1645.
[20] LI B,LIU Y,WANG X.Gradient harmonized single-stage detector[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2019,33(1):8577-8584.
[21] SHRIVASTAVA A,GUPTA A,GIRSHICK R.Training re-gion-based object detectors with online hard example mining[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:761-769.
[22] EVERINGHAM M,VAN GOOL L,WILLIAMS C K I,et al.The pascal visual object classes (voc) challenge[J].InternationalJournal of Computer Vision,2010,88(2):303-338.
[23] LIN T Y,MAIRE M,BELONGIE S,et al.Microsoft coco:Common objects in context[C]//European Conference on Computer Vision.Cham:Springer,2014:740-755.
[24] SHEN L,LIN Z,HUANG Q.Relay backpropagation for effective learning of deep convolutional neural networks[C]//European Conference on Computer Vision.Cham:Springer,2016:467-482.
[1] 饶志双, 贾真, 张凡, 李天瑞.
基于Key-Value关联记忆网络的知识图谱问答方法
Key-Value Relational Memory Networks for Question Answering over Knowledge Graph
计算机科学, 2022, 49(9): 202-207. https://doi.org/10.11896/jsjkx.220300277
[2] 汤凌韬, 王迪, 张鲁飞, 刘盛云.
基于安全多方计算和差分隐私的联邦学习方案
Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy
计算机科学, 2022, 49(9): 297-305. https://doi.org/10.11896/jsjkx.210800108
[3] 徐涌鑫, 赵俊峰, 王亚沙, 谢冰, 杨恺.
时序知识图谱表示学习
Temporal Knowledge Graph Representation Learning
计算机科学, 2022, 49(9): 162-171. https://doi.org/10.11896/jsjkx.220500204
[4] 武红鑫, 韩萌, 陈志强, 张喜龙, 李慕航.
监督和半监督学习下的多标签分类综述
Survey of Multi-label Classification Based on Supervised and Semi-supervised Learning
计算机科学, 2022, 49(8): 12-25. https://doi.org/10.11896/jsjkx.210700111
[5] 王剑, 彭雨琦, 赵宇斐, 杨健.
基于深度学习的社交网络舆情信息抽取方法综述
Survey of Social Network Public Opinion Information Extraction Based on Deep Learning
计算机科学, 2022, 49(8): 279-293. https://doi.org/10.11896/jsjkx.220300099
[6] 郝志荣, 陈龙, 黄嘉成.
面向文本分类的类别区分式通用对抗攻击方法
Class Discriminative Universal Adversarial Attack for Text Classification
计算机科学, 2022, 49(8): 323-329. https://doi.org/10.11896/jsjkx.220200077
[7] 姜梦函, 李邵梅, 郑洪浩, 张建朋.
基于改进位置编码的谣言检测模型
Rumor Detection Model Based on Improved Position Embedding
计算机科学, 2022, 49(8): 330-335. https://doi.org/10.11896/jsjkx.210600046
[8] 孙奇, 吉根林, 张杰.
基于非局部注意力生成对抗网络的视频异常事件检测方法
Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection
计算机科学, 2022, 49(8): 172-177. https://doi.org/10.11896/jsjkx.210600061
[9] 胡艳羽, 赵龙, 董祥军.
一种用于癌症分类的两阶段深度特征选择提取算法
Two-stage Deep Feature Selection Extraction Algorithm for Cancer Classification
计算机科学, 2022, 49(7): 73-78. https://doi.org/10.11896/jsjkx.210500092
[10] 程成, 降爱莲.
基于多路径特征提取的实时语义分割方法
Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction
计算机科学, 2022, 49(7): 120-126. https://doi.org/10.11896/jsjkx.210500157
[11] 侯钰涛, 阿布都克力木·阿布力孜, 哈里旦木·阿布都克里木.
中文预训练模型研究进展
Advances in Chinese Pre-training Models
计算机科学, 2022, 49(7): 148-163. https://doi.org/10.11896/jsjkx.211200018
[12] 周慧, 施皓晨, 屠要峰, 黄圣君.
基于主动采样的深度鲁棒神经网络学习
Robust Deep Neural Network Learning Based on Active Sampling
计算机科学, 2022, 49(7): 164-169. https://doi.org/10.11896/jsjkx.210600044
[13] 苏丹宁, 曹桂涛, 王燕楠, 王宏, 任赫.
小样本雷达辐射源识别的深度学习方法综述
Survey of Deep Learning for Radar Emitter Identification Based on Small Sample
计算机科学, 2022, 49(7): 226-235. https://doi.org/10.11896/jsjkx.210600138
[14] 王君锋, 刘凡, 杨赛, 吕坦悦, 陈峙宇, 许峰.
基于多源迁移学习的大坝裂缝检测
Dam Crack Detection Based on Multi-source Transfer Learning
计算机科学, 2022, 49(6A): 319-324. https://doi.org/10.11896/jsjkx.210500124
[15] 楚玉春, 龚航, 王学芳, 刘培顺.
基于YOLOv4的目标检测知识蒸馏算法研究
Study on Knowledge Distillation of Target Detection Algorithm Based on YOLOv4
计算机科学, 2022, 49(6A): 337-344. https://doi.org/10.11896/jsjkx.210600204
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!