计算机科学 ›› 2022, Vol. 49 ›› Issue (3): 134-143.doi: 10.11896/jsjkx.210100001
韩洁1, 陈俊芬1, 李艳2, 湛泽聪1
HAN Jie1, CHEN Jun-fen1, LI Yan2, ZHAN Ze-cong1
摘要： 近年来,基于联合训练的深度聚类方法,如DEC(Deep Embedding Clustering)和DDC(Deep Denoising Clustering)算法,使基于特征提取的图像聚类取得了很多新进展,带来了聚类性能的突破,而且特征提取环节对后续聚类任务有直接影响。但是,这些方法的泛化能力较差,在不同数据集使用不同的网络结构,聚类性能相比分类性能仍有很大的提升空间。为此,文中提出了一种基于自注意力的自监督深度聚类方法(Self-attention Based Self-supervised Deep Clustering,SADC)。首先设计一个深度卷积自编码器用于提取特征,并且用带噪声的输入数据训练该网络来增强模型的鲁棒性;其次引入自注意力机制,辅助网络捕获对聚类有用的信息;最后编码器部分结合K-means算法形成一个深度聚类器,用于进行特征表示和聚类分配,通过迭代更新网络参数来提高聚类精度和网络的泛化能力。在6个图像数据集上验证所提聚类算法的性能,并与深度聚类算法DEC,DDC等进行比较。实验结果表明,SADC能提供令人满意的聚类结果,而且聚类性能与DEC和DDC相当。总之,统一的网络结构在保证聚类精度的同时降低了深度聚类算法的复杂度。
中图分类号:
| [1]ASANO Y M,RUPPRECHT C,VEDALDI A.Self-labelling via simultaneous clustering and repre-sentation learning[C]//Proceedings of the International Conference on Learning Representations (ICLR).2020:1-22. [2]WANG C,BAI X,DU J.Diffuse Interface Based Unsupervised Images Clustering Algorithm[J].Computer Science,2020,47(5):149-153. [3]MCCULLOCH W S,PITTS W H.A logical calculus of the ideas immanent in nervous activity[J].The Bulletin of Mathematical Biophysics,1988,5:115-133. [4]LECUN Y,BOTTOU L.Gradient-based learning applied to do-cument recognition[J].Proceedings of the IEEE,1998,86(11):2278-2324. [5]ZHAO H,JIA J,KOLTN V.Exploring self-attention for image recognition[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:10076-10085. [6]SHANG L,LU Z,LI H.Neural Responding Machine for Short-Text Conversation[C]//Proceedings of the Meeting of the Association for Computational Linguistics.2015. [7]VASWANI A,SHAZEER N,PARMAR N,et al.Attention Is All You Need[C]//Proceedings of the Advances in Neural Information Processing Systems(NeuIPS).2017:5998-6008. [8]XU K,BA J,KIROS R,et al.Show,Attend and Tell:NeuralImage Caption Generation with Visual Attention[C]//Procee-dings of the International Conference on Machine Learning(ICML).2015:2048-2057. [9]HUANG P,HUANG Y,WANG W,et al.Deep embedding network for clustering[C]//Proceedings of the IEEE International Conference on Pattern Recognition(ICPR).2014:1532-1537. [10]YANG B,FU X.Towards k-means-friendly spaces:Simul-taneous deep earning and clustering[C]//Proceedings of the International Conference on Machine Learning.2017:3861-3870. [11]XIE J,GIRSHICK R,FARHADI A.Unsupervised Deep Em-bedding for Clustering Analysis[C]//Proceedings of the International Conference on Machine Learning(ICML).2016:478-487. [12]LI F,QIAO H,ZHANG B.Discriminatively Boosted ImageClustering with Fully Convolutional Auto-Encoders[J].Pattern Recognition,2018,83:161-173. [13]CHEN J,ZHANG M,ZHAO J.A Deep Clustering AlgorithmBased on Denoising and Self-attention[J].Computer Science and Techno-logy,2020,15(9):1117-1727. [14]XIE J,HOU Q,CAO J.Image Clustering Algorithms by Deep Convolutional Autoencoders[J].Computer Science and Techno-logy,2019,13(4):586-595. [15]LI B, PI D, CUI L,et al.DNC:A Deep Neural Network-based Clustering-oriented Network Embedding Algorithm[J].Journal of Network and Computer Applications,2020,173:102854. [16]DING Y,WEI H,PAN Z,et al.Survey of Net-work Representation Learning[J].Computer Science,2020,47(9):52-59. [17]HE K,SUN J.Convolutional neural networks at constrainedtime cost[C]//Proceedings of the IEEE Conference on Compu-ter Vision and Pattern Recognition.2015:5353-5360. [18]HU W,MIYATO T,TOKUI S,et al.Learning Discrete Representations via Information Maximizing Self-Augmented Training[C]//Proceedings of the 34th International Conference on Machine Learning (ICML).2017:1-15. [19]BALDI P.Autoencoders,Unsupervised learning and deep architectures[C]//Proceedings of the International Conference on Unsupervised and Transfer Learning Workshop.2011:37-50. [20]VICENT P,LAROCHELLE H,LAJOIE I,et al.Stacked De-noising Autoencoders:Learning Useful Representations in a Deep Network with a Local Denoising Criterion[J].Journal of Machine Learning Research,2010,11(12):3371-3408. [21]DIZAJI K G,HERANDI A,HUANG H. Deep clustering viajoint convolutional autoencoder embedding and relative entropy minimization[C]//Proceedings of the IEEE International Conference on Computer Vision.2017:5736-5745. [22]LIN Z,FENG M,SANTOS C N D,et al.A Structured Self-attentive Sentence Embedding[C]//Proceedings of the International Conference on Learning Representations.2017:1-15. [23]WANG X,GIRSHICK R,GUPTA A,et al.Non-local Neuralnetworks[C]//Proceedings of the IEEE Conference on Compu-ter Vision and Pattern Recognition.2018:7794-7803. [24]ZHANG H,GOODFELLOW I,METAXAS D,et al.Self-Attention Generative Adversarial Networks[C]//International Conference on Machine Learning.PMLR,2019:7354-7363. [25]BOTTOU L,CURTIS F E,NOCEDAL J.Optimization methods for large-scale machine learning[J].Siam Review,2018,60(2):223-311. [26]RUDER S.An overview of gradient descent optimization algo-rithms[J].arXiv:1609.04747,2016. [27]RUMELHARTT D,HINTON G,WILLIAMS R.Learning rep-resentations by back-propagating errors[J].Nature,1986,323(6088):533-536. [28]LECUN Y,BOTTOU L.Gradient-based learning applied to do-cument recognition[J].Proceedings of the IEEE,1998,86(11):2278-2324. [29]NAIR V,HINTON G.Rectified linear units i-mprove restricted boltzmann machines[C]//Proceedings of the 27th International Conference on Machine Learning.Madison:Omni press,2010:807-814. [30]KINGMA D,BA L.Adam:A method for sto-chastic optimiza-tion[C]//Proceedings of the 3rd International Conference on Learning Representations.2015:1-15. [31]LAURENS V,HINTON G.Visualizing Data using t-SNE[J].Journal of Machine Learning Research,2008,9(2605):2579-2605. |
| [1] | 吴子仪, 李邵梅, 姜梦函, 张建朋. 基于自注意力模型的本体对齐方法 Ontology Alignment Method Based on Self-attention 计算机科学, 2022, 49(9): 215-220. https://doi.org/10.11896/jsjkx.210700190 |
| [2] | 方义秋, 张震坤, 葛君伟. 基于自注意力机制和迁移学习的跨领域推荐算法 Cross-domain Recommendation Algorithm Based on Self-attention Mechanism and Transfer Learning 计算机科学, 2022, 49(8): 70-77. https://doi.org/10.11896/jsjkx.210600011 |
| [3] | 陈坤峰, 潘志松, 王家宝, 施蕾, 张锦. 基于双目叠加仿生的微换衣行人再识别 Moderate Clothes-Changing Person Re-identification Based on Bionics of Binocular Summation 计算机科学, 2022, 49(8): 165-171. https://doi.org/10.11896/jsjkx.210600140 |
| [4] | 金方焱, 王秀利. 融合RACNN和BiLSTM的金融领域事件隐式因果关系抽取 Implicit Causality Extraction of Financial Events Integrating RACNN and BiLSTM 计算机科学, 2022, 49(7): 179-186. https://doi.org/10.11896/jsjkx.210500190 |
| [5] | 张嘉淏, 刘峰, 齐佳音. 一种基于Bottleneck Transformer的轻量级微表情识别架构 Lightweight Micro-expression Recognition Architecture Based on Bottleneck Transformer 计算机科学, 2022, 49(6A): 370-377. https://doi.org/10.11896/jsjkx.210500023 |
| [6] | 黄华伟, 李春华. 一种基于热带半环的密钥建立协议的安全性分析 Security Analysis of A Key Exchange Protocol Based on Tropical Semi-ring 计算机科学, 2022, 49(6A): 571-574. https://doi.org/10.11896/jsjkx.210700046 |
| [7] | 赵丹丹, 黄德根, 孟佳娜, 董宇, 张攀. 基于BERT-GRU-ATT模型的中文实体关系分类 Chinese Entity Relations Classification Based on BERT-GRU-ATT 计算机科学, 2022, 49(6): 319-325. https://doi.org/10.11896/jsjkx.210600123 |
| [8] | 胡艳丽, 童谭骞, 张啸宇, 彭娟. 融入自注意力机制的深度学习情感分析方法 Self-attention-based BGRU and CNN for Sentiment Analysis 计算机科学, 2022, 49(1): 252-258. https://doi.org/10.11896/jsjkx.210600063 |
| [9] | 徐少伟, 秦品乐, 曾建朝, 赵致楷, 高媛, 王丽芳. 基于多级特征和全局上下文的纵膈淋巴结分割算法 Mediastinal Lymph Node Segmentation Algorithm Based on Multi-level Features and Global Context 计算机科学, 2021, 48(6A): 95-100. https://doi.org/10.11896/jsjkx.200700067 |
| [10] | 陈扬, 王金亮, 夏炜, 杨颢, 朱润, 奚雪峰. 基于特征自动提取的足迹图像聚类方法 Footprint Image Clustering Method Based on Automatic Feature Extraction 计算机科学, 2021, 48(6A): 255-259. https://doi.org/10.11896/jsjkx.200900033 |
| [11] | 尤凌, 管张均. 一种低复杂度的水声OFDM通信系统子载波分配算法 Low-complexity Subcarrier Allocation Algorithm for Underwater OFDM Acoustic CommunicationSystems 计算机科学, 2021, 48(6A): 387-391. https://doi.org/10.11896/jsjkx.201100064 |
| [12] | 王习, 张凯, 李军辉, 孔芳, 张熠天. 联合自注意力和循环网络的图像标题生成 Generation of Image Caption of Joint Self-attention and Recurrent Neural Network 计算机科学, 2021, 48(4): 157-163. https://doi.org/10.11896/jsjkx.200300146 |
| [13] | 柴冰, 李冬冬, 王喆, 高大启. 融合频率和通道卷积注意的脑电(EEG)情感识别 EEG Emotion Recognition Based on Frequency and Channel Convolutional Attention 计算机科学, 2021, 48(12): 312-318. https://doi.org/10.11896/jsjkx.201000141 |
| [14] | 周小诗, 张梓葳, 文娟. 基于神经网络机器翻译的自然语言信息隐藏 Natural Language Steganography Based on Neural Machine Translation 计算机科学, 2021, 48(11A): 557-564. https://doi.org/10.11896/jsjkx.210100015 |
| [15] | 张世豪, 杜圣东, 贾真, 李天瑞. 基于深度神经网络和自注意力机制的医学实体关系抽取 Medical Entity Relation Extraction Based on Deep Neural Network and Self-attention Mechanism 计算机科学, 2021, 48(10): 77-84. https://doi.org/10.11896/jsjkx.210300271 |
|
||
首页