计算机科学

计算机科学 ›› 2025, Vol. 52 ›› Issue (4): 129-137.doi: 10.11896/jsjkx.240100111

• 数据库&大数据&数据科学 • 上一篇    下一篇

基于双分支小波卷积自编码器和数据增强的深度聚类方法

安瑞1, 鲁进1,2, 杨晶晶1   

  1. 1 云南大学信息学院 昆明 650504
    2 云南省高校物联网技术及应用重点实验室 昆明 650504
  • 收稿日期:2024-01-12 修回日期:2024-06-27 出版日期:2025-04-15 发布日期:2025-04-14
  • 通讯作者: 鲁进(lujin211636@ynu.edu.cn)
  • 作者简介:(yunanrui1204@163.com)
  • 基金资助:
    国家自然科学基金(62261059,61966037);云南省基础研究专项重点项目(202301AS070025)

Deep Clustering Method Based on Dual-branch Wavelet Convolutional Autoencoder and DataAugmentation

AN Rui1, LU Jin1,2, YANG Jingjing1   

  1. 1 School of Information Science and Engineering,Yunnan University,Kunming 650504,China
    2 Key Laboratory of Internet of Things Technology and Applications in Higher Education Institutions of Yunnan Province,Kunming 650504,China
  • Received:2024-01-12 Revised:2024-06-27 Online:2025-04-15 Published:2025-04-14
  • About author:AN Rui,born in 2000,postgraduate,is a member of CCF(No.T3057G).Her main research interests include image clustering and machine learning.
    LU Jin,born in 1984,Ph.D,associate professor.His main research interests include machine learning,stochastic re-sonance and compressed sensing.
  • Supported by:
    National Natural Science Foundation of China(62261059,61966037) and Key Project of Yunnan Basic Research Program(202301AS070025).

PDF (PC)

94

摘要： 基于自编码器的深度聚类是无监督学习的代表算法,近年来在计算机视觉领域获得了诸多关注。相较于传统算法,自动编码器隐藏层紧凑的表示空间为聚类任务提供了更为灵活的条件。现有的自编码器聚类大多使用单分支编码器网络,而采用多个网络结合的双编码器结构还有较大的探索空间。为此,提出了一种基于双分支小波卷积自编码器和数据增强的深度聚类方法DB-WCAE-DA(Deep Clustering Method Based on Dual-Branch Wavelet Convolutional Autoencoder and Data Augmentation)。首先,融合小波变换设计了一种双分支的卷积自编码器结构,将数据映射到低维特征空间进行聚类。其次,在一个分支上采用VMF混合模型构建聚类软分配,保留数据的几何结构和方向信息;在另一个分支上引入数据增强技术,并在嵌入空间中添加噪声,提高编码器对特征的学习能力。通过这种双分支嵌套式优化过程不断提炼数据特征,使得聚类结果更加可靠。最后,在多个基准数据集上验证了该模型的有效性。

关键词: 无监督学习, 深度聚类, 数据增强, 小波变换, 双分支自编码器

Abstract: Deep clustering based on autoencoders is a representative algorithm for unsupervised learning.It has gained much attention in the field of computer vision in recent years.Compared to traditional algorithms,the compact representation space provided by the hidden layers of autoencoders offers a more flexible condition for clustering tasks.Existing autoencoder clustering methods mostly use a single-branch encoder network,while the exploration space for a dual-encoder structure combining multiple networks is still significant.To address this,a deep clustering method named DB-WCAE-DA(Deep Clustering Method Based on Dual-branch Wavelet Convolutional Autoencoder and Data Augmentation) is proposed.Firstly,a dual-branch convolutional autoencoder structure is designed by integrating wavelet transformation,mapping the data to a low-dimensional feature space for clustering.Secondly,on one branch,a von mises-fisher(VMF) mixture model is employed to construct a soft clustering assignment,preserving the geometric structure and directional information of the data.On the other branch,data augmentation techniques are introduced,along with the addition of noise in the embedded space,to enhance the encoder’s learning capabilities.Through this dual-branch nested optimization process,the features of the data are continuously refined,resulting in more reliable clustering outcomes.Finally,the effectiveness of the model is validated on multiple benchmark datasets.

Key words: Unsupervised learning, Deep clustering, Data augmentation, Wavelet transform, Dual-branch autoencoder

中图分类号: 

  • TP181
[1]APARNA R,IDICULA S M.Spatio-Temporal Data Clustering Using Deep Learning:A Review [C]//2022 IEEE International Conference on Evolving and Adaptive Intelligent Systems(EAIS).2022:1-10.
[2]XIE J,GIRSHICK R,FARHADI A.Unsupervised Deep Embedding for Clustering Analysis [C]//Proceedings of the International Conference on Machine Learning(ICML).2016:478-487.
[3]GUO X,GAO L,LIU X,et al.Improved Deep Embedded Clustering with Local Structure Preservation [C]//Proceedings of the International Conference on Machine Learning(ICML).2017:1753-1759.
[4]GUO X,LIU X,ZHU E,et al.Deep Clustering with Convolutional Autoencoders [C]//Neural Information Processing.2017:373-382.
[5]SUN Z,SUN H.Stacked Denoising Autoencoder with Density-Grid Based Clustering Method for Detecting Outlier of Wind Turbine Components[J].IEEE Access,2019,7(1):13078-13091.
[6]CAI J,WANG S,GUO W.Unsupervised Embedded FeatureLearning for Deep Clustering with Stacked Sparse Auto-Encoder[J].Expert Systems with Applications,2021,186(30):115729-115740.
[7]REN Y,HU K,DAI X,et al.Semi-Supervised Deep Embedded Clustering[J].Neurocomputing,2019,325(10):121-130.
[8]JIANG Z,ZHENG Y,TAN H,et al.Variational Deep Embedding:An Unsupervised and Generative Approach to Clustering [C]//Proceedings of the 26th International Joint Conference on Artificial Intelligence.2017:1965-1972.
[9]KANG Y,KOU Y Q,XIE S Y,et al.Deep Clustering Model Based on Fusion Variational Graph Attention Self-Encoder[J].Computer Science,2021,48(S2):81-87,116.
[10]LI X,ZHAO X,CHU D,et al.An Autoencoder-Based Spectral Clustering Algorithm[J].Soft Computing,2020,24(3):1661-1671.
[11]ZHENG F F.The Research on Modulation Signal RecognitionTechnology Based on Deep Autoencoders[D].Dalian:Dalian University of Technology,2021.
[12]ZHENG M,LUO L,ZHENG H,et al.A Dual Encoder-Decoder Network for Self-Supervised Monocular Depth Estimation[J].IEEE Sensors Journal,2023,23(17):19747-19756.
[13]YAN F,YAN B,PEI M.Dual Transformer Encoder Model for Medical Image Classification [C]//2023 IEEE International Conference on Image Processing(ICIP).2023:690-694.
[14]YANG L,FAN W,BOUGUILA N.Deep Clustering AnalysisVia Dual Variational Autoencoder with Spherical Latent Embeddings[J].IEEE Transactions on Neural Networks and Learning Systems,2023,34(9):6303-6312.
[15]KOBAYASHI T.T-Vmf Similarity for Regularizing Intra-Class Feature Distribution [C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).2021:6612-6621.
[16]MACQUEEN J.Some Methods for Classification and Analysis of Multivariate Observations [C]//Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability.1967:281-297.
[17]SONG C,LIU F,HUANG Y,et al.Auto-Encoder Based Data Clustering [C]//Progress in Pattern Recognition,Image Analysis,Computer Vision,and Applications.2013:117-124.
[18]VINCENT P,LAROCHELLE H,BENGIO Y,et al.Extracting and Composing Robust Features with Denoising Autoencoders [C]//Proceedings of the 25th International Conference on Machine learning.2008:1096-1103.
[19]ZHANG K,SONG C,QIU L.Self-Paced Deep Clustering with Learning Loss[J].Pattern Recognition Letters,2023,171(1):8-14.
[20]MASCI J,MEIER U,CIREŞAN D,et al.Stacked Convolutional Auto-Encoders for Hierarchical Feature Extraction [C]//ArtificialNeural Networks and Machine Learning(ICML).2011:52-59.
[21]DU X L,CHEN Z G,XU X,et al.Improved Fault DiagnosisMethod for Bearing Faults Using Deep Wavelet Autoencoder[J].Computer Engineering and Applications,2020,56(5):263-269.
[22]HAIDONG S,HONGKAI J,KE Z,et al.A Novel TrackingDeep Wavelet Auto-Encoder Method for Intelligent Fault Diagnosis of Electric Locomotive Bearings[J].Mechanical Systems and Signal Processing,2018,110(1):193-209.
[23]HASNAT A,BOHNé J,MILGRAM J,et al.Von Mises-Fisher Mixture Model-Based Deep Learning:Application to Face Verification[J].arXiv:1706.04264,2017.
[24]LECUN Y,BOTTOU L,BENGIO Y,et al.Gradient-BasedLearning Applied to Document Recognition[C]//Proceedings of the IEEE.1998:2278-2324.
[25]XIAO H,RASUL K,VOLLGRAF R.Fashion-Mnist:A Novel Image Dataset for Benchmarking Machine Learning Algorithms[J].arXiv:1708.07747,2017.
[26]HULL J J.A Database for Handwritten Text Recognition Research[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,1994,16(5):550-554.
[27]PADMANABHAN D,BHAT S,SHEVADE S K,et al.Topic Model Based Multi-Label Classification from the Crowd[J].arXiv:1604.00783,20167.
[28]NENE S A,NAYAR S K,MURASE H.Columbia Object Image Library(Coil-20):CUCS-005-96[R].1996.
[29]NENE S A,NAYAR S K,MURASE H.Columbia Object Image Library(Coil100):CUCS-006-96[R].1996.
[30]YANG J,SHI R,WEI D,et al.Medmnist V2-a Large-ScaleLightweight Benchmark for 2d and 3d Biomedical Image Classification[J].Scientific Data,2023,10(1):41-51.
[31]XU W,LIU X,GONG Y.Document Clustering Based on Non-Negative Matrix Factorization [C]//Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Informaion Retrieval.2003:267-273.
[32]MCLACHLAN G J,LEE S X,RATHNAYAKE S I.FiniteMixture Models[J].Annual Review of Statistics and Its Application,2019,6(1):355-378.
[33]TAGHIA J,MA Z,LEIJON A.Bayesian Estimation of the Von-Mises Fisher Mixture Model with Variational Inference[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2014,36(9):1701-1715.
[34]CHEN X,DUAN Y,HOUTHOOFT R,et al.Infogan:Inter-pretable Representation Learning by Information Maximizing Generative Adversarial Nets [C]//Proceedings of the 30th International Conference on Neural Information Processing Systems.2016:2180-2188.
[35]MUKHERJEE S,ASNANI H,LIN E,et al.Clustergan:Latent Space Clustering in Generative Adversarial Networks [C]//Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence and Thirty-First Innovative Applications of Artificial Intelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificial Intelligence.2019.
[36]BU H.Deep Clustering Based on Contractive Autoencoder and Self-PacedLearning [C]//2023 4th International Conference on Computer Vision,Image and Deep Learning(CVIDL).2023:458-462.
[37]MIKLAUTZ L,BAUER L G M,MAUTZ D,et al.Details(Don’t)Matter:Isolating Cluster Information in Deep Embedded Spaces [C]//International Joint Conference on Artificial Intelligence.2021:2826-2832.
[38]LAURENS V D M,HINTON G.Visualizing Data Using T-Sne[J].Journal of Machine Learning Research,2008,9(2605):2579-2605.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发