Computer Science ›› 2025, Vol. 52 ›› Issue (4): 129-137.doi: 10.11896/jsjkx.240100111

• Database & Big Data & Data Science • Previous Articles     Next Articles

Deep Clustering Method Based on Dual-branch Wavelet Convolutional Autoencoder and DataAugmentation

AN Rui1, LU Jin1,2, YANG Jingjing1   

  1. 1 School of Information Science and Engineering,Yunnan University,Kunming 650504,China
    2 Key Laboratory of Internet of Things Technology and Applications in Higher Education Institutions of Yunnan Province,Kunming 650504,China
  • Received:2024-01-12 Revised:2024-06-27 Online:2025-04-15 Published:2025-04-14
  • About author:AN Rui,born in 2000,postgraduate,is a member of CCF(No.T3057G).Her main research interests include image clustering and machine learning.
    LU Jin,born in 1984,Ph.D,associate professor.His main research interests include machine learning,stochastic re-sonance and compressed sensing.
  • Supported by:
    National Natural Science Foundation of China(62261059,61966037) and Key Project of Yunnan Basic Research Program(202301AS070025).

Abstract: Deep clustering based on autoencoders is a representative algorithm for unsupervised learning.It has gained much attention in the field of computer vision in recent years.Compared to traditional algorithms,the compact representation space provided by the hidden layers of autoencoders offers a more flexible condition for clustering tasks.Existing autoencoder clustering methods mostly use a single-branch encoder network,while the exploration space for a dual-encoder structure combining multiple networks is still significant.To address this,a deep clustering method named DB-WCAE-DA(Deep Clustering Method Based on Dual-branch Wavelet Convolutional Autoencoder and Data Augmentation) is proposed.Firstly,a dual-branch convolutional autoencoder structure is designed by integrating wavelet transformation,mapping the data to a low-dimensional feature space for clustering.Secondly,on one branch,a von mises-fisher(VMF) mixture model is employed to construct a soft clustering assignment,preserving the geometric structure and directional information of the data.On the other branch,data augmentation techniques are introduced,along with the addition of noise in the embedded space,to enhance the encoder’s learning capabilities.Through this dual-branch nested optimization process,the features of the data are continuously refined,resulting in more reliable clustering outcomes.Finally,the effectiveness of the model is validated on multiple benchmark datasets.

Key words: Unsupervised learning, Deep clustering, Data augmentation, Wavelet transform, Dual-branch autoencoder

CLC Number: 

  • TP181
[1]APARNA R,IDICULA S M.Spatio-Temporal Data Clustering Using Deep Learning:A Review [C]//2022 IEEE International Conference on Evolving and Adaptive Intelligent Systems(EAIS).2022:1-10.
[2]XIE J,GIRSHICK R,FARHADI A.Unsupervised Deep Embedding for Clustering Analysis [C]//Proceedings of the International Conference on Machine Learning(ICML).2016:478-487.
[3]GUO X,GAO L,LIU X,et al.Improved Deep Embedded Clustering with Local Structure Preservation [C]//Proceedings of the International Conference on Machine Learning(ICML).2017:1753-1759.
[4]GUO X,LIU X,ZHU E,et al.Deep Clustering with Convolutional Autoencoders [C]//Neural Information Processing.2017:373-382.
[5]SUN Z,SUN H.Stacked Denoising Autoencoder with Density-Grid Based Clustering Method for Detecting Outlier of Wind Turbine Components[J].IEEE Access,2019,7(1):13078-13091.
[6]CAI J,WANG S,GUO W.Unsupervised Embedded FeatureLearning for Deep Clustering with Stacked Sparse Auto-Encoder[J].Expert Systems with Applications,2021,186(30):115729-115740.
[7]REN Y,HU K,DAI X,et al.Semi-Supervised Deep Embedded Clustering[J].Neurocomputing,2019,325(10):121-130.
[8]JIANG Z,ZHENG Y,TAN H,et al.Variational Deep Embedding:An Unsupervised and Generative Approach to Clustering [C]//Proceedings of the 26th International Joint Conference on Artificial Intelligence.2017:1965-1972.
[9]KANG Y,KOU Y Q,XIE S Y,et al.Deep Clustering Model Based on Fusion Variational Graph Attention Self-Encoder[J].Computer Science,2021,48(S2):81-87,116.
[10]LI X,ZHAO X,CHU D,et al.An Autoencoder-Based Spectral Clustering Algorithm[J].Soft Computing,2020,24(3):1661-1671.
[11]ZHENG F F.The Research on Modulation Signal RecognitionTechnology Based on Deep Autoencoders[D].Dalian:Dalian University of Technology,2021.
[12]ZHENG M,LUO L,ZHENG H,et al.A Dual Encoder-Decoder Network for Self-Supervised Monocular Depth Estimation[J].IEEE Sensors Journal,2023,23(17):19747-19756.
[13]YAN F,YAN B,PEI M.Dual Transformer Encoder Model for Medical Image Classification [C]//2023 IEEE International Conference on Image Processing(ICIP).2023:690-694.
[14]YANG L,FAN W,BOUGUILA N.Deep Clustering AnalysisVia Dual Variational Autoencoder with Spherical Latent Embeddings[J].IEEE Transactions on Neural Networks and Learning Systems,2023,34(9):6303-6312.
[15]KOBAYASHI T.T-Vmf Similarity for Regularizing Intra-Class Feature Distribution [C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).2021:6612-6621.
[16]MACQUEEN J.Some Methods for Classification and Analysis of Multivariate Observations [C]//Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability.1967:281-297.
[17]SONG C,LIU F,HUANG Y,et al.Auto-Encoder Based Data Clustering [C]//Progress in Pattern Recognition,Image Analysis,Computer Vision,and Applications.2013:117-124.
[18]VINCENT P,LAROCHELLE H,BENGIO Y,et al.Extracting and Composing Robust Features with Denoising Autoencoders [C]//Proceedings of the 25th International Conference on Machine learning.2008:1096-1103.
[19]ZHANG K,SONG C,QIU L.Self-Paced Deep Clustering with Learning Loss[J].Pattern Recognition Letters,2023,171(1):8-14.
[20]MASCI J,MEIER U,CIREŞAN D,et al.Stacked Convolutional Auto-Encoders for Hierarchical Feature Extraction [C]//ArtificialNeural Networks and Machine Learning(ICML).2011:52-59.
[21]DU X L,CHEN Z G,XU X,et al.Improved Fault DiagnosisMethod for Bearing Faults Using Deep Wavelet Autoencoder[J].Computer Engineering and Applications,2020,56(5):263-269.
[22]HAIDONG S,HONGKAI J,KE Z,et al.A Novel TrackingDeep Wavelet Auto-Encoder Method for Intelligent Fault Diagnosis of Electric Locomotive Bearings[J].Mechanical Systems and Signal Processing,2018,110(1):193-209.
[23]HASNAT A,BOHNé J,MILGRAM J,et al.Von Mises-Fisher Mixture Model-Based Deep Learning:Application to Face Verification[J].arXiv:1706.04264,2017.
[24]LECUN Y,BOTTOU L,BENGIO Y,et al.Gradient-BasedLearning Applied to Document Recognition[C]//Proceedings of the IEEE.1998:2278-2324.
[25]XIAO H,RASUL K,VOLLGRAF R.Fashion-Mnist:A Novel Image Dataset for Benchmarking Machine Learning Algorithms[J].arXiv:1708.07747,2017.
[26]HULL J J.A Database for Handwritten Text Recognition Research[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,1994,16(5):550-554.
[27]PADMANABHAN D,BHAT S,SHEVADE S K,et al.Topic Model Based Multi-Label Classification from the Crowd[J].arXiv:1604.00783,20167.
[28]NENE S A,NAYAR S K,MURASE H.Columbia Object Image Library(Coil-20):CUCS-005-96[R].1996.
[29]NENE S A,NAYAR S K,MURASE H.Columbia Object Image Library(Coil100):CUCS-006-96[R].1996.
[30]YANG J,SHI R,WEI D,et al.Medmnist V2-a Large-ScaleLightweight Benchmark for 2d and 3d Biomedical Image Classification[J].Scientific Data,2023,10(1):41-51.
[31]XU W,LIU X,GONG Y.Document Clustering Based on Non-Negative Matrix Factorization [C]//Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Informaion Retrieval.2003:267-273.
[32]MCLACHLAN G J,LEE S X,RATHNAYAKE S I.FiniteMixture Models[J].Annual Review of Statistics and Its Application,2019,6(1):355-378.
[33]TAGHIA J,MA Z,LEIJON A.Bayesian Estimation of the Von-Mises Fisher Mixture Model with Variational Inference[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2014,36(9):1701-1715.
[34]CHEN X,DUAN Y,HOUTHOOFT R,et al.Infogan:Inter-pretable Representation Learning by Information Maximizing Generative Adversarial Nets [C]//Proceedings of the 30th International Conference on Neural Information Processing Systems.2016:2180-2188.
[35]MUKHERJEE S,ASNANI H,LIN E,et al.Clustergan:Latent Space Clustering in Generative Adversarial Networks [C]//Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence and Thirty-First Innovative Applications of Artificial Intelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificial Intelligence.2019.
[36]BU H.Deep Clustering Based on Contractive Autoencoder and Self-PacedLearning [C]//2023 4th International Conference on Computer Vision,Image and Deep Learning(CVIDL).2023:458-462.
[37]MIKLAUTZ L,BAUER L G M,MAUTZ D,et al.Details(Don’t)Matter:Isolating Cluster Information in Deep Embedded Spaces [C]//International Joint Conference on Artificial Intelligence.2021:2826-2832.
[38]LAURENS V D M,HINTON G.Visualizing Data Using T-Sne[J].Journal of Machine Learning Research,2008,9(2605):2579-2605.
[1] WU Pengyuan, FANG Wei. Study on Graph Collaborative Filtering Model Based on FeatureNet Contrastive Learning [J]. Computer Science, 2025, 52(5): 139-148.
[2] FU Kun, CUI Jingyuan, DANG Xing, CHENG Xiao, YING Shicong, LI Jianwei. Study on Graph Data Augmentation Based on Graph Entropy Theory [J]. Computer Science, 2025, 52(5): 149-160.
[3] WANG Haiyong, DING Gufei. ECG Signal Denoising Method Based on Stationary Wavelet Transform with Hyperbolic TangentThreshold Function [J]. Computer Science, 2025, 52(5): 179-186.
[4] YANG Yingxiu, CHEN Hongmei, ZHOU Lihua , XIAO Qing. Heterogeneous Graph Attention Network Based on Data Augmentation [J]. Computer Science, 2025, 52(3): 180-187.
[5] HE Liren, PENG Bo, CHI Mingmin. Unsupervised Multi-class Anomaly Detection Based on Prototype Reverse Distillation [J]. Computer Science, 2025, 52(2): 202-211.
[6] DING Xinyu, KONG Bing, CHEN Hongmei, BAO Chongming, ZHOU Lihua. Path-masked Autoencoder Guiding Unsupervised Attribute Graph Node Clustering [J]. Computer Science, 2025, 52(1): 160-169.
[7] YE Lishuo, HE Zhixue. Multi-granularity Time Series Contrastive Learning Method Incorporating Time-Frequency Features [J]. Computer Science, 2025, 52(1): 170-182.
[8] DAI Chaofan, DING Huahua. Domain-adaptive Entity Resolution Algorithm Based on Semi-supervised Learning [J]. Computer Science, 2024, 51(9): 214-222.
[9] LI Dongyang, NIE Rencan, PAN Linna, LI He. UMGN:An Infrared and Visible Image Fusion Network Based on Unsupervised Significance MaskGuidance [J]. Computer Science, 2024, 51(6A): 230600170-5.
[10] YIN Xudong, CHEN Junyang, ZHOU Bo. Study on Industrial Defect Augmentation Data Filtering Based on OOD Scores [J]. Computer Science, 2024, 51(6A): 230700111-7.
[11] DUAN Pengsong, DIAO Xianguang, ZHANG Dalong, CAO Yangjie, LIU Guangyi, KONG Jinsheng. WiCare:Non-contact Fall Monitoring Model for Elderly in Toilet [J]. Computer Science, 2024, 51(6A): 230700044-8.
[12] LOU Ren, HE Renqiang, ZHAO Sanyuan, HAO Xin, ZHOU Yueqi, WANG Xinyuan, LI Fangfang. Single Stage Unsupervised Visible-infrared Person Re-identification [J]. Computer Science, 2024, 51(6A): 230600138-7.
[13] HE Yifan, HE Yulin, CUI Laizhong, HUANG Zhexue. Subspace-based I-nice Clustering Algorithm [J]. Computer Science, 2024, 51(6): 153-160.
[14] PAN Lei, LIU Xin, CHEN Junyi, CHENG Zhangtao, LIU Leyuan, ZHOU Fan. Event Prediction Based on Dynamic Graph with Local Data Augmentation [J]. Computer Science, 2024, 51(3): 118-127.
[15] CAI Jiacheng, DONG Fangmin, SUN Shuifa, TANG Yongheng. Unsupervised Learning of Monocular Depth Estimation:A Survey [J]. Computer Science, 2024, 51(2): 117-134.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!