计算机科学

计算机科学 ›› 2025, Vol. 52 ›› Issue (9): 47-53.doi: 10.11896/jsjkx.241000046

• 智能医学工程 • 上一篇    下一篇

M2T-Net:基于多源数据的跨任务迁移学习舌象诊断方法

曾丽莉1, 夏佳楠1, 李韶雯1, 敬迈科1, 赵慧辉2, 周雪忠1   

  1. 1 北京交通大学计算机科学与技术学院 北京 100044
    2 北京中医药大学中医学院 北京 100029
  • 收稿日期:2024-10-11 修回日期:2025-02-08 出版日期:2025-09-15 发布日期:2025-09-11
  • 通讯作者: 夏佳楠(xiajn@bjtu.edu.cn)
  • 作者简介:(22120463@bjtu.edu.cn)
  • 基金资助:
    中央高校基本科研业务费(2024JBMC007);国家重点研发计划(2023YFC3502604)

M2T-Net:Cross-task Transfer Learning Tongue Diagnosis Method Based on Multi-source Data

ZENG Lili1, XIA Jianan1, LI Shaowen1, JING Maike1, ZHAO Huihui2, ZHOU Xuezhong1   

  1. 1 School of Computer Science and Technology,Beijing Jiaotong University,Beijing 100044,China
    2 School of Traditional Chinese Medicine,Beijing University of Chinese Medicine,Beijing 100029,China
  • Received:2024-10-11 Revised:2025-02-08 Online:2025-09-15 Published:2025-09-11
  • About author:ZENG Lili,born in 2000,postgraduate,is a member of CCF(No.R0493G).Her main research interests include medical image analysis and domain generalization.
    XIA Jianan,born in 1990,Ph.D,lectu-rer,master’s supervisor,is a member of CCF(No.P2378M).Her main research interests include time series analysis and medical image analysis.
  • Supported by:
    Fundamental Research Funds for the Central Universities(2024JBMC007) and National Key Research and Deve-lopment Program of China(2023YFC3502604).

PDF (PC)

11

摘要： 冠心病是临床常见的心血管疾病,冠脉介入术是其常见治疗方法之一。然而,糖尿病是冠心病的危险因素,与冠心病合并会显著增加治疗风险,尽早诊断和采取相应措施对这类患者具有重要的临床意义。临床指标是目前诊疗冠心病及其合并病的重要参考依据,而这些指标的获取大多是有创的。舌象作为人体健康的外在表现,不仅反映舌色、苔色等特征,还与心脏的各种生理和病理特征关联。深度学习的发展为客观化与可重复性获取舌象表征提供了帮助。然而,现有舌象分类方法受限于数据集标签的单一性,导致模型泛化能力不足。为此,提出了一种基于多源数据的跨任务迁移学习舌象诊断方法M2T-Net。该方法包括两个阶段:在多源数据的预训练阶段,获取不同任务下的高质图像编码器;在跨任务迁移阶段,结合交叉注意力机制,融合不同任务的特征表示,用于疾病分类。实验表明,M2T-Net模型在冠心病和冠心病伴随糖尿病两种人群的分类任务上的分类准确率达到93%,优于现有先进方法,具备较强的泛化能力与实用性,并且跨任务获得疾病表征更符合中医舌诊的整体观诊断思想,为舌象分析领域提供了更具实用性的解决方案。

关键词: 迁移学习, 舌象诊断, 交叉注意力, 深度学习, 舌诊智能化

Abstract: Coronary artery disease is a common clinical cardiovascular disease,and coronary intervention is one of its common treatment methods.However,diabetes mellitus is a risk factor for coronary artery disease,and the combination of diabetes mellitus and coronary artery disease significantly increases the risk of treatment,so early diagnosis and corresponding measures are of great clinical significance for these patients.Clinical indicators are important references for the diagnosis and treatment of coronary heart disease and their comorbidities,and most of these indicators are invasive.Tongue image,as an external manifestation of human health,not only reflects the tongue color,moss color and other characteristics,but also correlates with various physiological and pathological features of the heart.The development of deep learning provides assistance for objectivized and reproducible acquisition of tongue representations.However,existing tongue image classification methods are limited by the singularity of dataset labels,which leads to the lack of model generalization ability.To this end,a cross-task migration learning tongue diagnosis method M2T-Net based on multi-source data is proposed.Specifically it consists of two phases,in the pre-training phase of multi-source data,high quality image encoders under different tasks are acquired.In the cross-task migration phase,the feature representations from different tasks are fused for disease classification by combining the cross-attention mechanism.Experiments show that the performance of the M2T-Net model in the classification tasks of coronary heart disease and coronary heart disease accompanied by diabetes mellitus reaches a classification accuracy of 93%,which is better than the existing state-of-the-art methods,with strong generalization ability and practicality,and the cross-task acquisition of disease representations is more in line with the holistic diagnostic idea of Chinese medicine tongue diagnosis,which provides a more practical solution for the field of tongue image analysis.

Key words: Transfer learning, Tongue image diagnosis, Cross attention, Deep learning, Intelligent tongue diagnosis

中图分类号: 

  • TP391
[1]HEUSCH G.Myocardial ischaemia-reperfusion injury and cardioprotection in perspective[J].Nature Reviews Cardiology,2020,17(12):773-789.
[2]GUO J,WANG H,LI Y,et al.Nanotechnology in coronaryheart disease[J].Acta Biomaterialia,2023,171:37-67.
[3]ZHOU H,LI L,ZHAO H,et al.A large-scale,multi-centerurine biomarkers identification of coronary heart disease in TCM syndrome differentiation[J].Journal of Proteome Research,2019,18(5):1994-2003.
[4]ZHANG D,ZHANG H,ZHANG B.Introduction to tongue image analysis[M]//Tongue Image Analysis.Singapore:Springer,2017:3-18.
[5]XU Q,ZENG Y,TANG W,et al.Multi-task joint learning mo-del for segmenting and classifying tongue images using a deep neural network[J].IEEE Journal of Biomedical and Health Informatics,2020,24(9):2481-2489.
[6]HUANG Y,LI X,ZHENG S,et al.Tongue size and shape classification fusing segmentation features for traditional Chinese medicine diagnosis[J].Neural Computing and Applications,2023,35(10):7581-7594.
[7]WANG J,JIANG L,CHANG T,et al.Advances in the Application of Traditional Chinese Medicine Tongue Diagnosis in Coronary Heart Disease[J].Practical Journal of Cardiology,2024,32(3):99-102.
[8]XU J.Color altas of tongue diagnosis in traditional Chinese me-dicine[M].Shanghai:Shanghai University of TCM Press,2009.
[9]LI Y,LUO Y,ZHU Z.Tongue Image Feature Analysis Based on Deep Learning[J].Computer Science,2020,47(11):148-158.
[10]YOSINSKI J,CLUNE J,BENGIO Y,et al.How transferable are features in deep neural networks?[EB/OL].https://proceedings.neurips.cc/paper_files/paper/2014/file/532a2f85b6977104bc93f8580abbb330-Paper.pdf.
[11]YIN X,YU X,SOHN K,et al.Feature transfer learning for face recognition with under-represented data[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019.
[12]WEI W,MENG D,ZHAO Q,et al.Semi-supervised transferlearning for image rain removal[C]//2019 IEEE/CVF Confe-rence on Computer Vision and Pattern Recognition.2019.
[13]PERERA P,PATEL V M.Deep transfer learning for multiple class novelty detection[C]//IEEE/CVF Conference on Compu-ter Vision and Pattern Recognition.2019.
[14]DENG J,DONG W,SOCHER R,et al.Imagenet:A large-scale hierarchical image database[C]//2009 IEEE Conference on Computer Vision and Pattern Recognition.2009.
[15]XIE M,JEAN N,BURKE M,et al.Transfer learning from deep features for remote sensing and poverty mapping[C]//Procee-dings of the AAAI Conference on Artificial Intelligence.2016.
[16]XU R,LI G,YANG J,et al.Larger norm more transferable:An adaptive feature norm approach for unsupervised domain adaptation[C]//2019 IEEE/CVF International Conference on Computer Vision.2019.
[17]CSURKA G.A comprehensive survey on domain adaptation for visual applications[J].arXiv:1702.05374,2017.
[18]KUMAR A,LYNDON D,KIM J,et al.Subfigure and Multi-Label Classification using a Fine-Tuned Convolutional Neural Network[C]//Proceedings of the CLEF.2016.
[19]WANG S,DONG L,WANG X,et al.Classification of pathological types of lung cancer from CT images by deep residual neural networks with transfer learning strategy[J].Open Medicine,2020,15(1):190-197.
[20]HOU J,SU H Y,YAN B,et al.Classification of tongue color based on CNN[C]//Proceedings of the 2017 IEEE 2nd International Conference on Big Data Analysis(ICBDA).2017.
[21]WANG X,LIU J,WU C,et al.Artificial intelligence in tongue diagnosis:Using deep convolutional neural network for recognizing unhealthy tongue with tooth-mark[J].Computational and Structural Biotechnology Journal,2020,18:973-980.
[22]ZHANG X,ZHANG J,HU G,et al.Preliminary study of tongue image classification based on multi-label learning[C]//Procee-dings of the Advanced Intelligent Computing Theories and Applications:11th International Conference.2015.
[23]YU Y.Advances in application of artificial intelligence in medical image analysis[J].Chinese Journal of Medical Imaging Technology,2019,35(12):1808-1812.
[24]DE MOURA J,NOVO J,ORTEGA M.Deep feature analysis in a transfer learning-based approach for the automatic identification of diabetic macular edema[C]//Proceedings of the 2019 International Joint Conference on Neural Networks.2019.
[25]KHAN N M,ABRAHAM N,HON M.Transfer learning with intelligent training data selection for prediction of Alzheimer’s disease[J].IEEE Access,2019,7:72726-72735.
[26]ZHANG X,CHEN Z,GAO J,et al.A two-stage deep transfer learning model and its application for medical image processing in Traditional Chinese Medicine[J].Knowledge-based Systems,2022,239:108060.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发