Computer Science ›› 2020, Vol. 47 ›› Issue (11): 148-158.doi: 10.11896/jsjkx.191000104

• Computer Graphics & Multimedia • Previous Articles     Next Articles

Tongue Image Analysis in Traditional Chinese Medicine Based on Deep Learning

LI Yuan-tong1, LUO Yu-sheng2, ZHU Zhen-min2,3   

  1. 1 College of Information Engineering,Xiangtan University,Xiangtan,Hunan 411105,China
    2 Institute of Computing Technology,Chinese Academy of Sciences,Beijing 100080,China
    3 Beijing Key Laboratory of Mobile Computing and Pervasive Device,Beijing 100080,China
  • Received:2019-10-16 Revised:2020-03-07 Online:2020-11-15 Published:2020-11-05
  • About author:LI Yuan-tong,born in 1993,postgradua-te,is a member of China Computer Fe-deration.His main research interests include deep learning and pervasive computing.
    ZHU Zhen-min,born in 1962,Ph.D,professor,is a member of China Computer Federation.His main research interests include embedded system technology and pervasive computing.
  • Supported by:
    This work was supported by the National Key R&D Program of China (2018YFC2000605).

Abstract: The traditional Chinese medicine tongue diagnosis,because of its intuition and easy to be observed,as well as its high clinical value,convenience and practicability,has become one of the important research subjects.At present,the combination of medical image processing technology,artificial intelligence technology and clinical experience of Chinese medicine experts to achieve objectification,quantification and automation of TCM tongue diagnosis is the mainstream of modernization research of TCM tongue diagnosis.In this paper,the key techniques of tongue segmentation and tongue image feature recognition based on migration learning and deep learning are studied.A tongue segmentation method based on region-based single pixel loss function is proposed.It can instruct the training and learning of the model by combining the color correlation and the semantic correlation between neighboring pixels,and the semantic information of target pixel labels.The experiments show that it partly improves the segmentation effect of the model,the MIoU index on the test set reached 96.32%.Then,a classification model of the tongue image geometric features,which combines spatial transformation network and VGG16 model,is proposed to identify and extract the geometric features of tongue image,providing a basis for syndromic inference of tongue image.Considering the orderliness of the geometric features of the data on the two-dimensional plane,the spatial transformation network is used to explicitly learn the spatial invariance in the model.And the convolution part of the VGG16 model is reused,so that the knowledge learned from the tongue segmentation task can be used for parameter transfer learning.Through two sets of comparative experiments,the validity of the spatial transformation network is proved to improve the spatial invariance of the model,and the knowledge of transfer learning is proved to make the model converge faster and more smoothly.At the same time,a classification model of the tongue image texture features,based on the deep texture coding network and VGG16 model,is proposed to recognize and extract the texture features of tongue image,providing a basis for syndromic inference of tongue image.According to the disorder of texture features in two-dimensional plane,a deep texture coding network is used to encode the ordered feature map,obtained by convolution layers,into an orderless texture semantic representation,which can express texture information more effectively.And the deep texture encoding network can enable the whole model to input images of any size,which gets rid of the loss of texture information caused by scaling operations of fixed input size.The validity of the orderless encoding of the deep texture encoding network for texture semantic representation is verified by the comparative analysis of experiments.

Key words: Deep learning, Tongue image analysis, Tongue segmentation, Traditional chinese medicine tongue diagnosis, Transfer learning

CLC Number: 

  • TP391
[1] XU J T.Clinical map of traditional Chinese medicine tonguediagnosis[M].BeiJing:Chemical Industry Press,2017.
[2] ZHOU Z,HUANG F.Application of mathematical morphology method on the segmentationof tongue images [J].Technological Development of Enterprise,2009,28(3):164-166.
[3] ZHANG L,QIN J.Tongue-image segmentation based on grayprojection and threshold-adaptive method[J].Chinese Tissue Engineering Research and Clinical Rehabilitation,2010,14(9):1638-1641.
[4] WU J,ZHANG Y H,BAI J,et al.Tongue contour image extraction using a watershed transform and an active contour model[J].Journal of Tsinghua University (Science and Technology),2008(6):1040-1043.
[5] ZHANG X Y.Tongue image segmentation based on randomwalk algorithm [D].Beijing:Beijing Institute of Technology,2016.
[6] HUANG Z P,HUANG Y S,YI F L,et al.An automatic tongue segmentation algorithm based on OTSU and region growing[J].Lishizhen Medicine and Materia Medica Research,2017,28(12):3062-3064.
[7] LIU Y B,YANG S.New algorithm on image automatic segmentation of body of tongue[J].Journal of Shenyang Normal University (Natural Science Edition),2011,29(4):514-517.
[8] SUN X L,PANG C Y.An improved snake model method on tongue segmentation[J].Journal of Changchun University of Science and Technology (Natural Science Edition),2013,36(5):154-156.
[9] WEI B G,SHEN L S.Automatic analysis for plumpness and slenderness of tongue [J].Computer Engineering,2004(11):25-26.
[10] XU J T,ZHANG Z F,REN H F,et al.An imaging diagnostic method about analyzing for plumpness and slenderness of tongue[J].Chinese Imaging Journal of Integrated Traditional and Western Medicine,2009,7(06):407-410.
[11] LU P.Study of teeth-marked tongue based on image processing and pattern recognition [D].Shanghai:East China University of Science and Technology, 2014.
[12] ZHAI T T,XIA C M,WANG Y Q.Recognition of Greasy or curdy tongue coating based on gabor wavelet transformation [J].Computer Applications and Software,2016,33(10):162-166.
[13] YANG Z H.Dissertation for the doctoral degree in engineering [D].Harbin:Harbin Institute of Technology, 2010.
[14] LIU G Z.Research on application of traditional Chinese medicine tongue images classification based om CNN [D].Jilin:Jilin University,2018.
[15] HU J L,KAN H X.Tongue classification based on convolutional neural network[J].Journal of Anqing Normal University (Natural Science Edition),2018,24(4):44-49.
[16] SUN L Y,CHENG Z,GAO F S,et al.Discussion on Objective Research of Tongue Diagnosis by Computer Image Recognition Technology[J].Journal of Anhui Traditional Chinese Medical College,1986(4):5-7.
[17] WANG Y Q,WEI B G,CAI Y H,et al.A knowledge-basedarithmetic for automatic tongue segmentation[J].Acta Electronica Sinica,2004(3):489-491.
[18] GAO L,LING X M.A tongue segmentation method based on improved fuzzy operator and morphology[J].Journal of Lanzhou Jiaotong University,2006(3):89-91.
[19] LI Q L,XUE Y Q,WANG J Y,et al.Automated tongue segmentation algorithm based on hyperspectral image[J].Journal of Infrared and Millimeter Waves,2007(1):77-80.
[20] LIU Z,CHEN J X,ZHAO Y M,et al.Automatic tongue image segmentation based on visual attention and support vector machine[J].Journal of Beijing University of Traditional Chinese Medicine,2013,36(1):18-20.
[21] WANG P,YANG W C,SUN C K,et al.Tongue segmentation and tongue crack extraction of tongue 3D color point cloud[J].Infrared and Laser Engineering,2017,46(S1):88-95.
[22] WANG L R,TANG Y P,CHEN P,et al.Two-phase convolutional neural network design for tongue segmentation[J].Journal of Image and Graphics,2018,23(10):1571-1581.
[23] GONG Y P,CHEN S Z,LIAN Y S,et al.Quantity study on the pathological nature of the tongue fur[J].Chinese Journal of Information on Traditional Chinese Medicine,2006(11):28-29.
[24] SHENG S Y,LI B,YUE X Q,et al.Color feature extraction of tongue image based on manifold learning[J].Space Medicine & Medical Engineering,2008(5):435-439.
[25] HU S N.Research on color recognition for tongue image in traditional Chinese medicine [D].Huangzhou:Zhejiang Sci-Tech University,2010.
[26] ZHANG J,QIAN J,DONG H Y,et al.Analysis of traditional Chinese medicine digital tongue texture based on fractal theory[J].China Journal of Traditional Chinese Medicine and Pharmacy,2016,31(1):104-106.
[27] XIE T.A new approach to the tongue-image segmentation and moistening analysis based on image processing [D].Shanghai:East China University of Science and Technology, 2017.
[28] WANG S,LIU K H,WANG L T.Tongue spots and petechiae recognition and extraction in tongue diagnosis images[J].Computer Engineering & Science,2017,39(6):1126-1132.
[29] LIU B,HU G Q,ZHANG X F,et al.An improved automatic description method of tongue coating thickness in Chinese medicine[J].Beijing Biomedical Engineering,2018,37(2):157-163.
[30] LIN T Y,GOYAL P,GIRSHICK R,et al.Focal loss for dense object detection[C]//IEEE transactions on pattern analysis and machine intelligence.2018.
[31] ZJADERBERG M, SIMONYAN K, ZISSERMAN A. Spatialtransformer networks[C]//Advances in Neural Information Processing Systems.2015.
[32] ZHANG H, XUE J, DANA K. Deep TEN: Texture encoding Network[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR).IEEE,2017.
[1] XU Yong-xin, ZHAO Jun-feng, WANG Ya-sha, XIE Bing, YANG Kai. Temporal Knowledge Graph Representation Learning [J]. Computer Science, 2022, 49(9): 162-171.
[2] RAO Zhi-shuang, JIA Zhen, ZHANG Fan, LI Tian-rui. Key-Value Relational Memory Networks for Question Answering over Knowledge Graph [J]. Computer Science, 2022, 49(9): 202-207.
[3] TANG Ling-tao, WANG Di, ZHANG Lu-fei, LIU Sheng-yun. Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy [J]. Computer Science, 2022, 49(9): 297-305.
[4] WANG Jian, PENG Yu-qi, ZHAO Yu-fei, YANG Jian. Survey of Social Network Public Opinion Information Extraction Based on Deep Learning [J]. Computer Science, 2022, 49(8): 279-293.
[5] HAO Zhi-rong, CHEN Long, HUANG Jia-cheng. Class Discriminative Universal Adversarial Attack for Text Classification [J]. Computer Science, 2022, 49(8): 323-329.
[6] JIANG Meng-han, LI Shao-mei, ZHENG Hong-hao, ZHANG Jian-peng. Rumor Detection Model Based on Improved Position Embedding [J]. Computer Science, 2022, 49(8): 330-335.
[7] FANG Yi-qiu, ZHANG Zhen-kun, GE Jun-wei. Cross-domain Recommendation Algorithm Based on Self-attention Mechanism and Transfer Learning [J]. Computer Science, 2022, 49(8): 70-77.
[8] SUN Qi, JI Gen-lin, ZHANG Jie. Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection [J]. Computer Science, 2022, 49(8): 172-177.
[9] HOU Yu-tao, ABULIZI Abudukelimu, ABUDUKELIMU Halidanmu. Advances in Chinese Pre-training Models [J]. Computer Science, 2022, 49(7): 148-163.
[10] ZHOU Hui, SHI Hao-chen, TU Yao-feng, HUANG Sheng-jun. Robust Deep Neural Network Learning Based on Active Sampling [J]. Computer Science, 2022, 49(7): 164-169.
[11] SU Dan-ning, CAO Gui-tao, WANG Yan-nan, WANG Hong, REN He. Survey of Deep Learning for Radar Emitter Identification Based on Small Sample [J]. Computer Science, 2022, 49(7): 226-235.
[12] HU Yan-yu, ZHAO Long, DONG Xiang-jun. Two-stage Deep Feature Selection Extraction Algorithm for Cancer Classification [J]. Computer Science, 2022, 49(7): 73-78.
[13] CHENG Cheng, JIANG Ai-lian. Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction [J]. Computer Science, 2022, 49(7): 120-126.
[14] LIU Wei-ye, LU Hui-min, LI Yu-peng, MA Ning. Survey on Finger Vein Recognition Research [J]. Computer Science, 2022, 49(6A): 1-11.
[15] SUN Fu-quan, CUI Zhi-qing, ZOU Peng, ZHANG Kun. Brain Tumor Segmentation Algorithm Based on Multi-scale Features [J]. Computer Science, 2022, 49(6A): 12-16.
Full text



No Suggested Reading articles found!