Computer Science ›› 2021, Vol. 48 ›› Issue (7): 256-269.doi: 10.11896/jsjkx.200800223

• Artificial Intelligence • Previous Articles     Next Articles

Summary of Computer-assisted Tongue Diagnosis Solutions for Key Problems

ZHANG Li-qian1, LI Meng-hang1, GAO Shan-shan1,2,3, ZHANG Cai-ming 2,4,5   

  1. 1 School of Computer Science and Technology,Shandong University of Finance and Economics,Jinan 250014,China
    2 Shandong Provincial Key Laboratory of Digital Media Technology,Jinan 250014,China
    3 Shandong China-U.S.Digital Media International Cooperation Research Center,Jinan 250014,China
    4 School of Software,Shandong University,Jinan 250101,China
    5 Shandong Province Future Intelligent Computing Collaborative Innovation Center,Yantai,Shandong 264025,China
  • Received:2020-08-31 Revised:2020-11-07 Online:2021-07-15 Published:2021-07-02
  • About author:ZHANG Li-qian,born in 1995,postgraduate,is a member of China Computer Federation.Her main research interests include image identification and so on.(
    GAO Shan-shan,born in 1980,Ph.D,professor,Ph.D supervisor,is a member of China Computer Federation. Her main research interests include Computer graphics,image processing,intelligent information processing and visua-lization.
  • Supported by:
    NSFC-Zhejiang Joint Fund of the Integration of Informatization and Industrialization(U1909210,U1609218),National Natural Science Foundation of China(61772309,61902217),Key Research and Development Project of Shandong Province(2017GGX10109,2019GGX101007),Shandong Provincial University Excellent Young Talents Joint Fund(ZR2018JL022) and Introduction and Education Plan of Young Creative Talents in Colleges and Universities of Shandong Province.

Abstract: Tongue diagnosis is one of the important contents of the four diagnostic methods of “looking,listening,asking and fee-ling the pulse”,and it is also a major feature of TCM (traditional Chinese medicine) diagnosis.TCM physicians need to make clinical diagnosis through visual observation,which makes traditional tongue diagnosis have the disadvantages of strong subjective dependence and lack of quantification.With the development of Wise Information Technology of 120 (WIT 120),researchers have focused on how to use computers to assist in the diagnosis of tongue images,realize intelligent tongue diagnosis,and then realize smart Chinese medicine.In recent years,the intelligent tongue diagnosis and its relevant research have become more and more popular.In order to assist researchers in this field to explore computer-aided tongue diagnosis in a more in-depth manner,this paper systematically and comprehensively reviewed them.Firstly,the specific process of computer-aided diagnosis of tongue image of traditional Chinese medicine is introduced.Secondly,based on the extensive study on the existing literature,the latest achievements and existing applications,this paper classifies and discusses different steps of computer-aided tongue diagnosis in mainstream methods,and summarizes the basic ideas,advantages and disadvantages of these methods.Then a relatively complete Computer-aided tongue diagnosis system is designed and implemented after enumerating some tongue image analysis systems that have been developed so far.Finally,this paper summarizes and prospects the possible development direction in the future.

Key words: Assisted tongue diagnosis, Feature extraction, Tongue analysis system, Tongue coating and body separation, Tongue segmentation

CLC Number: 

  • TP391
[1]XIE T.A new approach to the tongue-image segmentation and moisrening analysis based on image processing[D].Shanghai:East China University of Science and Technology,2017.
[2]SUN L Y,CHENG Z,GAO F S,et al.Discussion on the objective research of tongue diagnosis using computer image recognition technology[J].Journal of Anhui College of Traditional Chinese Medicine,1986,5(4):5-7.
[3]DU J Q,LU Y S,ZHU M F,et al.A novel algorithm of color tongue image segmentation based on HSI[C]//Proceedings of International Conference on Biomedical Engineering & Informatics.Sanya:IEEE Computer Society Press,2008:733-737.
[4]ZHU M F,DU J Q,ZHANG K,et al.A Novel approach fortongue image extraction based on combination of color and space information[C]//Proceedings of International Conference on Bioinformatics and Biomedical Engineering.Beijing:IEEE Computer Society Press,2009.
[5]LI C,WANG D Y,LIU Y Q,et al.A novel automatic tongue ima-ge segmentation algorithm:color enhancement method based on L*a*b* color space[C]//Proceedings of International Conference on Bioinformatics and Biomedicine.Washington:IEEE Computer Society Press,2015:990-993.
[6]WANG Y Z,YANG J,ZHOU Y,et al.Automatic tongue area extraction in tongue images[J].Computer Simulation,2005,22(2):232-235.
[7]MA C,TANG Z D,TANG L.Application of image segmentation technique in tongue diagnosis[J].Computer Simulation,2008,25(2):215-218.
[8]WY W J,MA L Z,XIAO X Z.Method of tongue image segmentation based on luminance and roughness information[J].Journal of System Simulation,2006,18(S1):374-376,379.
[9]ZHI L,YAN J Q,ZHOU T,et al.Tongue shape detection based on B-spline[C]//Proceedings of International Conference on Machine Learing and Cybernetics.Dalian:IEEE ComputerSocie-ty Press,2006:3829-3832.
[10]WU K,ZHANG D.Robust tongue segmentation by fusing region-based and edge-based approaches[J].Expert Systems with Applications,2015,42(21):8027-8038.
[11]KIM K H,DO J H,RYU H,et al.Tongue diagnosis method for extraction of effective region and classification of tongue coating[C]//Proceedings of Image Processing Theory,Tools and Applications.Sousse:IEEE Computer Society Press,2008.
[12]HUANG Z P,HUAGN Y S,YI F L,et al.An automatic tongue segmentation algorithm based on OTSU and region growing[J].LiShiZhen Medicine and Materia Medica Research,2017,28(12):3062-3064.
[13]LIU W X,ZHOU C E,LI Z Y,et al.Patch-driven tongue image segmentation using sparse representation[J].IEEE Access,2020,8:41372-41383.
[14]WANG Y Q,WEI B G,CAI Y H,et al.A knowledge-basedarithmetic for automatic tongue segmentation[J].Chinese Journal of Electronics,2004,32(3):489-491.
[15]PANG B,WANG K,ZHANG D,et al.On automated tongueimage segmentation in Chinese medicine[C]//Proceedings of International Conference on Pattern Recognition.Quebec:IEEE Computer Society Press,2002.
[16]PANG B,ZHANG D,WANG K.The bielliptical deformablecontour and its application to automated tongue segmentation in Chinese medicine[J].IEEE Transactions on Medical Imaging,2005,24(8):946-956.
[17]YU S Y,YANG J,WANG Y G.Color active contour models based tongue segmentation in traditional Chinese medicine[C]//Proceedings of International Conference on Bioinformatics & Bio-medical Engineering.Wuhan:IEEE Computer Society Press,2007:1081-1084.
[18]FU Z C,LI X Q,LI F F.Tougue image segmentation based on Snake model[J].Journal of Image And Graphics,2009,14(4):688-693.
[19]ZUO W,WANG K,ZHANG D,et al.Combination of polar edge detection and active contour model for automated tongue segmentation[C]//Proceedings of International Conference on Ima-ge and Graphics.Hong Kong:IEEE Computer Society Press,2005.
[20]ZHANG H,ZUO W,WANG K,et al.A snake-based approach to automated segmentation of tongue image using polar edge detector[J].International Journal of Imaging Systems and Technology,2006,16(4):103-112.
[21]QIN W X,LI B,YUE X Q.A hybrid tongue image segmentation algorithm based on initialization of Snake contours[J].Journal of University of Science and Technology of China,2010,40(8):807-811.
[22]GUO J,YANG Y,WU Q,et al.Adaptive active contour model based automatic tongue image segmentation[C]//Proceedings of International Congress on Image and Signal Processing,BioMe-dical Engineering and Informatics.Datong:IEEE Computer Socie-ty Press,2016:1386-1390.
[23]ZHU F,DAI P W,PAN B,et al.Tongue segmentation algorithm based on improved bacteria forageng optimization algorithm[J].Journal of Graphics,2019,40(1):70-77.
[24]YAN J J,XU Z,GUO R,et al.Research on tongue image segmentation based on active contour model of force field[J].Chinese Journal of Traditional Chinese Medicine,2019,34(8):3725-3727.
[25]WU J,ZHANG Y H,BAI J.Tongue Area Extraction in Tongue Diagnosis of Traditional Chinese Medicine[C]//Proceedings of International Conference of the IEEE Engineering in Medicine &Biology Society.Shanghai:IEEE Computer Society Press,2005:4955-4957.
[26]WU J,ZHANG Y H,BAI J,et al.Tongue contour image extraction using a watershed transform and an active contour mo-del[J].Journal of Tsinghua University (Natural Science edition),2008,48(6):1040-1043.
[27]NING J,ZHANG D,WU C,et al.Automatic tongue image segmentation based on gradient vector flow and region merging[J].Neural Computing and Applications,2012,21(8):1819-1826.
[28]SHI M J,LI G Z,LI F F.C2G2FSnake:autom-atic tongue image segmentation utilizing prior knowledge[J].SCIENCE CHINA Information Sciences(English Edition),2013(9):150-163.
[29]SHI M J,LI G Z,LI F F,et al.Computerized tongue image segmentation via the double geovector flow[J].Chinese Medicine,2014,9(7).
[30]LIU Z,YAN J Q,ZHANG D,et al.Automated tongue segmentation in hyperspectral images for medicine[J].Applied Optics,2007,46(34):8328-8334.
[31]FU Z C,LI W,LI X Q,et al.Automatic tongue location and segmentation[C]//Proceedings of International Conference on Audio,Language & Image Processing.Shanghai:IEEE Computer Society Press,2008:1050-1055.
[32]LI X Q,LI J D,WANG D.Automatic Tongue Image Segmentation Based on Histogram Projection and Matting[C]//Procee-dings of International Conference on Bioinformatics and Biomedicine.Belfast:IEEE Computer Society Press,2014:76-81.
[33]LI J P,PAN B C,WEI Y K.Tongue Image Segmentation Based on Fuzzy Rough Sets[C]//Proceedings of International Confe-rence on Environmental Science and Information Application Technology.Wuhan:IEEE Computer Society Press,2009:367-369.
[34]KANAWONG R,XU W T,XU D,et al.An automatic tongue detection and segmentation framework for computer-aided tongue image analysis[J].International Journal of Functional Informatics & Personalised Medicine,2012,4(1):56-68.
[35]ZHU M F,DU J Q,DING C H,et al.Improved Fast Random Walk Tongue Image Extraction Algorithm[J].Journal of Copu-ter-Aided Design & Computer Graphics,2015,27(4):633-639.
[36]XIE T,XIA C M,CHEN F F,et al.A method of tongue image segmentation based on kernel FCM[C]//Proceedings of International Congress on Image and Signal Processing,BioMedical Engineering and Informatics.Datong:IEEE Computer Society Press,2017:319-324.
[37]FU W B,SUN T,LANG J,et al.Review of Principle and Application of Deep Learning[J].Computer Science,2018,45(S1):11-15,40.
[38]LI J,XU B,BAN X,et al.A Tongue Image Segmentation Me-thod Based on Enhanced HSV Convolutional Neural Network[C]//Proceedings of International Conference on Cooperative Design,Visualization and Engineering.Mallorca:IEEE Compu-ter Society Press,2017:252-260.
[39]QU P L,ZHANG H,ZHUO L,et al.Automatic Tongue Image Segmentation for Traditional Chinese Medicine Using Deep Neural Network[J].Springer,2017,10361:247-259.
[40]XUE Y S,LI X Q,WU P,et al.Automated Tongue Segmentation in Chinese Medicine Based on Deep Learning[C]//Procee-dings of International Conference on Neural Information Processing.Siem Reap:IEEE Computer Society Press,2018:542-553.
[41]LIN B Q,QU Y Y,XIE J W,et al.Deeptongue:tongue segmentation via resnet[C]//Proceedings of International Conference on Acoustics,Speech and Signal Processing.Calgary:IEEE Computer Society Press,2018:1035-1039.
[42]ZHOU C,FAN H,LI Z.TongueNet:Accurate Localization and Segmentation for Tongue Images using Deep Neural Networks[J].IEEE Access,2019,7:148779-148789.
[43]WEI Y,LIU C S.Cascaded CNN for Real-time Tongue Segmentation Based on Key Points Localization[C]//Proceedings of International Conference on Big Data Analytics.Suzhou:IEEE Press,2019:303-307.
[44]MA L X,YANG H,SONG T T,et al.Research on tongue image segmentation algorithm of tongue image based on high resolution feature[J].Computer Engineering,2020,46(10):248-252.
[45]LI Y T,LUO Y S,ZHU Z M.Tongue Image Analysis in Traditional Chinese Medicine Based on Deep Learning[J].Computer Science 2020,47(11):148-158.
[46]ZHANG X H,SHI Q L,WANG B,et al.Review of Mchine Learing Algorithms in Traditional Chinese Medicine[J].Computer Science,2018,45(S2):32-36.
[47]HSU Y C,CHEN Y C,Lo L C,et al.Automatic tongue feature extraction[C]//Proceedings of International Computer Symposium.Tainan:IEEE Computer Society Press,2010:936-941.
[48]HAN F.Tongue color space analysis and color feature study[D].Harbin:Harbin Institute of Technology,2011.
[49]HAN L B,HU G Q,ZHANG X F,et al.Separation method of tongue coating and body of tongue image based on histogram equalization and gamma correction and K-means clustering[J].Beijing Biomedical Engineering,2019,38(1):1-6.
[50]LI Q,LIU Z.Tongue color analysis and discrimination based on hyperspectral images[J].Computerized Medical Imaging and Graphics:the Official Journal of the Computerized Medical Imaging Society,2009,33(3):217-221.
[51]CHIU C C.A novel approach based on computerized image ana-lysis for traditional Chinese medical diagnosis of the tongue[J].Computer Methods & Programs in Biomedicine,2000,61(2):77-89.
[52]LIU Y W,ZHANG X F,SHEN L S,et al.Comparisons forMulti-Class Support Vector Machine on Tongue Color Recognition of Traditional Chinese Medicine[J].Beijing Biomedical Engineering,2009,28(3):253-258.
[53]WANG X Z,ZHANG B,YANG Z M,et al.Statistical Analysis of Tongue Images for Feature Extraction and Diagnostics[J].IEEE Transactions on Image Processing,2013,22(12):5336-5347.
[54]SUN L Y,XIE H C,GAO F S,et al.Discussion on the objective research of tongue diagnosis using computer image recognition technology[J].Journal of Anhui Traditional Chinese Medical College,1986,5(4):5-7.
[55]LIU M A,XU J P,ZHAO Y,et al.The clinical research of glossoscopy of acute cerebrovascular disease[J].Journal of Emergency in Traditional Chinese Medicine,2008,17(11):1552-1554.
[56]HHUANG B,WU J S,ZHANG D,et al.Tongue shape classification by geometric features[J].Information Sciences,2010,180(2):312-324.
[57]LI W S,YAO J F,SONG H.The recognition of the teeth marks of tongue based on the improved level set in TCM[C]//Proceedings of International Congress on Image and Signal Proces-sing.Yantai:IEEE Computer Society Press,2010:2700-2704.
[58]LI H H,ZHANG X F,HU G Q,et al.The improvement of the tooth-marked recognition method on the tongue images[C]//Proceedings of International Conference on Complex Medical Engineering.Beijing:IEEE Computer Society Press,2013:412-415.
[59]SHAO Q,LI X Q,FU Z C.Recognition of Teeth-MarkedTongue Based on Gradient of Concave Region[C]//Proceedings of International Conference on Audio,Language and Image Processing.Shanghai:IEEE Computer Society Press,2014:968-972.
[60]ZHU M L M,LU P,XIA C M,et al.Research on Douglas-Peucker Method in Feature Extraction from 55 Cases of Tooth-Marked Tongue Images[J].Chinese Archives of Traditional Chinese Medicine,2014,32(9):2138-2140.
[61]WANG S,LIU K H,WANG L T.Tongue spots and petechiae recognition and extraction in tongue diagnosis images[J].Computer Engineering & Science,2017,39(6):1126-1132.
[62]YAN Z,ZHANG D P,LT N M.Kernel False-Colour Transformation and Line Extraction for Fissured Tongue Image[J].Journal of Computer-Aided Design & Computer Graphics,2010,22(5):771-776.
[63]QIN H S,HUANG Z C,ZHAO Y Q,et al.New MLBP-Otsumethod and its application in tongue crack image segmentation[J].Computer Engineering and Applications,2014,50(23):151-155.
[64]ZHANG H K,HU Y Y,LI X,et al.Computer Identification and Quantification of Fissured Tongue Diagnosis[C]//Proceedings of International Conference on Bioinformatics & Biomedicine.Madrid:IEEE Computer Society Press,2018:1953-1958.
[65]CHANG W H,CHU H T,CHANG H H.Tongue Fissure Visua-lization with Deep Learning[C]//Proceedings of Conference on Technologies and Applications of Artificial Intelligence.Taichung:IEEE Computer Society Press,2018:14-17.
[66]SELVARAJU R R,COGSWELL M,DAS A,et al.Grad-CAM:Visual Explanations from Deep Networks via Gradient-Based Localization[C]//Proceedings of International Conference on Computer Vision.Venice:IEEE Computer Society Press,2018:14-17.
[67]CAO M L,ZHAGN X F,SHEN L S.Application Survey of Information Combination in the Toughness and Tenderness of Tongue Manifestation Recognition[J].Beijing Biomedical Engineering,2006,25(6):644-648.
[68]WEI B G,SHEN L S.Automatic Analysis for Plumpness and Slenderness of Tongue[J].Computer Engineering,2004,30(11):25-26,68.
[69]ZHANG K,ZHANG H L,JIN S,et al.Analysis plumpness and slenderness of tongue based on the neural net[J].Chinese Journal of Traditional Chinese Medicine,2014,29(10):3111-3114.
[70]SUN D P,WU J,ZHANG Y H,et al.Automatic Sublingual Venae Extraction Method Based on Clustering[J].Chinese Journal of Biomedical Engineering,2008,27(2):265-269.
[71]YAN Z F.Feature Acquisition and Analysis of Sublingual Vein for Tongue Diagnosis in Traditional Chinese Medicine[D].Harbin:Harbin Institute of Technology,2010.
[72]ZHANG Q.A new tongue diagnosis method based on adaptive outline extraction and multiple features synthesis[D].Shanghai:Fudan University,2014.
[73]BAI L Y,SHI Y,WU J,et al.Automatic extraction of tongue coatings from digital images:A traditional Chinese medicine dia-gnostic tool[J].Tsinghua Science and Technology,2009,14(2):170-175.
[74]TANG R S.Classification of the thickness of tongue coatingusing 2D-Gabor wavelet[C]//Proceedings of Heilongjiang Computer Society Academic Exchange Annual Meeting.2007:262-266.
[75]LIU B,HU G Q,ZHANG X F,et al.An improved automatic description method of tongue coating thickness in Chinese medicine[J].Beijing Biomedical Engineering,2018,37(2):157-163.
[76]WEI B G,SHEN L S,CAI Y H,et al.Research on Curdy and Greas Tongue Fur Analysis for Traditional Chinese Medicine[J].Acta Electronica Sinica,2003,31(12A):2083-2086.
[77]QU T T,XIA C M,WANG Y Q,et al.Recognition of greasy or curdy tongue coating based on gabor wavelet transformation[J].Computer Applications and Software,2016,33(10):162-166.
[78]HUANG C W,CHEN Y J,YEN T T,et al.Region-based hie-rarchical tongue feature extraction[C]//Proceedings of International Conference on Machine Learning and Cybernetics.Lanzhou:IEEE Computer Society Press,2014:867-870.
[79]PANG B,ZHANG D,LI N,et al.Computerized tongue diagnosis based on Bayesian networks[J].Transactions on Biomedical Engineering,2004,51(10):1803-1810.
[80]GAO Z,PO L M,JIANG W,et al.A Novel Computerized Me-thod Based on Support Vector Machine for Tongue Diagnosis[C]//Proceedings of International IEEE Conference on Signal-Image Technologies and Internet-Based System.Shanghai:IEEE Computer Society Press,2007:797-802.
[81]TANG Y P,WANG L R,HE X,et al.Classification of tongue image based on multi-task deep convolutional Neural Network[J].Computer Science,2018,45(12):255-261.
[82]GAO Z,CUI M,LU G.A Novel Computerized System forTongue Diagnosis[C]//Proceedings of International Seminar on Future Information Technology and Management Engineering.Loughborough:IEEE Computer Society Press,2008:364-367.
[83]ZHANG Q,SHANG H L,ZHU J J,et al.A new tongue diagno-sis application on Android platform[C]//Proceedings of International Conference on Bioinformatics & Biomedicine.Shanghai:IEEE Computer Society Press,2014:324-327.
[84]SONG H B,WEN C B,CHENG X E.The construction of the auxiliary diagnosis and treatment system of tongue image and face image of traditional Chinese medicine based on AI[J].Lishi-zhen Medicine and Materia Medica Research,2020,31(2):502-505.
[1] ZHANG Yuan, KANG Le, GONG Zhao-hui, ZHANG Zhi-hong. Related Transaction Behavior Detection in Futures Market Based on Bi-LSTM [J]. Computer Science, 2022, 49(7): 31-39.
[2] ZENG Zhi-xian, CAO Jian-jun, WENG Nian-feng, JIANG Guo-quan, XU Bin. Fine-grained Semantic Association Video-Text Cross-modal Entity Resolution Based on Attention Mechanism [J]. Computer Science, 2022, 49(7): 106-112.
[3] CHENG Cheng, JIANG Ai-lian. Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction [J]. Computer Science, 2022, 49(7): 120-126.
[4] LIU Wei-ye, LU Hui-min, LI Yu-peng, MA Ning. Survey on Finger Vein Recognition Research [J]. Computer Science, 2022, 49(6A): 1-11.
[5] GAO Yuan-hao, LUO Xiao-qing, ZHANG Zhan-cheng. Infrared and Visible Image Fusion Based on Feature Separation [J]. Computer Science, 2022, 49(5): 58-63.
[6] ZUO Jie-ge, LIU Xiao-ming, CAI Bing. Outdoor Image Weather Recognition Based on Image Blocks and Feature Fusion [J]. Computer Science, 2022, 49(3): 197-203.
[7] REN Shou-peng, LI Jin, WANG Jing-ru, YUE Kun. Ensemble Regression Decision Trees-based lncRNA-disease Association Prediction [J]. Computer Science, 2022, 49(2): 265-271.
[8] ZHANG Shi-peng, LI Yong-zhong. Intrusion Detection Method Based on Denoising Autoencoder and Three-way Decisions [J]. Computer Science, 2021, 48(9): 345-351.
[9] FENG Xia, HU Zhi-yi, LIU Cai-hua. Survey of Research Progress on Cross-modal Retrieval [J]. Computer Science, 2021, 48(8): 13-23.
[10] BAO Yu-xuan, LU Tian-liang, DU Yan-hui, SHI Da. Deepfake Videos Detection Method Based on i_ResNet34 Model and Data Augmentation [J]. Computer Science, 2021, 48(7): 77-85.
[11] CHEN Yang, WANG Jin-liang, XIA Wei, YANG Hao, ZHU Run, XI Xue-feng. Footprint Image Clustering Method Based on Automatic Feature Extraction [J]. Computer Science, 2021, 48(6A): 255-259.
[12] LI Na-na, WANG Yong, ZHOU Lin, ZOU Chun-ming, TIAN Ying-jie, GUO Nai-wang. DDoS Attack Random Forest Detection Method Based on Secondary Screening of Feature Importance [J]. Computer Science, 2021, 48(6A): 464-467.
[13] LEI Jian-mei, ZENG Ling-qiu, MU Jie, CHEN Li-dong, WANG Cong, CHAI Yong. Reverse Diagnostic Method Based on Vehicle EMC Standard Test and Machine Learning [J]. Computer Science, 2021, 48(6): 190-195.
[14] LI Meng-he, XU Hong-ji, SHI Lei-xin, ZHAO Wen-jie, LI Juan. Multi-person Activity Recognition Based on Bone Keypoints Detection [J]. Computer Science, 2021, 48(4): 138-143.
[15] LU Bo-ren, HU Shi-zhe, LOU Zheng-zheng, YE Yang-dong. Character-level Feature Extraction Method for Railway Text Classification [J]. Computer Science, 2021, 48(3): 220-226.
Full text



No Suggested Reading articles found!