基于多任务卷积神经网络的舌象分类研究

doi:10.11896／j.issn.1002-137X.2018.12.042

摘要/Abstract

摘要： 针对现有技术难以并行实现舌象多标签的高效分类和识别,难以利用标签间的相关性进行综合分析等问题,提出了一种基于多任务卷积神经网络的舌象分类方法,构建了一种多任务联合学习模型,尝试实现传统中医舌诊中对舌色、苔色、裂纹和齿痕等多个标签的同时辨识。首先,在共享网络层对所有标签进行联合学习,从特征提取的角度自动挖掘和利用标签间的相关性;然后,在不同子网络层分别完成特定类别的学习任务,从而消除多标签分类中的歧义性;最后,训练多个Softmax分类器以实现对所有标签的并行预测。研究表明,所提方法能以端到端的方式同时提取舌象的多个特征并直接进行分类识别,在各分类评价指标上的最低值约为0.96,多任务的总体识别时间为34ms,因此该方法在精度和速度上均具有明显优势。

关键词: 多标签, 多任务网络, 迁移学习, 舌象分类, 相关性

Abstract: It is difficult to exploit the existing methods to achieve efficient classification and identification of tongue ima-ge’ labels in parallel,and it is also difficult to utilize the correlation between labels for comprehensive analysis.Aiming at the problems above,this paper proposed a classification method of tongue image based on multi-task deep convolutional neural network and constructed a multi-task joint learning model based on deep convolutional neural network to realize the simultaneous identification of tongue color,moss color,tongue crack and tooth marks in tongue diagnosis of Chinese medicine.First,the shared network layer is used to learn all labels,and the correlation between the tags is extracted and utilized automatically from the perspective of feature extraction.Then,the learning tasks of specific labels are completed in different sub-network layers to eliminate the ambiguity in the multi-label classification problem.Finally,multiple Softmax classifiers are trained to achieve parallel prediction of all labels.Experimental results suggest that the proposed method can simultaneous extract multiple features of tongue image and implement classification by means of end to end.The lowest value is about 0.96 in several evaluation indexes and the multi-task recognition rate is about 34ms.Therefore,this algorithm has obvious advantages in accuracy and speed.

Key words: Correlation, Multi-label, Multi-task network, Tongue classification, Transfer learning

中图分类号:

TP391.41

汤一平, 王丽冉, 何霞, 陈朋, 袁公萍. 基于多任务卷积神经网络的舌象分类研究[J]. 计算机科学, 2018, 45(12): 255-261. https://doi.org/10.11896／j.issn.1002-137X.2018.12.042

TANG Yi-ping, WANG Li-ran, HE Xia, CHEN Peng, YUAN Gong-ping. Classification of Tongue Image Based on Multi-task Deep Convolutional Neural Network[J]. Computer Science, 2018, 45(12): 255-261. https://doi.org/10.11896／j.issn.1002-137X.2018.12.042

参考文献

[1]LEE J,KIM H,KIM N R,et al.An approach for multi-labelclassification by directed acyclic graph with label correlation maximization[J].Information Sciences,2016,351(C):101-114.
[2]WU J,YE C,SHENG V S,et al.Active learning with label correlation exploration for multi-label image classification[J].Iet Computer Vision,2017,11(7):577-584.
[3]LI S L,LIU R,LIU H.Multi-Label Learning for Improved RBF Neural Networks[J].Computer Science,2015,42(4):316-320.(in Chinese)
李书玲,刘蓉,刘红.改进型RBF神经网络的多标签算法研究[J].计算机科学,2015,42(4):316-320.
[4]XU X D,YAO M H,LIU H W,et al.Pre-processing Method of Multi-label Classification Based on kNN[J].Computer Science,2015,42(5):106-108,131.(in Chinese)
徐晓丹,姚明海,刘华文,等.基于kNN的多标签分类预处理方法[J].计算机科学,2015,42(5):106-108,131.
[5]ZHANG Y F,HU G Q,ZHANG X F,et al.An algorithm study on tongue color recognition of patients[J].Beijing Biomedical Engineering,2016,35(1):7-11.(in Chinese)
张艺凡,胡广芹,张新峰,等.基于支持向量机的痤疮患者舌色苔色识别算法研究[J].北京生物医学工程,2016,35(1):7-11.
[6]WANG Y G,YANG J,ZHOU Y,et al.Tongue Image Color Recognition in Traditional Chinese Medicine[J].Journal of Biomedical Engineering,2005,22(6):1116-1120.(in Chinese)
王永刚,杨杰,周越,等.中医舌象颜色识别的研究[J].生物医学工程学杂志,2005,22(6):1116-1120.
[7]YANG Z H,ZHANG D P,LI N M.Kernel False-Colour Transformation and Line Extraction for Fissured Tongue Image[J].Journal of Computer-Aided Design and Computer Graphics,2010,22(5):771-776.(in Chinese)
杨朝辉,张大鹏,李乃民.裂纹舌图像的核假彩色变换及其纹线提取[J].计算机辅助设计与图形学学报,2010,22(5):771-776.
[8]ZHUMU L M,LU P,XIA C M,et al.Research on Douglas-Peucker Method in Feature Extraction from 55 Cases of Tooth-Marked Tongue Image[J].Chinese Archives of Traditional Chinese Medicine,2014,32(9):2138-2140.(in Chinese)
朱穆朗玛,陆萍,夏春明,等.基于道格拉斯-普克法提取55例齿痕舌图像特征研究[J].中华中医药学刊,2014,32(9):2138-2140.
[9]LIU W,ANGUELOV D,ERHAN D,et al.SSD:Single shotmultibox detector[C]∥Proceedings of European Conference on Computer Vision.Amsterdam:Springer International Publi-shing,2016:21-37.
[10]JONATHAN L,EVAN S,TREVOR D.Fully convolutional networks for semantic segmentation[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Boston:IEEE,2015:3431-3440.
[11]SUN Y,WANG X G,TANG X O.Deep Learning Face Representation from Predicting 10,000 Classes[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Columbus:IEEE,2014:1891-1898.
[12]GIRSHICK R,DONAHUE J,DARRELL T,et al.Rich feature hierarchies for accurate object detection and semantic segmentation[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Columbus:IEEE,2014:580-587.
[13]GIRSHICK R.Fast R-CNN[C]∥Proceedings of the IEEE Conference on International Conference on Computer Vision.Boston:IEEE,2015:1440-1448.
[14]REN S Q,HE K M,GIRSHICK R,et al.Faster R-CNN:Towards Real-Time Object Detection with Region Proposal Networks[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2017,39(6):1137-1149.
[15]HOOCHANG S,ROTH H R,GAO M,et al.Deep convolutional neural networks for computer-aided detection:CNN architectures,dataset characteristics and transfer learning[J].IEEE Transactions on Medical Imaging,2016,35(5):1285-1298.
[16]DONAHUE J,JIA Y Q,VINYALS O,et al.DeCAF:A Deep Convolutional Activation Feature for Generic Visual Recognition[C]∥Proceedings of the 31st International Conference on Machine Learning.2014:I-647-I-655.
[17]RAZAVIAN A S,AZIZPOUR H,SULLIVAN J,et al.CNN features off-the-shelf:An Astounding Baseline for Recognition[C]∥2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops(CV PRW).2014:512-519.
[18]YOSINSKI J,CLUNE J,BENGIO Y,et al.How transferableare features in deep neural networks?[C]∥Proceedings of the IEEE Conference on Neural Information Processing Systems.Montreal:MIT Press,2014:3320-3328.
[19]DENG J,DONG W,SOCHER R,et al.ImageNet:A large-scale hierarchical image database[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Miami:IEEE,2009:248-255.
[20]KAN H X,ZHANG L Y,DONG C W.A Tongue Image Recognition Method Based on Type 2 Diabetes Traditional Chinese Medicine Syndrome Classification[J].Chinese Journal of Biomedical Engineering,2016,35(6):658-664.(in Chinese)
阚红星,张璐瑶,董昌武.一种2型糖尿病中医证型的舌图像识别方法[J].中国生物医学工程学报,2016,35(6):658-664.

相关文章 15

[1]	方义秋, 张震坤, 葛君伟. 基于自注意力机制和迁移学习的跨领域推荐算法 Cross-domain Recommendation Algorithm Based on Self-attention Mechanism and Transfer Learning 计算机科学, 2022, 49(8): 70-77. https://doi.org/10.11896/jsjkx.210600011
[2]	陈莹, 郝应光, 王洪玉, 王坤. 基于局部梯度强度图的动态规划检测前跟踪算法 Dynamic Programming Track-Before-Detect Algorithm Based on Local Gradient and Intensity Map 计算机科学, 2022, 49(8): 150-156. https://doi.org/10.11896/jsjkx.210700135
[3]	武红鑫, 韩萌, 陈志强, 张喜龙, 李慕航. 监督和半监督学习下的多标签分类综述 Survey of Multi-label Classification Based on Supervised and Semi-supervised Learning 计算机科学, 2022, 49(8): 12-25. https://doi.org/10.11896/jsjkx.210700111
[4]	杨啸, 王翔坤, 胡浩, 朱敏. 面向设备状态监测的可视化技术综述 Survey on Visualization Technology for Equipment Condition Monitoring 计算机科学, 2022, 49(7): 89-99. https://doi.org/10.11896/jsjkx.210900167
[5]	王君锋, 刘凡, 杨赛, 吕坦悦, 陈峙宇, 许峰. 基于多源迁移学习的大坝裂缝检测 Dam Crack Detection Based on Multi-source Transfer Learning 计算机科学, 2022, 49(6A): 319-324. https://doi.org/10.11896/jsjkx.210500124
[6]	朱旭东, 熊贇. 基于样本分布损失的图像多标签分类研究 Study on Multi-label Image Classification Based on Sample Distribution Loss 计算机科学, 2022, 49(6): 210-216. https://doi.org/10.11896/jsjkx.210300267
[7]	赵耿, 王超, 马英杰. 基于混沌序列相关性的峰均比抑制研究 Study on PAPR Reduction Based on Correlation of Chaotic Sequences 计算机科学, 2022, 49(5): 250-255. https://doi.org/10.11896/jsjkx.210400292
[8]	彭云聪, 秦小林, 张力戈, 顾勇翔. 面向图像分类的小样本学习算法综述 Survey on Few-shot Learning Algorithms for Image Classification 计算机科学, 2022, 49(5): 1-9. https://doi.org/10.11896/jsjkx.210500128
[9]	谭珍琼, 姜文君, 任演纳, 张吉, 任德盛, 李晓鸿. 基于二分图的个性化学习任务分配 Personalized Learning Task Assignment Based on Bipartite Graph 计算机科学, 2022, 49(4): 269-281. https://doi.org/10.11896/jsjkx.210500125
[10]	左杰格, 柳晓鸣, 蔡兵. 基于图像分块与特征融合的户外图像天气识别 Outdoor Image Weather Recognition Based on Image Blocks and Feature Fusion 计算机科学, 2022, 49(3): 197-203. https://doi.org/10.11896/jsjkx.201200263
[11]	张舒萌, 余增, 李天瑞. 跨领域文本的可迁移情绪分析方法 Transferable Emotion Analysis Method for Cross-domain Text 计算机科学, 2022, 49(3): 218-224. https://doi.org/10.11896/jsjkx.210400034
[12]	李星燃, 张立言, 姚树婧. 结合特征融合和注意力机制的微表情识别方法 Micro-expression Recognition Method Combining Feature Fusion and Attention Mechanism 计算机科学, 2022, 49(2): 4-11. https://doi.org/10.11896/jsjkx.210900028
[13]	方仲礼, 王喆, 迟子秋. 面向多标签小样本学习的双流重构网络 Dual-stream Reconstruction Network for Multi-label and Few-shot Learning 计算机科学, 2022, 49(1): 212-218. https://doi.org/10.11896/jsjkx.201100143
[14]	侯宏旭, 孙硕, 乌尼尔. 蒙汉神经机器翻译研究综述 Survey of Mongolian-Chinese Neural Machine Translation 计算机科学, 2022, 49(1): 31-40. https://doi.org/10.11896/jsjkx.210900006
[15]	刘意, 毛莺池, 程杨堃, 高建, 王龙宝. 基于邻域一致性的异常检测序列集成方法 Locality and Consistency Based Sequential Ensemble Method for Outlier Detection 计算机科学, 2022, 49(1): 146-152. https://doi.org/10.11896/jsjkx.201000156

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed