Computer Science ›› 2020, Vol. 47 ›› Issue (11A): 416-420.doi: 10.11896/jsjkx.200200020

• Big Data & Data Science • Previous Articles     Next Articles

Extraction and Automatic Classification of TCM Medical Records Based on Attention Mechanism of BERT and Bi-LSTM

DU Lin1, CAO Dong1, LIN Shu-yuan2, QU Yi-qian2, YE Hui1   

  1. 1 School of Medical Information Engineering,Guangzhou University of Traditional Chinese Medicine,Guangzhou 510000,China
    2 School of Basic Medical,Zhejiang Chinese Medical University,Hangzhou 310000,China
  • Online:2020-11-15 Published:2020-11-17
  • About author:DU Lin,born in 1998,undergraduate.Her main research interests include artificial intelligence and natural language processing.
    YE Hui,born in 1978,postgraduate,is a member of China Computer Federation.His main research interests include medical natural language processing.
  • Supported by:
    This work was supported by the National Science and Technology Major Project of the Ministry of Science and Technology of China (2017YFB1002302) and National Key Research and Development Project (2019YFC1710400).

Abstract: The development of traditional Chinese medicine has gradually become a hot topic,among which the medical records of traditional Chinese medicine contain huge and valuable medical information.However,in terms of the text mining and utilization of TCM medical records,it is always difficult to extract effective information and classify them.To solve this problem,it is of great clinical value to study a method of extracting and automatically classifying TCM medical records.This paper attempts to propose a short medical record classification model based on BERT+ Bi-LSTM +Attention fusion.BERT preprocessing is used to obtain the short text vector as the input of the model,to compare the pre-training effect of BERT and word2vec model,and to compare the effect of Bi-LSTM +Attention and LSTM model.The experimental results show that BERT+ Bi-LSTM +Attention fusion model achieves the highest Average F1 value of 89.52% in the extraction and classification of TCM medical records.Through comparison,it is found that the pre-training effect of BERT is significantly improved compared with that of word2vec model,and the effect of Bi-LSTM +Attention model is significantly improved compared with that of LSTM model.Therefore,the BERT+ Bi-LSTM +Attention fusion model proposed in this paper has certain medical value in the extraction and classification of medical records.

Key words: Attention, BERT, Bi-LSTM, LSTM

CLC Number: 

  • TP391.1
[1] ZHOU Y.Research on medical text analysis mining technology based on machine learning [D].Beijing:Beijing Jiaotong University,2019.
[2] SUN C A,DING Y,TIAN G.Analysis of emotional tendency of neural network based on GLU-CNN and attention-bilstm [J].Shandong University of Science and Technology,2019,40(7):62-66.
[3] YANG P,YANG Z H,LUO L,et al.Recognition of chemical drug named entity based on attention mechanism [J].Computer Research and Development,2008,55(7):1548-1556.
[4] WANG Y,WANG M X,ZHANG S,et al.BERT based alerttext named entity recognition [J/OL].Computer application:1-7 [2019-11-21].https://kns.cnki.net/kcms/detail/detail.aspx?dbcode=CJFD&dbname=CJFDLAST2020&filename=JSJY202002040&v=p5eBltYjMg96L0PeNbfh3J3eRmFBqc7Bb8ovNkpxL0WtdeRGGLxEJhgL4xxvx4DQ.
[5] YANG P,DONG W Y.Recognition method of Chinese namedentities based on BERT embedding [J/OL].Computer Engineering:1-7.[2019-11-21].https://kns.cnki.net/kcms/detail/detail.aspx?dbcode=CJFD&dbname=CJFDLAST2020&filename=JSJC202004006&v=fpDLQvPDFGf6wfMKb3vnBnPPBJfHHDXcSaF%25mmd2Bu59DcVlTutrMRBjr1z9Ri0PG2Gqa.
[6] DEVLIN J,CHANG M W,LEE K,et al.BERT:Pre-training of DeepBidirectional Transformers for Language Understanding[J].CS.CL,2018:4171-4186.
[7] JIANG Y.Research on Chinese medicine text mining for TCM prescription compatibility [D].University of Electronic Science and Technology of China,2019.
[8] YAO L,JIN Z,MAO C S,et al.Traditional Chinese medicine clinical records classification with BERT and domain specific corpora[J].Journal of the American Medical Informatics Association:JAMIA,2019,26(12).
[9] GUO X P.On constructing electronic medical records confor-ming to the characteristicsof traditional Chinese medicine [J].Journal of Traditional Chinese Medicine Management,2009,17(5):469-470.
[10] BIN Y.Intelligent Judicial Research Based on BERT SentenceEmbedding and Multi-Level Attention CNNs[C]//International Informatization and Engineering Associations:Computer Science and Electronic Technology International Society.2019:7.
[11] ULLAHA,AHMAD J,MUHAMMAD K,et al.Action Recognition in Video Sequences using Deep Bi-directional LSTM with CNN Features[J].IEEE Access,2017,PP(99):1-1.
[12] HU T T,FENG Y Q,SHEN L J,et al.Selection of main features of LSTM speech emotion based on attention mechanism [J].Acoustic Technology,2019,38(4):414-421.
[1] ZHOU Fang-quan, CHENG Wei-qing. Sequence Recommendation Based on Global Enhanced Graph Neural Network [J]. Computer Science, 2022, 49(9): 55-63.
[2] DAI Yu, XU Lin-feng. Cross-image Text Reading Method Based on Text Line Matching [J]. Computer Science, 2022, 49(9): 139-145.
[3] ZHOU Le-yuan, ZHANG Jian-hua, YUAN Tian-tian, CHEN Sheng-yong. Sequence-to-Sequence Chinese Continuous Sign Language Recognition and Translation with Multi- layer Attention Mechanism Fusion [J]. Computer Science, 2022, 49(9): 155-161.
[4] XIONG Li-qin, CAO Lei, LAI Jun, CHEN Xi-liang. Overview of Multi-agent Deep Reinforcement Learning Based on Value Factorization [J]. Computer Science, 2022, 49(9): 172-182.
[5] RAO Zhi-shuang, JIA Zhen, ZHANG Fan, LI Tian-rui. Key-Value Relational Memory Networks for Question Answering over Knowledge Graph [J]. Computer Science, 2022, 49(9): 202-207.
[6] WU Zi-yi, LI Shao-mei, JIANG Meng-han, ZHANG Jian-peng. Ontology Alignment Method Based on Self-attention [J]. Computer Science, 2022, 49(9): 215-220.
[7] LI Rong-fan, ZHONG Ting, WU Jin, ZHOU Fan, KUANG Ping. Spatio-Temporal Attention-based Kriging for Land Deformation Data Interpolation [J]. Computer Science, 2022, 49(8): 33-39.
[8] WANG Ming, PENG Jian, HUANG Fei-hu. Multi-time Scale Spatial-Temporal Graph Neural Network for Traffic Flow Prediction [J]. Computer Science, 2022, 49(8): 40-48.
[9] SHI Dian-xi, ZHAO Chen-ran, ZHANG Yao-wen, YANG Shao-wu, ZHANG Yong-jun. Adaptive Reward Method for End-to-End Cooperation Based on Multi-agent Reinforcement Learning [J]. Computer Science, 2022, 49(8): 247-256.
[10] JIANG Meng-han, LI Shao-mei, ZHENG Hong-hao, ZHANG Jian-peng. Rumor Detection Model Based on Improved Position Embedding [J]. Computer Science, 2022, 49(8): 330-335.
[11] FANG Yi-qiu, ZHANG Zhen-kun, GE Jun-wei. Cross-domain Recommendation Algorithm Based on Self-attention Mechanism and Transfer Learning [J]. Computer Science, 2022, 49(8): 70-77.
[12] ZHU Cheng-zhang, HUANG Jia-er, XIAO Ya-long, WANG Han, ZOU Bei-ji. Deep Hash Retrieval Algorithm for Medical Images Based on Attention Mechanism [J]. Computer Science, 2022, 49(8): 113-119.
[13] WEI Kai-xuan, FU Ying. Re-parameterized Multi-scale Fusion Network for Efficient Extreme Low-light Raw Denoising [J]. Computer Science, 2022, 49(8): 120-126.
[14] LIU Dong-mei, XU Yang, WU Ze-bin, LIU Qian, SONG Bin, WEI Zhi-hui. Incremental Object Detection Method Based on Border Distance Measurement [J]. Computer Science, 2022, 49(8): 136-142.
[15] CHEN Kun-feng, PAN Zhi-song, WANG Jia-bao, SHI Lei, ZHANG Jin. Moderate Clothes-Changing Person Re-identification Based on Bionics of Binocular Summation [J]. Computer Science, 2022, 49(8): 165-171.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!