计算机科学 ›› 2022, Vol. 49 ›› Issue (11A): 211000209-6.doi: 10.11896/jsjkx.211000209

• 人工智能 • 上一篇    下一篇

基于transformer的门控双塔模型预测H1N1流感抗原性

李川, 李维华, 王迎晖, 陈伟, 文俊颖   

  1. 云南大学信息学院 昆明 650503
  • 出版日期:2022-11-10 发布日期:2022-11-21
  • 通讯作者: 李维华(lywey@163.com)
  • 作者简介:(lichuanfirst163@163.com)
  • 基金资助:
    国家自然科学基金(32060151)

Gated Two-tower Transformer-based Model for Predicting Antigenicity of Influenza H1N1

LI Chuan, LI Wei-hua, WANG Ying-hui, CHEN Wei, WEN Jun-ying   

  1. School of Information Science and Engineering,Yunnan University,Kunming 650503,China
  • Online:2022-11-10 Published:2022-11-21
  • About author:LI Chuan,born in 1998,postgraduate.His main research interests include deep learning and bioinformatics.
    LI Wei-hua,born in 1977,Ph.D,asso-ciate professor.Her main research interests include data mining and bioinformatics.
  • Supported by:
    National Natural Science Foundation of China(32060151).

摘要: 流感病毒血凝素蛋白的快速演变导致新的病毒株不断产生,新的病毒株可能引起季节性流感甚至全球流感大爆发。及时检测出抗原变异体对疫苗的筛选和设计至关重要。鲁棒的抗原性预测模型是应对疫苗挑战的有效方法。各种端到端的特征学习工具为蛋白组学提供了良好的特征表示方法,但是现有的甲型流感预测模型还不能有效地提取并利用血凝素蛋白氨基酸序列中的特征。基于transformer设计一个门控双塔模型,通过输入甲型流感病毒血凝素蛋白的氨基酸序列,利用两个并行的编码器分别从血凝素蛋白氨基酸序列的时间维和空间维上捕捉抗原特征,并学习特征与预测结果间的非线性关系。为了减少数据中的噪声,融合时间维与空间维上的特征时,通过门机制自适应地获取衡量它们相对重要性的权重进行选择性融合,最后使用融合特征预测H1N1流感抗原变异株。在H1N1数据集上的实验结果表明,该模型利用优秀的非线性特征学习能力提高了抗原变异的预测性能,同时具有良好的鲁棒性。

关键词: 甲型流感, H1N1, 抗原性预测, transformer, 双塔模型, 门机制

Abstract: The rapid evolution of influenza virus hemagglutinin protein has led to the continuous production of new virus strains,which may cause seasonal influenza and even global influenza outbreaks.Timely detection of antigen variants is essential for vaccine screening and design.Therefore,a robust predictive model of antigenicity is an effective method to deal with the challenge of vaccines.Various end-to-end feature learning tools provide good feature representation methods for proteomics,but the existing influenza A prediction models cannot effectively extract and utilize features in amino acid sequences.In this paper,a gated two-tower model is designed based on the transformer.By inputting the amino acid sequence of the influenza A virus hemagglutinin protein,two parallel encoders are used to capture the antigenic characteristics from the time and space dimensions of the hemagglutinin protein amino acid sequence,and learn the nonlinear relationship between features and prediction results.In order to reduce the noise in the data,when fusing the features in the time dimension and the space dimension,the weights that measure their relative importance are adaptively obtained through the gate mechanism for selective fusion,and finally the fusion features are used to predict the H1N1 influenza antigen variants.Experimental results on the H1N1 data set show that the use of the model’sexcellent non-linear feature learning ability improves the predictive performance of antigenic variation,and at the same time has good robustness.

Key words: Influenza A, H1N1, Antigenicity prediction, Transformer, Two-tower mode, Gate mechanism

中图分类号: 

  • TP391
[1]AGOR J K,OZALTIN O Y.Models for predicting the evolution of influenza to inform vaccine strain selection[J].Hum Vaccin Immunother,2018,14(3):678-683.
[2]YIN R,ZHOU X,IVAN F X,et al.Identification of PotentialCritical Virulent Sites Based on Hemagglutinin of Influenza a Virus in Past Pandemic Strains[C]//Proceedings of the 6th International Conference on Bioinformatics and Biomedical Science.Singapore,Association for Computing Machiner,2017:30-36.
[3]NEHER R A,BEDFORD T.Nextflu:real-time tracking of seasonal influenza virus evolution in humans[J].Bioinformatics,2015,31(21):3546-3548.
[4]SAUTTO G A,KIRCHENBAUM G A,ROSS T M.Towards a universal influenza vaccine:different approaches for one goal[J].Virology Journal,2018,15(1):17.
[5]YIN R,LUUSUA E,DABROWSKI J,et al.Tempel:time-series mutation prediction of influenza A viruses via attention-based recurrent neural networks[J].Bioinformatics,2020,36(9):2697-2704.
[6]DE JONG J C,PALACHE A M.Haemagglutination-inhibitingantibody to influenza virus[J].Developments in Biologicals,2003,115:63-73.
[7]SMITH D J,LAPEDES A S,DE JONG J C,et al.Mapping theantigenic andgenetic evolution of influenza virus[J].Science 2004,305(5682):371-376.
[8]LEES W D,MOSS D S,SHEPHERD A J.A computationalanalysis of the antigenic properties of haemagglutinin in influenza A H3N2[J].Bioinformatics,2010,26(11):1403-1408.
[9]LIAO Y C,LEE M S,KO C Y,et al.Bioinformatics models for predicting antigenic variants of influenza A/H3N2 virus[J].Bioinformatics,2008,24(4):505-512.
[10]ZHOU X,YIN R,KWOH C K,et al.A context-free encoding scheme of protein sequences for predicting antigenicity of diverse influenza A viruses[C]//Proceedings of the 29th International Conference on Genome Informatics(GIW 2018):genomics.BMC Genomics,2018:936.
[11]PENG Y,WANG D,WANG J,et al.A universal computational model for predicting antigenic variants of influenza A virus based on conserved antigenic structures[J].Scientific Reports,2017,7:42051
[12]YIN R,TRAN V H,ZHOU X,et al.Predicting antigenic variants of H1N1 influenza virus based on epidemics and pandemics using a stacking model[J/OL].https://doi.org/10.1371/journal.pone.0207777,2018.
[13]LECUN Y,BENGIO Y,HINTON G.Deep learning[J].Nature,2015,521(7553):436-444.
[14]HOCHREITER S,SCHMIDHUBER J.Long short-term memory[J].Neural Computation,1997,9(8):1735-1780.
[15]CHO K,VAN MERRIENBOER B,BAHDANAU D.On theproperties of neural machine translation:Encoder-decoder approaches[J].arXiv 2014:14091259.
[16]YIN R,THWIN N N,ZHUANG P,et al.IAV-CNN:a 2D convolutional neural network model to predict antigenic variants of influenza A virus[J].IEEE/ACM Trans Comput Biol Bioinform,2021,9:1-1.
[17]ASWANI A V,SHAZEER N,PARMAR N,et al.Attention is all you need[J].arXiv:1706.03762,2017.
[18]ASGARI E,MOFRAD M R K.ProtVec:A Continuous Distributed Representation of Biological Sequences[J].PloS One,2015,10:11.
[19]MARCAIS G,KINGSFORD C.A fast,lock-free approach for efficient parallel counting of occurrences of k-mers[J].Bioinformatics,2011,27(6):764-770.
[20]KINGMA D P,BA J.Adam:A method for stochastic optimization[J].arXiv:1412.6980,2015.
[21]SRIVASTAVA N,HINTON G,KRIZHEVSKY A.Dropout:a simple way to prevent neural networks from overfitting[J].The Journal of Machine Learning Research,2014,15(1):1929-1958.
[22]RADZICKA A,WOLFENDEN R.Comparing the polarities of theamino acids:Side-chain distribution coefficients between the vapor phase,cyclohexane,1-octanol,and neutral aqueous sol-ution[J].Biochemistry,1988,27(5):1664-1670.
[23]MEILER J,MLLER M,ZEIDLER A,et al.Generation and evaluation of dimension-reduced amino acid parameter representations by artificial neural networks[J].Mol.Model.Annu.,2001,7(9):360-369.
[24]ATCHLEY W R,ZHAO J,FERNANDES A D,et al.Solvingthe protein sequence metric problem[J].Proc.Nat.Acad.Sci.United States America,2005,102(18):395-6400.
[25]DAYHOFF M O.A model of evolutionary change in proteins[J].Atlas Protein Sequence Structure,1978,5:89-99.
[26]HENIKOFF S,HENIKOFF J G.Amino acid substitution matrices from protein blocks[J].Proceedings of the National Academy of Sciences,1992,89(22):10915-10919.
[27]ALTSCHUL S F,KOONIN E V.Iterated profile searches withPSI-BLAST—a tool for discovery in protein databases[J].Trends in biochemical sciences,1998,23(11):444-447.
[28]MIYAZAWA S,JERNIGAN R L.Self-consistent estimation ofinter-residue protein contact energies based on an equilibrium mixture approximation of residues[J].Proteins:Structure Function Bioinf,1999,34(1):49-68.
[29]MICHELETTI C,SENO F,BANAVAR J R,et al.Learning effective amino acid interactions through iterative stochastic techniques[J].Proteins:Structure Function Bioinf,2001,42(3):422-431.
[30]LIN K,MAY A C W,TAYLOR W R.Amino acidencodingschemes from protein structure alignments:Multi-dimensional vectors to describe residue types[J].Theoretical Biol,2002,216(3):361-365.
[1] 汪鸣, 彭舰, 黄飞虎.
基于多时间尺度时空图网络的交通流量预测模型
Multi-time Scale Spatial-Temporal Graph Neural Network for Traffic Flow Prediction
计算机科学, 2022, 49(8): 40-48. https://doi.org/10.11896/jsjkx.220100188
[2] 张嘉淏, 刘峰, 齐佳音.
一种基于Bottleneck Transformer的轻量级微表情识别架构
Lightweight Micro-expression Recognition Architecture Based on Bottleneck Transformer
计算机科学, 2022, 49(6A): 370-377. https://doi.org/10.11896/jsjkx.210500023
[3] 康雁, 徐玉龙, 寇勇奇, 谢思宇, 杨学昆, 李浩.
基于Transformer和LSTM的药物相互作用预测
Drug-Drug Interaction Prediction Based on Transformer and LSTM
计算机科学, 2022, 49(6A): 17-21. https://doi.org/10.11896/jsjkx.210400150
[4] 赵小虎, 叶圣, 李晓.
多算法融合的骨骼重建信息动作分类方法
Multi-algorithm Fusion Behavior Classification Method for Body Bone Information Reconstruction
计算机科学, 2022, 49(6): 269-275. https://doi.org/10.11896/jsjkx.210500070
[5] 陆亮, 孔芳.
面向对话的融入知识的实体关系抽取
Dialogue-based Entity Relation Extraction with Knowledge
计算机科学, 2022, 49(5): 200-205. https://doi.org/10.11896/jsjkx.210300198
[6] 王帅, 张淑军, 叶康, 郭淇.
基于改进Transformer的连续手语识别方法
Continuous Sign Language Recognition Method Based on Improved Transformer
计算机科学, 2022, 49(11A): 211200198-6. https://doi.org/10.11896/jsjkx.211200198
[7] 胡新荣, 陈志恒, 刘军平, 彭涛, 叶鹏, 朱强.
基于多模态表示学习的情感分析框架
Sentiment Analysis Framework Based on Multimodal Representation Learning
计算机科学, 2022, 49(11A): 210900107-6. https://doi.org/10.11896/jsjkx.210900107
[8] 王迎晖, 李维华, 李川, 陈伟, 文俊颖.
基于注意力机制与集成学习的甲型H5N1流感病毒抗原相似性预测
Prediction of Antigenic Similarity of Influenza A/H5N1 Virus Based on Attention Mechanism and Ensemble Learning
计算机科学, 2022, 49(11A): 210900032-6. https://doi.org/10.11896/jsjkx.210900032
[9] 方仲俊, 张静, 李冬冬.
基于空间和多层级联合编码的图像描述算法
Spatial Encoding and Multi-layer Joint Encoding Enhanced Transformer for Image Captioning
计算机科学, 2022, 49(10): 151-158. https://doi.org/10.11896/jsjkx.210900159
[10] 杨慧敏, 马廷淮.
融合检索与生成的复合对话模型
Compound Conversation Model Combining Retrieval and Generation
计算机科学, 2021, 48(8): 234-239. https://doi.org/10.11896/jsjkx.200700162
[11] 杨进才, 曹元, 胡泉, 沈显君.
基于Transformer模型与关系词特征的汉语因果类复句关系自动识别
Relation Classification of Chinese Causal Compound Sentences Based on Transformer Model and Relational Word Feature
计算机科学, 2021, 48(6A): 295-298. https://doi.org/10.11896/jsjkx.200500019
[12] 霍帅, 庞春江.
基于Transformer和多通道卷积神经网络的情感分析研究
Research on Sentiment Analysis Based on Transformer and Multi-channel Convolutional Neural Network
计算机科学, 2021, 48(6A): 349-356. https://doi.org/10.11896/jsjkx.200800004
[13] 蒋琪, 苏伟, 谢莹, 周弘安平, 张久文, 蔡川.
基于Transformer的汉字到盲文端到端自动转换
End-to-End Chinese-Braille Automatic Conversion Based on Transformer
计算机科学, 2021, 48(11A): 136-141. https://doi.org/10.11896/jsjkx.210100025
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!