计算机科学 ›› 2023, Vol. 50 ›› Issue (12): 97-103.doi: 10.11896/jsjkx.221100112
段梦梦1, 金城2
DUAN Mengmeng1, JIN Cheng2
摘要： 在时间序列分类任务中,模型集成方法通过训练多个基础模型并利用一定的规则来聚合基础模型的输出,从而得到比单一基础模型更准确的结果。目前模型集成方法主要关注基础模型的选择以及如何提高基础模型的差异性和多样性,忽视了对聚合规则的探索。针对这一问题,提出了基于Transformer特征融合的时间序列分类网络(Transformer Feature Fusion Network,TFFN)。该网络包含二重Transformer编解码器(Dual Transformer Encoder Decoder,Dual TED)和基于Transformer的具有样本分布感知特性的分类模块(Transformer Encoder Head,TEH)两个核心组件。Dual TED利用Transformer的注意力模块对基础特征进行提取和融合,得到具有更强辨别性的融合特征。具有样本分布感知特性的分类模块根据融合特征对时间序列进行更准确的分类,从而弥补现有集成模型方法忽视特征融合、集成规则过于简单的不足。实验结果表明,TFFN在多个主流时间序列分类数据集上取得了最好的成绩。
中图分类号:
| [1]BAJWA M N,KHURRAM S,MUNIR M,et al.Confident classification using a hybrid between deterministic and probabilistic convolutional neural networks[J].IEEE Access,2020,8:115476-115485. [2]LIU M H,ZENG A L,LAI Q X,et al.T-WaveNet:A Tree-Structured Wavelet Neural Network for Time Series Signal Analysis[C]//International Conference on Learning Representations(ICLR).2021. [3]XIAO Z,XU X,XING H,et al.RNTS:Robust Neural Temporal Search for Time Series Classification[C]//International Joint Conference on Neural Networks(IJCNN).2021:1-8. [4]YAN W,LI G,WU Z,et al.Extracting diverse-shapelets for early classification on time series[J].World Wide Web,2020,23(6):3055-3081. [5]LI H,JIA R,WAN X.Time series classification based on complex network[J].Expert Systems with Applications,2022,194:116502. [6]YAGHOUBI V,CHENG L,VAN PAEPEGEM W,et al.An ensemble classifier for vibration-based quality monitoring[J].Mechanical Systems and Signal Processing,2022,165:108341. [7]MELIN P,MONICA J C,SANCHEZ D,et al.Multiple ensemble neural network models with fuzzy response aggregation for predicting COVID-19 time series:the case of Mexico[J].Healthcare,2020,8(2):181. [8]TSAI Y H H,BAI S,LIANG P P,et al.Multimodal transformer for unaligned multimodal language sequences[C]//Proceedings of the Conference.Association for Computational Linguistics.Meeting.NIH Public Access,2019:6558. [9]YU Z,YU J,CUI Y,et al.Deep modular co-attention networks for visual question answering[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:6281-6290. [10]SCHÄFER P,LESER U.Multivariate Time Series Classification with WEASEL+MUSE[EB/OL].(2017-11)[2022-11].https://arxiv.org/abs/1711.11343. [11]FAWAZ H I,LUCAS B,FORESTIER G,et al.InceptionTime:Finding AlexNet for time series classification[J].Data Mining and Knowledge Discovery,2020,34(6):1936-1962. [12]TANG W S,LONG G D,LIU L,et al.Omni-Scale CNNs:A Simple and Effective Kernel Size Configuration for Time Series Classification[C]//International Conference on Learning Representations(ICLR).2022. [13]ZHANG X,GAO Y,LIN J,et al.TapNet:Multivariate Time Series Classification with Attentional Prototypical Network[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2020:6845-6852. [14]DEMPSTER A,PETITJEAN F,WEBB G I.ROCKET:exceptionally fast and accurate time series classification using random convolutional kernels[J].Data Mining and Knowledge Discove-ry,2020,34(5):1454-1495. [15]DEMPSTER A,SCHMIDT D F,WEBB G I.MiniRocket:A Very Fast(Almost) Deterministic Transform for Time Series Classification[C]//The 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining(KDD' 21).ACM,2021:248-257. [16]ZERVEAS G,JAYARAMAN S,PATEL D,et al.A Transfor-mer-based Framework for Multivariate Time Series Representation Learning[C]//The 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining(KDD'21).ACM,2021:2114-2124. [17]WORTSMAN M,ILHARCO G,GADRE S Y,et al.Modelsoups:averaging weights of multiple fine-tuned models improves accuracy without increasing inference time[C]//International Conference on Machine Learning.PMLR,2022:23965-23998. [18]LINES J,TAYLOR S,BAGNALL A.Hive-cote:The hierarchical vote collective of transformation-based ensembles for time series classification[C]//2016 IEEE 16th International Confe-rence on Data Mining(ICDM).IEEE,2016:1041-1046. [19]SHIFAZ A,PELLETIER C,PETITJEAN F,et al.TS-CHIEF:a scalable and accurate forest algorithm for time series classification[J].Data Mining and Knowledge Discovery,2020,34(3):742-775. [20]MIDDLEHURST M,LARGE J,FLYNN M,et al.HIVE-COTE 2.0:a new meta ensemble for time series classification[J].Machine Learning,2021,110(11):3211-3243. [21]LI X,JIANG H,NIU M,et al.An enhanced selective ensemble deep learning method for rolling bearing fault diagnosis with beetle antennae search algorithm[J].Mechanical Systems and Signal Processing,2020,142:106752. [22]ZHOU Y,CHENG G,JIANG S,et al.Building an efficient intrusion detection system based on feature selection and ensemble classifier[J].Computer Networks,2020,174:107247. [23]ISHAQ M,KWON S.Short-term energy forecasting framework using an ensemble deep learning approach[J].IEEE Access,2021,9:94262-94271. [24]XIA T,SONG Y,ZHENG Y,et al.An ensemble frameworkbased on convolutional bi-directional LSTM with multiple time windows for remaining useful life estimation[J].Computers in Industry,2020,115:103182. [25]DOSOVITSKIY A,BEYER L,KOLESNIKOV A,et al.Animage is worth 16x16 words:Transformers for image recognition at scale[C]//International Conference on Learning Representations(ICLR).2021. [26]DAU H A,BAGNALL A,KAMGAR K,et al.The UCR Time Series Archive[J].IEEE/CAA Journal of Automatica Sinica,2019,6(6):6-18. [27]BAGNALL A,DAU H A,LINES J,et al.The UEA multiva-riate time series classification archive,2018[A/OL].[2018-10].https://arxiv.org/abs/1811.00075. [28]CHEN Y P,EAMONN K,HU B,et al.The ucr time series classification archive[DS/OL].[2015-07].http://www.cs.ucr.edu/~eamonn/time_seires_data/. [29]LUCAS B,SHIFAZ A,CHARLOTTE P,et al.Proximity fo-rest:an effective and scalable distance-based classifier for time series[J].Data Mining and Knowledge Discovery,2019,33(3):607-635. [30]BENAVOLI A,CORANI G,MANGILI F.Should we really use post-hoc tests based on mean-ranks?[J].The Journal of Machine Learning Research,2016,17(1):152-161. |
|
||
首页