动态数据流分析的在线超限学习算法综述

doi:10.11896/j.issn.1002-137X.2019.04.001

摘要/Abstract

摘要： 动态数据流分析是一个具有广泛应用价值的研究课题,在线学习方法是其中的一种关键技术。在众多在线学习方法中,在线贯序超限学习机(Online Sequential Extreme Learning Machine,OSELM)是一种新颖且实用的在线学习算法,目前已在动态数据流分析中得到了成功应用。首先,介绍了OSELM的理论基础和算法执行过程;然后,以动态数据流分析为应用背景,对各种改进OSELM算法进行了分类综述,包括基于滑动窗口的OSELM、基于遗忘因子的OSELM、基于样本加权的OSELM以及其他方法,重点论述了各类算法的设计思路和实现策略,并对其优缺点进行了比较和分析;最后,探讨了值得进一步研究的问题。

关键词: 动态数据流分析, 滑动窗口, 样本加权, 遗忘因子, 在线贯序超限学习机

Abstract: Dynamic data stream analysis has become a research focus for its widespread application prospects,and online learning method is key to solve this problem.Among existing online learning methods,online sequential extreme lear-ning machine (OSELM) is a novel and practical online learning algorithm,and it has been successfully applied in the field of dynamic data stream analysis.Firstly,the theoretical foundation and the execution process of OSELM were reviewed.Then,regarding dynamic data flow analysis as the application background,this paper classified and summarized different kinds of improved OSELM algorithms,including the sliding window based OSELM,forgetting factor based OSELM,sample weighting based OSELM and other methods.This paper focused on the design ideas and implementation strategies of different kinds of algorithms,compared and analyzed the advantages and disadvantages of main algorithms.Finally,the possible works for future research were presented.

Key words: Dynamic data stream analysis, Forgettingfactor, Online sequential extreme learning machine, Sample weighting, Sliding window

中图分类号:

TP181

郭威, 于建江, 汤克明, 徐涛. 动态数据流分析的在线超限学习算法综述[J]. 计算机科学, 2019, 46(4): 1-7. https://doi.org/10.11896/j.issn.1002-137X.2019.04.001

GUO Wei, YU Jian-jiang, TANG Ke-ming, XU Tao. Survey of Online Sequential Extreme Learning Algorithms for Dynamic Data Stream Analysis[J]. Computer Science, 2019, 46(4): 1-7. https://doi.org/10.11896/j.issn.1002-137X.2019.04.001

参考文献

[1]OLORUNNIMBE M K,VIKTOR H L,PAQUET E.Dynamic adaptation of online ensembles for drifting data streams[J].Journal of Intelligent Information Systems,2018,50(2):291-313.
[2]MEJRI D,LIMAM M,WEIHS C.A new dynamic weighted majority control chart for data streams[J].Soft Computing,2018,22(2):511-522.
[3]MENCAGLI G,TORQUATI M,DANELUTTO M.Elastic-
PPQ:A two-level autonomic system for spatial preference query processing over dynamic data streams[J].Future Generation Computer Systems,2018,79(3):862-877.
[4]MOHAMAD S,BOUCHACHIA A,SAYEDMOUCHAWEH
M.A Bi-Criteria Active Learning Algorithm for Dynamic Data Streams[J].IEEE Transactions on Neural Networks & Lear-ning Systems,2018,29(1):74-86.
[5]KHAMASSI I,SAYED-MOUCHAWEH M,HAMMAMI M,et al.Discussion and review on evolving data streams and concept drift adapting[J].Evolving Systems,2018,9(1):1-23.
[6]LI C,WEI F,DONG W,et al.Dynamic Structure Embedded Online Multiple-Output Regression for Streaming Data[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2018,doi:10.1109/TPAMI.2018.2794446.
[7]LIANG N Y,HUANG G B,SARATCHANDRAN P,et al.A fast and accurate online sequential learning algorithm for feedforward networks[J].IEEE Transactions on Neural Networks,2006,17(6):1411-23.
[8]HUANG G B,ZHU Q Y,SIEW C K.Extreme learning ma-
chine:Theory and applications[J].Neurocomputing,2006,70(1):489-501.
[9]HUYNH H T,WON Y.Regularized online sequential learning algorithm for single-hidden layer feedforward neural networks[J].Pattern Recognition Letters,2011,32(14):1930-1935.
[10]HUANG S,WANG B,QIU J,et al.Parallel ensemble of online sequential extreme learning machine based on MapReduce[J].Neurocomputing,2016,174:352-367.
[11]SCARDAPANE S,COMMINIELLO D,SCARPINITI M,et al.Online Sequential Extreme Learning Machine With Kernels[J].IEEE Transactions on Neural Networks & Learning Systems,2015,26(9):2214-2220.
[12]JIA X,WANG R,LIU J,et al.A semi-supervised online sequential extreme learning machine method[J].Neurocomputing,2016,174:168-178.
[13]WONG P K,XIANG H G,WONG K I,et al.Online extreme learning machine based modeling and optimization for point-by-point engine calibration[J].Neurocomputing,2018,277:187-197.
[14]OUYANG J Q,ZHOU Y,TANG H R.A Meteorological Predication Model Based on Storm and Online Sequential Extreme Learning Machine[J].Journal of Computer Research & Deve-lopment,2017,54(8):1736-1743.(in Chinese)
欧阳建权,周勇,唐欢容.基于Storm的在线序列极限学习机的气象预测模型[J].计算机研究与发展,2017,54(8):1736-1743.
[15]ZHOU X,LIU Z,ZHU C.Online Regularized and Kernelized
Extreme Learning Machines with Forgetting Mechanism[J].Mathematical Problems in Engineering,2014,2014:1-11.
[16]ZHOU X,WANG C.Cholesky factorization based online regularized and kernelized extreme learning machines with forgetting mechanism[J].Neurocomputing,2016,174:1147-1155.
[17]ZHANG H,ZHANG S,YIN Y.Kernel online sequential ELM algorithm with sliding window subject to time-varying environments[J].Memetic Computing,2016,10(1):43-52.
[18]WANG X,HAN M.Online sequential extreme learning machine with kernels for nonstationary time series prediction[J].Neurocomputing,2014,145:90-97.
[19]ZHANG X,WANG H L.Fixed-memory extreme learning machine and its applications[J].Control and Decision,2012,27(8):1206-1210.(in Chinese)
张弦,王宏力.限定记忆极端学习机及其应用[J].控制与决策,2012,27(8):1206-1210.
[20]ZHAO J,WANG Z,PARK D S.Online sequential extreme
learning machine with forgetting mechanism[J].Neurocompu-ting,2012,87:79-89.
[21]WANG J,RAN R,ZHOU Y.A Short-Term Photovoltaic Power Prediction Model Based on an FOS-ELM Algorithm[J].Applied Sciences,2017,7(4):423.
[22]XIAO D,LI B,ZHANG S.An online sequential multiple hidden layers extreme learning machine method with forgetting mechanism[J].Chemometrics and Intelligent Laboratory Systems,2018,176:126-133.
[23]MIRZA B,LIN Z.Meta-cognitive online sequential extreme
learning machine for imbalanced and concept-drifting data classification[J].Neural Networks,2016,80:79-94.
[24]ZHANG X,WANG H L.Selective forgetting extreme learning machine and its application to time series prediction[J].Acta Physica Sinica,2011,60(8):68-74.(in Chinese)
张弦,王宏力.具有选择与遗忘机制的极端学习机在时间序列预测中的应用[J].物理学报,2011,60(8):68-74.
[25]LU J,HUANG J,LU F.Sensor Fault Diagnosis for Aero Engine Based on Online Sequential Extreme Learning Machine with Memory Principle[J].Energies,2017,10(1):1-15.
[26]TIAN Z,WANG G,REN Y,et al.An Adaptive Online Sequential Extreme Learning Machine for Short-Term Wind Speed Prediction Based on Improved Artificial Bee Colony Algorithm[J].Neural Network World,2018,28(3):191-212.
[27]GUO W,XU T,TANG K M,et al.Online sequential extreme learning machine with generalized regularization and forgetting mechanism[J].Control & Decision,2017,32(2):247-254.(in Chinese)
郭威,徐涛,汤克明,等.具有广义正则化与遗忘机制的在线贯序超限学习机[J].控制与决策,2017,32(2):247-254.
[28]DU Z L,LI X M,ZHENG Z G,et al.Extreme learning machine based on regularization and forgetting factor and its application in fault prediction[J].Chinese Journal of Scientific Instrument,2015,36(7):1546-1553.(in Chinese)
杜占龙,李小民,郑宗贵,等.基于正则化与遗忘因子的极限学习机及其在故障预测中的应用[J].仪器仪表学报,2015,36(7):1546-1553.
[29]WU Z,TANG H,HE S,et al.Fast dynamic hysteresis modeling using a regularized online sequential extreme learning machine with forgetting property[J].The International Journal of Advanced Manufacturing Technology,2018,94:3473-3484.
[30]ZHANG H,ZHANG S,YIN Y.Online sequential ELM algo-
rithm with forgetting factor for real applications[J].Neurocomputing,2017,261:144-152.
[31]LIU D,WU Y,JIANG H.FP-ELM:An online sequential lear-
ning algorithm for dealing with concept drift[J].Neurocompu-ting,2016,207:322-334.
[32]LI Y,ZHANG S,YIN Y,et al.A Novel Online Sequential Extreme Learning Machine for Gas Utilization Ratio Prediction in Blast Furnaces[J].Sensors,2017,17(8):1847.
[33]LU J,HUANG J,LU F.Time Series Prediction Based on Adaptive Weight Online Sequential Extreme Learning Machine[J].Applied Sciences,2017,7(3):217.
[34]GUO W,XU T,YU J J,et al.Online Sequential Extreme Lear-
ning Machine Based on M-estimator and Variable Forgetting Factor[J].Journal of Electronics & Information Technology,2018,40(6):1360-1367.(in Chinese)
郭威,徐涛,于建江,等.基于M-estimator与可变遗忘因子的在线贯序超限学习机[J].电子与信息学报,2018,40(6):1360-1367.
[35]SOARES S G,ARA JO R.An adaptive ensemble of online Extreme Learning Machines with variable forgetting factor for dynamic system prediction[J].Neurocomputing,2016,171:693-707.
[36]LIMJS,LEES K,PANGH S,et al.Low complexity adaptive forgetting factor for online sequential extremelearning machine (OS-ELM) for application to nonstationary systemestimations[J].Neural Computing & Applications,2013,22(3-4):569-576.
[37]LIU J.Adaptive forgetting factor OS-ELM and bootstrap for
time series prediction[J].International Journal of Modeling Simulation & Scientific Computing,2017,8(3):1-19.
[38]GUO W,XU T,TANG K,et al.Online Sequential Extreme
Learning Machine with Generalized Regularization and Adaptive Forgetting Factor for Time-Varying System Prediction[J].Mathematical Problems in Engineering,2018,2018:1-22.
[39]ZOU Q Y,WANG X J,ZHOU C J,et al.The memory degradation based online sequential extreme learning machine[J].Neurocomputing,2018,275:2864-2879.
[40]GU Y,LIU J,CHEN Y,et al.TOSELM:Timeliness Online Sequential Extreme Learning Machine[J].Neurocomputing,2014,128:119-127.
[41]LUO X,YANG X,JIANG C,et al.Timeliness online regularized extreme learning machine[J].International Journal of Machine Learning and Cybernetics,2016,9(3):465-476.
[42]MAO W,WANG J,WANG L,et al.Online Sequential Prediction for Nonstationary Time Series with New Weight-Setting Strategy Using Extreme Learning Machine[J].Mathematical Problems in Engineering,2015,2015:1-13.
[43]LU X,ZHOU C,HUANG M,et al.Regularized online sequential extreme learning machine with adaptive regulation factor for time-varying nonlinear system[J].Neurocomputing,2016,174:617-626.
[44]SAHANI M,DASH P K.Variational mode decomposition and
weighted online sequential extreme learning machine for power quality event patterns recognition[J].Neurocomputing,2018,310:10-27.
[45]YIN J C.A variable-structure online sequential extreme learning machine for time-varying system prediction[J].Neurocompu-ting,2017,261:115-125.
[46]XU S,WANG J.Dynamic extreme learning machine for data
stream classification[J].Neurocomputing,2017,238:433-449.
[47]YE Y,SQUARTINI S,PIAZZA F.Online sequential extreme learning machine in nonstationary environments[J].Neurocomputing,2013,116:94-101.
[48]YIN J C,ZOU Z J,XU F,et al.Online ship roll motion prediction based on grey sequential extreme learning machine[J].Neurocomputing,2014,129:168-174.
[49]MIRZA B,LIN Z,LIU N.Ensemble of subset online sequential extreme learning machine for class imbalance and concept drift[J].Neurocomputing,2015,149:316-329.
[50]GHIMIRE D,LEE J.Online sequential extreme learning ma-
chine-based co-training for dynamic moving cast shadow detection[J].Multimedia Tools and Applications,2015,75(18):11181-11197.
[51]CAI Y,LAMARE R C D,ZHAO M,et al.Low-Com-
plexity Variable Forgetting Factor Mechanism for Blind Adaptive Constrained Constant Modulus Algorithms[J].IEEE Transactions on Signal Processing,2012,60(8):3988-4002.
[52]GUO W,XU T,TANG K.M-estimator-based online sequential extreme learning machine for predicting chaotic time series with outliers[J].Neural Computing & Applications,2017,28(12):4093-4110.

相关文章 15

[1]	龚建锋. 抗能量分析的带符号滑动窗口标量乘算法 Resisting Power Analysis Algorithm of Scalar Multiplication Based on Signed Sliding Window 计算机科学, 2021, 48(6A): 533-537. https://doi.org/10.11896/jsjkx.191200097
[2]	宋莺, 钟忺, 孙宝林, 桂超. MANET中基于滑动窗口的网络编码协作算法 Sliding Window-based Network Coding Cooperative Algorithm in MANET 计算机科学, 2020, 47(11): 322-326. https://doi.org/10.11896/jsjkx.191000181
[3]	王旭, 杨晓春. 基于FLINK的滑动窗口内三角形计数算法研究 Study of Triangle Counting Algorithm with Sliding Windows Based on FLINK 计算机科学, 2020, 47(10): 83-90. https://doi.org/10.11896/jsjkx.190900014
[4]	彭成, 贺婧, 池昊. 一种确定滑动窗口规模的边界距离算法 Boundary Distance Algorithm for Determining Sliding Window Size 计算机科学, 2019, 46(6A): 482-487.
[5]	杜娟, 沈思昀. 基于改进多权值滑动窗口的立体匹配方法的实现及应用 Implementation and Application of Stereo Matching Method Based onImproved Multi-weight Sliding Window 计算机科学, 2019, 46(11A): 241-245.
[6]	毛义坪, 余磊, 官泽瑾. 基于分数阶微分的多聚焦图像融合 Multi-focus Image Fusion Based on Fractional Differential 计算机科学, 2019, 46(11A): 315-319.
[7]	李青,肖迎元,王晓晔,李玉坤. 无线传感器网络中基于聚簇结构的Skyline查询方法 Clustering Architecture-based Skyline Query Processing in Wireless Sensor Networks 计算机科学, 2017, 44(10): 177-181. https://doi.org/10.11896/j.issn.1002-137X.2017.10.033
[8]	张建辉,王会青,孙宏伟,郭芷榕,白莹莹. 基于二分迭代SAX的时序相似性度量算法 Similarity Measure Algorithm of Time Series Based on Binary-dividing SAX 计算机科学, 2017, 44(1): 247-252. https://doi.org/10.11896/j.issn.1002-137X.2017.01.046
[9]	李彦,周艺,刘雨声,梁智. 无线体域网中基于时隙滑动窗口的概率路由协议 Probability Routing Protocol Based on Slotted Sliding Window in Wireless Body Area Network 计算机科学, 2016, 43(Z11): 259-263. https://doi.org/10.11896/j.issn.1002-137X.2016.11A.060
[10]	王丽珍,周丽华,邓世昆. 一种基于滑动窗口模式匹配的加权预测方法 Weighted Prediction Method Based on Sliding Window and Pattern Matching 计算机科学, 2016, 43(Z11): 591-596. https://doi.org/10.11896/j.issn.1002-137X.2016.11A.134
[11]	钟忺,杨光,卢炎生. 基于双阈值滑动窗口子镜头分割和完全连通图的关键帧提取方法 Method of Key Frames Extraction Based on Double-threshold Values Sliding Window Sub-shot Segmentation and Fully Connected Graph 计算机科学, 2016, 43(6): 289-293. https://doi.org/10.11896/j.issn.1002-137X.2016.06.057
[12]	曲晓雅,刘真. 基于阈值滑动窗口机制的虚拟机迁移判决算法 Dynamic Threshold Window Algorithm for Virtual Machines Migration Decision 计算机科学, 2016, 43(4): 64-69. https://doi.org/10.11896/j.issn.1002-137X.2016.04.013
[13]	陈小东,孙力娟,韩崇,郭剑. 基于模糊聚类的数据流概念漂移检测算法 Detecting Concept Drift of Data Stream Based on Fuzzy Clustering 计算机科学, 2016, 43(4): 219-223. https://doi.org/10.11896/j.issn.1002-137X.2016.04.045
[14]	吕太之. 基于滑动窗口Hurst指数的心电分析研究 Research of Heart Rate Variable Analysis Based on Sliding Window Hurst 计算机科学, 2016, 43(2): 259-262. https://doi.org/10.11896/j.issn.1002-137X.2016.02.054
[15]	常东亚,严建峰,杨璐,刘晓升. 基于滑动窗口的主题模型 Sliding-window Based Topic Modeling 计算机科学, 2016, 43(12): 101-107. https://doi.org/10.11896/j.issn.1002-137X.2016.12.018

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed