计算机科学 ›› 2022, Vol. 49 ›› Issue (11): 212-220.doi: 10.11896/jsjkx.210900054
魏军胜1, 刘琰1, 陈静2, 段顺然1
WEI Jun-sheng1, LIU Yan1, CHEN Jing2, DUAN Shun-ran1
摘要： 集成学习一直是构建强大和稳定的预测模型的策略之一,它能通过融合多个模型来提升结果的准确性和稳定性。但是,现有的集成方法在权重计算上还存在一定的缺陷,面对多种分类问题时无法自适应地选择集成权重,不具有通用性。针对以上问题,提出了一种自适应权重的多分类通用集成方法(UMEAW)。与通常的集成分类方法只针对一种分类任务不同,UMEAW面对不同的分类问题,首先根据分类个数计算权重调配系数,然后利用指数函数分布特性,根据模型评价指标与权重调配系数自动计算一次模型融合的权重,最后通过不断迭代的方法自适应地调整融合权重,实现不同分类任务下的模型融合。实验结果表明,UMEAW在9个不同分类个数、不同领域、不同规模的数据集上都能实现模型融合,其融合效果在大部分任务上都优于基线方法。与单个模型相比,用UMEAW融合后的结果F1值稳定增加了3%~25%;与其他集成方法相比,F1值稳定提升了1%~2%,证明了UMEAW的通用性和有效性。
中图分类号:
| [1]OPITZ D,MACLIN R.Popular Ensemble Methods:An Empirical Study[J].Journal of Artificial Intelligence Research,1999,11:169-198. [2]HANSEN L K,SALAMON P.Neural network ensembles[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2002,12(10):993-1001. [3]FREUND Y.Boosting a weak learning algorithm by majority[J].Information and Computation,1995,121(2):256-285. [4]SHAPIRE R.The strength of weak learnability[J].Machine Learning,1990,5(2):197-227. [5]JI Z,HUI Z,ROSSET S,et al.Multi-class adaboost[J].Statistics and its Interface,2009,2(3):349-360. [6]WOLPERT D H.Stacked generalization[J].Neural Networks,2017,5(2):241-259. [7]LITTLESTONE N,WARMUTH M K.The weighted majority algorithm[J].Information and Computation,1994,108(2):212-261. [8]YU Z,WANG D,ZHAO Z,et al.Hybrid incremental ensemble learning for noisy real-world data classification[J].IEEE Transa-ctions on Cybernetics,2017,49(2):403-416. [9]DONG M G,ZHANG W,JING C.Dynamic weight ensemble classification algorithm for unbalanced data streams[J].Miniature Microcomputer Systems,2020,41(8):1649-1655. [10]GAO W,ZHOU Z H.Approximation stability and boosting[C]//International Conference on Algorithmic Learning Theory.Berlin:Springer,2010:59-73. [11]YIN X C,HUANG K,HAO H W,et al.A novel classifier ensemble method with sparsity and diversity[J].Neurocomputing,2014,134:214-221. [12]ZHANG L,SUGANTHAN P N.Random Forests with ensemble of feature spaces[J].Pattern Recognition,2014,47(10):3429-3437. [13]KOLTER J Z,MALOOF M A.Dynamic weighted majority:An ensemble method for drifting concepts[J].Journal of Machine Learning Research,2007,8(Dec):2755-2790. [14]MAO S,JIAO L,XIONG L,et al.Weighted classifier ensemble based on quadratic form[J].Pattern Recognition,2015,48(5):1688-1706. [15]MUHLBAIER M D,TOPALIS A,POLIKAR R.Learn++.NC:Combining Ensemble of Classifiers With Dynamically Weighted Consult-and-Vote for Efficient Incremental Learning of New Classes[J].IEEE Transactions on Neural Networks,2009,20(1):152-168. [16]HUANG Y,MONEKOSSO D,WANG H.Clustering Ensembles Based on Multi-classifier Fusion[C]//IEEE International Conference on Intelligent Computing and Intelligent Systems(ICIS 2010).Shanghai:IEEE,2010:393-397. [17]NGUYEN T T,LIEW A W C,TRAN M T,et al.Combining multi classifiers based on a genetic algorithm-a gaussian mixture model framework[C]//International Conference on Intelligent Computing.Cham:Springer,2014:56-67. [18]SONG J Z,GUO C Y,LIU H S.Selective SVM Ensemble Base on Clustering Analysis Apply for Analog Circuit Fault Diagnosis with Small Samples[J].Applied Mechanics & Materials,2013,380-384:841-845. [19]YANG J,WANG F.Auto-Ensemble:An Adaptive LearningRate Scheduling Based Deep Learning Model Ensembling[J].IEEE Access,2020,8(12):217499-217509. [20]LU Z,WU X,ZHU X,et al.Ensemble Pruning via Individual Contribution Ordering[C]//ACM Sigkdd International Confe-rence on Knowledge Discovery & Data Mining.Washington:ACM,2010:871. [21]MARTÍNEZ-MUÑOZ G,HERNÁNDEZ-LOBATO D,SUÁREZA.An Analysis of Ensemble Pruning Techniques Based on Ordered Aggregation[J].IEEE Transactions on Pattern Analysis &Machine Intelligence,2009,31(2):245-259. [22]HASAN M A,ABDULLAH N,RAHMAN M,et al.Dental Impression Tray Selection From Maxillary Arch Images Using Multi-Feature Fusion and Ensemble Classifier[J].IEEE Access,2021,9(2):30573-30586. [23]BANIA R K,HALDER A.R-HEFS:Rough set based heteroge-neous ensemble feature selection method for medical data classification[J].Artificial Intelligence in Medicine,2021,114:102049-102080. [24]LAI C M,HUANG H P.A gene selection algorithm using simplified swarm optimization with multi-filter ensemble technique[J].Applied Soft Computing,2021,100:106994-107042. [25]ZHOUG,GUO F L.Research on high-dimensional data integration learning method based on feature selection[J].Computer Science,2021,48(S1):250-254. [26]DEVLIN J,CHANG M,LEE K,et al.Bert:Pre-training of deep bidirectional transformers for language understanding[J].ar-Xiv:1810.04805,2018. [27]CUI Y,CHE W,LIU T,et al.Pre-training with whole word masking for chinese bert[J].arXiv:1906.08101,2019. [28]YANG Z,DAI Z,YANG Y,et al.XLNet:Generalized Autoregressive Pretraining for Language Understanding[J].Advances in Neural Information Processing Systems,2019,32:5753-5763. |
| [1] | 陈志强, 韩萌, 李慕航, 武红鑫, 张喜龙. 数据流概念漂移处理方法研究综述 Survey of Concept Drift Handling Methods in Data Streams 计算机科学, 2022, 49(9): 14-32. https://doi.org/10.11896/jsjkx.210700112 |
| [2] | 曹晓雯, 梁美玉, 鲁康康. 基于细粒度语义推理的跨媒体双路对抗哈希学习模型 Fine-grained Semantic Reasoning Based Cross-media Dual-way Adversarial Hashing Learning Model 计算机科学, 2022, 49(9): 123-131. https://doi.org/10.11896/jsjkx.220600011 |
| [3] | 周旭, 钱胜胜, 李章明, 方全, 徐常胜. 基于对偶变分多模态注意力网络的不完备社会事件分类方法 Dual Variational Multi-modal Attention Network for Incomplete Social Event Classification 计算机科学, 2022, 49(9): 132-138. https://doi.org/10.11896/jsjkx.220600022 |
| [4] | 周乐员, 张剑华, 袁甜甜, 陈胜勇. 多层注意力机制融合的序列到序列中国连续手语识别和翻译 Sequence-to-Sequence Chinese Continuous Sign Language Recognition and Translation with Multi- layer Attention Mechanism Fusion 计算机科学, 2022, 49(9): 155-161. https://doi.org/10.11896/jsjkx.210800026 |
| [5] | 吴子仪, 李邵梅, 姜梦函, 张建朋. 基于自注意力模型的本体对齐方法 Ontology Alignment Method Based on Self-attention 计算机科学, 2022, 49(9): 215-220. https://doi.org/10.11896/jsjkx.210700190 |
| [6] | 武红鑫, 韩萌, 陈志强, 张喜龙, 李慕航. 监督和半监督学习下的多标签分类综述 Survey of Multi-label Classification Based on Supervised and Semi-supervised Learning 计算机科学, 2022, 49(8): 12-25. https://doi.org/10.11896/jsjkx.210700111 |
| [7] | 汪鸣, 彭舰, 黄飞虎. 基于多时间尺度时空图网络的交通流量预测模型 Multi-time Scale Spatial-Temporal Graph Neural Network for Traffic Flow Prediction 计算机科学, 2022, 49(8): 40-48. https://doi.org/10.11896/jsjkx.220100188 |
| [8] | 秦琪琦, 张月琴, 王润泽, 张泽华. 基于知识图谱的层次粒化推荐方法 Hierarchical Granulation Recommendation Method Based on Knowledge Graph 计算机科学, 2022, 49(8): 64-69. https://doi.org/10.11896/jsjkx.210600111 |
| [9] | 魏恺轩, 付莹. 基于重参数化多尺度融合网络的高效极暗光原始图像降噪 Re-parameterized Multi-scale Fusion Network for Efficient Extreme Low-light Raw Denoising 计算机科学, 2022, 49(8): 120-126. https://doi.org/10.11896/jsjkx.220200179 |
| [10] | 王灿, 刘永坚, 解庆, 马艳春. 基于软标签和样本权重优化的Anchor Free目标检测算法 Anchor Free Object Detection Algorithm Based on Soft Label and Sample Weight Optimization 计算机科学, 2022, 49(8): 157-164. https://doi.org/10.11896/jsjkx.210600240 |
| [11] | 沈祥培, 丁彦蕊. 多检测器融合的深度相关滤波视频多目标跟踪算法 Multi-detector Fusion-based Depth Correlation Filtering Video Multi-target Tracking Algorithm 计算机科学, 2022, 49(8): 184-190. https://doi.org/10.11896/jsjkx.210600004 |
| [12] | 檀莹莹, 王俊丽, 张超波. 基于图卷积神经网络的文本分类方法研究综述 Review of Text Classification Methods Based on Graph Convolutional Network 计算机科学, 2022, 49(8): 205-216. https://doi.org/10.11896/jsjkx.210800064 |
| [13] | 闫佳丹, 贾彩燕. 基于双图神经网络信息融合的文本分类方法 Text Classification Method Based on Information Fusion of Dual-graph Neural Network 计算机科学, 2022, 49(8): 230-236. https://doi.org/10.11896/jsjkx.210600042 |
| [14] | 郝志荣, 陈龙, 黄嘉成. 面向文本分类的类别区分式通用对抗攻击方法 Class Discriminative Universal Adversarial Attack for Text Classification 计算机科学, 2022, 49(8): 323-329. https://doi.org/10.11896/jsjkx.220200077 |
| [15] | 张源, 康乐, 宫朝辉, 张志鸿. 基于Bi-LSTM的期货市场关联交易行为检测方法 Related Transaction Behavior Detection in Futures Market Based on Bi-LSTM 计算机科学, 2022, 49(7): 31-39. https://doi.org/10.11896/jsjkx.210400304 |
|
||
首页