计算机科学 ›› 2022, Vol. 49 ›› Issue (11A): 211000205-9.doi: 10.11896/jsjkx.211000205
孙开伟1, 郭豪1, 曾雅苑1, 方阳1, 刘期烈2
SUN Kai-wei1, GUO Hao1, ZENG Ya-yuan1, FANG Yang1, LIU Qi-lie2
摘要: 多目标回归(Multi-target Regression,MTR)是一种同时预测多个相互关联的连续型输出目标的机器学习问题。在多目标回归中,多个输出目标共享同一个特征表示,其主要挑战在于如何有效地发掘和利用输出目标之间的关联,以提高所有输出目标的预测准确性。文中提出了一种基于超网络的多目标回归方法(Multi-target Regression Method based on Hypernetwork,MTR-HN)。首先采用k-means算法对每个连续型输出目标进行一维聚类,然后根据聚类结果将多目标回归问题转化成多类别多标签分类问题,最后采用超网络模型对多类别多标签分类问题进行建模,构建最终的多目标回归预测模型。MTR-HN方法的优点在于:1)对输出空间离散化,能够降低模型过拟合的风险;2)采用超网络模型,能更有效地对输出目标之间的关联进行建模。在18个多目标回归数据集上进行的对比实验表明,文中提出的MTR-HN方法能够取得比现有方法更高的预测准确性。
中图分类号:
[1]SUN K W,DENG M X,LI H,et al.Learning local instance correlations for multi-target regression [J].Applied Intelligence,2021,51:6124-6135. [2]XIAO X,XU Y.Multi-target regression via self-parameterizedLasso and refactored target space [J].Applied Intelligence,2021,51:6743-6751. [3]WANG Y,WIPF D P,LING Q,et al.Multitask learning forsubspace segmentation [C]//Proceedings of the 32nd International Conference on Machine Learning(ICML).2015:1209-1217. [4]XIONG T,BAO Y,HU Z.Multiple-output support vector regression with a firefly algorithm for interval-valued stock price index forecasting [J].Knowledge-Based Systems,2014,55:87-100. [5]HADAVANDI E,SHAHRABI J,SHAMSHIRBAND S.A novel Boosted-neural network ensemble for modeling multi-target regression problems [J].Engineering Applications of Artificial Intelligence,2015,45:204-219. [6]YAN Y,RICCI E,SUBRAMANIAN R,et al.A multi-tasklearning framework for head pose estimation under target motion [J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2016,38(6):1070-1083. [7]WANG X,ZHEN X,LI Q,et al.Cognitive assessment prediction in Alzheimer’s disease by multi-layer multitarget regression [J].Neuroinformatics,2018,16:285-294. [8]ZHEN X,YU M,HE X,et al.Multi-target regression via robust low-rank learning [J].IEEE Transactions on Pattern Analysis &Machine Intelligence,2018,40(2):497-504. [9]SPYROMITROS-XIOUFIS E,TSOUMAKAS G,GROVES W,et al.Multi-target regression via input space expansion:treating targets as inputs [J].Machine Learning,2016,104(1):55-98. [10]YUAN H,ZHENG J,LAI L,et al.Sparse structural feature selection for multitarget regression [J].Knowledge-Based Systems,2018,160:200-209. [11]ZHANG M L,ZHOU Z H.A Review on Multi-Label Learning Algorithms [J].IEEE Transactions on Knowledge & Data Engineering,2014,26(8):1819-1837. [12]LV J,WU T,PENG C,et al.Compact Learning for Multi-Label Classification [J].Pattern Recognition,2021,113:107833. [13]SPYROMITROS-XIOUFIS E,SECHIDIS K,VLAHAVAS I.Multi-target regression via output space quantization [C]//International Joint Conference on Neural Networks.2020. [14]ZHANG B T.Hypernetworks:A Molecular Evolutionary Architecture for Cognitive Learning and Memory[J].IEEE Computational Intelligence Magazine,2008,3(3):49-63. [15]SUN K W,LEE C,XIE X X.MLHN:A Hypernetwork Model for Multi-Label Classification [J].International Journal of Pattern Recognition and Artificial Intelligence,2015,29(6):1550020.1. [16]SUN K W,LEE C,WANG J.Multilabel Classification via Co-Evolutionary Multilabel Hypernetwork [J].IEEE Transactions on Knowledge & Data Engineering,2016,28(9):2438-2451. [17]SUN K W,LEE C.Addressing class-imbalance in multi-label learning via two-stage multi-label hypernetwork [J].Neurocomputing,2017,266(29):375-389. [18]XU D,SHI Y,TSANG I W,et al.A Survey on Multi-outputLearning [J].IEEE Transactions on Neural Networks and Learning Systems,2020,31(7):2409-2429 [19]ZHANG Y,YANG Q.A Survey on Multi-Task Learning [J].IEEE Transactions on Knowledge and Data Engineering,2021,arXiv:1707.08114. [20]TSOUMAKAS G,KATAKIS I.Multi-Label Classification:An Overview [J].International Journal of Data Warehousing and Mining,2009,3(3):1-13. [21]BRINKER K,MENCIA E L,FUERNKRANZ J,et al.Multilabel classification via calibrated label ranking [J].Machine Learning,2008,73(2):133-153. [22]TSOUMAKAS G,KATAKIS I,VLAHAVAS I.Randomk-La-belsets for Multi-Label Classification [J].IEEE Transactions on Knowledge and Data Engineering,2011,23(7):1079-1089. [23]READ J,PFAHRINGER B,HOLMES G,et al.Classifier chains for multi-label classification [J].Machine Learning,2011,85(3):333-359. [24]ZHANG M L,ZHOU Z H.ML-KNN:A lazy learning approach to multi-label learning [J].Pattern Recognition,2007,40(7):2038-2048. [25]ZHANG M L,ZHOU Z H.Multilabel Neural Networks withApplications to Functional Genomics and Text Categorization [J].IEEE Transactions on Knowledge & Data Engineering,2006,18(10):1338-1351. [26]WANG H,XU Y.Sparse elastic net multi-label rank supportvector machine with pinball loss and its applications [J].Applied Soft Computing,2021,104(9):107232. [27]ARGYRIOU A,EVGENIOU T,PONTIL M.Convex multi-task feature learning [J].Machine Learning,2008,73(3):243-272. [28]SU F,SHANG H Y,WANG J Y.Low-Rank Deep Convolutional Neural Network for Multi-Task Learning[J].Computational Intelligence and Neuroscience,2019,2019:1-10. [29]WANG D,NIE F,HUANG H.Learning Task Relational Structure for Multi-task Feature Learning [C]//IEEE International Conference on Data Mining.2016:1239-1244. [30]BICKEL S,BOGOJESKA J,LENGAUER T,et al.Multi-tasklearning for HIV therapy screening [C]//Proceedings of the 25th International Conference on Machine Learning.2008:56-63. [31]TSOUMAKAS G,SPYROMITROS X E,VREKOU A,et al.Multi-target regression via random linear target combinations [C]//Joint European Conference on Machine Learning and Know-ledge Discovery in Databases.2014:225-240. [32]BRESKVAR M,KOCEV D,DZEROSKI S.Ensembles formulti-target regression with random output selections [J].Machine Learning,2018,107(11):1673-1709. [33]ZHANG Y,YEUNG D Y.A convex formulation for learningtask relationships in multi-task learning[C]//Proceedings of UAI.2010:73-742. [34]TODOROVSKI L,BLOCKEEL H,DZEROSKI S.Ranking with Predictive Clustering Trees [C]//European Conference on Machine Learning.2002:444-455. [35]OSOJNIK A,PANOV P,DZEROSKI S.Tree-based methods for online multi-target regression [J].Journal of Intelligent Information Systems,2018,50:315-339 [36]LEVATIC J,CECI M,KOCEV D,et al.Self-training for multi-target regression with tree ensembles [J].Knowledge-Based Systems,2017,123:41-60. [37]LIU H T,LI H,WANG J,et al.Multi-Target Regression via Sparse Integration and Label-Specific Features [J].Acta Electronica Sinica,2020,48(5):906-913. [38]WANG J,GAO X R,ZHANG R,et al.Multi-Target Regression via Specific Features and Inter-Target Correlations [J].Acta Electronica Sinica,2020,48(11):2092-2100. [39]WANG J,CHEN Z L,SUN K W,et al.Multi-target regression via target specific features [J].Knowledge-Based Systems,2019,170(15):70-78. [40]DEMSAR J.Statistical comparisons of classifiers over multiple data sets [J].Journal of Machine Learning Research,2006,7:1-30. |
[1] | 柴慧敏, 张勇, 方敏. 基于特征相似度聚类的空中目标分群方法 Aerial Target Grouping Method Based on Feature Similarity Clustering 计算机科学, 2022, 49(9): 70-75. https://doi.org/10.11896/jsjkx.210800203 |
[2] | 鲁晨阳, 邓苏, 马武彬, 吴亚辉, 周浩浩. 基于分层抽样优化的面向异构客户端的联邦学习 Federated Learning Based on Stratified Sampling Optimization for Heterogeneous Clients 计算机科学, 2022, 49(9): 183-193. https://doi.org/10.11896/jsjkx.220500263 |
[3] | 冷典典, 杜鹏, 陈建廷, 向阳. 面向自动化集装箱码头的AGV行驶时间估计 Automated Container Terminal Oriented Travel Time Estimation of AGV 计算机科学, 2022, 49(9): 208-214. https://doi.org/10.11896/jsjkx.210700028 |
[4] | 宁晗阳, 马苗, 杨波, 刘士昌. 密码学智能化研究进展与分析 Research Progress and Analysis on Intelligent Cryptology 计算机科学, 2022, 49(9): 288-296. https://doi.org/10.11896/jsjkx.220300053 |
[5] | 何强, 尹震宇, 黄敏, 王兴伟, 王源田, 崔硕, 赵勇. 基于大数据的进化网络影响力分析研究综述 Survey of Influence Analysis of Evolutionary Network Based on Big Data 计算机科学, 2022, 49(8): 1-11. https://doi.org/10.11896/jsjkx.210700240 |
[6] | 武红鑫, 韩萌, 陈志强, 张喜龙, 李慕航. 监督和半监督学习下的多标签分类综述 Survey of Multi-label Classification Based on Supervised and Semi-supervised Learning 计算机科学, 2022, 49(8): 12-25. https://doi.org/10.11896/jsjkx.210700111 |
[7] | 李瑶, 李涛, 李埼钒, 梁家瑞, Ibegbu Nnamdi JULIAN, 陈俊杰, 郭浩. 基于多尺度的稀疏脑功能超网络构建及多特征融合分类研究 Construction and Multi-feature Fusion Classification Research Based on Multi-scale Sparse Brain Functional Hyper-network 计算机科学, 2022, 49(8): 257-266. https://doi.org/10.11896/jsjkx.210600094 |
[8] | 张光华, 高天娇, 陈振国, 于乃文. 基于N-Gram静态分析技术的恶意软件分类研究 Study on Malware Classification Based on N-Gram Static Analysis Technology 计算机科学, 2022, 49(8): 336-343. https://doi.org/10.11896/jsjkx.210900203 |
[9] | 陈明鑫, 张钧波, 李天瑞. 联邦学习攻防研究综述 Survey on Attacks and Defenses in Federated Learning 计算机科学, 2022, 49(7): 310-323. https://doi.org/10.11896/jsjkx.211000079 |
[10] | 刘丽, 李仁发. 医疗CPS协作网络控制策略优化 Control Strategy Optimization of Medical CPS Cooperative Network 计算机科学, 2022, 49(6A): 39-43. https://doi.org/10.11896/jsjkx.210300230 |
[11] | 李亚茹, 张宇来, 王佳晨. 面向超参数估计的贝叶斯优化方法综述 Survey on Bayesian Optimization Methods for Hyper-parameter Tuning 计算机科学, 2022, 49(6A): 86-92. https://doi.org/10.11896/jsjkx.210300208 |
[12] | 赵璐, 袁立明, 郝琨. 多示例学习算法综述 Review of Multi-instance Learning Algorithms 计算机科学, 2022, 49(6A): 93-99. https://doi.org/10.11896/jsjkx.210500047 |
[13] | 鲁晨阳, 邓苏, 马武彬, 吴亚辉, 周浩浩. 基于DBSCAN聚类的集群联邦学习方法 Clustered Federated Learning Methods Based on DBSCAN Clustering 计算机科学, 2022, 49(6A): 232-237. https://doi.org/10.11896/jsjkx.211100059 |
[14] | 郁舒昊, 周辉, 叶春杨, 王太正. SDFA:基于多特征融合的船舶轨迹聚类方法研究 SDFA:Study on Ship Trajectory Clustering Method Based on Multi-feature Fusion 计算机科学, 2022, 49(6A): 256-260. https://doi.org/10.11896/jsjkx.211100253 |
[15] | 毛森林, 夏镇, 耿新宇, 陈剑辉, 蒋宏霞. 基于密度敏感距离和模糊划分的改进FCM算法 FCM Algorithm Based on Density Sensitive Distance and Fuzzy Partition 计算机科学, 2022, 49(6A): 285-290. https://doi.org/10.11896/jsjkx.210700042 |
|