计算机科学 ›› 2022, Vol. 49 ›› Issue (11A): 211000205-9.doi: 10.11896/jsjkx.211000205

• 人工智能 • 上一篇    下一篇

一种基于超网络的多目标回归方法

孙开伟1, 郭豪1, 曾雅苑1, 方阳1, 刘期烈2   

  1. 1 重庆邮电大学数据工程与可视计算重点实验室 重庆 400065
    2 重庆邮电大学通信与信息工程学院 重庆 400065
  • 出版日期:2022-11-10 发布日期:2022-11-21
  • 通讯作者: 孙开伟 (sunkw@cqupt.edu.cn)
  • 基金资助:
    国家自然科学基金(61806033);重庆市自然科学基金面上项目(cstc2019jcyj-msxmX0021);重庆市科技创新领军人才支持计划(CSTCCXLJRC201908);重庆市自然基金重点项目(cstc2019jcyj-zdxm0008);重庆市教委重点项目(KJZD-K201900605);重庆市教委“成渝地区双城经济圈建设”科技创新项目(KJCXZD2020027)

Multi-target Regression Method Based on Hypernetwork

SUN Kai-wei1, GUO Hao1, ZENG Ya-yuan1, FANG Yang1, LIU Qi-lie2   

  1. 1 Key Laboratory of Data Engineering and Visual Computing,Chongqing University of Posts and Telecommunications,Chongqing 400065,China
    2 School of Communication and Information Engineering,Chongqing University of Posts and Telecommunications,Chongqing 400065,China
  • Online:2022-11-10 Published:2022-11-21
  • About author:SUN Kai-wei,born in 1987,Ph.D,associate professor,master supervisor,is a member of China Computer Federation.His main research interests include machine learning,natural language processing and big data analysis.
  • Supported by:
    National Natural Science Foundation of China(61806033),Natural Science Foundation of Chongqing(cstc2019jcyj-msxmX0021),Chongqing Science and Technology Innovation Leading Talent Support Program(CSTCCXLJRC201908),Basic and Advanced Research Projects of CSTC(cstc2019jcyj-zdxmX0008),Science and Technology Research Program of Chongqing Municipal Education Commission(KJZD-K201900605) and Science Innovation Program of Chengdu-Chongqing Economic Circle in Southwest China(KJCXZD2020027).

摘要: 多目标回归(Multi-target Regression,MTR)是一种同时预测多个相互关联的连续型输出目标的机器学习问题。在多目标回归中,多个输出目标共享同一个特征表示,其主要挑战在于如何有效地发掘和利用输出目标之间的关联,以提高所有输出目标的预测准确性。文中提出了一种基于超网络的多目标回归方法(Multi-target Regression Method based on Hypernetwork,MTR-HN)。首先采用k-means算法对每个连续型输出目标进行一维聚类,然后根据聚类结果将多目标回归问题转化成多类别多标签分类问题,最后采用超网络模型对多类别多标签分类问题进行建模,构建最终的多目标回归预测模型。MTR-HN方法的优点在于:1)对输出空间离散化,能够降低模型过拟合的风险;2)采用超网络模型,能更有效地对输出目标之间的关联进行建模。在18个多目标回归数据集上进行的对比实验表明,文中提出的MTR-HN方法能够取得比现有方法更高的预测准确性。

关键词: 机器学习, 多目标回归, 聚类, 多标签分类, 超网络

Abstract: Multi-target regression(MTR) is a kind of machine learning problem which predicts multiple relevant continuous output targets simultaneously.In MTR,multiple output targets share the same input feature representation,the main challenge of MTR lies in how to effectively explore and utilize the correlations among multiple output targets to improve the prediction accuracy of all output targets.In this paper,a multi-target regression method based on hypernetwork(MTR-HN) is proposed.First,the k-means method is applied to each output target to divide it into multiple clusters.Then,according to the clustering results,MTR problem is transformed into a multi-class multi-label classification problem.Finally,hypernetwork model is utilized to model the multi-class multi-label classification problem,and the final prediction model for MTR is built based on hypernetwork.The main merits of MTR-HN lie in:1)discretizing the output space,can reduce the risk of overfitting;2)hypernetwork can model the inter-target correlations more effectively.Comparative experiments on 18 multi-target regression datasets show that the proposed MTR-HN achieves better prediction performance than existing state-of-the-art multi-target regression methods.

Key words: Machine learning, Multi-target regression, Clustering, Multi-label classification, Hypernetwork

中图分类号: 

  • TP181
[1]SUN K W,DENG M X,LI H,et al.Learning local instance correlations for multi-target regression [J].Applied Intelligence,2021,51:6124-6135.
[2]XIAO X,XU Y.Multi-target regression via self-parameterizedLasso and refactored target space [J].Applied Intelligence,2021,51:6743-6751.
[3]WANG Y,WIPF D P,LING Q,et al.Multitask learning forsubspace segmentation [C]//Proceedings of the 32nd International Conference on Machine Learning(ICML).2015:1209-1217.
[4]XIONG T,BAO Y,HU Z.Multiple-output support vector regression with a firefly algorithm for interval-valued stock price index forecasting [J].Knowledge-Based Systems,2014,55:87-100.
[5]HADAVANDI E,SHAHRABI J,SHAMSHIRBAND S.A novel Boosted-neural network ensemble for modeling multi-target regression problems [J].Engineering Applications of Artificial Intelligence,2015,45:204-219.
[6]YAN Y,RICCI E,SUBRAMANIAN R,et al.A multi-tasklearning framework for head pose estimation under target motion [J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2016,38(6):1070-1083.
[7]WANG X,ZHEN X,LI Q,et al.Cognitive assessment prediction in Alzheimer’s disease by multi-layer multitarget regression [J].Neuroinformatics,2018,16:285-294.
[8]ZHEN X,YU M,HE X,et al.Multi-target regression via robust low-rank learning [J].IEEE Transactions on Pattern Analysis &Machine Intelligence,2018,40(2):497-504.
[9]SPYROMITROS-XIOUFIS E,TSOUMAKAS G,GROVES W,et al.Multi-target regression via input space expansion:treating targets as inputs [J].Machine Learning,2016,104(1):55-98.
[10]YUAN H,ZHENG J,LAI L,et al.Sparse structural feature selection for multitarget regression [J].Knowledge-Based Systems,2018,160:200-209.
[11]ZHANG M L,ZHOU Z H.A Review on Multi-Label Learning Algorithms [J].IEEE Transactions on Knowledge & Data Engineering,2014,26(8):1819-1837.
[12]LV J,WU T,PENG C,et al.Compact Learning for Multi-Label Classification [J].Pattern Recognition,2021,113:107833.
[13]SPYROMITROS-XIOUFIS E,SECHIDIS K,VLAHAVAS I.Multi-target regression via output space quantization [C]//International Joint Conference on Neural Networks.2020.
[14]ZHANG B T.Hypernetworks:A Molecular Evolutionary Architecture for Cognitive Learning and Memory[J].IEEE Computational Intelligence Magazine,2008,3(3):49-63.
[15]SUN K W,LEE C,XIE X X.MLHN:A Hypernetwork Model for Multi-Label Classification [J].International Journal of Pattern Recognition and Artificial Intelligence,2015,29(6):1550020.1.
[16]SUN K W,LEE C,WANG J.Multilabel Classification via Co-Evolutionary Multilabel Hypernetwork [J].IEEE Transactions on Knowledge & Data Engineering,2016,28(9):2438-2451.
[17]SUN K W,LEE C.Addressing class-imbalance in multi-label learning via two-stage multi-label hypernetwork [J].Neurocomputing,2017,266(29):375-389.
[18]XU D,SHI Y,TSANG I W,et al.A Survey on Multi-outputLearning [J].IEEE Transactions on Neural Networks and Learning Systems,2020,31(7):2409-2429
[19]ZHANG Y,YANG Q.A Survey on Multi-Task Learning [J].IEEE Transactions on Knowledge and Data Engineering,2021,arXiv:1707.08114.
[20]TSOUMAKAS G,KATAKIS I.Multi-Label Classification:An Overview [J].International Journal of Data Warehousing and Mining,2009,3(3):1-13.
[21]BRINKER K,MENCIA E L,FUERNKRANZ J,et al.Multilabel classification via calibrated label ranking [J].Machine Learning,2008,73(2):133-153.
[22]TSOUMAKAS G,KATAKIS I,VLAHAVAS I.Randomk-La-belsets for Multi-Label Classification [J].IEEE Transactions on Knowledge and Data Engineering,2011,23(7):1079-1089.
[23]READ J,PFAHRINGER B,HOLMES G,et al.Classifier chains for multi-label classification [J].Machine Learning,2011,85(3):333-359.
[24]ZHANG M L,ZHOU Z H.ML-KNN:A lazy learning approach to multi-label learning [J].Pattern Recognition,2007,40(7):2038-2048.
[25]ZHANG M L,ZHOU Z H.Multilabel Neural Networks withApplications to Functional Genomics and Text Categorization [J].IEEE Transactions on Knowledge & Data Engineering,2006,18(10):1338-1351.
[26]WANG H,XU Y.Sparse elastic net multi-label rank supportvector machine with pinball loss and its applications [J].Applied Soft Computing,2021,104(9):107232.
[27]ARGYRIOU A,EVGENIOU T,PONTIL M.Convex multi-task feature learning [J].Machine Learning,2008,73(3):243-272.
[28]SU F,SHANG H Y,WANG J Y.Low-Rank Deep Convolutional Neural Network for Multi-Task Learning[J].Computational Intelligence and Neuroscience,2019,2019:1-10.
[29]WANG D,NIE F,HUANG H.Learning Task Relational Structure for Multi-task Feature Learning [C]//IEEE International Conference on Data Mining.2016:1239-1244.
[30]BICKEL S,BOGOJESKA J,LENGAUER T,et al.Multi-tasklearning for HIV therapy screening [C]//Proceedings of the 25th International Conference on Machine Learning.2008:56-63.
[31]TSOUMAKAS G,SPYROMITROS X E,VREKOU A,et al.Multi-target regression via random linear target combinations [C]//Joint European Conference on Machine Learning and Know-ledge Discovery in Databases.2014:225-240.
[32]BRESKVAR M,KOCEV D,DZEROSKI S.Ensembles formulti-target regression with random output selections [J].Machine Learning,2018,107(11):1673-1709.
[33]ZHANG Y,YEUNG D Y.A convex formulation for learningtask relationships in multi-task learning[C]//Proceedings of UAI.2010:73-742.
[34]TODOROVSKI L,BLOCKEEL H,DZEROSKI S.Ranking with Predictive Clustering Trees [C]//European Conference on Machine Learning.2002:444-455.
[35]OSOJNIK A,PANOV P,DZEROSKI S.Tree-based methods for online multi-target regression [J].Journal of Intelligent Information Systems,2018,50:315-339
[36]LEVATIC J,CECI M,KOCEV D,et al.Self-training for multi-target regression with tree ensembles [J].Knowledge-Based Systems,2017,123:41-60.
[37]LIU H T,LI H,WANG J,et al.Multi-Target Regression via Sparse Integration and Label-Specific Features [J].Acta Electronica Sinica,2020,48(5):906-913.
[38]WANG J,GAO X R,ZHANG R,et al.Multi-Target Regression via Specific Features and Inter-Target Correlations [J].Acta Electronica Sinica,2020,48(11):2092-2100.
[39]WANG J,CHEN Z L,SUN K W,et al.Multi-target regression via target specific features [J].Knowledge-Based Systems,2019,170(15):70-78.
[40]DEMSAR J.Statistical comparisons of classifiers over multiple data sets [J].Journal of Machine Learning Research,2006,7:1-30.
[1] 柴慧敏, 张勇, 方敏.
基于特征相似度聚类的空中目标分群方法
Aerial Target Grouping Method Based on Feature Similarity Clustering
计算机科学, 2022, 49(9): 70-75. https://doi.org/10.11896/jsjkx.210800203
[2] 鲁晨阳, 邓苏, 马武彬, 吴亚辉, 周浩浩.
基于分层抽样优化的面向异构客户端的联邦学习
Federated Learning Based on Stratified Sampling Optimization for Heterogeneous Clients
计算机科学, 2022, 49(9): 183-193. https://doi.org/10.11896/jsjkx.220500263
[3] 冷典典, 杜鹏, 陈建廷, 向阳.
面向自动化集装箱码头的AGV行驶时间估计
Automated Container Terminal Oriented Travel Time Estimation of AGV
计算机科学, 2022, 49(9): 208-214. https://doi.org/10.11896/jsjkx.210700028
[4] 宁晗阳, 马苗, 杨波, 刘士昌.
密码学智能化研究进展与分析
Research Progress and Analysis on Intelligent Cryptology
计算机科学, 2022, 49(9): 288-296. https://doi.org/10.11896/jsjkx.220300053
[5] 何强, 尹震宇, 黄敏, 王兴伟, 王源田, 崔硕, 赵勇.
基于大数据的进化网络影响力分析研究综述
Survey of Influence Analysis of Evolutionary Network Based on Big Data
计算机科学, 2022, 49(8): 1-11. https://doi.org/10.11896/jsjkx.210700240
[6] 武红鑫, 韩萌, 陈志强, 张喜龙, 李慕航.
监督和半监督学习下的多标签分类综述
Survey of Multi-label Classification Based on Supervised and Semi-supervised Learning
计算机科学, 2022, 49(8): 12-25. https://doi.org/10.11896/jsjkx.210700111
[7] 李瑶, 李涛, 李埼钒, 梁家瑞, Ibegbu Nnamdi JULIAN, 陈俊杰, 郭浩.
基于多尺度的稀疏脑功能超网络构建及多特征融合分类研究
Construction and Multi-feature Fusion Classification Research Based on Multi-scale Sparse Brain Functional Hyper-network
计算机科学, 2022, 49(8): 257-266. https://doi.org/10.11896/jsjkx.210600094
[8] 张光华, 高天娇, 陈振国, 于乃文.
基于N-Gram静态分析技术的恶意软件分类研究
Study on Malware Classification Based on N-Gram Static Analysis Technology
计算机科学, 2022, 49(8): 336-343. https://doi.org/10.11896/jsjkx.210900203
[9] 陈明鑫, 张钧波, 李天瑞.
联邦学习攻防研究综述
Survey on Attacks and Defenses in Federated Learning
计算机科学, 2022, 49(7): 310-323. https://doi.org/10.11896/jsjkx.211000079
[10] 刘丽, 李仁发.
医疗CPS协作网络控制策略优化
Control Strategy Optimization of Medical CPS Cooperative Network
计算机科学, 2022, 49(6A): 39-43. https://doi.org/10.11896/jsjkx.210300230
[11] 李亚茹, 张宇来, 王佳晨.
面向超参数估计的贝叶斯优化方法综述
Survey on Bayesian Optimization Methods for Hyper-parameter Tuning
计算机科学, 2022, 49(6A): 86-92. https://doi.org/10.11896/jsjkx.210300208
[12] 赵璐, 袁立明, 郝琨.
多示例学习算法综述
Review of Multi-instance Learning Algorithms
计算机科学, 2022, 49(6A): 93-99. https://doi.org/10.11896/jsjkx.210500047
[13] 鲁晨阳, 邓苏, 马武彬, 吴亚辉, 周浩浩.
基于DBSCAN聚类的集群联邦学习方法
Clustered Federated Learning Methods Based on DBSCAN Clustering
计算机科学, 2022, 49(6A): 232-237. https://doi.org/10.11896/jsjkx.211100059
[14] 郁舒昊, 周辉, 叶春杨, 王太正.
SDFA:基于多特征融合的船舶轨迹聚类方法研究
SDFA:Study on Ship Trajectory Clustering Method Based on Multi-feature Fusion
计算机科学, 2022, 49(6A): 256-260. https://doi.org/10.11896/jsjkx.211100253
[15] 毛森林, 夏镇, 耿新宇, 陈剑辉, 蒋宏霞.
基于密度敏感距离和模糊划分的改进FCM算法
FCM Algorithm Based on Density Sensitive Distance and Fuzzy Partition
计算机科学, 2022, 49(6A): 285-290. https://doi.org/10.11896/jsjkx.210700042
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!