Computer Science

Computer Science ›› 2022, Vol. 49 ›› Issue (11A): 211000205-9.doi: 10.11896/jsjkx.211000205

• Artificial Intelligence • Previous Articles     Next Articles

Multi-target Regression Method Based on Hypernetwork

SUN Kai-wei1, GUO Hao1, ZENG Ya-yuan1, FANG Yang1, LIU Qi-lie2   

  1. 1 Key Laboratory of Data Engineering and Visual Computing,Chongqing University of Posts and Telecommunications,Chongqing 400065,China
    2 School of Communication and Information Engineering,Chongqing University of Posts and Telecommunications,Chongqing 400065,China
  • Online:2022-11-10 Published:2022-11-21
  • About author:SUN Kai-wei,born in 1987,Ph.D,associate professor,master supervisor,is a member of China Computer Federation.His main research interests include machine learning,natural language processing and big data analysis.
  • Supported by:
    National Natural Science Foundation of China(61806033),Natural Science Foundation of Chongqing(cstc2019jcyj-msxmX0021),Chongqing Science and Technology Innovation Leading Talent Support Program(CSTCCXLJRC201908),Basic and Advanced Research Projects of CSTC(cstc2019jcyj-zdxmX0008),Science and Technology Research Program of Chongqing Municipal Education Commission(KJZD-K201900605) and Science Innovation Program of Chengdu-Chongqing Economic Circle in Southwest China(KJCXZD2020027).

PDF (PC)

366

Abstract: Multi-target regression(MTR) is a kind of machine learning problem which predicts multiple relevant continuous output targets simultaneously.In MTR,multiple output targets share the same input feature representation,the main challenge of MTR lies in how to effectively explore and utilize the correlations among multiple output targets to improve the prediction accuracy of all output targets.In this paper,a multi-target regression method based on hypernetwork(MTR-HN) is proposed.First,the k-means method is applied to each output target to divide it into multiple clusters.Then,according to the clustering results,MTR problem is transformed into a multi-class multi-label classification problem.Finally,hypernetwork model is utilized to model the multi-class multi-label classification problem,and the final prediction model for MTR is built based on hypernetwork.The main merits of MTR-HN lie in:1)discretizing the output space,can reduce the risk of overfitting;2)hypernetwork can model the inter-target correlations more effectively.Comparative experiments on 18 multi-target regression datasets show that the proposed MTR-HN achieves better prediction performance than existing state-of-the-art multi-target regression methods.

Key words: Machine learning, Multi-target regression, Clustering, Multi-label classification, Hypernetwork

CLC Number: 

  • TP181
[1]SUN K W,DENG M X,LI H,et al.Learning local instance correlations for multi-target regression [J].Applied Intelligence,2021,51:6124-6135.
[2]XIAO X,XU Y.Multi-target regression via self-parameterizedLasso and refactored target space [J].Applied Intelligence,2021,51:6743-6751.
[3]WANG Y,WIPF D P,LING Q,et al.Multitask learning forsubspace segmentation [C]//Proceedings of the 32nd International Conference on Machine Learning(ICML).2015:1209-1217.
[4]XIONG T,BAO Y,HU Z.Multiple-output support vector regression with a firefly algorithm for interval-valued stock price index forecasting [J].Knowledge-Based Systems,2014,55:87-100.
[5]HADAVANDI E,SHAHRABI J,SHAMSHIRBAND S.A novel Boosted-neural network ensemble for modeling multi-target regression problems [J].Engineering Applications of Artificial Intelligence,2015,45:204-219.
[6]YAN Y,RICCI E,SUBRAMANIAN R,et al.A multi-tasklearning framework for head pose estimation under target motion [J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2016,38(6):1070-1083.
[7]WANG X,ZHEN X,LI Q,et al.Cognitive assessment prediction in Alzheimer’s disease by multi-layer multitarget regression [J].Neuroinformatics,2018,16:285-294.
[8]ZHEN X,YU M,HE X,et al.Multi-target regression via robust low-rank learning [J].IEEE Transactions on Pattern Analysis &Machine Intelligence,2018,40(2):497-504.
[9]SPYROMITROS-XIOUFIS E,TSOUMAKAS G,GROVES W,et al.Multi-target regression via input space expansion:treating targets as inputs [J].Machine Learning,2016,104(1):55-98.
[10]YUAN H,ZHENG J,LAI L,et al.Sparse structural feature selection for multitarget regression [J].Knowledge-Based Systems,2018,160:200-209.
[11]ZHANG M L,ZHOU Z H.A Review on Multi-Label Learning Algorithms [J].IEEE Transactions on Knowledge & Data Engineering,2014,26(8):1819-1837.
[12]LV J,WU T,PENG C,et al.Compact Learning for Multi-Label Classification [J].Pattern Recognition,2021,113:107833.
[13]SPYROMITROS-XIOUFIS E,SECHIDIS K,VLAHAVAS I.Multi-target regression via output space quantization [C]//International Joint Conference on Neural Networks.2020.
[14]ZHANG B T.Hypernetworks:A Molecular Evolutionary Architecture for Cognitive Learning and Memory[J].IEEE Computational Intelligence Magazine,2008,3(3):49-63.
[15]SUN K W,LEE C,XIE X X.MLHN:A Hypernetwork Model for Multi-Label Classification [J].International Journal of Pattern Recognition and Artificial Intelligence,2015,29(6):1550020.1.
[16]SUN K W,LEE C,WANG J.Multilabel Classification via Co-Evolutionary Multilabel Hypernetwork [J].IEEE Transactions on Knowledge & Data Engineering,2016,28(9):2438-2451.
[17]SUN K W,LEE C.Addressing class-imbalance in multi-label learning via two-stage multi-label hypernetwork [J].Neurocomputing,2017,266(29):375-389.
[18]XU D,SHI Y,TSANG I W,et al.A Survey on Multi-outputLearning [J].IEEE Transactions on Neural Networks and Learning Systems,2020,31(7):2409-2429
[19]ZHANG Y,YANG Q.A Survey on Multi-Task Learning [J].IEEE Transactions on Knowledge and Data Engineering,2021,arXiv:1707.08114.
[20]TSOUMAKAS G,KATAKIS I.Multi-Label Classification:An Overview [J].International Journal of Data Warehousing and Mining,2009,3(3):1-13.
[21]BRINKER K,MENCIA E L,FUERNKRANZ J,et al.Multilabel classification via calibrated label ranking [J].Machine Learning,2008,73(2):133-153.
[22]TSOUMAKAS G,KATAKIS I,VLAHAVAS I.Randomk-La-belsets for Multi-Label Classification [J].IEEE Transactions on Knowledge and Data Engineering,2011,23(7):1079-1089.
[23]READ J,PFAHRINGER B,HOLMES G,et al.Classifier chains for multi-label classification [J].Machine Learning,2011,85(3):333-359.
[24]ZHANG M L,ZHOU Z H.ML-KNN:A lazy learning approach to multi-label learning [J].Pattern Recognition,2007,40(7):2038-2048.
[25]ZHANG M L,ZHOU Z H.Multilabel Neural Networks withApplications to Functional Genomics and Text Categorization [J].IEEE Transactions on Knowledge & Data Engineering,2006,18(10):1338-1351.
[26]WANG H,XU Y.Sparse elastic net multi-label rank supportvector machine with pinball loss and its applications [J].Applied Soft Computing,2021,104(9):107232.
[27]ARGYRIOU A,EVGENIOU T,PONTIL M.Convex multi-task feature learning [J].Machine Learning,2008,73(3):243-272.
[28]SU F,SHANG H Y,WANG J Y.Low-Rank Deep Convolutional Neural Network for Multi-Task Learning[J].Computational Intelligence and Neuroscience,2019,2019:1-10.
[29]WANG D,NIE F,HUANG H.Learning Task Relational Structure for Multi-task Feature Learning [C]//IEEE International Conference on Data Mining.2016:1239-1244.
[30]BICKEL S,BOGOJESKA J,LENGAUER T,et al.Multi-tasklearning for HIV therapy screening [C]//Proceedings of the 25th International Conference on Machine Learning.2008:56-63.
[31]TSOUMAKAS G,SPYROMITROS X E,VREKOU A,et al.Multi-target regression via random linear target combinations [C]//Joint European Conference on Machine Learning and Know-ledge Discovery in Databases.2014:225-240.
[32]BRESKVAR M,KOCEV D,DZEROSKI S.Ensembles formulti-target regression with random output selections [J].Machine Learning,2018,107(11):1673-1709.
[33]ZHANG Y,YEUNG D Y.A convex formulation for learningtask relationships in multi-task learning[C]//Proceedings of UAI.2010:73-742.
[34]TODOROVSKI L,BLOCKEEL H,DZEROSKI S.Ranking with Predictive Clustering Trees [C]//European Conference on Machine Learning.2002:444-455.
[35]OSOJNIK A,PANOV P,DZEROSKI S.Tree-based methods for online multi-target regression [J].Journal of Intelligent Information Systems,2018,50:315-339
[36]LEVATIC J,CECI M,KOCEV D,et al.Self-training for multi-target regression with tree ensembles [J].Knowledge-Based Systems,2017,123:41-60.
[37]LIU H T,LI H,WANG J,et al.Multi-Target Regression via Sparse Integration and Label-Specific Features [J].Acta Electronica Sinica,2020,48(5):906-913.
[38]WANG J,GAO X R,ZHANG R,et al.Multi-Target Regression via Specific Features and Inter-Target Correlations [J].Acta Electronica Sinica,2020,48(11):2092-2100.
[39]WANG J,CHEN Z L,SUN K W,et al.Multi-target regression via target specific features [J].Knowledge-Based Systems,2019,170(15):70-78.
[40]DEMSAR J.Statistical comparisons of classifiers over multiple data sets [J].Journal of Machine Learning Research,2006,7:1-30.
[1] CHAI Hui-min, ZHANG Yong, FANG Min. Aerial Target Grouping Method Based on Feature Similarity Clustering [J]. Computer Science, 2022, 49(9): 70-75.
[2] LU Chen-yang, DENG Su, MA Wu-bin, WU Ya-hui, ZHOU Hao-hao. Federated Learning Based on Stratified Sampling Optimization for Heterogeneous Clients [J]. Computer Science, 2022, 49(9): 183-193.
[3] LENG Dian-dian, DU Peng, CHEN Jian-ting, XIANG Yang. Automated Container Terminal Oriented Travel Time Estimation of AGV [J]. Computer Science, 2022, 49(9): 208-214.
[4] NING Han-yang, MA Miao, YANG Bo, LIU Shi-chang. Research Progress and Analysis on Intelligent Cryptology [J]. Computer Science, 2022, 49(9): 288-296.
[5] HE Qiang, YIN Zhen-yu, HUANG Min, WANG Xing-wei, WANG Yuan-tian, CUI Shuo, ZHAO Yong. Survey of Influence Analysis of Evolutionary Network Based on Big Data [J]. Computer Science, 2022, 49(8): 1-11.
[6] WU Hong-xin, HAN Meng, CHEN Zhi-qiang, ZHANG Xi-long, LI Mu-hang. Survey of Multi-label Classification Based on Supervised and Semi-supervised Learning [J]. Computer Science, 2022, 49(8): 12-25.
[7] LI Yao, LI Tao, LI Qi-fan, LIANG Jia-rui, Ibegbu Nnamdi JULIAN, CHEN Jun-jie, GUO Hao. Construction and Multi-feature Fusion Classification Research Based on Multi-scale Sparse Brain Functional Hyper-network [J]. Computer Science, 2022, 49(8): 257-266.
[8] ZHANG Guang-hua, GAO Tian-jiao, CHEN Zhen-guo, YU Nai-wen. Study on Malware Classification Based on N-Gram Static Analysis Technology [J]. Computer Science, 2022, 49(8): 336-343.
[9] CHEN Ming-xin, ZHANG Jun-bo, LI Tian-rui. Survey on Attacks and Defenses in Federated Learning [J]. Computer Science, 2022, 49(7): 310-323.
[10] MAO Sen-lin, XIA Zhen, GENG Xin-yu, CHEN Jian-hui, JIANG Hong-xia. FCM Algorithm Based on Density Sensitive Distance and Fuzzy Partition [J]. Computer Science, 2022, 49(6A): 285-290.
[11] CHEN Jing-nian. Acceleration of SVM for Multi-class Classification [J]. Computer Science, 2022, 49(6A): 297-300.
[12] XIAO Zhi-hong, HAN Ye-tong, ZOU Yong-pan. Study on Activity Recognition Based on Multi-source Data and Logical Reasoning [J]. Computer Science, 2022, 49(6A): 397-406.
[13] WANG Fei, HUANG Tao, YANG Ye. Study on Machine Learning Algorithms for Life Prediction of IGBT Devices Based on Stacking Multi-model Fusion [J]. Computer Science, 2022, 49(6A): 784-789.
[14] LI Ya-ru, ZHANG Yu-lai, WANG Jia-chen. Survey on Bayesian Optimization Methods for Hyper-parameter Tuning [J]. Computer Science, 2022, 49(6A): 86-92.
[15] ZHAO Lu, YUAN Li-ming, HAO Kun. Review of Multi-instance Learning Algorithms [J]. Computer Science, 2022, 49(6A): 93-99.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.