Computer Science ›› 2019, Vol. 46 ›› Issue (6): 69-74.doi: 10.11896/j.issn.1002-137X.2019.06.009

Previous Articles     Next Articles

Combined Feature Extraction Method for Ordinal Regression

ZENG Qing-tian1,2, LIU Chen-zheng1, NI Wei-jian1, DUAN Hua3   

  1. (College of Computer Science and Engineering,Shandong University of Science and Technology,Qingdao,Shandong 266590,China)1
    (College of Electronic and Information Engineering,Shandong University of Science and Technology,Qingdao,Shandong 266590,China)2
    (College of Mathematics and Systems Science,Shandong University of Science and Technology,Qingdao,Shandong 266590,China)3
  • Received:2018-06-20 Published:2019-06-24

Abstract: Ordinal regression,also known as ordinal classification, is a supervised learning task that uses the labels with a natural order to classify data items.Ordinal regression is closely related to many practical problems.In recent years,the research on ordinal regression has attracted more and more attention.Ordinal regression,like other supervised lear-ning tasks(classification,regression,etc.),requires feature extraction to improve the efficiency and accuracy of the model.However,while feature extraction has been extensively studied for other classification tasks,there are few researches in ordinal regression.It is well known that the combined features could capture more underlying data semantics than single features,but it is difficult to improve the accuracy of the model by adding general combined features.Based on the frequent mining patterns,this paper used the K-L divergence value to select the most discriminative frequent patterns for feature combination,and proposed a new ordinal regression combination feature extraction method.Multiple ordinal regression models are used for validation on both the public and our own datasets.The experimental results show that using the most distinguishing frequent pattern combination features can effectively improve the training effect of most ordinal regression models.

Key words: Feature combination, Feature selection, Frequent pattern, Ordinal regression

CLC Number: 

  • TP391
[1]GUTIÉRREZ P A,PÉREZ-ORTIZ M,SÁNCHEZ-MONEDERO J,et al.Ordinal Regression Methods:Survey and Experimental Study[J].IEEE Transactions on Knowledge & Data Engineering,2016,28(1):127-146.
[2]SMOLA A J,SCHOELKOPF B.A tutorial on support vector regression[J].Statistics and Computing,2004,14(3):199-222.
[3]HSU C W,LIN C J.A Comparison of Methods for Multiclass Support Vector Machines[C]∥IEEE TRANS.on Neural Networks.2002:415-425.
[4]CHENG J,WANG Z,POLLASTRI G.A neural network approach to ordinal regression[C]∥IEEE International Joint Conference on Neural Networks.IEEE,2008:1279-1284.
[5]DENG W Y,ZHENG Q H,LIAN S,et al.Ordinal extreme learning machine[J].Neurocomputing,2010,74(1-3):447-456.
[6]MCCULLAGH P.Regression Models for Ordinal Data[J].Journal of the Royal Statistical Society,1980,42(2):109-142.
[7]MATHIESON M.Ordinal Models for Neural Networks[C]∥ Neural Networks in Finanicial Engineering.1996:523-536.
[8]WEI C,KEERTHI S S.Support Vector Ordinal Regression [M].MIT Press,2007.
[9]MUKRAS R,WIRATUNGA N,LOTHIAN R,et al.Information Gain Feature Selection for Ordinal Text Classification using Probability Redistribution[C]∥Proceedings of the Textlink Workshop at IJCAI 2007.Hyderabad,2007:1-10.
[10]BACCIANELLA S,ESULI A,SEBASTIANI F.Multi-facet Ra-ting of Product Reviews[C]∥European Conference on Ir Research on Advances in Information Retrieval.Springer-Verlag,2009:461-472.
[11]BACCIANELLA S,ESULI A,SEBASTIANI F.Feature selec-tion for ordinal regression[C]∥ACM Symposium on Applied Computing.DBLP,2010:1748-1754.
[12]BACCIANELLA S,ESULI A,SEBASTIANI F.Feature Selection for Ordinal Text Classification[J].Neural Computation,2014,26(3):557-591.
[13]AGRAWAL R,IMIELIN'SKI T,SWAMI A.Mining association rules between sets of items in large databases[C]∥ACM SIGMOD International Conference on Management of Data.ACM,1993:207-216.
[14]AGRAWAL R.Fast algorithms for mining association rules[C]∥ Proc.VLDB Conference.1994:487-499.
[15]HAN J,PEI J,YIN Y.Mining frequent patterns without candidate generation[C]∥ACM SIGMOD International Conference on Management of Data.ACM,2000:1-12.
[16]ZAKI M J.Hsiao:CHARM:An efficient algorithm for closed itemset mining[C]∥Proceedings of the Second SIAM International Conference on Data Mining.Arlington,2002:457-473.
[17]YAN X,YU P S,HAN J.Graph indexing:a frequent structure-based approach[C]∥SIGMOD’04 Proceedings of the 2004 ACM SIGMOD international conference on Management of data.New York:ACM,2004:335-346.
[18]WANG K,XU C,LIU B.Clustering transactions using large items[C]∥International Conference on Information and Know-ledge Management,Proceedings.Staff Publications,1999:483-490.
[19]LIUB,HSUW,MAY M.Integrating classification and association rule mining[C]∥4th International Conference on Know-ledge Discovery and Data Mining.1998:80-86.
[20]LODHI H,SAUNDERS C,SHAWE-TAYLOR J,et al.Text classification using string kernels[J].Journal of Machine Lear-ning Research,2002,2(3):419-444.
[21]DEHKORDI M N,SHENASSA M H.CLoPAR:Classification based on Predictive Association Rules[C]∥2006 3rd International IEEE Conference Intelligent Systems.IEEE,2007:483-487.
[22]WANG J,KARYPIS G.HARMONY:Efficiently Mining the Best Rules for Classification[C]∥Siam Conference on Data Mining.2005:205-216.
[23]PASCAL.Pascal(Pattern Analysis,Statistical Modelling and Computational Learning) machine learning benchmarks repository[EB/OL].Available:http://mldata.org.
[24] ASUNCIONA,NEWMAND.UCI Machine Learning Reposito-ry[EB/OL].http://www.ics.uci.edu/~mlearn/MLRepository.html.
[1] LI Bin, WAN Yuan. Unsupervised Multi-view Feature Selection Based on Similarity Matrix Learning and Matrix Alignment [J]. Computer Science, 2022, 49(8): 86-96.
[2] HU Yan-yu, ZHAO Long, DONG Xiang-jun. Two-stage Deep Feature Selection Extraction Algorithm for Cancer Classification [J]. Computer Science, 2022, 49(7): 73-78.
[3] KANG Yan, WANG Hai-ning, TAO Liu, YANG Hai-xiao, YANG Xue-kun, WANG Fei, LI Hao. Hybrid Improved Flower Pollination Algorithm and Gray Wolf Algorithm for Feature Selection [J]. Computer Science, 2022, 49(6A): 125-132.
[4] CHU An-qi, DING Zhi-jun. Application of Gray Wolf Optimization Algorithm on Synchronous Processing of Sample Equalization and Feature Selection in Credit Evaluation [J]. Computer Science, 2022, 49(4): 134-139.
[5] SUN Lin, HUANG Miao-miao, XU Jiu-cheng. Weak Label Feature Selection Method Based on Neighborhood Rough Sets and Relief [J]. Computer Science, 2022, 49(4): 152-160.
[6] LI Zong-ran, CHEN XIU-Hong, LU Yun, SHAO Zheng-yi. Robust Joint Sparse Uncorrelated Regression [J]. Computer Science, 2022, 49(2): 191-197.
[7] ZHANG Ye, LI Zhi-hua, WANG Chang-jie. Kernel Density Estimation-based Lightweight IoT Anomaly Traffic Detection Method [J]. Computer Science, 2021, 48(9): 337-344.
[8] YANG Lei, JIANG Ai-lian, QIANG Yan. Structure Preserving Unsupervised Feature Selection Based on Autoencoder and Manifold Regularization [J]. Computer Science, 2021, 48(8): 53-59.
[9] WU Cheng-feng, CAI Li, LI Jin, LIANG Yu. Frequent Pattern Mining of Residents’ Travel Based on Multi-source Location Data [J]. Computer Science, 2021, 48(7): 155-163.
[10] HOU Chun-ping, ZHAO Chun-yue, WANG Zhi-peng. Video Abnormal Event Detection Algorithm Based on Self-feedback Optimal Subclass Mining [J]. Computer Science, 2021, 48(7): 199-205.
[11] HU Yan-mei, YANG Bo, DUO Bin. Logistic Regression with Regularization Based on Network Structure [J]. Computer Science, 2021, 48(7): 281-291.
[12] ZHOU Gang, GUO Fu-liang. Research on Ensemble Learning Method Based on Feature Selection for High-dimensional Data [J]. Computer Science, 2021, 48(6A): 250-254.
[13] DING Si-fan, WANG Feng, WEI Wei. Relief Feature Selection Algorithm Based on Label Correlation [J]. Computer Science, 2021, 48(4): 91-96.
[14] TENG Jun-yuan, GAO Meng, ZHENG Xiao-meng, JIANG Yun-song. Noise Tolerable Feature Selection Method for Software Defect Prediction [J]. Computer Science, 2021, 48(12): 131-139.
[15] ZHANG Ya-chuan, LI Hao, SONG Chen-ming, BU Rong-jing, WANG Hai-ning, KANG Yan. Hybrid Artificial Chemical Reaction Optimization with Wolf Colony Algorithm for Feature Selection [J]. Computer Science, 2021, 48(11A): 93-101.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!