基于最大间隔和流形假设的半监督学习算法

doi:10.11896/jsjkx.221100136

Abstract

Abstract: Semi-supervised learning is a weakly supervised learning pattern between supervised learning and unsupervised lear-ning.It combines a small number of labeled instances with a large number of unlabeled instances to build a model during the process of learning,hoping to achieve better learning accuracy than supervised learning using only labeled instances.In this lear-ning pattern,this paper proposes a semi-supervised learning algorithm that combines the maximum margin with manifold hypo-thesis of the instance space.The algorithm utilizes the manifold structure of instances to estimate the labeling confidence over unlabeled instances,at the same time utilizes the maximum margin to derive the classification model.And alternating optimization is adopted to address the quadratic programming problem of the model parameters and the labeling confidence in an iterative manner.On 12 UCI datasets and 4 datasets generated by the MNIST database of handwritten digits,in semi-supervised transductive learning,the proposed algorithm’s performance outperforms the comparison algorithms for 60.5% of the configurations in semi-supervised inductive learning,the proposed algorithm’s performance outperforms the comparison algorithms for 42.6% of the configurations.

Key words: Semi-supervised learning, Maximum margin, Manifold hypothesis, Labeling confidence, Support vector machine

CLC Number:

TP181

DAI Wei, CHAI Jing, LIU Yajiao. Semi-supervised Learning Algorithm Based on Maximum Margin and Manifold Hypothesis[J].Computer Science, 2024, 51(2): 259-267.

References

[1]VAN ENGELEN J E,HOOS H H.A survey on semi-supervised learning[J].Machine Learning,2020,109:373-440.
[2] CHAPELLE O,SCHOLKOPF B,ZIEN A.Semi-supervisedlearning[M].Cambridge,MA:MIT Press,2006.
[3]ZHU X J.Semi-supervised learning literature survey[M].Madison,USA:Department of Computer Sciences,University of Wisconsin at Madison,2005.
[4]ZHU X J,GOLDBERG A B.Introduction to semi-supervisedlearning[J].Synthesis Lectures on Artificial Intelligence and Machine Learning,2009,3(1):1-130.
[5]CARON M,BOJANOWSKI P,JOULIN A,et al.Deep clustering for unsupervised learning of visual features[C]//Procee-dings of the European Conference on Computer Vision(ECCV 2018) .Berlin:Springer,2018:139-156.
[6]HASTIE T,TIBSHIRANI R,FRIEDMAN J.The elements of Statistical Learning:Data Mining,Inference,and Prediction[M].Berlin:Springer,2019.
[7]FIGUEIREDO M A T.Adaptive sparseness for supervisedlearning[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2003,25(9):1150-1159.
[8]LI Y F,ZHOU Z H.Towards making unlabeled data never hurt[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2014,37(1):175-188.
[9]LI M,ZHOU Z H.Improve computer-aided diagnosis with machine learning techniques using undiagnosed samples[J].IEEE Transactions on Systems,Man,and Cybernetics－Part A:Systems and Humans,2007,37(6):1088-1098.
[10]JOACHIMS T.Transductive inference for text classificationusing support vector machines[C]//Proceedings of the 16th International Conference on Machine Learning.San Francisco,USA:Morgan Kaufmann Publishers Inc,1999:200-209.
[11]WANG L,CHAN K L,ZHANG Z.Bootstrapping SVM active learning by incorporating unlabelled images for image retrieval[C]//IEEE Computer Society Conference on Computer Vision and Pattern Recognition.2003:629-634.
[12]KASABOV N,PANG S.Transductive support vector machines and applications in bioinformatics for promoter recognition[C]//Proceedings of 2003 International Conference on Neural Networks and Signal Processing.Piscataway,NJ:IEEE 1:1-6.
[13]SINGLA M,GHOSH D,SHUKLA K K.pin-TSVM:A Robust Transductive Support Vector Machine and its Application to the Detection of COVID-19 Infected Patients[J].Neural Processing Letters,2021,53(6):3981-4010.
[14]GOUTTE C,DÉJEAN H,GAUSSIER E,et al.Combining labelled and unlabelled data:a case study on Fisher kernels and transductive inference for biological entity recognition[C]//Proceedings of the 6th Conference on Natural Language Learning.Stroudsburg,PA:ACL,2002:1-7.
[15]KOCKELKORN M,LÜNEBURG A,SCHEFFER T.Usingtransduction and multi-view learning to answer emails[C]//Proceedings of Knowledge Discovery in Databases:PKDD 2003.Berlin:Springer,2003:266-277.
[16]ZHOU D,BOUSQUET O,LAL T,et al.Learning with local and global consistency[J].Advances in Neural Information Processing Systems,2003,16:321-328.
[17]ZHU X,GHAHRAMANI Z.Learning from labeled and unlabeled data with label propagation[R].Pittsburgh,PA:Carnegie Mellon University,Technical Report:CMU-CALD-02-107,2002.
[18]SUN S,HUSSAIN Z,SHAWE-TAYLOR J.Manifold-preser-ving graph reduction for sparse semi-supervised learning[J].Neurocomputing,2014,124:13-21.
[19]ZHU X,GHAHRAMANI Z,LAFFERTY J D.Semi-supervised learning using gaussian fields and harmonic functions[C]//Proceedings of the 20th International Conference on Machine Learning.Menlo Park:AAAI Press,2003:912-919.
[20]ZHU X.Semi-supervised learning with graphs[M].Pittsburgh,PA:Carnegie Mellon University,2005.
[21]CAI X,WEN G,WEI J,et al.Relative manifold based semi-supervised dimensionality reduction[J].Frontiers of Computer Science,2014,8:923-932.
[22]VAPNIK V N.An overview of statistical learning theory[J].IEEE Transactions on Neural Networks,1999,10(5):988-999.
[23]DING S,ZHU Z,ZHANG X.An overview on semi-supervisedsupport vector machine[J].Neural Computing and Applications,2017,28(5):969-978.
[24]LI Y F,GUO L Z,ZHOU Z H.Towards safe weakly supervised learning[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2021,43(1):334-346.
[25]WANG W,ZHANG M L.Semi-supervised partial label learning via confidence-rated margin maximization[C]//Proceedings of the 34th International Conference on Neural Information Processing Systems.Red Hook,NY:Curran Associates Inc,2020:6982-6993.
[26]MILLER D J,UYAR H.A mixture of experts classifier withlearning based on both labelled and unlabelled data[J].Advances in Neural Information Processing Systems,1996,9:571-577.
[27]NIGAM K,MCCALLUM A K,THRUN S,et al.Text classification from labeled and unlabeled documents using EM[J].Machine Learning,2000,39(2):103-134.
[28]SHAHSHAHANI B M,LANDGREBE D A.The effect of unlabeled samples in reducing the small sample size problem and mitigating the Hughes phenomenon[J].IEEE Transactions on Geoscience and Remote Sensing,1994,32(5):1087-1095.
[29]BELKIN M,NIYOGI P.Laplacian eigenmaps and spectral techniques for embedding and clustering[C]//Proceedings of the 14th International Conference on Neural Information Processing Systems Natural and Synthetic.Cambridge,MA:MIT Press,2001:585-591.
[30]HINTON G E,SALAKHUTDINOV R R.Using deep beliefnets to learn covariance kernels for Gaussian processes[C]//Proceedings of the 20th International Conference on Neural Information Processing Systems.Red Hook,NY:Curran Asso-ciates Inc,2007:20:1249-1256.
[31]COATES A,NG A Y.The importance of encoding versus trai-ning with sparse coding and vector quantization[C]//Procee-dings of the 28th International Conference on International Conference on Machine Learning.Madison.Wisconsin:Omnipress,2011:921-928.
[32]BLUM A,MITCHELL T.Combining labeled and unlabeled data with co-training[C]//Proceedings of the 11th Annual Confe-rence on Computational Learning Theory.New York,NY:ACM,1998:92-100.
[33]NIGAM K,GHANI R.Analyzing the effectiveness and applicability of co-training[C]//Proceedings of the 9th International Conference on Information and Knowledge Management.New York,NY:ACM,2000:86-93.
[34]GOLDMAN S,ZHOU Y.Enhancing supervised learning withunlabeled data[C]//Proceedings of the 17th International Conference on Machine Learning.San Francisco,CA:Morgan Kaufmann Publishers Inc,2000:327-334.
[35]ZHOU Z H,LI M.Tri-training:Exploiting unlabeled data using three classifiers[J].IEEE Transactions on Knowledge and Data Engineering,2005,17(11):1529-1541.
[36]WANG F,ZHANG C.Label propagation through linear neighborhoods[J].IEEE Transactions on Knowledge and Data Engineering,2008,20(1):55-67.
[37]GONG C,LIU T,TAO D,et al.Deformed graph Laplacian for semisupervised learning[J].IEEE Transactions on Neural Networks and Learning Systems,2015,26(10):2261-2274.
[38] CALDER J,COOK B,THORPE M,et al.Poisson learning:Graph based semi-supervised learning at very low label rates[C]//Proceedings of the 37th International Conference on Machine Learning.Clearwater Beach,USA:PMLR,2020:1306-1316.
[39]BENNETT K,DEMIRIZ A.Semi-supervised support vector machines[J].Advances in Neural Information Processing Systems.1998,11:368-374.
[40]LI Y F,KWOK J T,ZHOU Z H.Semi-supervised learning using label mean[C]//Proceedings of the 26th Annual International Conference on Machine Learning.New York,NY:ACM,2009:633-640.
[41]LI Y F,KWOK J,ZHOU Z H.Cost-sensitive semi-supervisedsupport vector machine[C]//Proceedings of the 24th AAAI Conference on Artificial Intelligence.Menlo Park,CA:AAAI Press,2010:500-505.
[42]WANG Q W,LI Y F,ZHOU Z H.Partial Label Learning with Unlabeled Data[C]//Proceedings of the 28th International Joint Conference on Artificial Intelligence.Menlo Park,CA:AAAI Press,2019:3755-3761.

Related Articles 15

[1]	KANG Wei, LI Lihui, WEN Yimin. Semi-supervised Classification of Data Stream with Concept Drift Based on Clustering Model Reuse [J]. Computer Science, 2024, 51(4): 124-131.
[2]	LI Hui, LI Wengen, GUAN Jihong. Dually Encoded Semi-supervised Anomaly Detection [J]. Computer Science, 2023, 50(7): 53-59.
[3]	GU Yuhang, HAO Jie, CHEN Bing. Semi-supervised Semantic Segmentation for High-resolution Remote Sensing Images Based on DataFusion [J]. Computer Science, 2023, 50(6A): 220500001-6.
[4]	WANG Qingyu, WANG Hairui, ZHU Guifu, MENG Shunjian. Study on SQL Injection Detection Based on FlexUDA Model [J]. Computer Science, 2023, 50(6A): 220600172-6.
[5]	QIN Liang, XIE Liang, CHEN Shengshuang, XU Haijiao. Online Semi-supervised Cross-modal Hashing Based on Anchor Graph Classification [J]. Computer Science, 2023, 50(6): 183-193.
[6]	ZHANG Renbin, ZUO Yicong, ZHOU Zelin, WANG Long, CUI Yuhang. Multimodal Generative Adversarial Networks Based Multivariate Time Series Anomaly Detection [J]. Computer Science, 2023, 50(5): 355-362.
[7]	CUI Jingsong, ZHANG Tongtong, GUO Chi, GUO Wenfei. Network Equipment Anomaly Detection Based on Time Delay Feature [J]. Computer Science, 2023, 50(3): 371-379.
[8]	LI Haitao, WANG Ruimin, DONG Weiyu, JIANG Liehui. Semi-supervised Network Traffic Anomaly Detection Method Based on GRU [J]. Computer Science, 2023, 50(3): 380-390.
[9]	WANG Xiangwei, HAN Rui, Chi Harold LIU. Hierarchical Memory Pool Based Edge Semi-supervised Continual Learning Method [J]. Computer Science, 2023, 50(2): 23-31.
[10]	XU Huajie, XIAO Yifeng. Semi-supervised Semantic Segmentation Method Based on Multiple Teacher Network Model [J]. Computer Science, 2023, 50(12): 279-284.
[11]	SONG Faxing, MIAO Duoqian, ZHANG Hongyun. Semi-supervised Object Detection with Sequential Three-way Decision [J]. Computer Science, 2023, 50(10): 1-6.
[12]	HE Yulin, ZHU Penghui, HUANG Zhexue, Fournier-Viger PHILIPPE. Classification Uncertainty Minimization-based Semi-supervised Ensemble Learning Algorithm [J]. Computer Science, 2023, 50(10): 88-95.
[13]	WU Hong-xin, HAN Meng, CHEN Zhi-qiang, ZHANG Xi-long, LI Mu-hang. Survey of Multi-label Classification Based on Supervised and Semi-supervised Learning [J]. Computer Science, 2022, 49(8): 12-25.
[14]	HOU Xia-ye, CHEN Hai-yan, ZHANG Bing, YUAN Li-gang, JIA Yi-zhen. Active Metric Learning Based on Support Vector Machines [J]. Computer Science, 2022, 49(6A): 113-118.
[15]	SHAN Xiao-ying, REN Ying-chun. Fishing Type Identification of Marine Fishing Vessels Based on Support Vector Machine Optimized by Improved Sparrow Search Algorithm [J]. Computer Science, 2022, 49(6A): 211-216.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 0

No Suggested Reading articles found!

Semi-supervised Learning Algorithm Based on Maximum Margin and Manifold Hypothesis

PDF (PC)

Abstract

Cite this article

share this article

References

Related Articles 15

Metrics

Comments

Recommended 0