Computer Science ›› 2016, Vol. 43 ›› Issue (12): 91-96.doi: 10.11896/j.issn.1002-137X.2016.12.016

Previous Articles     Next Articles

Sparse Feature Learning for Restricted Boltzmann Machine

KANG Li-ping, XU Guang-luan and SUN Xian   

  • Online:2018-12-01 Published:2018-12-01

Abstract: As a basic model for deep learning algorithms,restricted boltzmann machine (RBM) is widely applied in the field of machine learning.However,the traditional RBM algorithm does not take full account of the sparse feature lear-ning for data.Therefore,the algorithm performance is greatly influenced by the sparsity of the dataset.In this study,a sparse feature learning method for restricted Boltzmann machine(sRBM) was proposed.Firstly,the sparse coefficient of the dataset is determined by the mean of normalized input data.Then the dense dataset with the sparse coefficient being greater than threshold will be converted to sparse dataset automatically.As a result,sRBM makes the input data sparse without information loss.We performed experiments on MNIST dataset and attribute discovery dataset.The experimental results show that sRBM improves the performance of sparse feature learning for RBM effectively.

Key words: Restricted Boltzmann machine(RBM),Sparsity,Feature learning,Belief network,Stability

[1] Poultney C,Chopra S,Cun Y L.Efficient Learning of SparseRepresentations with an Energy-Based Model[C]∥Proceedings of Advances in Neural Information Processing Systems.Vancouver,B.C:NIPS,2006:1137-1144
[2] Doi E,Balcan D C,Lewicki M S.A Theoretical Analysis of Robust Coding over Noisy Overcomplete Channels[C]∥Procee-dings of Advances in Neural Information Processing Systems.Vancouver,B.C:NIPS,2006:37-3311
[3] Olshausen B A,Field D J.Sparse Coding with an Overcomplete Basis Set:A Strategy Employed by V1?[J].Vision Research,1997,37(23):3311-3325
[4] Hinton G E,Salakhutdinov R R.Reducing the Dimensionality of Data with Neural Networks[J].Science,2006,313(5786):504-507
[5] Guo L L,Ding S F.Research progress on deep learning[J].Computer Science,2015,42(5):28-33(in Chinese) 郭丽丽,丁世飞.深度学习研究进展[J].计算机科学,2015,42(5):28-33
[6] Hinton G.A Practical Guide to Training Restricted Boltzmann Machines[J].Momentum,2010,9(1):599-619
[7] Chen Y,Zheng D Q,Zhao T J.Chinese relation extraction based on Deep Belief Nets[J].Journal of Software,2012,23(10):2572-2585(in Chinese) 陈宇,郑德权,赵铁军.基于 Deep Belief Nets 的中文名实体关系抽取[J].软件学报,2012,23(10):2572-2585
[8] Mohamed A R,Dahl G E,Hinton G.Acoustic Modeling Using Deep Belief Networks[J].IEEE Transactions on Audio,Speech,and Language Processing,2012,20(1):14-22
[9] Schmah T,Hinton G E,Small S L,et al.Generative Versus Discriminative Training of Rbms for Classification of Fmri Images[C]∥Proceedings of Advances in Neural Information Proces-sing Systems.Vancouver,B.C.:NIPS,2008:1409-1416
[10] Memisevic R,Hinton G.Unsupervised Learning of Image Trans-formations[C]∥Proceedings of Computer Vision and Pattern Re-cognition(CVPR’07).Minneapolis,MN:IEEE,2007:1-8
[11] Taylor G W,Sigal L,Fleet D J,et al.Dynamical Binary Latent Variable Models for 3d Human Pose Tracking[C]∥2010 IEEE Conference on Proceedings of Computer Vision and Pattern Re-cognition(CVPR).San Francisco,CA:IEEE,2010:631-638
[12] Lee H,Ekanadham C,Ng A Y.Sparse Deep Belief Net Model for Visual Area V2[C]∥Proceedings of Advances in Neural Information Processing Systems.Vancouver,B.C:NIPS,2008:873-880
[13] Luo H,Shen R,Niu C,et al.Sparse Group Restricted Boltzmann Machines[C]∥Proceedings of AAAI Conference on Artificial Intelligence.Georgia,USA:AAAI,2010
[14] Guo R,Qi H.Partially-Sparse Restricted Boltzmann Machine for Background Modeling and Subtraction[C]∥2013 12th International Conference on Proceedings of Machine Learning and Applications (ICMLA).Miami,FL:IEEE,2013:209-214
[15] Carreira-Perpinan M A,Hinton G E.On Contrastive Divergence Learning[C]∥Proceedings of Proceedings of the tenth international workshop on Artificial Intelligence and Statistics.Barbados:AISTATS,2005:33-40
[16] Salakhutdinov R,Mnih A,Hinton G.Restricted Boltzmann Machines for Collaborative Filtering[C]∥Procee-dings of Procee-dings of the 24th International Conference on Machine Learning.New York:ACM,2007:791-798
[17] Pineda F J.Generalization of Back-Propagation to RecurrentNeural Networks[J].Physical Review Letters,1987,59(19):307-308
[18] LeCun Y,Cortes C,Burges C J.The Mnist Database of Handwritten Digits.http://yann.lecun.com/exdb/mnist
[19] Berg T L,Berg A C,Shih J.Automatic Attribute Discovery and Characterization from Noisy Web Data[C]∥Proceedings of Computer Vision(ECCV 2010).Crete,Greece:ECCV,2010:663-676

No related articles found!
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!