计算机科学 ›› 2016, Vol. 43 ›› Issue (12): 91-96.doi: 10.11896/j.issn.1002-137X.2016.12.016

• 机器学习 • 上一篇    下一篇

受限玻尔兹曼机的稀疏化特征学习

康丽萍,许光銮,孙显   

  1. 中国科学院电子学研究所 北京100190;中国科学院大学 北京100190,中国科学院电子学研究所 北京100190,中国科学院电子学研究所 北京100190
  • 出版日期:2018-12-01 发布日期:2018-12-01
  • 基金资助:
    本文受国家自然科学基金(41301493),高分对地观测领域学术交流项目(GFEX04060103)资助

Sparse Feature Learning for Restricted Boltzmann Machine

KANG Li-ping, XU Guang-luan and SUN Xian   

  • Online:2018-12-01 Published:2018-12-01

摘要: 受限玻尔兹曼机(RBM)作为深度学习算法的一种基础模型被广泛应用,但传统RBM算法没有充分考虑数据的稀疏化特征学习,使得算法性能受数据集的稀疏性影响较大。提出一种RBM稀疏化特征学习方法(sRBM),通过归一化的输入数据均值确定数据集的稀疏系数,将稀疏系数大于阈值的稠密数据集自动转化为稀疏数据集,在不损失信息量的情况下实现输入数据的稀疏化。在手写字符数据集和自然图像数据集上的实验结果表明,sRBM通过输入数据稀疏化有效提升了RBM的稀疏化特征学习性能。

关键词: 受限玻尔兹曼机(RBM),稀疏化,特征学习,置信网络,稳定性

Abstract: As a basic model for deep learning algorithms,restricted boltzmann machine (RBM) is widely applied in the field of machine learning.However,the traditional RBM algorithm does not take full account of the sparse feature lear-ning for data.Therefore,the algorithm performance is greatly influenced by the sparsity of the dataset.In this study,a sparse feature learning method for restricted Boltzmann machine(sRBM) was proposed.Firstly,the sparse coefficient of the dataset is determined by the mean of normalized input data.Then the dense dataset with the sparse coefficient being greater than threshold will be converted to sparse dataset automatically.As a result,sRBM makes the input data sparse without information loss.We performed experiments on MNIST dataset and attribute discovery dataset.The experimental results show that sRBM improves the performance of sparse feature learning for RBM effectively.

Key words: Restricted Boltzmann machine(RBM),Sparsity,Feature learning,Belief network,Stability

[1] Poultney C,Chopra S,Cun Y L.Efficient Learning of SparseRepresentations with an Energy-Based Model[C]∥Proceedings of Advances in Neural Information Processing Systems.Vancouver,B.C:NIPS,2006:1137-1144
[2] Doi E,Balcan D C,Lewicki M S.A Theoretical Analysis of Robust Coding over Noisy Overcomplete Channels[C]∥Procee-dings of Advances in Neural Information Processing Systems.Vancouver,B.C:NIPS,2006:37-3311
[3] Olshausen B A,Field D J.Sparse Coding with an Overcomplete Basis Set:A Strategy Employed by V1?[J].Vision Research,1997,37(23):3311-3325
[4] Hinton G E,Salakhutdinov R R.Reducing the Dimensionality of Data with Neural Networks[J].Science,2006,313(5786):504-507
[5] Guo L L,Ding S F.Research progress on deep learning[J].Computer Science,2015,42(5):28-33(in Chinese) 郭丽丽,丁世飞.深度学习研究进展[J].计算机科学,2015,42(5):28-33
[6] Hinton G.A Practical Guide to Training Restricted Boltzmann Machines[J].Momentum,2010,9(1):599-619
[7] Chen Y,Zheng D Q,Zhao T J.Chinese relation extraction based on Deep Belief Nets[J].Journal of Software,2012,23(10):2572-2585(in Chinese) 陈宇,郑德权,赵铁军.基于 Deep Belief Nets 的中文名实体关系抽取[J].软件学报,2012,23(10):2572-2585
[8] Mohamed A R,Dahl G E,Hinton G.Acoustic Modeling Using Deep Belief Networks[J].IEEE Transactions on Audio,Speech,and Language Processing,2012,20(1):14-22
[9] Schmah T,Hinton G E,Small S L,et al.Generative Versus Discriminative Training of Rbms for Classification of Fmri Images[C]∥Proceedings of Advances in Neural Information Proces-sing Systems.Vancouver,B.C.:NIPS,2008:1409-1416
[10] Memisevic R,Hinton G.Unsupervised Learning of Image Trans-formations[C]∥Proceedings of Computer Vision and Pattern Re-cognition(CVPR’07).Minneapolis,MN:IEEE,2007:1-8
[11] Taylor G W,Sigal L,Fleet D J,et al.Dynamical Binary Latent Variable Models for 3d Human Pose Tracking[C]∥2010 IEEE Conference on Proceedings of Computer Vision and Pattern Re-cognition(CVPR).San Francisco,CA:IEEE,2010:631-638
[12] Lee H,Ekanadham C,Ng A Y.Sparse Deep Belief Net Model for Visual Area V2[C]∥Proceedings of Advances in Neural Information Processing Systems.Vancouver,B.C:NIPS,2008:873-880
[13] Luo H,Shen R,Niu C,et al.Sparse Group Restricted Boltzmann Machines[C]∥Proceedings of AAAI Conference on Artificial Intelligence.Georgia,USA:AAAI,2010
[14] Guo R,Qi H.Partially-Sparse Restricted Boltzmann Machine for Background Modeling and Subtraction[C]∥2013 12th International Conference on Proceedings of Machine Learning and Applications (ICMLA).Miami,FL:IEEE,2013:209-214
[15] Carreira-Perpinan M A,Hinton G E.On Contrastive Divergence Learning[C]∥Proceedings of Proceedings of the tenth international workshop on Artificial Intelligence and Statistics.Barbados:AISTATS,2005:33-40
[16] Salakhutdinov R,Mnih A,Hinton G.Restricted Boltzmann Machines for Collaborative Filtering[C]∥Procee-dings of Procee-dings of the 24th International Conference on Machine Learning.New York:ACM,2007:791-798
[17] Pineda F J.Generalization of Back-Propagation to RecurrentNeural Networks[J].Physical Review Letters,1987,59(19):307-308
[18] LeCun Y,Cortes C,Burges C J.The Mnist Database of Handwritten Digits.http://yann.lecun.com/exdb/mnist
[19] Berg T L,Berg A C,Shih J.Automatic Attribute Discovery and Characterization from Noisy Web Data[C]∥Proceedings of Computer Vision(ECCV 2010).Crete,Greece:ECCV,2010:663-676

No related articles found!
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!