计算机科学 ›› 2016, Vol. 43 ›› Issue (12): 91-96.doi: 10.11896/j.issn.1002-137X.2016.12.016
康丽萍,许光銮,孙显
KANG Li-ping, XU Guang-luan and SUN Xian
摘要: 受限玻尔兹曼机(RBM)作为深度学习算法的一种基础模型被广泛应用,但传统RBM算法没有充分考虑数据的稀疏化特征学习,使得算法性能受数据集的稀疏性影响较大。提出一种RBM稀疏化特征学习方法(sRBM),通过归一化的输入数据均值确定数据集的稀疏系数,将稀疏系数大于阈值的稠密数据集自动转化为稀疏数据集,在不损失信息量的情况下实现输入数据的稀疏化。在手写字符数据集和自然图像数据集上的实验结果表明,sRBM通过输入数据稀疏化有效提升了RBM的稀疏化特征学习性能。
[1] Poultney C,Chopra S,Cun Y L.Efficient Learning of SparseRepresentations with an Energy-Based Model[C]∥Proceedings of Advances in Neural Information Processing Systems.Vancouver,B.C:NIPS,2006:1137-1144 [2] Doi E,Balcan D C,Lewicki M S.A Theoretical Analysis of Robust Coding over Noisy Overcomplete Channels[C]∥Procee-dings of Advances in Neural Information Processing Systems.Vancouver,B.C:NIPS,2006:37-3311 [3] Olshausen B A,Field D J.Sparse Coding with an Overcomplete Basis Set:A Strategy Employed by V1?[J].Vision Research,1997,37(23):3311-3325 [4] Hinton G E,Salakhutdinov R R.Reducing the Dimensionality of Data with Neural Networks[J].Science,2006,313(5786):504-507 [5] Guo L L,Ding S F.Research progress on deep learning[J].Computer Science,2015,42(5):28-33(in Chinese) 郭丽丽,丁世飞.深度学习研究进展[J].计算机科学,2015,42(5):28-33 [6] Hinton G.A Practical Guide to Training Restricted Boltzmann Machines[J].Momentum,2010,9(1):599-619 [7] Chen Y,Zheng D Q,Zhao T J.Chinese relation extraction based on Deep Belief Nets[J].Journal of Software,2012,23(10):2572-2585(in Chinese) 陈宇,郑德权,赵铁军.基于 Deep Belief Nets 的中文名实体关系抽取[J].软件学报,2012,23(10):2572-2585 [8] Mohamed A R,Dahl G E,Hinton G.Acoustic Modeling Using Deep Belief Networks[J].IEEE Transactions on Audio,Speech,and Language Processing,2012,20(1):14-22 [9] Schmah T,Hinton G E,Small S L,et al.Generative Versus Discriminative Training of Rbms for Classification of Fmri Images[C]∥Proceedings of Advances in Neural Information Proces-sing Systems.Vancouver,B.C.:NIPS,2008:1409-1416 [10] Memisevic R,Hinton G.Unsupervised Learning of Image Trans-formations[C]∥Proceedings of Computer Vision and Pattern Re-cognition(CVPR’07).Minneapolis,MN:IEEE,2007:1-8 [11] Taylor G W,Sigal L,Fleet D J,et al.Dynamical Binary Latent Variable Models for 3d Human Pose Tracking[C]∥2010 IEEE Conference on Proceedings of Computer Vision and Pattern Re-cognition(CVPR).San Francisco,CA:IEEE,2010:631-638 [12] Lee H,Ekanadham C,Ng A Y.Sparse Deep Belief Net Model for Visual Area V2[C]∥Proceedings of Advances in Neural Information Processing Systems.Vancouver,B.C:NIPS,2008:873-880 [13] Luo H,Shen R,Niu C,et al.Sparse Group Restricted Boltzmann Machines[C]∥Proceedings of AAAI Conference on Artificial Intelligence.Georgia,USA:AAAI,2010 [14] Guo R,Qi H.Partially-Sparse Restricted Boltzmann Machine for Background Modeling and Subtraction[C]∥2013 12th International Conference on Proceedings of Machine Learning and Applications (ICMLA).Miami,FL:IEEE,2013:209-214 [15] Carreira-Perpinan M A,Hinton G E.On Contrastive Divergence Learning[C]∥Proceedings of Proceedings of the tenth international workshop on Artificial Intelligence and Statistics.Barbados:AISTATS,2005:33-40 [16] Salakhutdinov R,Mnih A,Hinton G.Restricted Boltzmann Machines for Collaborative Filtering[C]∥Procee-dings of Procee-dings of the 24th International Conference on Machine Learning.New York:ACM,2007:791-798 [17] Pineda F J.Generalization of Back-Propagation to RecurrentNeural Networks[J].Physical Review Letters,1987,59(19):307-308 [18] LeCun Y,Cortes C,Burges C J.The Mnist Database of Handwritten Digits.http://yann.lecun.com/exdb/mnist [19] Berg T L,Berg A C,Shih J.Automatic Attribute Discovery and Characterization from Noisy Web Data[C]∥Proceedings of Computer Vision(ECCV 2010).Crete,Greece:ECCV,2010:663-676 |
No related articles found! |
|