Computer Science ›› 2015, Vol. 42 ›› Issue (5): 28-33.doi: 10.11896/j.issn.1002-137X.2015.05.006

Previous Articles     Next Articles

Research Progress on Deep Learning

GUO Li-li and DING Shi-fei   

  • Online:2018-11-14 Published:2018-11-14

Abstract: Deep learning plays an important role in machine learning.If shallow learning is a wave of machine learning, as a new field of machine learning,the deep learning will set off another wave of machine learning.Deep learning establishes and simulates the human brain’s hierarchical structure to extract the external input data’s features from lower to higher,which can explain the external data.Firstly,this paper discussed the origin of deep learning.Secondly,it described the common methods of deep learning illustrated by the example of supervised lear-ning and unsupervised learning.Then it generalized deep learning’s recent research and applications.Finally,it concluded the problems of development.

Key words: Machine learning,Shallow learning,Deep learning,CNNs,DBNs

[1] 丁世飞.人工智能[M].北京:清华大学出版社,2011
[2] 史忠值.神经网络[M].北京:高等教育出版社,2009
[3] Rumelhart D,Hinton G,Williams R.Learning representationsby back-propagating errors[J].Nature,1986,323(6088):533-536
[4] 余凯,贾磊,陈雨强.深度学习的昨天、今天和明天[J].计算机研究与发展,2013,50(9):1799-1804
[5] Hinton G,Salakhutdinov R.Reducing the dimensionality of data with neural networks[J].Science,2006,313(5786):504-507
[6] Ding Shi-fei,Zhang Yan-an,Chen Jin-rong,et al.Research onUsing Genetic Algorithms to Optimize Elman Neural Networks[J].Neural Computing and Applications,2013,23(2):293-297
[7] Ding Shi-fei,Jia Wei-kuan,Su Chun-yang,et al.Research ofNeural Network Algorithm Based on Factor Analysis and Cluster Analysis[J].Neural Computing and Applications,2011,20(2):297-302
[8] Lee T S,Mumford D.Hierarchical Bayesian inference in the vi-sual cortex[J].Optical Society of America,2003,20(7):1434-1448
[9] Serre T,Wolf L,Bileschi S,et al.Robust object recognition with cortex-like mechanisms[J].IEEE Trans on Pattern Analysis and Machine Intelligence,2007,29(3):411-426
[10] Lee T S,Mumford D,Romero R,et al.The role of the primary visual cortex in higher level vision[J].Vision Research,1998,38 (15):2429-2454
[11] Bengio Y.Learning deep architectures for AI[J].Foundations and Trends in Machine Learning,2009,2(1):1-127
[12] Bengio Y,LeCun Y.Scaling learning algorithms towards AI[M]∥Bottou L,Chapelle O,Decoste D,et al.Large-Scale Kernel Machines.Cambridge:MIT Press,2007:321-358
[13] 李海峰,李纯果.深度学习结构和算法比较分析[J].河北大学学报:自然科学版,2012,32(5):538-544
[14] Hinton G E.Learning distributed representations of concepts[C]∥Proc.of the 8th Annual Conference of the Cognitive ScienceSociety.1986:1-12
[15] 孙志军,薛磊,许阳明.深度学习研究综述[J].计算机应用研究,2012,29(8):2806-2810
[16] Bengio Y,Delalleau O.On the expressive power of deep architectures[C]∥Proceedings of the 22nd International Conference on Algorithmic Learning Theory.Berlin Heidelberg,2011:18-36
[17] Vincent P,Larochelle H,Lajoie I,et al.Stacked denoising autoencoders:learning useful representations in a deep network with a local denoising criterion[J].Journal of Machine Learning Research,2010,11(12):3371-3408
[18] Hubel D H,Wiesel T N.Receptive Fields,Binocular Interaction and Functional Architecture in the Cat’s Visual Cortex [J].Journal of Physiology,1962,160:106-154
[19] Fukushima K.Neocognition:A Self-Organizing Neural Network Model for a mechanism of Pattern Recognition Unaffected by Shift in Postion[J].Biological Cybermetics,1980,36:193-202
[20] LeCun Y,Bottou L,Bengio Y,et al.Granient-based learning applied to document recognition[J].Proceedings of IEEE,1988,6(11):2278-2324
[21] LeCun Y,Boser B,Denker J S,et al.Backpropagation Applied to Handwritten Zip Code Recognition[J].Neural Computation,1989,1(4):541-551
[22] Serre T,Keriman G,Kouch M.A Quantitative Theory of Immediate Visual Recognition” Progress in Brain Research,Computational Neuroscience[J].Theoretical Insights into Brain Function,2007,165:33-56
[23] LeCun Y,Kavukcuogl U K,Farabe C.Convolutional networksand applications in vision[Z].International Symposium on Circuits and Systems,Paris,2010
[24] Kwolek B.Face Detection Using Convolutional Neural Net-works And Gobor Filters[J].Artificial Neural Networks:Biological Inspirations,2005,3699:551-556
[25] Rosenblatt F.The Perceptron:A Probabilistic Model For Information Storage and Organization in the Brain[J].Psychological Review,1958,65:386-408
[26] Neubauer C.Shape,position and size invariant visual patternrecognition based on principles of neocognitron and perception in artificial neural networks[M].Netherlands:North Holland,1992:833-837
[27] Laserson J.From neural networks to deep learning:zeroing in on the human brain[J].ACM Crossroads Student Magazine,2011,18(1):29-34
[28] LeCun Y,Botton L,Bengio Y,et al.Gradient-based learning applied to document recognition[J].Proceedings of IEEE,1998,86(11):2278-2324
[29] Vincent P,Larochelle H,Bengio Y,et al.Extracting and composing robust features with denoising autoencoders[C]∥Proceedings of the 25th International Conference on Machine Learning (ICML’2008).New York:ACM Press,2008:1096-1103
[30] Huang Fu-jie,LeCun Y.Large-scale learning with SVM andconvolutional for generic object categorization [C]∥Proc.of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.Washington DC:IEEE Computer Society,2006:284-291
[31] 聂仁灿,姚绍文,周冬明.基于简化脉冲耦合神经网络的人脸识别[J].计算机科学,2014,41(2):297-301
[32] Simard P Y,Steinkraus D,Platt J C.Best Practice for Convolutional Neural Networks Applied to Visual Document Analysis[C]∥Seventh International Conference on Document Analysis and Recognition.2003:963-985
[33] Sukittanon S,Surendran A C,Burges C J C,et al.Convolutional Networks for speech Detection.http://131.107.65.14/pubs/68033/convnet_speechdetect.pdf
[34] Chen Y,Han C,Wang C,et al.The application of a convolution neural network on face and license plate detection[C]∥18th International Conference on Pattern Recognition(ICPR).Hong Kong,China:IEEE Computer Society,2006:552-555
[35] 赵元庆,吴华.多尺度特征和神经网络相融合的手写体数字识别[J].计算机科学,2013,40(8):316-318
[36] Ji S,Xu W,Yang M,et al.3D Convolutional Neural Networks for Human Action Recognition[J].IEEE Transaction on Pattern Analysis and Machine Intelligence,2013,35(1):221-231
[37] Hinton G E.Distributed representations[R].Tech.Report,Uni-versity of Toronto,1984
[38] Hinton G E,Osindero S.A fast learning algorithm for deep belief nets[J].Neural Computation,2006,18:1527-1554
[39] Lawrence C E,Altschul S F,Boguski M S,et al.Detecting Subtle Sequence Signals:A Gibbs Sampling Strategy for Multiple Alignment[J].Science,1993,262:208-214
[40] Bishop C M.Pattern recognition and machine learning[M].New York:Springer,2006
[41] Hinton G E,Dayanp,Frey B,et al.The wake-sleep algorithm for unsupervised neural network[J].Science,1995,268:1158-1161
[42] Liu Yan,Zhou Shu-sen,Chen Qing-cai.Discriminative deep belief networks for visual data classification[J].Pattern Recognition,2010,44(10):2287-2296
[43] Yu Dong,Deng Li.Deep convex net:a scalable architecture for speech pattern classification[C]∥Proc.of the 12th Annual Conference of International Speech Communication Association.2011:2296-2299
[44] Zhou Shu-sen,Chen Qing-cai,Wang Xiao-long.Convolutional Deep Networks for Visual Data Classification[J].Neural Process Lett,2013,38:17-27
[45] Wong W K,Sun M M.Deep learning regularized fisher mappings[J].IEEE Transactions on Neural Networks,2011,22(10):1668-1675
[46] 孙志军,薛磊,许阳明.基于深度学的边际Fisher分析特征提取算法[J].电子与信息学报,2013,35(4):805-811
[47] Zhou Shu-sen,Chen Qing-cai,Wang Xiao-long.Active deeplearning method for semi-supervised sentiment classification[J].Neurocomputing,2013,120:536-546
[48] Tom A.Stanford algorithm analyzes sentence sentiment,ad-vances machine learning [N].Stanford University,2013
[49] Li Fei-fei,Fergus R,Perona P.Learning generative visual mo-dels from few training examples:an incremental Bayesian approach tested on 101 object categories[J].Computer Vision and Image Understanding,2004,106(1):59-70
[50] Hinton G E,Li Deng,Dong Yu,et al.Deep neural networks for acoustic modeling in speech recognition [J].IEEE Signal Processing Magazine,2012,29(6):82-97
[51] Markoff J.Scientists see promise in deep-learning programs[N].The New York Times,2012-11-23
[52] Bach F,Jenatton R,Obozinski G.Structured sparsity throughconvex optimization.http:arxiv.org/pdf.1109.2397/pdf
[53] Bengio Y,Courville A,Vincent P.Representation Learning:AReview and New Perspectives[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2013,35(8):1798-1828
[54] Markoff J.How many computers to identify a cat?[N].The new York Times,2012-06-25
[55] 李彦宏.2012百度年会主题报告:相信技术的力量[R].北京:百度,2013
[56] 10 Breakthrough Technologies 2013 [N].MIT Technology Review,2013-04-2

No related articles found!
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!