Computer Science ›› 2018, Vol. 45 ›› Issue (6A): 11-15.

• Review • Previous Articles     Next Articles

Review of Principle and Application of Deep Learning

FU Wen-bo1,SUN Tao2,LIANG Ji1,YAN Bao-wei1,FAN Fu-xin1   

  1. Huazhong University of Science and Technology,Wuhan 430074,China1
    China Institute of Water Resources and Hydropower Research,Beijing 100044,China2
  • Online:2018-06-20 Published:2018-08-03

Abstract: As an important technical means of machine learning,deep learning has a broad application prospect.This article briefly described the development of deep learning,introduced convolutional neural network,restricted boltzmann machine,auto encoder and its derived series method model,andsix kinds of mainstream depth frame such as Caffe,TensorFlow,Torch.This paper also discussed the application of deep learning in image,speech,video,text and data analysis,analyzed the existing problems and future trends of deep learning,providing a more comprehensive method guidance and literature index support for beginners.

Key words: Application, Auto encoder, Convolutional neural networks, Deep learning, Framework, Neural networks, Restricted Boltzmann machine

CLC Number: 

  • TP18
[1]MCCULLOCH W S,PITTS W H.A logical calculus of ideas imminent in nervous activity.Biol Math Biophys,1943,5:115-133 [2]ROSENBLATT F.Two Theorems of Statistical Separability in the Perceptron.https://www.researchgate.net/publication/243614378_Two_Theorems_of_Statistical_Separability_in_the_Perceptron.
[3]MINSKY M L,PAPERT S.Perceptrons:an introduction to com- putational geometry[M]∥Perceptrons:An Introduction to Computational Geometry.The MIT Press,1969:3356-62.
[4]RUMELHART D E,HINTON G E,WILLIAMS R J.Learning internal representation by back-propagation of errors.Nature,1986,323(323):533-536.
[5]HINTON G E,OSINDERO S,TEH Y W.A Fast Learning Algorithm for Deep Belief Nets.Neural Computation,2006,18(7):1527.
[6]BROWNE M,GHIDARY S S.Convolutional Neural Networks for Image Processing:An Application in Robot Vision[C]∥Conference on Ai:Advances in Artificial Intelligence.DBLP,2003:641-652.
[7]HAYKIN S,KOSKO B.Gradient Based Learning Applied to Document Recognition.New Jersey:Wiley-IEEE Press,2009:306-351.
[8]LECUN Y,BENGIO Y,HINTON G.Deep learning.Nature,2015,521(7553):436-444.
[9]PALIY I.Face detection using Haar-like features cascade and convolutional neural network[C]∥Proceedings of International Conference on Modern Problems of Radio Engineering,Telecommunications and Computer Science.IEEE,2008:375-377.
[10]HUBEL D H,WIESEL T N.Receptive fields,binocular interaction and functional architecture in the cat’s visual cortex.Journal of Physiology,1962,160(1):106-154.
[11]KRUTHIVENTI S S S,AYUSH K,BABU R V.DeepFix:A Fully Convolutional Neural Network for Predicting Human Eye Fixations.IEEE Transactions on Image Processing,2015,26(9):4446-4456.
[12]郭昕,陈琳,沉敏.分层自适应深度卷积神经网络及其在轴承故障诊断中的应用.测量,2016,93:490-502.
[13]SMOLENSKY P.Information processing in dynamical systems: foundations of harmony theory[M]∥Parallel distributed processing:explorations in the microstructure of cognition.MIT Press,1986:194-281.
[14]SALAKHUTDINOV R,LAROCHELLE H.Efficient Learning of Deep Boltzmann Machines.https://www.researchgate.net/publication/220320264_Efficient_Learning_of_Deep_Boltzmann_Machines.
[15]罗恒.基于协同过滤视角的受限玻尔兹曼机研究.上海:上海交通大学,2011.
[16]LAROCHELLE H,MANDEL M,PASCANU R,et al.Learning algorithms for the classification restricted Boltzmann machine.Journal of Machine Learning Research,2012,13(1):643-669.
[17]SCHLKOPF B,PLATT J,HOFMANN T.Greedy Layer- Wise Training of Deep Networks[C]∥International Conference on Neural Information Processing Systems.MIT Press,2006:153-160.
[18]LEE H,GROSSE R,RANGANATH R,et al.Unsupervised learning of hierarchical representations with convolutional deep belief networks.Communications of the Acm,2011,54(10):95-103.
[19]HALKIAS X,PARIS S ,GLOTIN H.Sparse Penalty in Deep Belief Networks:Using the Mixed Norm Constraint .(2013-01-16).http://arxiv.org/pdf/1301.3533vl.pdf.
[20]GOODFELLOW I J,POUGET-ABADIE J,MIRZA M,et al. Generative Adversarial Networks.Advances in Neural Information Processing Systems,2014,3:2672-2680.
[21]HINTON G E,SALAKHUTDINOV R R.Reducing the Dimensionality of Data with Neural Networks.Science,2006,313(5786):504-507.
[22]BENGIO Y,LAMBLIN P,POPOVICI D,et al.Greedy layer-wise training of deep networks[C]∥International Conference on Neural Information Processing Systems.MIT Press,2006:153-160.
[23]RANZATO M,BOUREAU Y L,LECUN Y.Sparse feature learning for deep belief networks[C]∥Advances in Neural Information Processing Systems.2007:1185-1192.
[24]VINCENT P,LAROCHELLE H,BENGIO Y,et al.Extracting and composing robust features with denoising autoencoders[C]∥27th International Conference on Machine Learning.ACM.2008:1096-1103.
[25]VINCENT P,LAROCHELLE H,LAJOIE I,et al.Stacked Denoising Autoencoders:Learning Useful Representations in a Deep Network with a Local Denoising Criterion..Journal of Machine Learning Research,2010,11(12):3371-3408.
[26]MASCI J,MEIER U,DAN C,et al.Stacked Convolutional Auto-Encoders for Hierarchical Feature Extraction[C]∥International Conference on Artificial Neural Networks.2011:52-59.
[27]JIANG X,ZHANG Y,ZHANG W,et al.A novel sparse auto-encoder for deep unsupervised learning[C]∥Sixth International Conference on Advanced Computational Intelligence.IEEE,2014:256-261.
[28]JIA Y,SHELHAMER E,DONAHUE J,et al.Caffe:Convolutional Architecture for Fast Feature Embedding[C]∥Acm International Conference on Multimedia.ACM,2014:675-678.
[29]ABADI M.TensorFlow:learning functions at scale.AcmSigplan Notices,2016,51(9):1.
[30]SEIDE F.Keynote:The computer science behind the Microsoft Cognitive Toolkit:An open source large-scale deep learning toolkit for Windows and Linux[C]∥Ieee/acm International Symposium on Code Generation and Optimization.IEEE,2017.
[31]YANG H,FRITZSCHE M,BARTZ C,et al.BMXNet:An Open- Source Binary Neural Network Implementation Based on MXNet[C]∥ACM.2017:1209-1212.
[32]SANKARAN A,ARALIKATTE R,MANI S,et al.DARVIZ:deep abstract representation,visualization,and verification of deep learning models[C]∥Ieee/acm,International Conference on Software Engineering:New Ideas and Emerging Technologies Results Track.IEEE,2017:47-50.
[33]BOUFENAR C,BATOUCHE M.Investigation on deep learning for off-line handwritten Arabic Character Recognition using Theano research platform[C]∥Intelligent Systems and Computer Vision.IEEE,2017.
[34]魏英姿,谭龙田,欧阳海飞,等.玉米籽粒完整性识别的深度学习方法.沈阳理工大学学报,2016,35(4):1-6.
[35]李楠.基于深度学习框架Caffe的路面裂缝识别研究.工程技术研究,2017(3):20,28.
[36]杜敬.基于深度学习的无人机遥感影像水体识别.江西科学,2017,35(1):158-161.
[37]寇广,汤光明,王硕,等.深度学习在僵尸云检测中的应用研究.通信学报,2016,37(11):114-128.
[38]刘大伟,韩玲,韩晓勇.基于深度学习的高分辨率遥感影像分类研究.光学学报,2016,36(4):298-306.
[39]樊恒,徐俊,邓勇,等.基于深度学习的人体行为识别.武汉大学学报(信息科学版),2016,41(4):492-497.
[40]李婷.基于深度学习的盲文识别方法.计算机与现代化,2015,238(6):37-40.
[41]阚希,张永宏,曹庭,等.利用多光谱卫星遥感和深度学习方法进行青藏高原积雪判识.测绘学报,2016,45(10):1210-1221.
[42]HINTON G,DENG L,YU D,et al.Deep Neural Networks for Acoustic Modeling in Speech Recognition:The Shared Views of Four Research Groups.IEEE Signal Processing Magazine,2012,29(6):82-97.
[43]王山海,景新幸,杨海燕.基于深度学习神经网络的孤立词语音识别的研究.计算机应用研究,2015,32(8):2289-2291.
[44]王强,曾向阳.深度学习方法及其在水下目标识别中的应用[C]∥中国声学学会水声学分会学术会议.2015:139-141.
[45]张剑.连续语音识别中的循环神经网络语言模型技术研究.郑州:解放军信息工程大学,2014.
[46]LU X G,TSAO Y,MATSUDA S,et al.Speech enhancement based on deep denoising autoencoder∥The 14th Annual Conference of the International Speech Communication Association.Lyon:ISCA Archive,2013:436-440 [47]GREFF K,SRIVASTAVA R K,KOUTN K J,et al.LSTM:A Search Space Odyssey.IEEE Transactions on Neural Networks & Learning Systems,2015,28(10):2222-2232.
[48]杨红红,曲仕茹.基于稀疏约束深度学习的交通目标跟踪.中国公路学报,2016,29(6):253-261.
[49]司朋举,胡伟.深度学习在仿生眼监控系统中的应用.测控技术,2016,35(12):139-143.
[50]王慧燕,杨宇涛,张政,等.深度学习辅助的多行人跟踪算法.中国图象图形学报,2017,22(3):349-357.
[51]文孟飞,胡超,刘伟荣.一种基于深度学习的异构多模态目标识别方法.中南大学学报(自然科学版),2016,47(5):1580-1587.
[52]张立民,刘凯.基于深度玻尔兹曼机的文本特征提取研究.微电子学与计算机,2015,32(2):142-147.
[53]周盈盈,范磊.基于改进词向量模型的深度学习文本主题分类.计算机科学与应用,2016,6(11):629-637.
[54]李阳辉,谢明,易阳.基于深度学习的社交网络平台细粒度情感分析.计算机应用研究,2017,34(3):743-747.
[55]雷亚国,贾峰,周昕,等.基于深度学习理论的机械装备大数据健康监测方法.机械工程学报,2015,51(21):49-56.
[56]潘儒扬,李乡儒.基于深度学习技术的恒星大气物理参数自动估计.天文学报,2016,57(4):379-388.
[57]谭娟,王胜春.基于深度学习的交通拥堵预测模型研究.计算机应用研究,2015,32(10):2951-2954.
[58]潘志刚,刘三明,李莹,等.基于深度学习网络的风电场功率短期预测研究.科技与创新,2015,43(19):4-6.
[59]夏春江,王培良,张媛.基于深度学习的木材含水率预测.杭州电子科技大学学报,2015,35(1):31-35.
[60]姚俊杨,许继平,王小艺,等.基于深度学习的湖库藻类水华预测研究.计算机与应用化学,2015,32(10):1265-1268.
[1] XU Yong-xin, ZHAO Jun-feng, WANG Ya-sha, XIE Bing, YANG Kai. Temporal Knowledge Graph Representation Learning [J]. Computer Science, 2022, 49(9): 162-171.
[2] RAO Zhi-shuang, JIA Zhen, ZHANG Fan, LI Tian-rui. Key-Value Relational Memory Networks for Question Answering over Knowledge Graph [J]. Computer Science, 2022, 49(9): 202-207.
[3] HU Yu-jiao, JIA Qing-min, SUN Qing-shuang, XIE Ren-chao, HUANG Tao. Functional Architecture to Intelligent Computing Power Network [J]. Computer Science, 2022, 49(9): 249-259.
[4] NING Han-yang, MA Miao, YANG Bo, LIU Shi-chang. Research Progress and Analysis on Intelligent Cryptology [J]. Computer Science, 2022, 49(9): 288-296.
[5] TANG Ling-tao, WANG Di, ZHANG Lu-fei, LIU Sheng-yun. Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy [J]. Computer Science, 2022, 49(9): 297-305.
[6] WANG Jian, PENG Yu-qi, ZHAO Yu-fei, YANG Jian. Survey of Social Network Public Opinion Information Extraction Based on Deep Learning [J]. Computer Science, 2022, 49(8): 279-293.
[7] HAO Zhi-rong, CHEN Long, HUANG Jia-cheng. Class Discriminative Universal Adversarial Attack for Text Classification [J]. Computer Science, 2022, 49(8): 323-329.
[8] JIANG Meng-han, LI Shao-mei, ZHENG Hong-hao, ZHANG Jian-peng. Rumor Detection Model Based on Improved Position Embedding [J]. Computer Science, 2022, 49(8): 330-335.
[9] ZHU Cheng-zhang, HUANG Jia-er, XIAO Ya-long, WANG Han, ZOU Bei-ji. Deep Hash Retrieval Algorithm for Medical Images Based on Attention Mechanism [J]. Computer Science, 2022, 49(8): 113-119.
[10] SUN Qi, JI Gen-lin, ZHANG Jie. Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection [J]. Computer Science, 2022, 49(8): 172-177.
[11] HOU Yu-tao, ABULIZI Abudukelimu, ABUDUKELIMU Halidanmu. Advances in Chinese Pre-training Models [J]. Computer Science, 2022, 49(7): 148-163.
[12] ZHOU Hui, SHI Hao-chen, TU Yao-feng, HUANG Sheng-jun. Robust Deep Neural Network Learning Based on Active Sampling [J]. Computer Science, 2022, 49(7): 164-169.
[13] SU Dan-ning, CAO Gui-tao, WANG Yan-nan, WANG Hong, REN He. Survey of Deep Learning for Radar Emitter Identification Based on Small Sample [J]. Computer Science, 2022, 49(7): 226-235.
[14] HU Yan-yu, ZHAO Long, DONG Xiang-jun. Two-stage Deep Feature Selection Extraction Algorithm for Cancer Classification [J]. Computer Science, 2022, 49(7): 73-78.
[15] CHENG Cheng, JIANG Ai-lian. Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction [J]. Computer Science, 2022, 49(7): 120-126.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!