深度学习原理及应用综述

摘要/Abstract

摘要： 深度学习作为机器学习领域中重要的技术手段,有着广阔的应用前景。文中简述了深度学习的发展历程,介绍了卷积神经网络、受限玻尔兹曼机、自动编码器及其衍生的系列方法模型,以及Caffe,TensorFlow,Torch等6种主流深度框架;论述了深度学习在图像、语音、视频、文本、数据分析方面的应用情况,分析了深度学习现阶段存在的问题以及未来的发展趋势,为初学者提供了较全面的方法指导与文献索引支持。

关键词: 卷积神经网络, 框架, 深度学习, 神经网络, 受限玻尔兹曼机, 应用, 自动编码器

Abstract: As an important technical means of machine learning,deep learning has a broad application prospect.This article briefly described the development of deep learning,introduced convolutional neural network,restricted boltzmann machine,auto encoder and its derived series method model,andsix kinds of mainstream depth frame such as Caffe,TensorFlow,Torch.This paper also discussed the application of deep learning in image,speech,video,text and data analysis,analyzed the existing problems and future trends of deep learning,providing a more comprehensive method guidance and literature index support for beginners.

Key words: Application, Auto encoder, Convolutional neural networks, Deep learning, Framework, Neural networks, Restricted Boltzmann machine

中图分类号:

TP18

付文博, 孙涛, 梁藉, 闫宝伟, 范福新. 深度学习原理及应用综述[J]. 计算机科学, 2018, 45(6A): 11-15. https://doi.org/

FU Wen-bo, SUN Tao, LIANG Ji, YAN Bao-wei, FAN Fu-xin. Review of Principle and Application of Deep Learning[J]. Computer Science, 2018, 45(6A): 11-15. https://doi.org/

参考文献

[1]MCCULLOCH W S,PITTS W H.A logical calculus of ideas imminent in nervous activity.Biol Math Biophys,1943,5:115-133 [2]ROSENBLATT F.Two Theorems of Statistical Separability in the Perceptron.https://www.researchgate.net/publication/243614378_Two_Theorems_of_Statistical_Separability_in_the_Perceptron.
[3]MINSKY M L,PAPERT S.Perceptrons:an introduction to com- putational geometry[M]∥Perceptrons:An Introduction to Computational Geometry.The MIT Press,1969:3356-62.
[4]RUMELHART D E,HINTON G E,WILLIAMS R J.Learning internal representation by back-propagation of errors.Nature,1986,323(323):533-536.
[5]HINTON G E,OSINDERO S,TEH Y W.A Fast Learning Algorithm for Deep Belief Nets.Neural Computation,2006,18(7):1527.
[6]BROWNE M,GHIDARY S S.Convolutional Neural Networks for Image Processing:An Application in Robot Vision[C]∥Conference on Ai:Advances in Artificial Intelligence.DBLP,2003:641-652.
[7]HAYKIN S,KOSKO B.Gradient Based Learning Applied to Document Recognition.New Jersey:Wiley-IEEE Press,2009:306-351.
[8]LECUN Y,BENGIO Y,HINTON G.Deep learning.Nature,2015,521(7553):436-444.
[9]PALIY I.Face detection using Haar-like features cascade and convolutional neural network[C]∥Proceedings of International Conference on Modern Problems of Radio Engineering,Telecommunications and Computer Science.IEEE,2008:375-377.
[10]HUBEL D H,WIESEL T N.Receptive fields,binocular interaction and functional architecture in the cat’s visual cortex.Journal of Physiology,1962,160(1):106-154.
[11]KRUTHIVENTI S S S,AYUSH K,BABU R V.DeepFix:A Fully Convolutional Neural Network for Predicting Human Eye Fixations.IEEE Transactions on Image Processing,2015,26(9):4446-4456.
[12]郭昕,陈琳,沉敏.分层自适应深度卷积神经网络及其在轴承故障诊断中的应用.测量,2016,93:490-502.
[13]SMOLENSKY P.Information processing in dynamical systems: foundations of harmony theory[M]∥Parallel distributed processing:explorations in the microstructure of cognition.MIT Press,1986:194-281.
[14]SALAKHUTDINOV R,LAROCHELLE H.Efficient Learning of Deep Boltzmann Machines.https://www.researchgate.net/publication/220320264_Efficient_Learning_of_Deep_Boltzmann_Machines.
[15]罗恒.基于协同过滤视角的受限玻尔兹曼机研究.上海:上海交通大学,2011.
[16]LAROCHELLE H,MANDEL M,PASCANU R,et al.Learning algorithms for the classification restricted Boltzmann machine.Journal of Machine Learning Research,2012,13(1):643-669.
[17]SCHLKOPF B,PLATT J,HOFMANN T.Greedy Layer- Wise Training of Deep Networks[C]∥International Conference on Neural Information Processing Systems.MIT Press,2006:153-160.
[18]LEE H,GROSSE R,RANGANATH R,et al.Unsupervised learning of hierarchical representations with convolutional deep belief networks.Communications of the Acm,2011,54(10):95-103.
[19]HALKIAS X,PARIS S ,GLOTIN H.Sparse Penalty in Deep Belief Networks:Using the Mixed Norm Constraint .(2013-01-16).http://arxiv.org/pdf/1301.3533vl.pdf.
[20]GOODFELLOW I J,POUGET-ABADIE J,MIRZA M,et al. Generative Adversarial Networks.Advances in Neural Information Processing Systems,2014,3:2672-2680.
[21]HINTON G E,SALAKHUTDINOV R R.Reducing the Dimensionality of Data with Neural Networks.Science,2006,313(5786):504-507.
[22]BENGIO Y,LAMBLIN P,POPOVICI D,et al.Greedy layer-wise training of deep networks[C]∥International Conference on Neural Information Processing Systems.MIT Press,2006:153-160.
[23]RANZATO M,BOUREAU Y L,LECUN Y.Sparse feature learning for deep belief networks[C]∥Advances in Neural Information Processing Systems.2007:1185-1192.
[24]VINCENT P,LAROCHELLE H,BENGIO Y,et al.Extracting and composing robust features with denoising autoencoders[C]∥27th International Conference on Machine Learning.ACM.2008:1096-1103.
[25]VINCENT P,LAROCHELLE H,LAJOIE I,et al.Stacked Denoising Autoencoders:Learning Useful Representations in a Deep Network with a Local Denoising Criterion..Journal of Machine Learning Research,2010,11(12):3371-3408.
[26]MASCI J,MEIER U,DAN C,et al.Stacked Convolutional Auto-Encoders for Hierarchical Feature Extraction[C]∥International Conference on Artificial Neural Networks.2011:52-59.
[27]JIANG X,ZHANG Y,ZHANG W,et al.A novel sparse auto-encoder for deep unsupervised learning[C]∥Sixth International Conference on Advanced Computational Intelligence.IEEE,2014:256-261.
[28]JIA Y,SHELHAMER E,DONAHUE J,et al.Caffe:Convolutional Architecture for Fast Feature Embedding[C]∥Acm International Conference on Multimedia.ACM,2014:675-678.
[29]ABADI M.TensorFlow:learning functions at scale.AcmSigplan Notices,2016,51(9):1.
[30]SEIDE F.Keynote:The computer science behind the Microsoft Cognitive Toolkit:An open source large-scale deep learning toolkit for Windows and Linux[C]∥Ieee/acm International Symposium on Code Generation and Optimization.IEEE,2017.
[31]YANG H,FRITZSCHE M,BARTZ C,et al.BMXNet:An Open- Source Binary Neural Network Implementation Based on MXNet[C]∥ACM.2017:1209-1212.
[32]SANKARAN A,ARALIKATTE R,MANI S,et al.DARVIZ:deep abstract representation,visualization,and verification of deep learning models[C]∥Ieee/acm,International Conference on Software Engineering:New Ideas and Emerging Technologies Results Track.IEEE,2017:47-50.
[33]BOUFENAR C,BATOUCHE M.Investigation on deep learning for off-line handwritten Arabic Character Recognition using Theano research platform[C]∥Intelligent Systems and Computer Vision.IEEE,2017.
[34]魏英姿,谭龙田,欧阳海飞,等.玉米籽粒完整性识别的深度学习方法.沈阳理工大学学报,2016,35(4):1-6.
[35]李楠.基于深度学习框架Caffe的路面裂缝识别研究.工程技术研究,2017(3):20,28.
[36]杜敬.基于深度学习的无人机遥感影像水体识别.江西科学,2017,35(1):158-161.
[37]寇广,汤光明,王硕,等.深度学习在僵尸云检测中的应用研究.通信学报,2016,37(11):114-128.
[38]刘大伟,韩玲,韩晓勇.基于深度学习的高分辨率遥感影像分类研究.光学学报,2016,36(4):298-306.
[39]樊恒,徐俊,邓勇,等.基于深度学习的人体行为识别.武汉大学学报(信息科学版),2016,41(4):492-497.
[40]李婷.基于深度学习的盲文识别方法.计算机与现代化,2015,238(6):37-40.
[41]阚希,张永宏,曹庭,等.利用多光谱卫星遥感和深度学习方法进行青藏高原积雪判识.测绘学报,2016,45(10):1210-1221.
[42]HINTON G,DENG L,YU D,et al.Deep Neural Networks for Acoustic Modeling in Speech Recognition:The Shared Views of Four Research Groups.IEEE Signal Processing Magazine,2012,29(6):82-97.
[43]王山海,景新幸,杨海燕.基于深度学习神经网络的孤立词语音识别的研究.计算机应用研究,2015,32(8):2289-2291.
[44]王强,曾向阳.深度学习方法及其在水下目标识别中的应用[C]∥中国声学学会水声学分会学术会议.2015:139-141.
[45]张剑.连续语音识别中的循环神经网络语言模型技术研究.郑州:解放军信息工程大学,2014.
[46]LU X G,TSAO Y,MATSUDA S,et al.Speech enhancement based on deep denoising autoencoder∥The 14th Annual Conference of the International Speech Communication Association.Lyon:ISCA Archive,2013:436-440 [47]GREFF K,SRIVASTAVA R K,KOUTN K J,et al.LSTM:A Search Space Odyssey.IEEE Transactions on Neural Networks & Learning Systems,2015,28(10):2222-2232.
[48]杨红红,曲仕茹.基于稀疏约束深度学习的交通目标跟踪.中国公路学报,2016,29(6):253-261.
[49]司朋举,胡伟.深度学习在仿生眼监控系统中的应用.测控技术,2016,35(12):139-143.
[50]王慧燕,杨宇涛,张政,等.深度学习辅助的多行人跟踪算法.中国图象图形学报,2017,22(3):349-357.
[51]文孟飞,胡超,刘伟荣.一种基于深度学习的异构多模态目标识别方法.中南大学学报(自然科学版),2016,47(5):1580-1587.
[52]张立民,刘凯.基于深度玻尔兹曼机的文本特征提取研究.微电子学与计算机,2015,32(2):142-147.
[53]周盈盈,范磊.基于改进词向量模型的深度学习文本主题分类.计算机科学与应用,2016,6(11):629-637.
[54]李阳辉,谢明,易阳.基于深度学习的社交网络平台细粒度情感分析.计算机应用研究,2017,34(3):743-747.
[55]雷亚国,贾峰,周昕,等.基于深度学习理论的机械装备大数据健康监测方法.机械工程学报,2015,51(21):49-56.
[56]潘儒扬,李乡儒.基于深度学习技术的恒星大气物理参数自动估计.天文学报,2016,57(4):379-388.
[57]谭娟,王胜春.基于深度学习的交通拥堵预测模型研究.计算机应用研究,2015,32(10):2951-2954.
[58]潘志刚,刘三明,李莹,等.基于深度学习网络的风电场功率短期预测研究.科技与创新,2015,43(19):4-6.
[59]夏春江,王培良,张媛.基于深度学习的木材含水率预测.杭州电子科技大学学报,2015,35(1):31-35.
[60]姚俊杨,许继平,王小艺,等.基于深度学习的湖库藻类水华预测研究.计算机与应用化学,2015,32(10):1265-1268.

相关文章 15

[1]	饶志双, 贾真, 张凡, 李天瑞. 基于Key-Value关联记忆网络的知识图谱问答方法 Key-Value Relational Memory Networks for Question Answering over Knowledge Graph 计算机科学, 2022, 49(9): 202-207. https://doi.org/10.11896/jsjkx.220300277
[2]	宁晗阳, 马苗, 杨波, 刘士昌. 密码学智能化研究进展与分析 Research Progress and Analysis on Intelligent Cryptology 计算机科学, 2022, 49(9): 288-296. https://doi.org/10.11896/jsjkx.220300053
[3]	汤凌韬, 王迪, 张鲁飞, 刘盛云. 基于安全多方计算和差分隐私的联邦学习方案 Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy 计算机科学, 2022, 49(9): 297-305. https://doi.org/10.11896/jsjkx.210800108
[4]	周芳泉, 成卫青. 基于全局增强图神经网络的序列推荐 Sequence Recommendation Based on Global Enhanced Graph Neural Network 计算机科学, 2022, 49(9): 55-63. https://doi.org/10.11896/jsjkx.210700085
[5]	周乐员, 张剑华, 袁甜甜, 陈胜勇. 多层注意力机制融合的序列到序列中国连续手语识别和翻译 Sequence-to-Sequence Chinese Continuous Sign Language Recognition and Translation with Multi- layer Attention Mechanism Fusion 计算机科学, 2022, 49(9): 155-161. https://doi.org/10.11896/jsjkx.210800026
[6]	徐涌鑫, 赵俊峰, 王亚沙, 谢冰, 杨恺. 时序知识图谱表示学习 Temporal Knowledge Graph Representation Learning 计算机科学, 2022, 49(9): 162-171. https://doi.org/10.11896/jsjkx.220500204
[7]	李宗民, 张玉鹏, 刘玉杰, 李华. 基于可变形图卷积的点云表征学习 Deformable Graph Convolutional Networks Based Point Cloud Representation Learning 计算机科学, 2022, 49(8): 273-278. https://doi.org/10.11896/jsjkx.210900023
[8]	王剑, 彭雨琦, 赵宇斐, 杨健. 基于深度学习的社交网络舆情信息抽取方法综述 Survey of Social Network Public Opinion Information Extraction Based on Deep Learning 计算机科学, 2022, 49(8): 279-293. https://doi.org/10.11896/jsjkx.220300099
[9]	郝志荣, 陈龙, 黄嘉成. 面向文本分类的类别区分式通用对抗攻击方法 Class Discriminative Universal Adversarial Attack for Text Classification 计算机科学, 2022, 49(8): 323-329. https://doi.org/10.11896/jsjkx.220200077
[10]	姜梦函, 李邵梅, 郑洪浩, 张建朋. 基于改进位置编码的谣言检测模型 Rumor Detection Model Based on Improved Position Embedding 计算机科学, 2022, 49(8): 330-335. https://doi.org/10.11896/jsjkx.210600046
[11]	王润安, 邹兆年. 基于物理操作级模型的查询执行时间预测方法 Query Performance Prediction Based on Physical Operation-level Models 计算机科学, 2022, 49(8): 49-55. https://doi.org/10.11896/jsjkx.210700074
[12]	陈泳全, 姜瑛. 基于卷积神经网络的APP用户行为分析方法 Analysis Method of APP User Behavior Based on Convolutional Neural Network 计算机科学, 2022, 49(8): 78-85. https://doi.org/10.11896/jsjkx.210700121
[13]	朱承璋, 黄嘉儿, 肖亚龙, 王晗, 邹北骥. 基于注意力机制的医学影像深度哈希检索算法 Deep Hash Retrieval Algorithm for Medical Images Based on Attention Mechanism 计算机科学, 2022, 49(8): 113-119. https://doi.org/10.11896/jsjkx.210700153
[14]	孙奇, 吉根林, 张杰. 基于非局部注意力生成对抗网络的视频异常事件检测方法 Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection 计算机科学, 2022, 49(8): 172-177. https://doi.org/10.11896/jsjkx.210600061
[15]	檀莹莹, 王俊丽, 张超波. 基于图卷积神经网络的文本分类方法研究综述 Review of Text Classification Methods Based on Graph Convolutional Network 计算机科学, 2022, 49(8): 205-216. https://doi.org/10.11896/jsjkx.210800064

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed