计算机科学 ›› 2019, Vol. 46 ›› Issue (7): 206-210.doi: 10.11896/j.issn.1002-137X.2019.07.031
麦应潮,陈云华,张灵
MAI Ying-chao,CHEN Yun-hua,ZHANG Ling
摘要: 当前人工神经网络虽然在图像识别等方面媲美人脑,但因其所采用的激活函数ReLU和Softplus等只是对生物神经元输出响应特性的高度简化与模拟,使其在抗噪性、不确定性信息处理及功耗等方面与人脑仍存在巨大差距。通过分析生物神经元仿真实验,以其响应特性为基础,引入反映每个神经元随机性的参数η,构建出一种具有生物真实性的强抗噪性激活函数Rand Softplus。最后将该激活函数应用于深度残差网络,并基于人脸表情数据集对其进行验证。结果表明,在输入无噪声或具有少量噪声时,文中提出的激活函数与当前主流激活函数的识别精度基本持平,当输入包含较大噪声时,文中所提激活函数的识别精度远高于其他激活函数,表现出了良好的抗噪性能。
中图分类号:
[1]LECUN Y,BENGIO Y,HINTON G.Deep learning [J].Nature,2015,521(7553):436. [2]SCHMIDHUBER J.Deep learning in neural networks:An overview[J].Neural Networks,2015,61:85-117. [3]BADJATIYA P,GUPTA S,GUPTA M,et al.Deep Learning for Hate Speech Detection in Tweets[C]∥Proceedings of the 26th International Conference on World Wide Web Companion.Perth:International World Wide Web Conferences Steering Committee,2017:759-760. [4]KIM B K,LEE H,ROH J,et al.Hierarchical Committee of Deep CNNs with Exponentially-Weighted Decision Fusion for Static Facial Expression Recognition[C]∥Proceedings of the 2015 ACM on International Conference on Multimodal Interaction.Seattle:ACM,2015:427-434. [5]HODGKIN A L,HUXLEY A F.A quantitative description of membrane current and its application to conduction and excitation in nerve[J].The Journal of Physiology,1952,117(4):500-544. [6]STEIN R B.A Theoretical analysis of neuronal variability[J].Biophysical Journal,1965,5(2):173-194. [7]IZHIKEVICH E M.Which model to use for cortical spiking neurons[J].IEEE Transactions on Neural Networks,2004,15(5):1063-1070. [8]ACHARD P,DE SCHUTTER E.Complex parameter landscape for a complex neuron model[J].PLOS Computational Biology,2006,2(7):794-804. [9]GLOROT X,BENGIO Y.Understanding the difficulty of trai- ning deep feedforward neural networks[C]∥Proceedings of the thirteenth International Conferenceon Artificial Intelligenceand Statistics.Sardinia:PMLR,2010:249-256. [10]GLOROT X,BORDES A,BENGIO Y.Deep sparse rectifier neural networks[C]∥Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics.Fort Lauderdale:PMLR,2011:315-323. [11]ERHAN D,BENGIO Y,COURVILLE A,et al.Why does unsupervised pre-training help deep learning?[J].Journal of Machine Learning Research,2010,11:625-660. [12]TROTTIER L,GIGU P,CHAIB-DRAA B.Parametric expo- nential linear unit for deep convolutional neural networks[C]∥Machine Learning and Applications.Cancun:IEEE Press,2017:207-214. [13]STROMATIAS E,NEIL D,PFEIFFER M,et al.Robustness of spiking deep belief networks to noise and reduced bit precision of neuro-inspired hardware platforms[J].Frontiers in neuroscience,2015,9:222. [14]LIU Q,FURBER S.Noisy softplus:a biology inspired activation function[C]∥InInternational Conference on Neural Information Processing.Kyoto:Springer,2016:405-412. [15]LA C G,GIUGLIANO M,SENN W,et al.The response of cortical neurons to in vivo-like input current:theory and experiment:I.Noisy inputs with stationary statistics[J].Biological Cybernetics,2008,99(5):279-301. [16]GEWALTIG M O,DIESMANN M.Nest (neural simulation tool)[J].Scholarpedia,2007,2(4):1430. [17]DAVISON A P,BRUDERLE D,EPPLER J M,et al.PyNN:a common interface for neuronal network simulators[J].Frontiers in Neuroinformatics,2009,2(11):204-123. [18]LUCEY P,COHN J F,KANADE T,et al.The Extended Cohn-Kanade Dataset (CK+):A complete dataset for action unit and emotion-specified expression[C]∥Computer Vision and Pattern Recognition Workshops.San Francisco:IEEE Press 2010:94-101. [19]CALVO M G,LUNDQVIS D.Facial expressions of emotion (KDEF):Identification under different display-duration conditions[J].Behavior Research Methods,2008,40(1):109-115. [20]WHITEHILL J,MOVELLAN J R.A discriminative approach to frame-by-frame head pose tracking[C]∥IEEE International Conference on Automatic Face & Gesture Recognition.Amsterdam:IEEE Press,2008:1-7. [21]HE K,ZHANG X,REN S,et al.Deep residual learning for ima- ge recognition[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,Boston:IEEE Press,2016:770-778. |
[1] | 周芳泉, 成卫青. 基于全局增强图神经网络的序列推荐 Sequence Recommendation Based on Global Enhanced Graph Neural Network 计算机科学, 2022, 49(9): 55-63. https://doi.org/10.11896/jsjkx.210700085 |
[2] | 周乐员, 张剑华, 袁甜甜, 陈胜勇. 多层注意力机制融合的序列到序列中国连续手语识别和翻译 Sequence-to-Sequence Chinese Continuous Sign Language Recognition and Translation with Multi- layer Attention Mechanism Fusion 计算机科学, 2022, 49(9): 155-161. https://doi.org/10.11896/jsjkx.210800026 |
[3] | 宁晗阳, 马苗, 杨波, 刘士昌. 密码学智能化研究进展与分析 Research Progress and Analysis on Intelligent Cryptology 计算机科学, 2022, 49(9): 288-296. https://doi.org/10.11896/jsjkx.220300053 |
[4] | 李宗民, 张玉鹏, 刘玉杰, 李华. 基于可变形图卷积的点云表征学习 Deformable Graph Convolutional Networks Based Point Cloud Representation Learning 计算机科学, 2022, 49(8): 273-278. https://doi.org/10.11896/jsjkx.210900023 |
[5] | 郝志荣, 陈龙, 黄嘉成. 面向文本分类的类别区分式通用对抗攻击方法 Class Discriminative Universal Adversarial Attack for Text Classification 计算机科学, 2022, 49(8): 323-329. https://doi.org/10.11896/jsjkx.220200077 |
[6] | 王润安, 邹兆年. 基于物理操作级模型的查询执行时间预测方法 Query Performance Prediction Based on Physical Operation-level Models 计算机科学, 2022, 49(8): 49-55. https://doi.org/10.11896/jsjkx.210700074 |
[7] | 陈泳全, 姜瑛. 基于卷积神经网络的APP用户行为分析方法 Analysis Method of APP User Behavior Based on Convolutional Neural Network 计算机科学, 2022, 49(8): 78-85. https://doi.org/10.11896/jsjkx.210700121 |
[8] | 朱承璋, 黄嘉儿, 肖亚龙, 王晗, 邹北骥. 基于注意力机制的医学影像深度哈希检索算法 Deep Hash Retrieval Algorithm for Medical Images Based on Attention Mechanism 计算机科学, 2022, 49(8): 113-119. https://doi.org/10.11896/jsjkx.210700153 |
[9] | 檀莹莹, 王俊丽, 张超波. 基于图卷积神经网络的文本分类方法研究综述 Review of Text Classification Methods Based on Graph Convolutional Network 计算机科学, 2022, 49(8): 205-216. https://doi.org/10.11896/jsjkx.210800064 |
[10] | 闫佳丹, 贾彩燕. 基于双图神经网络信息融合的文本分类方法 Text Classification Method Based on Information Fusion of Dual-graph Neural Network 计算机科学, 2022, 49(8): 230-236. https://doi.org/10.11896/jsjkx.210600042 |
[11] | 齐秀秀, 王佳昊, 李文雄, 周帆. 基于概率元学习的矩阵补全预测融合算法 Fusion Algorithm for Matrix Completion Prediction Based on Probabilistic Meta-learning 计算机科学, 2022, 49(7): 18-24. https://doi.org/10.11896/jsjkx.210600126 |
[12] | 杨炳新, 郭艳蓉, 郝世杰, 洪日昌. 基于数据增广和模型集成策略的图神经网络在抑郁症识别上的应用 Application of Graph Neural Network Based on Data Augmentation and Model Ensemble in Depression Recognition 计算机科学, 2022, 49(7): 57-63. https://doi.org/10.11896/jsjkx.210800070 |
[13] | 张颖涛, 张杰, 张睿, 张文强. 全局信息引导的真实图像风格迁移 Photorealistic Style Transfer Guided by Global Information 计算机科学, 2022, 49(7): 100-105. https://doi.org/10.11896/jsjkx.210600036 |
[14] | 戴朝霞, 李锦欣, 张向东, 徐旭, 梅林, 张亮. 基于DNGAN的磁共振图像超分辨率重建算法 Super-resolution Reconstruction of MRI Based on DNGAN 计算机科学, 2022, 49(7): 113-119. https://doi.org/10.11896/jsjkx.210600105 |
[15] | 刘月红, 牛少华, 神显豪. 基于卷积神经网络的虚拟现实视频帧内预测编码 Virtual Reality Video Intraframe Prediction Coding Based on Convolutional Neural Network 计算机科学, 2022, 49(7): 127-131. https://doi.org/10.11896/jsjkx.211100179 |
|