Computer Science ›› 2018, Vol. 45 ›› Issue (12): 177-181.doi: 10.11896/j.issn.1002-137X.2018.12.028

• Artificial Intelligence • Previous Articles     Next Articles

Deep Learning Model for Typhon Grade Classification Based on Improved Activation Function

ZHENG Zong-sheng, LIU Zhao-rong, HUANG Dong-mei, SONG Wei, ZOU Guo-liang, HOU Qian, HAO Jian-bo   

  1. (College of Information Technology,Shanghai Ocean University,Shanghai 201306,China)
  • Received:2017-11-23 Online:2018-12-15 Published:2019-02-25

Abstract: Aiming at the issue that it is difficult to select the activation function in deep learning model for specific task,on the basis of analyzing the advantages and disadvantages of traditional activation function and the popular activation function at the present stage,this paper constructed an activation function T-ReLU which can make up for the shortcomings of Tanh function and ReLU function by combining the Tanh activation function with the widely used ReLU function.By constructing the deep learning model Typ-CNNs for typhoon grade classification,using the Typhoon satellite image published by the Japan Meteorological Agency as the self-built sample data,this paper made use of several different activation functions to conduct comparison experiments.The results show that the test accuracy of typhoon grade classification using the T-ReLU function is 1.124% higher than that of using ReLU activation function,which is 2.102% higher than that of using Tanh function.In order to further verify the reliability of the results,the MNIST general data set was utilized to carry out the comparison experiment of activation function.The final results show that 99.855% training accuracy and 98.620% test accuracy can be obtained by using T-ReLU function,and it performs better than other activation functions.

Key words: Activation function, Convolution neural network, Deep learning, MNIST dataset, Typhoon grade

CLC Number: 

  • TP391
[1]GUO L L,DING S F.Research Progress in Deep Learning [J].Computer Science,2015,42(5):28-33.(in Chinese)
郭丽丽,丁世飞.深度学习研究进展[J].计算机科学,2015,42(5):28-33.
[2]LECUN Y,BENGIO Y,HINTON G.Deep learning[J].Nature,2015,521(7553):436-444.
[3]SCHULZ H,BEHNKE S.Deep Learning[J].KI -Künstliche Intelligenz,2012,26(4):357-363.
[4]CIRSTEA B I,LIKFORMANSULEM L.Improving a deep convolutional neural network architecture for character recognition[J].Electronic Imaging,2016,2016(17):1-7.
[5]PENG Q,JI G S,XIE L J,et al.Application of Convolution Neural Network in Vehicle Identification [J/OL].http://kns.cnki.net/kcms/detail/11.5602.TP.20170807.1008.002.html.(in Chinese)
彭清,季桂树,谢林江,等.卷积神经网络在车辆识别中的应用[J/OL].http://kns.cnki.net/kcms/detail/11.5602.TP.20170807.1008.002.html.
[6]LI J C,YUANG C,SONG Y.Automatic Labeling of Multi-label Images Based on Convolutional Neural Network [J].Computer Science,2016,43(7):41-45.(in Chinese)
黎健成,袁春,宋友.基于卷积神经网络的多标签图像自动标注[J].计算机科学,2016,43(7):41-45.
[7]LI H,LIU F,YANG S Y,et al.Remote sensing image fusion based on deep supportive value learning network [J].Acta Automatica Sinica,2016,39(8):1583-1596.(in Chinese)
李红,刘芳,杨淑媛,等.基于深度支撑值学习网络的遥感图像融合[J].计算机学报,2016,39(8):1583-1596.
[8]SHAFIE A S,MOHTAR I A,MASROM S,et al.Backpropagation neural network with new improved error function and activation function for classification problem[C]∥Humanities,Scien-ce and Engineering Research.IEEE,2012:1359-1364.
[9]GONG Z T,CHEN G X,CAO J S.Application of Convolutional Neural Network in Image Classification of Cerebrospinal Fluid [J].Computer Engineering and Design,2017,38 (4):1056-1061.(in Chinese)
龚震霆,陈光喜,曹建收.卷积神经网络在脑脊液图像分类上的应用[J].计算机工程与设计,2017,38(4):1056-1061.
[10]WANG F F.Research and application of improved convolutionneural network algorithm [D].Nanjing:Nanjing University of Posts and Telecommunications,2016.(in Chinese)
王飞飞.基于改进卷积神经网络算法的研究与应用[D].南京:南京邮电大学,2016.
[11]NAIR V,HINTON G E.Rectified linear units improve restric-ted boltzmann machines[C]∥International Conference on International Conference on Machine Learning.Omnipress,2010:807-814.
[12]REHN M,SOMMER F T.A network that uses few active neurones to code visual input predicts the diverse shapes of cortical receptive fields[J].Journal of Computational Neuroscience,2007,22(2):135-146.
[13]LENNIE P.Supplemental Data The Cost of Cortical Computation[J].Current Biology,2003,13(6):493-497.
[14]HUANG Y,DUAN X S,SUN S Y,et al.Research on training algorithm of deep neural networks based on improved sigmoid activation function [J].Computer Measurement and Control,2017,25(2):126-129.(in Chinese)
黄毅,段修生,孙世宇,等.基于改进sigmoid激活函数的深度神经网络训练算法研究[J].计算机测量与控制,2017,25(2):126-129.
[15]GLOROT X,BORDES A,BENGIO Y.Deep sparse rectifierneural networks∥International Conference on Artificial Intelligence and Statistics.2012:315-323.
[16]JARRETT K,KAVUKCUOGLU K,RANZATO M,et al.What is the Best Multi-Stage Architecture for Object Recognition?[C]∥IEEE International Conference on Computer Vision.2009:2146-2153.
[17]OLSHAUSEN B A,FIELD D J.Sparse coding with an overcomplete basis set:a strategy employed by V1[J].Vision Research,1997,37(23):3311.
[18]KRIZHEVSKY A,SUTSKEVER I,HINTON G E.ImageNet classification with deep convolutional neural networks∥International Conference on Neural Information Processing Systems.Curran Associates Inc.,2012:1097-1105.
[19]MAAS A L,QI P,HANNUN A Y,et al.Building DNN acoustic models for large vocabulary speech recognition[J].Computer Speech & Language,2017,41(C):195-213.
[20]SZEGEDY C,LIU W,JIA Y,et al.Going deeper with convolutions∥IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2015:1-9.
[21]HE K,ZHANG X,REN S,et al.Delving Deep into Rectifiers:Surpassing Human-Level Performance on ImageNet Classification∥2015 IEEE International Conference on Computer Vision (ICCV).EEE,2015.
[22]XU B,WANG N,CHEN T,et al.Empirical Evaluation of Rectified Activations in Convolutional Network.https://arxiv.org/abs/1505.00853.
[23]POGGIO T,GIROSI F.Networks for approximation and lear-ning[J].Proceedings of the IEEE,1990,78(9):1481-1497.
[24]SU H,LI G,YU D,et al.Error back propagation for sequence training of Context-Dependent Deep NetworkS for conversatio-nal speech transcription[C]∥IEEE International Conference on Acoustics,Speech and Signal Processing.IEEE,2013:6664-6668.
[25]VEDALDI A,LENC K.MatConvNet:Convolutional NeuralNetworks for MATLAB[C]∥ACM International Conference on Multimedia.ACM,2015:689-692.
[1] RAO Zhi-shuang, JIA Zhen, ZHANG Fan, LI Tian-rui. Key-Value Relational Memory Networks for Question Answering over Knowledge Graph [J]. Computer Science, 2022, 49(9): 202-207.
[2] TANG Ling-tao, WANG Di, ZHANG Lu-fei, LIU Sheng-yun. Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy [J]. Computer Science, 2022, 49(9): 297-305.
[3] XU Yong-xin, ZHAO Jun-feng, WANG Ya-sha, XIE Bing, YANG Kai. Temporal Knowledge Graph Representation Learning [J]. Computer Science, 2022, 49(9): 162-171.
[4] WANG Jian, PENG Yu-qi, ZHAO Yu-fei, YANG Jian. Survey of Social Network Public Opinion Information Extraction Based on Deep Learning [J]. Computer Science, 2022, 49(8): 279-293.
[5] HAO Zhi-rong, CHEN Long, HUANG Jia-cheng. Class Discriminative Universal Adversarial Attack for Text Classification [J]. Computer Science, 2022, 49(8): 323-329.
[6] JIANG Meng-han, LI Shao-mei, ZHENG Hong-hao, ZHANG Jian-peng. Rumor Detection Model Based on Improved Position Embedding [J]. Computer Science, 2022, 49(8): 330-335.
[7] SUN Qi, JI Gen-lin, ZHANG Jie. Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection [J]. Computer Science, 2022, 49(8): 172-177.
[8] HOU Yu-tao, ABULIZI Abudukelimu, ABUDUKELIMU Halidanmu. Advances in Chinese Pre-training Models [J]. Computer Science, 2022, 49(7): 148-163.
[9] ZHOU Hui, SHI Hao-chen, TU Yao-feng, HUANG Sheng-jun. Robust Deep Neural Network Learning Based on Active Sampling [J]. Computer Science, 2022, 49(7): 164-169.
[10] SU Dan-ning, CAO Gui-tao, WANG Yan-nan, WANG Hong, REN He. Survey of Deep Learning for Radar Emitter Identification Based on Small Sample [J]. Computer Science, 2022, 49(7): 226-235.
[11] HU Yan-yu, ZHAO Long, DONG Xiang-jun. Two-stage Deep Feature Selection Extraction Algorithm for Cancer Classification [J]. Computer Science, 2022, 49(7): 73-78.
[12] ZHANG Ying-tao, ZHANG Jie, ZHANG Rui, ZHANG Wen-qiang. Photorealistic Style Transfer Guided by Global Information [J]. Computer Science, 2022, 49(7): 100-105.
[13] CHENG Cheng, JIANG Ai-lian. Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction [J]. Computer Science, 2022, 49(7): 120-126.
[14] ZHU Wen-tao, LAN Xian-chao, LUO Huan-lin, YUE Bing, WANG Yang. Remote Sensing Aircraft Target Detection Based on Improved Faster R-CNN [J]. Computer Science, 2022, 49(6A): 378-383.
[15] WANG Jian-ming, CHEN Xiang-yu, YANG Zi-zhong, SHI Chen-yang, ZHANG Yu-hang, QIAN Zheng-kun. Influence of Different Data Augmentation Methods on Model Recognition Accuracy [J]. Computer Science, 2022, 49(6A): 418-423.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!