Computer Science ›› 2022, Vol. 49 ›› Issue (11A): 211100009-5.doi: 10.11896/jsjkx.211100009

• Image Processing & Multimedia Technology • Previous Articles     Next Articles

Handwritten Digit Recognition Based on Attention Mechanism

LI Bo-yan1, ZHANG Yong2, YUAN De-rong2, XIONG Tang-tang1, HE Lang2   

  1. 1 School of Statistics,Jiangxi University of Finance and Economics,Nanchang 330000,China
    2 School of Software and Internet of Things Engineering,Jiangxi University of Finance and Economics,Nanchang 330000,China
  • Online:2022-11-10 Published:2022-11-21
  • About author:LI Bo-yan,born in 1999,master.Her main research interests include education statistics and artificial intelligence.
    ZHANG Yong,born in 1975,Ph.D,professor.His main research interests include information security and intelligent systems,quantum computing.
  • Supported by:
    National Natural Science Foundation of China(61762043),Natural Science Foundation of Jiangxi Province,China(20192BAB207022) and Key Science and Technology Research Project of Jiangxi Provincial Department of Education(GJJ190249).

Abstract: As an important branch of pattern recognition,handwritten digit recognition is in an unprecedented upsurge,and con-volutional neural networks are also widely used in related research.In view of the problem that gradient explosion and gradient dispersion are prone to occur in the training process of handwritten digit recognition,which leads to low image recognition accuracy,a model embedded with convolutional block attention module(CBAM)is newly proposed for handwritten digit recognition.The CBAM is embedded in the convolutional neural network in order to screen out effective features from the channel and spatial dimensions respectively,suppress irrelevant features,enhance the expression ability of features,and improve the recognition accuracy of the model.In order to further improve the accuracy of network identification,the batch normalization(BN) algorithm is fully applied in the entire network architecture to speed up the model convergence,in this way,the anti-over-fitting ability of the model gets improved.The results of experiments which are conducted on the MNIST dataset show that the overall recognition accuracy of the embedded CBAM attention module network is up to 99.87%,and compared with some traditional convolutional neural network models,its recognition accuracy is significantly improved.

Key words: Handwritten digit recognition, Attention mechanism, Convolutional neural network, Deep learning

CLC Number: 

  • TP391
[1]CHEN T X.Research on handwritten digit recognition based on integrated convolutional neural network[D].Wuhan:Central China Normal University,2020.
[2]DU X,GAO M F.Application of artificial neural network in number recognition[J].Computer System Applications,2007(2):21-22,27.
[3]LECUN Y,BOSER B,DENKER J S,et al.Hardwritten digit recognition with a back-propagation network[J].Advances in Neural Information Processing Systems,1900,2(2):369-404.
[4]LECUN Y,BOTTOU L,BENGIO Y,et al.Gradient-based learning applied to document recognition[J].Proceedings of the IEEE,1998,86(11):2278-2324.
[5]KRIZHEVSKY A,SUTSKEVER I,HINTON G E.Imagenetclassification with deep convolutional neural networks[J].Communications of the ACM,2017,60(6):84-90.
[6]SZEGEDY C,LIU W,JIA Y,et al.Going deeper with convolutions[C]//Computer Vision and Pattern Recognition.2015:1-9.
[7]HE K,ZHANG X,REN S,et al.Deep residual learning foriamge recognition[C]//IEEE Conference on Computer Vision and Pattern Recognition.IEEE Computer Society,2016:770-778.
[8]CHOLLET F.Xception:Deep learning with depth-wise separa-ble convolutions[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR).IEEE,2017:1800-1807.
[9]RU X Q,HUA G G,LI L H,et al.Research on handwritten di-git recognition based on deformable convolutional neural network[J].Microelectronics and Computer,2019,36(4):47-51.
[10]MA J Y,MENG X,ZHAO Y.Handwritten digit recognitionbased on spiking neural network[J].Digital Technology and Application,2019,37(5):81-83.
[11]YU S X,XIA C X,TANG Z T,et al.Handwritten digit recognition based on improved inception convolutional neural network [J].Computer Applications and Software,2019,36(12):143-149.
[12]FU Y Z.Research on handwritten digit recognition methodbased on deep learning[D].Yinchuan:Ningxia University,2020.
[13]WOO S,PARK J,LEE J Y,et al.CBAM:convolutional block attention module[C]//European Conference on Computer Vision.Cham:Springer,2018:3-19.
[14]IOFFE S,SZEGEDY C.Batch normalization:Accelerating deep network training by reducing internal covariate shift[C]//International Conference on Learning.PMLR,2015:448-456.
[15]LI H W,WU Q X.Implementation of neural network activation function in smart sensors[J].Sensors and Microsystems,2014,33(1):46-48.
[16]ZHOU F Y,JIN L P,DONG J.Summary of convolutional neural network research[J].Chinese Journal of Computers,2017,40(6):1229-1251.
[17]MAAS A L,HANNUN A Y,NG A Y.Rectifier nonlinea-rities improve neural network acoustic models[C]//Proceedings of the 30th International Conference on Machine Learning.Atlanta:ACM,2013:456-462.
[18]ZHANG H,ZHANG Q,YU J Y.Overview of the development of activation functions and analysis of their properties[J].Journal of Xihua University(Natural Science Edition),2021,40(4):1-10.
[19]NAIR V,HINTON G E.Rectified linear units improve restricted boltzmann machines[C]//Proceedings of the 27th International Conference on Machine Learning(ICML-10).Haifa,Israel:DBLP,2010:807-814.
[20]ZUBAIR S,YAN F,WANG W W.Dictionary learning basedsparse coefficients for audio classification with max and average pooling[J].Digital Signal Processing,2013,23(3):960-970.
[21]HANG S T,AONO M.Bi-linearly weighted fractional max pooling[J].Multimedia Tools and Applications,2017,76(21):22095-22117.
[22]DIETTERICH T G,BAKIRI G.Solving multiclass learningproblems via error-correcting output codes[J].Joural of Artificial Intelligence Research,1995,2(1):263-286.
[23]HE X Y,XIONG W,LI Y Q,et al.Handwritten digit recognition based on convolutional neural network[J].Electronic Components and Information Technology,2020,4(7):53-54.
[24]LV H.Design of Handwritten digit recognition system based on convolutional neural network[J].Intelligent Computers and Applications,2019,9(2):54-56,62.
[1] ZHOU Fang-quan, CHENG Wei-qing. Sequence Recommendation Based on Global Enhanced Graph Neural Network [J]. Computer Science, 2022, 49(9): 55-63.
[2] DAI Yu, XU Lin-feng. Cross-image Text Reading Method Based on Text Line Matching [J]. Computer Science, 2022, 49(9): 139-145.
[3] ZHOU Le-yuan, ZHANG Jian-hua, YUAN Tian-tian, CHEN Sheng-yong. Sequence-to-Sequence Chinese Continuous Sign Language Recognition and Translation with Multi- layer Attention Mechanism Fusion [J]. Computer Science, 2022, 49(9): 155-161.
[4] XU Yong-xin, ZHAO Jun-feng, WANG Ya-sha, XIE Bing, YANG Kai. Temporal Knowledge Graph Representation Learning [J]. Computer Science, 2022, 49(9): 162-171.
[5] XIONG Li-qin, CAO Lei, LAI Jun, CHEN Xi-liang. Overview of Multi-agent Deep Reinforcement Learning Based on Value Factorization [J]. Computer Science, 2022, 49(9): 172-182.
[6] RAO Zhi-shuang, JIA Zhen, ZHANG Fan, LI Tian-rui. Key-Value Relational Memory Networks for Question Answering over Knowledge Graph [J]. Computer Science, 2022, 49(9): 202-207.
[7] TANG Ling-tao, WANG Di, ZHANG Lu-fei, LIU Sheng-yun. Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy [J]. Computer Science, 2022, 49(9): 297-305.
[8] WANG Ming, PENG Jian, HUANG Fei-hu. Multi-time Scale Spatial-Temporal Graph Neural Network for Traffic Flow Prediction [J]. Computer Science, 2022, 49(8): 40-48.
[9] CHEN Yong-quan, JIANG Ying. Analysis Method of APP User Behavior Based on Convolutional Neural Network [J]. Computer Science, 2022, 49(8): 78-85.
[10] ZHU Cheng-zhang, HUANG Jia-er, XIAO Ya-long, WANG Han, ZOU Bei-ji. Deep Hash Retrieval Algorithm for Medical Images Based on Attention Mechanism [J]. Computer Science, 2022, 49(8): 113-119.
[11] SUN Qi, JI Gen-lin, ZHANG Jie. Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection [J]. Computer Science, 2022, 49(8): 172-177.
[12] YAN Jia-dan, JIA Cai-yan. Text Classification Method Based on Information Fusion of Dual-graph Neural Network [J]. Computer Science, 2022, 49(8): 230-236.
[13] WANG Jian, PENG Yu-qi, ZHAO Yu-fei, YANG Jian. Survey of Social Network Public Opinion Information Extraction Based on Deep Learning [J]. Computer Science, 2022, 49(8): 279-293.
[14] HAO Zhi-rong, CHEN Long, HUANG Jia-cheng. Class Discriminative Universal Adversarial Attack for Text Classification [J]. Computer Science, 2022, 49(8): 323-329.
[15] JIANG Meng-han, LI Shao-mei, ZHENG Hong-hao, ZHANG Jian-peng. Rumor Detection Model Based on Improved Position Embedding [J]. Computer Science, 2022, 49(8): 330-335.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!