Computer Science

Computer Science ›› 2021, Vol. 48 ›› Issue (11): 242-249.doi: 10.11896/jsjkx.201000019

• Computer Graphics & Multimedia • Previous Articles     Next Articles

Sign Language Recognition Based on Image-interpreted Mechanomyography and Convolution Neural Network

WANG Xin-ping, XIA Chun-ming, YAN Jian-jun   

  1. School of Mechinery and Power Eningeering,East China University of Science and Technology,Shanghai 200237,China
  • Received:2020-10-03 Revised:2020-12-03 Online:2021-11-15 Published:2021-11-10
  • About author:WANG Xin-ping,born in 1995,postgraduate.His main research interests include pattern recognition of mechanomyography,embedded system development.
    XIA Chun-ming,born in 1969,Ph.D,professor.His main research interests include mechanomyography,traditional Chinese medicine signal analysis.

PDF (PC)

1046

Abstract: Time series signals are widely used in various pattern recognition applications.In order to solve the problem of low pattern recognition rate of time series signals for a large number of targets,this article uses a variety of transform methods to convert time series signals into images,and performes pattern recognition using image classification algorithms.In the experiment,the mechanomyography (MMG) corresponding to 30 sign languages on the forearm are collected and converted into diffe-rent image styles,and a convolution neural network (CNN) framework is designed to establish pattern recognition classification models for the images.The models are optimized twice with the application of transfer learning algorithm,and the recognition rate of the best classification model reaches 98.7%,which is much higher than the recognition rate of traditional machine learning algorithms.The experimental results imply that the image processing of time series signals can effectively improve the recognition rate of multi-target pattern recognition of MMG.This paper can provide references for pattern recognition of other time series signal.

Key words: Convolution neural network, Image-interpreted time series signal, Mechanomyography, Pattern recognition, Transfer learning

CLC Number: 

  • TN911.7
[1]ZHANG C Z,GU X T,ZHANG Y M.Gesture RecognitionBased on Deep Convolutional Neural Network[J].Radio Engineering,2019,49(7):587-591.
[2]KAKOTY N M,SHARMA M D.Recognition of Sign Language Alphabets and Numbers based on Hand Kinematics using a Data Glove[J].Procedia Computer Science,2018,133:55-62.
[3]SURI K,GUPTA R.Continuous sign language recognition from wearable IMUs using deep capsule networks and game theory[J].Computers & Electrical Engineering,2019,78:493-503.
[4]LI L X,WANG Y,WU J J,et al.Micro-motion hand gesture recognition method based on improved multiple dimensional convolution neural network[J].Computer Engineering,2018,44(9):243-249.
[5]AZAR S G,SEYEDARABI H.Trajectory-based recognition of dynamic Persian sign language using hidden Markov model[J].Computer Speech & Language,2020,61:101053.
[6]WOŁCZOWSKI A,ZDUNEK R.Electromyography and mechanomyography signal recognition:Experimental analysis using multi-way array decomposition methods[J].Biocybernetics and Biomedical Engineering,2017,37(1):103-113.
[7]ZHANG Y,XIA C.A preliminary study of classification of upper limb motions and forces based on mechanomyography[J/OL].Medical Engineering & Physics.https://www.researchgate.net/publication/341471100_A_preliminary_study_of_classification_of_upper_limb_motions_and_forces_based_on_mechanomyography.
[8]XIE H B,ZHENG Y P,GUO J Y.Classification of the mechanomyogram signal using a wavelet packet transform and singular value decomposition for multifunction prosthesis control[J].Physiological Measurement,2009,30(5):441-457.
[9]JIANG W D,XIA C M,ZHANG Y,et al.Static Fatigue Analysis of Upper Trapezius Muscle Based on Fractal Dimension of Mechanomyography[J].Space Medicine & Medicine Enginee-ring,2019(3):259-264.
[10]YANG Y H,XIE H.Research on surface EMG signal gesture recognition based on convolution neural network [J/OL].Microcomputer and Applications.http://en.cnki.com.cn/Article_en/CJFDTotal-WXJY201715017.htm.
[11]DING H,HE Q,ZENG L,et al.Motion intent recognition of individual fingers based on mechanomyogram[J].Pattern Recognition Letters,2017,88(1):41-48.
[12]ALVES N,CHAU T.Uncovering patterns of forearm muscle activity using multi-channel mechanomyography[J].Journal of Electromyogr Kinesiol,2010,20(5):777-786.
[13]WANG Z,OATES T.Encoding time series as images for visual inspection and classification using tiled convolutional neural networks[C]//Workshops at the twenty-ninth AAAI conference on artificial intelligence.2015:1.
[14]YAN J J,CHEN S Y,YAN H X,et al.Wrist pulse analysis and recognition based on recurrent plot and convolution neural network[J].Computer Engineering and Applications,2020(7):26.
[15]ZANGENEH E,RAHMATI M,MOHSENZADEH Y.Low re-solution face recognition using a two-branch deep convolutional neural network architecture[J].Expert Systems with Applications,2020,139:112854.
[16]WANG Z,YAN W,OATES T.Time series classification from scratch with deep neural networks:A strong baseline[C]//2017 International Joint Conference on Neural Networks (IJCNN).IEEE,2017:1578-1585.
[17]FAHIM S R,SARKER Y,SARKER S K,et al.Self attention convolutional neural network with time series imaging based feature extraction for transmission line fault detection and classification[J].Electric Power Systems Research,2020,187:106437.
[18]SANAYHA M,VATEEKUL P.Remaining Useful Life Prediction Using Enhanced Convolutional Neural Network on Multivariate Time Series Sensor Data[J].Walailak Journal of Science and Technology (WJST),2019,16(9):669-679.
[19]LIU S,DENG W.Very deep convolutional neural network based image classification using small training sample size[C]//2015 3rd IAPR Asian conference on pattern recognition (ACPR).IEEE,2015:730-734.
[20]WANG S,ZHANG L,FU J.Adversarial transfer learning forcross-domain visual recognition[J].Knowledge-Based Systems,2020,204:106258.
[21]ZHAO B,ZHANG X,ZHAN Z,et al.Deep multi-scale convolutional transfer learning network:A novel method for intelligent fault diagnosis of rolling bearings under variable working conditions and domains[J].Neurocomputing,2020,407:24-38.
[1] FANG Yi-qiu, ZHANG Zhen-kun, GE Jun-wei. Cross-domain Recommendation Algorithm Based on Self-attention Mechanism and Transfer Learning [J]. Computer Science, 2022, 49(8): 70-77.
[2] ZHANG Ying-tao, ZHANG Jie, ZHANG Rui, ZHANG Wen-qiang. Photorealistic Style Transfer Guided by Global Information [J]. Computer Science, 2022, 49(7): 100-105.
[3] WANG Jun-feng, LIU Fan, YANG Sai, LYU Tan-yue, CHEN Zhi-yu, XU Feng. Dam Crack Detection Based on Multi-source Transfer Learning [J]. Computer Science, 2022, 49(6A): 319-324.
[4] WANG Shan, XU Chu-yi, SHI Chun-xiang, ZHANG Ying. Study on Cloud Classification Method of Satellite Cloud Images Based on CNN-LSTM [J]. Computer Science, 2022, 49(6A): 675-679.
[5] PENG Yun-cong, QIN Xiao-lin, ZHANG Li-ge, GU Yong-xiang. Survey on Few-shot Learning Algorithms for Image Classification [J]. Computer Science, 2022, 49(5): 1-9.
[6] TAN Zhen-qiong, JIANG Wen-Jun, YUM Yen-na-cherry, ZHANG Ji, YUM Peter-tak-shing, LI Xiao-hong. Personalized Learning Task Assignment Based on Bipartite Graph [J]. Computer Science, 2022, 49(4): 269-281.
[7] LI Si-quan, WAN Yong-jing, JIANG Cui-ling. Multiple Fundamental Frequency Estimation Algorithm Based on Generative Adversarial Networks for Image Removal [J]. Computer Science, 2022, 49(3): 179-184.
[8] ZUO Jie-ge, LIU Xiao-ming, CAI Bing. Outdoor Image Weather Recognition Based on Image Blocks and Feature Fusion [J]. Computer Science, 2022, 49(3): 197-203.
[9] ZHANG Shu-meng, YU Zeng, LI Tian-rui. Transferable Emotion Analysis Method for Cross-domain Text [J]. Computer Science, 2022, 49(3): 218-224.
[10] LIU Yang, LI Fan-zhang. Fiber Bundle Meta-learning Algorithm Based on Variational Bayes [J]. Computer Science, 2022, 49(3): 225-231.
[11] HU Yan-li, TONG Tan-qian, ZHANG Xiao-yu, PENG Juan. Self-attention-based BGRU and CNN for Sentiment Analysis [J]. Computer Science, 2022, 49(1): 252-258.
[12] XIONG Zhao-yang, WANG Ting. Image Recognition for Building Components Based on Convolutional Neural Network [J]. Computer Science, 2021, 48(6A): 51-56.
[13] WU Lan, WANG Han, LI Bin-quan. Unsupervised Domain Adaptive Method Based on Optimal Selection of Self-supervised Tasks [J]. Computer Science, 2021, 48(6A): 357-363.
[14] LI Da, LEI Ying-ke, ZHANG Hai-chuan. Outdoor Fingerprint Positioning Based on LTE Networks [J]. Computer Science, 2021, 48(6A): 404-409.
[15] LIU Yu-tong, LI Peng, SUN Yun-yun, HU Su-jun. Image Recognition with Deep Dynamic Joint Adaptation Networks [J]. Computer Science, 2021, 48(6): 131-137.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.