Computer Science ›› 2017, Vol. 44 ›› Issue (7): 299-303.doi: 10.11896/j.issn.1002-137X.2017.07.054

Previous Articles     Next Articles

Real-time Dynamic Sign Language Recognition Based on Hierarchical Matching Strategy

LIANG Wen-le, HUANG Yuan-yuan and HU Zuo-jin   

  • Online:2018-11-13 Published:2018-11-13

Abstract: Dynamic sign language can be described by its trajectory and the key hand-action.However,a large number of statistical data show that most of the commonly used sign languages can be recognized by its trajectory curve.Therefore,a hierarchical matching recognition strategy for dynamic sign language was proposed in this paper.First,the gesture trajectory can be obtained by the somatosensory equipment like Kinect.According to its point density,an algorithm of key frame detection is designed and is used to extract the key gestures.Thus,we can achieve a precise description of dynamic sign language through trajectory curve and key frames.Then the dynamic time warping(DTW) algorithm is optimized and used to do the first-level matching,i.e.trajectory matching.If the recognition results can be get currently,the recognition process can be finished,otherwise the process should go into the second-level,i.e.key frame matching,and then get the final recognition results.Experiments show that this algorithm not only has better real-time performance,but also has higher recognition accuracy.

Key words: Dynamic sign language recognition,Gesture trajectory,Key frame,Dynamic time warping

[1] STARNER T,PENTLAND A.Real-time american sign lan-guage recognition from video using hidden markov models[M]∥Motion-Based Recognition.Springer Netherlands,1997:227-243.
[2] MARAQA M,AL-ZBOUN F,DHYABAT M,et al.Recognition of Arabic Sign Language(ArSL) Using Recurrent Neural Networks[C]∥Applications of Digital Information and Web Techno-logyies,2008(ICADIWT 2008).IEEE,2008:41-52.
[3] ARGYROS A,LOURAKIS M I A.Binocular hand tracking and reconstruction based on 2D shape matching[C]∥18th International Conference on Pattern Recognition,2006(ICPR 2006).IEEE,2006:207-210.
[4] VOGLER C,METAXAS D.Parallel hidden markov models for american sign language recognition[C]∥IEEE International Conference on Computer Vision,1999.IEEE,1999:116-122.
[5] JANG Y.Gesture recognition using depth-based hand tracking for contactless controller application[C]∥2012 IEEE International Conference on Consumer Electronics(ICCE).2012:297-298.
[6] CHAI X,LI G,LIN Y,et al.Sign language recognition andtranslation with kinect[C]∥IEEE Conf.on AFGR.2013:1-2.
[7] MARIN G,DOMINIO F,ZANUTTIGH P.Hand gesture recognition with jointly calibrated Leap Motion and depth sensor[J].Multimedia Tools and Applications,2016,72(22):14991-15015.
[8] SHEN Y L.Analysis of Chinese sign language morpheme [J].Chinese Journal of Special Education,1993(1):1-13.(in Chinese) 沈玉林.中国手语语素分析[J].特殊教育研究,1993(1):1-13
[9] DOLIOTIS P,STEFAN A,MCMURROUGH C,et al.Comparing Gesture Recognition Accuracy Using Color and Depth Information[C]∥Proceedings of the Fourth International Confe-rence on Pervasive Technologies Related to Assistive Environments(PETR) Crete.Greece,2011:1-7.
[10] CARMONA J,CLIMENT J.A performance evaluation of hmm and dtw for gesture recognition[C]∥Progress in Pattern Recognition,Image Analysis,Computer Vision,and Applications.2012:236-243.
[11] RAHEJA J L,MINHAS M,PRASHANTH D,et al.Robustgesture recognition using Kinect:A comparison between DTW and HMM[J].Optik-International Journal for Light and Electron Optics,2015,126(11):1098-1104.
[12] ZHUANG Y T,RUI Y,HUANG T S,et al.Adaptive KeyFrame Extraction Using Unsupervised Clustering[C]∥Proc.of IEEE Int.Conf.on Image Processing.1998:866-870.

No related articles found!
Full text



[1] LEI Li-hui and WANG Jing. Parallelization of LTL Model Checking Based on Possibility Measure[J]. Computer Science, 2018, 45(4): 71 -75, 88 .
[2] XIA Qing-xun and ZHUANG Yi. Remote Attestation Mechanism Based on Locality Principle[J]. Computer Science, 2018, 45(4): 148 -151, 162 .
[3] LI Bai-shen, LI Ling-zhi, SUN Yong and ZHU Yan-qin. Intranet Defense Algorithm Based on Pseudo Boosting Decision Tree[J]. Computer Science, 2018, 45(4): 157 -162 .
[4] WANG Huan, ZHANG Yun-feng and ZHANG Yan. Rapid Decision Method for Repairing Sequence Based on CFDs[J]. Computer Science, 2018, 45(3): 311 -316 .
[5] SUN Qi, JIN Yan, HE Kun and XU Ling-xuan. Hybrid Evolutionary Algorithm for Solving Mixed Capacitated General Routing Problem[J]. Computer Science, 2018, 45(4): 76 -82 .
[6] ZHANG Jia-nan and XIAO Ming-yu. Approximation Algorithm for Weighted Mixed Domination Problem[J]. Computer Science, 2018, 45(4): 83 -88 .
[7] WU Jian-hui, HUANG Zhong-xiang, LI Wu, WU Jian-hui, PENG Xin and ZHANG Sheng. Robustness Optimization of Sequence Decision in Urban Road Construction[J]. Computer Science, 2018, 45(4): 89 -93 .
[8] LIU Qin. Study on Data Quality Based on Constraint in Computer Forensics[J]. Computer Science, 2018, 45(4): 169 -172 .
[9] ZHONG Fei and YANG Bin. License Plate Detection Based on Principal Component Analysis Network[J]. Computer Science, 2018, 45(3): 268 -273 .
[10] SHI Wen-jun, WU Ji-gang and LUO Yu-chun. Fast and Efficient Scheduling Algorithms for Mobile Cloud Offloading[J]. Computer Science, 2018, 45(4): 94 -99, 116 .