Computer Science ›› 2017, Vol. 44 ›› Issue (Z11): 178-183.doi: 10.11896/j.issn.1002-137X.2017.11A.037

Previous Articles     Next Articles

Research on Continuous Sign Language Sentence Recognition Algorithm Based on Key Frame

GUO Xin-peng, HUANG Yuan-yuan and HU Zuo-jin   

  • Online:2018-12-01 Published:2018-12-01

Abstract: At present,most of the dynamic sign language recognition is only for sign language words.The continuous sign language sentence recognition research and the corresponding results are less,because the segmentation of such sentence is very difficult.In this paper,a sign language sentence recognition algorithm was proposed based on weighted key frames.Key frames can be regarded as the basic unit of sign word,therefore,according to the key frames,we can get related vocabularies,and thus we can further organize these vocabularies into meaningful sentence.Such work can avoid the hard point of dividing sign language sentence directly.With the help of Kinect,i.e.motion-control device,a kind of self-adaptive algorithm of key frame extraction based on the trajectory of sign language was brought out in the paper.After that,the key frame was given to weight according to its semantic contribution.Finally,the recognition algorithm was designed based on these weighted key frames and thus got the continuous sign language sentence.Experiments show that the algorithm designed in this paper can realize real-time recognition of continuous sign language sentences.

Key words: Sign language sentence recognition,Key frame,Gesture trace,Kinect motion-control device

[1] STARNER T,PENTLAND A.Real-Time American Sign Language Recognition Using Desk and Wearable Computer Based Video[C]∥IEEE Transaction on Pattern Analysis and Machine Intelligence.1998:1371-1375.
[2] VON AGRIS U,ZIEREN J,CANZLER U,et al.Recent deve-lopments in visual sign language recognition[J].Universal Access in the Information Society,2008,6(4):323-362.
[3] GROBEL K,ASSAN M.Isolated sign language recognitionusing hidden Markov models[C]∥IEEE International Confe-rence on Systems.1997:162-167.
[4] RAVIKIRAN J,MAHESH K,MAHISHI S,et al.Finger Detection for Sign Language Recognition[J].International Association of Engineers,2009,2174(1):489-493.
[5] GUO X L,YANG T T.Gesture recognition based on HMM-FNN model using a Kinect[J].Journal on Multimodal User Interfaces,2016:1-7.
[6] 谈家谱,徐文胜.基于Kinect的指尖检测与手势识别方法[J].计算机应用,2015,35(6):1795-1800.
[7] HALIM Z,ABBAS G.A Kinect-Based Sign Language HandGesture Recognition System for Hearing- and Speech-Impaired:A Pilot Study of Pakistani Sign Language[J].Assistive Technology,2015,27(1):34-43.
[8] YAN H,ZHANG M,TONG J,et al.Real time robust multi-fingertips tracking in 3D space using Kinect[J].Journal of Computer-Aided Design and Computer Graphics,2013,5(12):1801-1809.
[9] JIANG F,GAO W,WANG C L,et al.Development in Signer-Independent Sign Language Recognition and the Ideas of Solving Some Key Problems[J].Journal of Software,2007,18(3):477-489.
[10] NASRI S,BEHRAD A,RAZZAZI F.A novel approach for dynamic hand gesture recognition using contour-based similarity images[J].International Journal of Computer Mathematics,2015,92(4):662-685.
[11] LIN Y,CHAI X,ZHOU Y,et al.Curve Matching from the Viewof Manifold for Sign Language Recognition[C]∥ Workshop on Feature and Similarity Learning for Computer Vision (FSLCV).ACCV,2014:233-246.
[12] PU J,ZHOU W,ZHANG J,et al.Sign Language RecognitionBased on Trajectory Modeling with HMMs[C]∥International Conference on Multimedia Modeling.2016:686-697.
[13] STARNER T,PENTL A.Visual Recognition of American Sign Language Using Hidden Markov Models[C]∥International Workshop on Automatic Face & Gesture Recognition.1995:189-194.
[14] 方高林,高文,陈熙霖,等.基于SRN/HMM的非特定人连续手语识别系统[J].软件学报,2002,13(11):2169-2175.
[15] 中国残疾人联合会教育就业部,中国聋人协会.中国手语[M].华夏出版社,2003.
[16] LI S R,HUANG Y Y,HU Z J,et al.Key Frame Detection Algorithm based on Dynamic Sign Language Video for the Non Specific Population[J].International Journal of Signal Processing:Image Processing and Pattern Recognition,2015,8(12):135-148.
[17] SHI M M,HUANG Y Y,HU Z J.Dynamic Sign Language Re-cognition Algorithm Using Weighted Gesture Units[J].Journal of Information and Computational Science,2015,12(15):5611-5621.

No related articles found!
Full text



[1] LEI Li-hui and WANG Jing. Parallelization of LTL Model Checking Based on Possibility Measure[J]. Computer Science, 2018, 45(4): 71 -75, 88 .
[2] XIA Qing-xun and ZHUANG Yi. Remote Attestation Mechanism Based on Locality Principle[J]. Computer Science, 2018, 45(4): 148 -151, 162 .
[3] LI Bai-shen, LI Ling-zhi, SUN Yong and ZHU Yan-qin. Intranet Defense Algorithm Based on Pseudo Boosting Decision Tree[J]. Computer Science, 2018, 45(4): 157 -162 .
[4] WANG Huan, ZHANG Yun-feng and ZHANG Yan. Rapid Decision Method for Repairing Sequence Based on CFDs[J]. Computer Science, 2018, 45(3): 311 -316 .
[5] SUN Qi, JIN Yan, HE Kun and XU Ling-xuan. Hybrid Evolutionary Algorithm for Solving Mixed Capacitated General Routing Problem[J]. Computer Science, 2018, 45(4): 76 -82 .
[6] ZHANG Jia-nan and XIAO Ming-yu. Approximation Algorithm for Weighted Mixed Domination Problem[J]. Computer Science, 2018, 45(4): 83 -88 .
[7] WU Jian-hui, HUANG Zhong-xiang, LI Wu, WU Jian-hui, PENG Xin and ZHANG Sheng. Robustness Optimization of Sequence Decision in Urban Road Construction[J]. Computer Science, 2018, 45(4): 89 -93 .
[8] LIU Qin. Study on Data Quality Based on Constraint in Computer Forensics[J]. Computer Science, 2018, 45(4): 169 -172 .
[9] ZHONG Fei and YANG Bin. License Plate Detection Based on Principal Component Analysis Network[J]. Computer Science, 2018, 45(3): 268 -273 .
[10] SHI Wen-jun, WU Ji-gang and LUO Yu-chun. Fast and Efficient Scheduling Algorithms for Mobile Cloud Offloading[J]. Computer Science, 2018, 45(4): 94 -99, 116 .