Computer Science ›› 2019, Vol. 46 ›› Issue (11A): 299-302.

• Pattern Recognition & Image Processing • Previous Articles     Next Articles

Continuous Sign Language Sentence Recognition Based on Double Transfer Probability of Key Actions

LI Chen1, HUANG Yuan-yuan1, HU Zuo-jin2   

  1. (College of Computer Science and Technology,Nanjing University of Aeronautics & Astronautics,Nanjing 210016,China)1;
    (College of Math and Information Science,Nanjing Normal University of Special Education,Nanjing 210038,China)2
  • Online:2019-11-10 Published:2019-11-20

Abstract: At present,the most difficult problem in continuous sign language recognition is how to split out the words effectively.In this paper,key actions were regarded as the basic units of sign language and an algorithm based on double transfer probability of key actions was proposed.After acquiring the sequence of basic units from continuous sign language,the boundaries of words can be effectively found by judging the intra-word and inter-word transfer relations of all adjacent basic units.Then the sequence of basic units are segmented by these boundaries and the candidate words of each group of basic units can be identified.Finally,according to the transfer probabilities between candidate words of different groups,the probability of corresponding synthetic sentence is calculated and then the final recognition result is output by the principle of maximum probability.The algorithm is easy to implement and has high execution efficiency.It can be applied to non-specific population through experimental verification.

Key words: Sign language sentence, Key actions, Transfer probability

CLC Number: 

  • TP391.41
[1]STARNER T,PENTLAND A.Visual Recognition of American Sign Language Using Hidden Markov Models[C]∥International Workshop on Automatic Face & Gesture Recognition.1995:189-194.
[2]KONG W W,RANGANATH S.Automatic Hand TrajectorySegmentation and Phoneme Transcription for Sign Language[C]∥IEEE International Conference on Automatic Face & Gesture Recognition.IEEE,2008:1-6.
[3]KONG W W,RANGANATH S.Towards Subject Independent Continuous Sign Language Recognition[J].Pattern Recognition,2014,47(3):1294-1308.
[4]YANG H D,SCLAROFF S,LEE S W.Sign Language Spotting with a Threshold Model Based on Conditional Random Fields[M].IEEE Computer Society,2009.
[5]YANG R,SARKAR S,LOEDING B.Handling Movement Epenthesis and Hand Segmentation Ambiguities in Continuous Sign Language Recognition Using Nested Dynamic Programming[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2010,32(3):462-477.
[6]KOLLER O,ZARGARAN S,NEY H,et al.Deep Sign:Enabling Robust Statistical Continuous Sign Language Recognition via Hybrid CNN-HMMs[J].International Journal of Computer Vision,2018,126(12):1311-1325.
[8]FANG G,GAO W,ZHAO D.Large-Vocabulary ContinuousSign Language Recognition Based on Transition-Movement Models[J].IEEE Transactions on Systems,Man,and Cyberne-tics-Part A:Systems and Humans,2007,37(1):1-9.
[9]YANG W,TAO J,YE Z.Continuous sign language recognition using level building based on fast hidden Markov model[J].Pattern Recognition Letters,2016,78(C):28-35.
[10]XU X X,HUANG Y Y,HU Z J.Research on Continuous Sign Language Sentence Recognition Algorithm Based on Weighted Key-Frame[J].Journal of Advanced Computational Intelligence and Intelligent Informatics,2018,22(4):483-490.
[11]HUANG J,ZHOU W,ZHANG Q,et al.Video-based Sign Language Recognition without Temporal Segmentation[C]∥32nd AAAI Conference on Artificial Intelligence.2018:2257-2264
[13]LI S R,HUANG Y Y,HU Z J,et al.Key Frame Detection Algorithm based on Dynamic Sign Language Video for the Non Specific Population[J].International Journal of Signal Proces-sing,Image Processing and Pattern Recognition,2015,8(12):135-148.
[14]SHI M M,HUANG Y Y,HU Z J.Dynamic Sign Language Recognition Algorithm Using Weighted Gesture Units[J].Journal of Information and Computational Science,2015,12(15):5611-5621.
[15]LIANG W L,HUANG Y Y,HU Z J.Real-Time Dynamic Sign Language Recognition Based on Hierarchical Matching Strategy[J].International Journal of Signal Processing,Image Processing and Pattern Recognition,2017,10(7):21-34.
[1] XU Xin-xin, HUANG Yuan-yuan, HU Zuo-jin. Extraction Algorithm of Key Actions in Continuous and Complex Sign Language [J]. Computer Science, 2018, 45(11A): 189-193.
[2] GUO Xin-peng, HUANG Yuan-yuan and HU Zuo-jin. Research on Continuous Sign Language Sentence Recognition Algorithm Based on Key Frame [J]. Computer Science, 2017, 44(Z11): 178-183.
[3] GUO Yan-qing, ZHAO Rui, KONG Xiang-wei, FU Hai-yan and JIANG Jin-ping. News-summarization Extraction Method Based on Weighted Event Elements Strategy [J]. Computer Science, 2016, 43(1): 237-241.
[4] WANG Guan-jun,WANG Mao-li,ZHAO Ying. Research on Novel Test Vector Ordering Approach Based on Markov Decision Processes [J]. Computer Science, 2010, 37(5): 287-290.
Full text



[1] DU Wei, DING Shi-fei. Overview on Multi-agent Reinforcement Learning[J]. Computer Science, 2019, 46(8): 1 -8 .
[2] GAO Li-zheng, ZHOU Gang, LUO Jun-yong, LAN Ming-jing. Survey on Meta-event Extraction[J]. Computer Science, 2019, 46(8): 9 -15 .
[3] CAI Li, LI Ying-zi, JIANG Fang, LIANG Yu. Study on Clustering Mining of Imbalanced Data Fusion Towards Urban Hotspots[J]. Computer Science, 2019, 46(8): 16 -22 .
[4] YANG Zhen, WANG Hong-jun. Important Location Identification of Mobile Users Based on Trajectory Division and Density Clustering Method[J]. Computer Science, 2019, 46(8): 23 -27 .
[5] DENG Cun-bin, YU Hui-qun, FAN Gui-sheng. Integrating Dynamic Collaborative Filtering and Deep Learning for Recommendation[J]. Computer Science, 2019, 46(8): 28 -34 .
[6] ZHONG Feng-yan, WANG Yan, LI Nian-shuang. Node Selection Scheme for Data Repair in Heterogeneous Distributed Storage Systems[J]. Computer Science, 2019, 46(8): 35 -41 .
[7] SUN Guo-dao, ZHOU Zhi-xiu, LI Si, LIU Yi-peng, LIANG Rong-hua. Spatio-Temporal Evolution of Geographical Topics[J]. Computer Science, 2019, 46(8): 42 -49 .
[8] ZHANG Hui-bing, ZHONG Hao, HU Xiao-li. User Reviews Clustering Method Based on Topic Analysis[J]. Computer Science, 2019, 46(8): 50 -55 .
[9] LI Bo-jia, ZHANG Yang-sen, CHEN Ruo-yu. Method for Generating Massive Data with Assignable Distribution[J]. Computer Science, 2019, 46(8): 56 -63 .
[10] LU Xian-guang, DU Xue-hui, WANG Wen-juan. Alert Correlation Algorithm Based on Improved FP Growth[J]. Computer Science, 2019, 46(8): 64 -70 .