采用SIFT-BoW和深度图像信息的中国手语识别研究

计算机科学 ›› 2014, Vol. 41 ›› Issue (2): 302-307.

采用SIFT-BoW和深度图像信息的中国手语识别研究

杨全,彭进业

西安文理学院软件学院西安710065;西北大学信息科学与技术学院西安710127

出版日期:2018-11-14 发布日期:2018-11-14
基金资助:
本文受国家自然科学基金项目(61075014),高等学校博士学科点专项科研基金(20116102110027)资助

Chinese Sign Language Recognition Research Using SIFT-BoW and Depth Image Information

YANG Quan and PENG Jin-ye

Online:2018-11-14 Published:2018-11-14

摘要/Abstract

摘要： 将深度图像信息引入手语识别的研究,提出了一种基于DI_CamShift(Depth Image CamShift)和SIFT-BoW(Scale Invariant Feature Transform-Bag of Words)的中国手语识别方法。该方法将Kinect作为视频采集设备,在获取手语彩色视频的同时得到其深度信息；首先计算深度图像中手语手势的主轴方向角和质心位置,通过调整搜索窗口对手势进行准确跟踪；然后使用基于深度积分图像的Ostu算法分割手势并提取其SIFT特征,进而构建SIFT-BoW作为手语特征并用SVM进行识别。实验结果表明,该方法单个手语字母最好识别率为99.87%,平均识别率96.21%。

关键词: SIFT-BoW,DI_CamShift,深度图像,Kinect,手语识别中图法分类号TP311文献标识码A

Abstract: Introducing the depth image information into sign language recognition research,a Chinese sign language recognition method based on DI_CamShift (Depth Image CamShift) and SIFT-BoW (Scale Invariant Feature Transform-Bag of Words) was presented．It uses Kinect as the video capture device to obtain both of the color video and depth im-age information of sign language．First,it calculates spindle direction angle and mass center position of the depth image correctly tracks gesture by adjusting the search window．Second,an Ostu algorithm based on depth integral image is used to gesture segmentation,and the SIFT features are extracted．Finally,it builds SIFT-BoW as the feature of sign language and uses SVM for recognition．The experimental results show that the best recognition rate of single manual alphabet can reach 99.87%,while the average recognition rate is 96.21%.

Key words: SIFT-BoW,DI_CamShift,Depth image,Kinect,Sign language recognition

杨全,彭进业. 采用SIFT-BoW和深度图像信息的中国手语识别研究[J]. 计算机科学, 2014, 41(2): 302-307. https://doi.org/

YANG Quan and PENG Jin-ye. Chinese Sign Language Recognition Research Using SIFT-BoW and Depth Image Information[J]. Computer Science, 2014, 41(2): 302-307. https://doi.org/

参考文献

[1] Wachs J P,Kolsch M,Stern H,et al．Vision-Based Hand-Gesture Applications [J]．Communications of the ACM,February 2011,54(2):60-72
[2] Ren Zhou,Yuan Jun-song,Zhang Zheng-you．Robust Hand Gesture Recognition Based on Finger-Earth Mover’s Distance with a Commodity Depth Camera[C]∥The 19th ACM International Conference on Multimedia (MM’11)．Scottsdale,Arizona,USA,November 28-December 1,2011:1093-1096
[3] Doliotis P,Stefan A,McMurrough C,et al．Comparing Gesture Recognition Accuracy Using Color and Depth Information[C]∥Conference on Pervasive Technologies Related to Assistive Environments (PETRA)．Crete,Greece,May 2011:1-7
[4] 杨筱林,姚鸿勋．基于多尺度形状描述子的手势识别[J]．计算机工程与应用,2004(32):76-78
[5] 张良国,高文,陈熙霖,等．面向中等词汇量的中国手语视觉识别系统[J]．计算机研究与发展,2006,43(3):476-482
[6] 姜峰,高文,姚鸿勋,等．非特定人手语识别问题中的合成数据驱动方法[J]．计算机研究与发展,2007,44(5):873-881
[7] Deng J W,Tsui H T．A two-step approach based on HMM for the recognition of ASL[C]∥The 5th Asian Conference on Computer Vision．Melbourne,Australia,Jan 2002:1-6
[8] Chen Qing,Georganas N D,Petriu E M．Real-time vision-based hand gesture recognition using Haar-like features[C]∥Instrumentation and Measurement Technology Conference Procee-dings．May 2007:1-6
[9] Silanon K,Suvonvorn N．Hand motion analysis for Thai alphabet recognition using HMM [J]．International Journal of Information and Electronics Engineering,2011,1(1):65-71
[10] Eng-Jon O,Nicolas C H P,et al．Sign Language Recognition using Sequential Pattern Trees[C]∥The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)．Providence,Rhode Island,June 2012:2200-2207
[11] 张毅,张烁,罗元,等．基于Kinect深度图像信息的手势轨迹识别及应用[J]．计算机应用研究,2012,29(9):3547-3550
[12] 邓瑞,周玲玲,应忍,等.基于Kinect深度信息的手势提取与识别研究[J]．计算机应用研究,2013,30(4):1263-1265
[13] 朱志亮,刘富国,陶向阳,等．基于积分图和粒子群优化的肤色分割[J/OL]．http://www.cnki.net/kcms/detail/11.2127.TP.20130129.1543.016.html,2013-01
[14] 郎咸朋,朱枫,都颖明,等．基于积分图像的快速二维Otsu算法[J]．仪器仪表学报,2009,30(1):39-43
[15] 王宇石,高文．用基于视觉单词上下文的核函数对图像分类[J]．中国图象图形学报,2010,15(4):607-616
[16] 刘扬闻,霍宏,方涛．词包模型中视觉单词歧义性分析[J]．计算机工程,2011,34(19):204-209
[17] 张秋余,王道东,张墨逸,等．基于特征包支持向量机的手势识别[J]．计算机应用,2012,32(12):3392-3396
[18] Juan L,Gwun O．A Comparison of SIFT,PCA-SIFT and SURF [J]．International Journal of Image Processing,2009,3(4):143-152
[19] Bastanlar Y,Temizel A,Yardimci Y．Improved SIFT matching for image pairs with scale difference．Electronics Letters [J]．2010,46(5):107-108
[20] Lin C J．LibSVM:A library for Support Vector Machines [EB/OL]．http://www.csie.ntu.edu.tw/~cjlin/libsvm,2012-11

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed