计算机科学 ›› 2014, Vol. 41 ›› Issue (10): 50-52.doi: 10.11896/j.issn.1002-137X.2014.10.011

• 2013’和谐人机环境联合学术会议 • 上一篇    下一篇

基于RGB-D摄像头的实时手指跟踪与手势识别

刘鑫辰,傅慧源,马华东   

  1. 北京邮电大学智能通信软件与多媒体北京市重点实验室 北京100876;北京邮电大学智能通信软件与多媒体北京市重点实验室 北京100876;北京邮电大学智能通信软件与多媒体北京市重点实验室 北京100876
  • 出版日期:2018-11-14 发布日期:2018-11-14
  • 基金资助:
    本文受国家自然科学基金创新研究群体项目(61121001),高等学校博士学科点专项科研基金(20120005130002),国家杰出青年科学基金项目:网络环境中多媒体计算(60925010)资助

Real-time Fingertip Tracking and Gesture Recognition Using RGB-D Camera

LIU Xin-chen,FU Hui-yuan and MA Hua-dong   

  • Online:2018-11-14 Published:2018-11-14

摘要: 近些年,基于视觉的手部跟踪与手势识别一直是人机交互和计算机视觉等领域的研究热点。传统方法主要是使用单目或多目RGB摄像头等设备获得手部位置、方向等信息,但RGB摄像头易受到复杂背景、光照变化、纹理的限制,导致其准确性、实时性和鲁棒性都较差。随着可获得场景深度信息的家用RGB-Depth(RGB-D)摄像头的发展和上市,可以利用深度信息较好地克服上述环境问题。首先定义了一个基于RGB-D摄像头的3D交互空间,根据深度信息将手部区域从复杂背景、多变的光照条件下进行分割;然后提出了一种基于深度摄像头的手指识别和跟踪方法,该方法基于手部轮廓对人手及手指进行识别和跟踪;最后通过对手指位置和轨迹的跟踪进行手势识别,从而实现人机交互。对提出的方法进行的实验验证了它的准确性、实时性和鲁棒性。

关键词: 手指跟踪,手势识别,计算机视觉,人机交互,RGB-D摄像头

Abstract: In recent decades,visual interpretation of finger and hand gestures has been an attractive direction in both computer vision and human-computer interaction areas.Traditional methods use a monocular RGB camera or multiple RGB cameras to get hand information.But it is limited by clustered backgrounds,lighting conditions,textures and other environment factors,which makes the accuracy,robustness and efficiency cannot satisfy real-time interactions.With the coming of consumer-level RGB-D camera,above limitations can be overcame with depth data got from a RGB-D camera.We first defineed a 3D interaction space with the RGB-D camera and segmented the hand region from backgrounds with the help of depth information.Then we proposed a real-time finger recognition and tracking approach using a depth camera which mainly use the contour of hands.At last,the human-computer interaction was achieved with the position and trajectory of fingers from the above method.Based on the proposed method,we designed several experiments.The results validate the accuracy,effectiveness and robustness of our approach.

Key words: Finger tracking,Gesture recognition,Human-computer interaction,Computer vision,RGB-D camera

[1] Vladimir I P,Rajeev S,Thomas S H.Visual Interpretation ofHand Gestures for Human-Computer Interaction:A Review[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,1997,9(7) (下转第61页)(上接第52页)
[2] Ali E,George B,Mircea N,et al.Vision-based hand pose estimation:A review[J].Computer Vision and Image Understanding,2007(108):52-73
[3] Sushmita M,Tinku A.Gesture Recognition:A Survey[J].IEEE Transactions on Systems,MAN,and Cybernetics—Part C:A pplications and Reviews,2007,37(3)
[4] Juan P W,Mathias K,Helman S,et al.Vision-Based Hand-Gesture Applications[J].Communications of the ACM,2011, 54(2)
[5] Lars B,Ivan L,Tony L.Hand Gesture Recognition using Multi-Scale Colour Features,Hierarchical Models and Particle Filtering,Automatic Face and Gesture Recognition[C]∥Procee-dings.Fifth IEEE International Conference.May 2002:423-428
[6] Yikai F,Kongqiao W,Jian C,et al.A Real-time H andgestureRecognition Method[C]∥2007 IEEE International Conference on Multimedia and Expo.2007:995-998
[7] Kenji O,Yoichi S,Hideki K.Real-Time Fingertip Tracking and Gesture Recognition[J].IEEE Computer Graphics and Applications,2002,22(6):64-71
[8] Robert Y W,Jovan P.Real-Time Hand-Tracking with a Color Glove[J].ACM Transactions on Graphics (TOG), 2009,28(3)
[9] Jesus S,Robin R M.Hand Gesture Recognition with Depth Images:A Review[C]∥RO-MAN,2012 IEEE.2012:411-417
[10] Iason O,Nikolaos K,Antonis A A.Efficient Model-based 3DTracking of Hand Articulations using Kinect[C]∥Proceeding of BMVC.2011
[11] Cem K,Furkan K,Yunus E K,et al.Real Time Hand Pose Estimation using Depth Sensors[C]∥2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).2011:1228-1234
[12] Zhou R,Junsong Y,Zhengyou Z.Robust Hand Gesture Recognition Based on Finger-Earth Mover’s Distance with a Commodi-ty Depth Camera[C]∥MM ’11 Proceedings of the 19th ACM International Conference on Multimedia.2011:1093-1096
[13] 吴莉婷,张宇,杨一平,等.深度图像中基于轮廓曲线和局部区域特征的3维物体识别[J].中国图象图形学报,2012,17(2):269-278
[14] 梁秀波,张顺,李启雷,等.运动传感驱动的3D直观手势交互[J].计算机辅助设计与图形学学报,2010,22(3):521-526
[15] 万华根,肖海英,邹松,等.面向新一代大众游戏的手势交互技术[J].计算机辅助设计与图形学学报,2011,23(7)
[16] 周瑾,潘建江,童晶,等.使用kinect快速重建三维人体[J].计算机辅助设计与图形学学报,2013,25(6)

No related articles found!
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] 雷丽晖,王静. 可能性测度下的LTL模型检测并行化研究[J]. 计算机科学, 2018, 45(4): 71 -75 .
[2] 孙启,金燕,何琨,徐凌轩. 用于求解混合车辆路径问题的混合进化算法[J]. 计算机科学, 2018, 45(4): 76 -82 .
[3] 张佳男,肖鸣宇. 带权混合支配问题的近似算法研究[J]. 计算机科学, 2018, 45(4): 83 -88 .
[4] 伍建辉,黄中祥,李武,吴健辉,彭鑫,张生. 城市道路建设时序决策的鲁棒优化[J]. 计算机科学, 2018, 45(4): 89 -93 .
[5] 史雯隽,武继刚,罗裕春. 针对移动云计算任务迁移的快速高效调度算法[J]. 计算机科学, 2018, 45(4): 94 -99 .
[6] 周燕萍,业巧林. 基于L1-范数距离的最小二乘对支持向量机[J]. 计算机科学, 2018, 45(4): 100 -105 .
[7] 刘博艺,唐湘滟,程杰仁. 基于多生长时期模板匹配的玉米螟识别方法[J]. 计算机科学, 2018, 45(4): 106 -111 .
[8] 耿海军,施新刚,王之梁,尹霞,尹少平. 基于有向无环图的互联网域内节能路由算法[J]. 计算机科学, 2018, 45(4): 112 -116 .
[9] 崔琼,李建华,王宏,南明莉. 基于节点修复的网络化指挥信息系统弹性分析模型[J]. 计算机科学, 2018, 45(4): 117 -121 .
[10] 王振朝,侯欢欢,连蕊. 抑制CMT中乱序程度的路径优化方案[J]. 计算机科学, 2018, 45(4): 122 -125 .