计算机科学 ›› 2014, Vol. 41 ›› Issue (10): 50-52.doi: 10.11896/j.issn.1002-137X.2014.10.011

• 2013’和谐人机环境联合学术会议 • 上一篇    下一篇

基于RGB-D摄像头的实时手指跟踪与手势识别

刘鑫辰,傅慧源,马华东   

  1. 北京邮电大学智能通信软件与多媒体北京市重点实验室 北京100876;北京邮电大学智能通信软件与多媒体北京市重点实验室 北京100876;北京邮电大学智能通信软件与多媒体北京市重点实验室 北京100876
  • 出版日期:2018-11-14 发布日期:2018-11-14
  • 基金资助:
    本文受国家自然科学基金创新研究群体项目(61121001),高等学校博士学科点专项科研基金(20120005130002),国家杰出青年科学基金项目:网络环境中多媒体计算(60925010)资助

Real-time Fingertip Tracking and Gesture Recognition Using RGB-D Camera

LIU Xin-chen,FU Hui-yuan and MA Hua-dong   

  • Online:2018-11-14 Published:2018-11-14

摘要: 近些年,基于视觉的手部跟踪与手势识别一直是人机交互和计算机视觉等领域的研究热点。传统方法主要是使用单目或多目RGB摄像头等设备获得手部位置、方向等信息,但RGB摄像头易受到复杂背景、光照变化、纹理的限制,导致其准确性、实时性和鲁棒性都较差。随着可获得场景深度信息的家用RGB-Depth(RGB-D)摄像头的发展和上市,可以利用深度信息较好地克服上述环境问题。首先定义了一个基于RGB-D摄像头的3D交互空间,根据深度信息将手部区域从复杂背景、多变的光照条件下进行分割;然后提出了一种基于深度摄像头的手指识别和跟踪方法,该方法基于手部轮廓对人手及手指进行识别和跟踪;最后通过对手指位置和轨迹的跟踪进行手势识别,从而实现人机交互。对提出的方法进行的实验验证了它的准确性、实时性和鲁棒性。

关键词: 手指跟踪,手势识别,计算机视觉,人机交互,RGB-D摄像头

Abstract: In recent decades,visual interpretation of finger and hand gestures has been an attractive direction in both computer vision and human-computer interaction areas.Traditional methods use a monocular RGB camera or multiple RGB cameras to get hand information.But it is limited by clustered backgrounds,lighting conditions,textures and other environment factors,which makes the accuracy,robustness and efficiency cannot satisfy real-time interactions.With the coming of consumer-level RGB-D camera,above limitations can be overcame with depth data got from a RGB-D camera.We first defineed a 3D interaction space with the RGB-D camera and segmented the hand region from backgrounds with the help of depth information.Then we proposed a real-time finger recognition and tracking approach using a depth camera which mainly use the contour of hands.At last,the human-computer interaction was achieved with the position and trajectory of fingers from the above method.Based on the proposed method,we designed several experiments.The results validate the accuracy,effectiveness and robustness of our approach.

Key words: Finger tracking,Gesture recognition,Human-computer interaction,Computer vision,RGB-D camera

[1] Vladimir I P,Rajeev S,Thomas S H.Visual Interpretation ofHand Gestures for Human-Computer Interaction:A Review[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,1997,9(7) (下转第61页)(上接第52页)
[2] Ali E,George B,Mircea N,et al.Vision-based hand pose estimation:A review[J].Computer Vision and Image Understanding,2007(108):52-73
[3] Sushmita M,Tinku A.Gesture Recognition:A Survey[J].IEEE Transactions on Systems,MAN,and Cybernetics—Part C:A pplications and Reviews,2007,37(3)
[4] Juan P W,Mathias K,Helman S,et al.Vision-Based Hand-Gesture Applications[J].Communications of the ACM,2011, 54(2)
[5] Lars B,Ivan L,Tony L.Hand Gesture Recognition using Multi-Scale Colour Features,Hierarchical Models and Particle Filtering,Automatic Face and Gesture Recognition[C]∥Procee-dings.Fifth IEEE International Conference.May 2002:423-428
[6] Yikai F,Kongqiao W,Jian C,et al.A Real-time H andgestureRecognition Method[C]∥2007 IEEE International Conference on Multimedia and Expo.2007:995-998
[7] Kenji O,Yoichi S,Hideki K.Real-Time Fingertip Tracking and Gesture Recognition[J].IEEE Computer Graphics and Applications,2002,22(6):64-71
[8] Robert Y W,Jovan P.Real-Time Hand-Tracking with a Color Glove[J].ACM Transactions on Graphics (TOG), 2009,28(3)
[9] Jesus S,Robin R M.Hand Gesture Recognition with Depth Images:A Review[C]∥RO-MAN,2012 IEEE.2012:411-417
[10] Iason O,Nikolaos K,Antonis A A.Efficient Model-based 3DTracking of Hand Articulations using Kinect[C]∥Proceeding of BMVC.2011
[11] Cem K,Furkan K,Yunus E K,et al.Real Time Hand Pose Estimation using Depth Sensors[C]∥2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).2011:1228-1234
[12] Zhou R,Junsong Y,Zhengyou Z.Robust Hand Gesture Recognition Based on Finger-Earth Mover’s Distance with a Commodi-ty Depth Camera[C]∥MM ’11 Proceedings of the 19th ACM International Conference on Multimedia.2011:1093-1096
[13] 吴莉婷,张宇,杨一平,等.深度图像中基于轮廓曲线和局部区域特征的3维物体识别[J].中国图象图形学报,2012,17(2):269-278
[14] 梁秀波,张顺,李启雷,等.运动传感驱动的3D直观手势交互[J].计算机辅助设计与图形学学报,2010,22(3):521-526
[15] 万华根,肖海英,邹松,等.面向新一代大众游戏的手势交互技术[J].计算机辅助设计与图形学学报,2011,23(7)
[16] 周瑾,潘建江,童晶,等.使用kinect快速重建三维人体[J].计算机辅助设计与图形学学报,2013,25(6)

No related articles found!
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!