基于视觉手势识别的人机交互系统

计算机科学 ›› 2019, Vol. 46 ›› Issue (11A): 570-574.

基于视觉手势识别的人机交互系统

宋一凡, 张鹏, 刘立波

(宁夏大学信息工程学院银川750000)

出版日期:2019-11-10 发布日期:2019-11-20
通讯作者: 张鹏(1975-),男,博士,副教授,CCF会员,主要研究方向为智能信息处理,E-mail:pengzhang123@foxmail.com。
作者简介:宋一凡(1989-),男,硕士生,主要研究方向为深度学习,E-mail:yifanwz@126.com。
基金资助:
本文受自然科学基金(61862050),宁夏智慧农业关键技术集成应用与示范项目(2017BY067),宁夏回族自治区重点研发项目(2018BBF02006),社会科学基金(16BTY111)资助。

Human-machine Interaction System with Vision-based Gesture Recognition

SONG Yi-fan, ZHANG Peng, LIU Li-bo

(School of Information Engineering,Ningxia University,Yinchuan 750000,China)

Online:2019-11-10 Published:2019-11-20

摘要/Abstract

摘要： 人机交互系统是人与机器之间交流与信息传递的桥梁,随着计算机技术的迅速发展,使用鼠标、键盘等传统的人机交互技术已经不满足时代发展的需求,人们需要一种更快捷、更自然、更舒适的人机交互技术。基于手势的人机交互是人机交互系统的重要技术之一,传统的手势识别方法存在识别准确率不高、识别过程复杂等问题。针对上述缺陷,文中提出了一种基于深度学习的手势识别算法,该算法通过姿态估计对手势关节特征进行快速检测,利用卷积神经网络对关节特征图进行分类,克服了复杂背景中手势图像分割困难等问题,提高了识别结果的准确率。实验结果表明,该方法对各种手势不同尺度的表现具有很好的识别准确率,识别结果的准确率达到了98%。最后文中基于该算法设计了一个人机交互系统,并展示了手势识别在该人机交互系统中的应用。

关键词: 机器人, 深度学习, 手势识别, 姿态分析, 姿态预测

Abstract: Human-machine interaction system is the connection between human and computer.As the advances in computer technology,uses of mouse or keyboard are insufficient,now people need some nature and comfortable methods to manipulate the computers and so on.Gesture recognition-based method is one of the important human-machine interaction system,but there are some issues in the traditional method,such as low prediction accuracy,complex process procedure.To solve these problems,this paper proposed a deep learning-based gesture recognition algorithm.This method abstracts gesture feature heat map by pose estimation,then classified them by using convolutional neural network,overcomes the difficulty of image segmentation in complex background,and improves the accuracy in recognition.The result shows a 98% accuracy in recognition.Finally,a gesture guessing game robot was designed with this method,and the application of gesture recognition in human-machine interaction was presented.

Key words: Deep learning, Gesture recognition, Pose estimation, Pose prediction, Robot

中图分类号:

TP242

宋一凡, 张鹏, 刘立波. 基于视觉手势识别的人机交互系统[J]. 计算机科学, 2019, 46(11A): 570-574. https://doi.org/

SONG Yi-fan, ZHANG Peng, LIU Li-bo. Human-machine Interaction System with Vision-based Gesture Recognition[J]. Computer Science, 2019, 46(11A): 570-574. https://doi.org/

参考文献

[1]AGAH A.Human interactions with intelligent systems:Re-search taxonomy[J].Computers and Electrical Engineering,2000,27(1):71-107.
[2]TADEUSIEWICZ R.Speech in human system interaction[C]∥Human System Interactions.IEEE,2010:2-13.
[3]齐静,徐坤,丁希仑.机器人视觉手势交互技术研究进展[J].机器人,2017,39(4):565-584.
[4]PANWAR M.Hand gesture recognition based on shape parameters[C]∥International Conference on Computing.2012:317-319.
[5]RAUTARAY S S,AGRAWAL A.Vision based hand gesturerecognition for human computer interaction:a survey[J].Artificial Intelligence Review,2012,43(1):1-54.
[6]PISHARADY P K,SAERBECK M.Recent methods and databases in vision-based hand gesture recognition:A review[J].Computer Vision and Image Understanding,2015,141:152-165.
[7]TSENG K T,HUANG W F,WU C H.Vision-based fingerguessing game in human machine interaction[C]∥2006 IEEE International Conference on Robotics and Biomimetics.IEEE,2006:619-624.
[8]LIN C Y,CHUANG L W,CHENG L C,et al.An interactive finger-gaming robot with real-time emotion feedback[C]∥Proceedings of the 2015 6th International Conference on Automation.IEEE,2015:513-518.
[9]WEN X,NIU Y.A method for hand gesture recognition based on morphology and Fingertip-Angle[C]∥2010 The 2nd International Conference on Computer and Automation Engineering (ICCAE).2010,1:688-691.
[10]陈睿.基于仿人机器人的博弈型人机交互研[D].上海:上海交通大学,2014.
[11]WEI S E,RAMAKRISHNA V,KANADE T,et al.Convolutional pose machines[C]∥Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition.2016.
[12]HOCHREITER S,SCHMIDHUBER J.Long Short-Term Me-mory[J].Neural Computation,1997,9(8):1735-1780.
[13]GRAVES A.Supervised Sequence Labeling with RecurrentNeural Networks[M].Springer Berlin Heidelberg,2012.
[14]蔡娟,蔡坚勇,廖晓东.基于卷积神经网络的手势识别初探[J].计算机系统应用,2015,24(4):113-117.
[15]王龙,刘辉,王彬.结合肤色模型和卷积神经网络的手势识别方法[J].计算机工程与应用,2017,53(6):209-214.

相关文章 15

[1]	饶志双, 贾真, 张凡, 李天瑞. 基于Key-Value关联记忆网络的知识图谱问答方法 Key-Value Relational Memory Networks for Question Answering over Knowledge Graph 计算机科学, 2022, 49(9): 202-207. https://doi.org/10.11896/jsjkx.220300277
[2]	汤凌韬, 王迪, 张鲁飞, 刘盛云. 基于安全多方计算和差分隐私的联邦学习方案 Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy 计算机科学, 2022, 49(9): 297-305. https://doi.org/10.11896/jsjkx.210800108
[3]	徐涌鑫, 赵俊峰, 王亚沙, 谢冰, 杨恺. 时序知识图谱表示学习 Temporal Knowledge Graph Representation Learning 计算机科学, 2022, 49(9): 162-171. https://doi.org/10.11896/jsjkx.220500204
[4]	王剑, 彭雨琦, 赵宇斐, 杨健. 基于深度学习的社交网络舆情信息抽取方法综述 Survey of Social Network Public Opinion Information Extraction Based on Deep Learning 计算机科学, 2022, 49(8): 279-293. https://doi.org/10.11896/jsjkx.220300099
[5]	郝志荣, 陈龙, 黄嘉成. 面向文本分类的类别区分式通用对抗攻击方法 Class Discriminative Universal Adversarial Attack for Text Classification 计算机科学, 2022, 49(8): 323-329. https://doi.org/10.11896/jsjkx.220200077
[6]	姜梦函, 李邵梅, 郑洪浩, 张建朋. 基于改进位置编码的谣言检测模型 Rumor Detection Model Based on Improved Position Embedding 计算机科学, 2022, 49(8): 330-335. https://doi.org/10.11896/jsjkx.210600046
[7]	孙奇, 吉根林, 张杰. 基于非局部注意力生成对抗网络的视频异常事件检测方法 Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection 计算机科学, 2022, 49(8): 172-177. https://doi.org/10.11896/jsjkx.210600061
[8]	侯钰涛, 阿布都克力木·阿布力孜, 哈里旦木·阿布都克里木. 中文预训练模型研究进展 Advances in Chinese Pre-training Models 计算机科学, 2022, 49(7): 148-163. https://doi.org/10.11896/jsjkx.211200018
[9]	周慧, 施皓晨, 屠要峰, 黄圣君. 基于主动采样的深度鲁棒神经网络学习 Robust Deep Neural Network Learning Based on Active Sampling 计算机科学, 2022, 49(7): 164-169. https://doi.org/10.11896/jsjkx.210600044
[10]	何晓, 周佳立, 吴超. 面向铣削机器人的低成本精确标定及刀路拟合方法 Low Cost Accurate Calibration and Tool Path Fitting Method for Milling Robot 计算机科学, 2022, 49(7): 187-195. https://doi.org/10.11896/jsjkx.210500135
[11]	苏丹宁, 曹桂涛, 王燕楠, 王宏, 任赫. 小样本雷达辐射源识别的深度学习方法综述 Survey of Deep Learning for Radar Emitter Identification Based on Small Sample 计算机科学, 2022, 49(7): 226-235. https://doi.org/10.11896/jsjkx.210600138
[12]	胡艳羽, 赵龙, 董祥军. 一种用于癌症分类的两阶段深度特征选择提取算法 Two-stage Deep Feature Selection Extraction Algorithm for Cancer Classification 计算机科学, 2022, 49(7): 73-78. https://doi.org/10.11896/jsjkx.210500092
[13]	程成, 降爱莲. 基于多路径特征提取的实时语义分割方法 Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction 计算机科学, 2022, 49(7): 120-126. https://doi.org/10.11896/jsjkx.210500157
[14]	王君锋, 刘凡, 杨赛, 吕坦悦, 陈峙宇, 许峰. 基于多源迁移学习的大坝裂缝检测 Dam Crack Detection Based on Multi-source Transfer Learning 计算机科学, 2022, 49(6A): 319-324. https://doi.org/10.11896/jsjkx.210500124
[15]	楚玉春, 龚航, 王学芳, 刘培顺. 基于YOLOv4的目标检测知识蒸馏算法研究 Study on Knowledge Distillation of Target Detection Algorithm Based on YOLOv4 计算机科学, 2022, 49(6A): 337-344. https://doi.org/10.11896/jsjkx.210600204

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed