计算机科学 ›› 2020, Vol. 47 ›› Issue (7): 103-110.doi: 10.11896/jsjkx.200100073

• 计算机图形学&多媒体 • 上一篇    下一篇

基于自适应多分类器融合的手势识别

刘肖1, 袁冠1,2, 张艳梅1, 闫秋艳1, 王志晓1   

  1. 1 中国矿业大学计算机科学与技术学院 江苏 徐州221116
    2 教育部矿山数字化工程研究中心 江苏 徐州221116
  • 收稿日期:2020-01-13 出版日期:2020-07-15 发布日期:2020-07-16
  • 通讯作者: 袁冠(yuanguan@cumt.edu.cn)
  • 作者简介:liuxiaocumt2018@163.com
  • 基金资助:
    国家自然科学基金(71774159,61876186,61977061);中国博士后科学基金(2018M642358);绿色安全管理与政策科学智库(2018WHCC03)

Hand Gesture Recognition Based on Self-adaptive Multi-classifiers Fusion

LIU Xiao1, YUAN Guan1,2, ZHANG Yan-mei1, YAN Qiu-yan1, WANG Zhi-xiao1   

  1. 1 College of Computer Science and Technology,China University of Mining and Technology,Xuzhou,Jiangsu 221116,China
    2 Digitization of Mine,Engineering Research Center of Ministry of Education,Xuzhou,Jiangsu 221116,China
  • Received:2020-01-13 Online:2020-07-15 Published:2020-07-16
  • About author:LIU Xiao,born in 1994,postgraduate.His main research interests include data mining,pattern recognition and perceptual computing.
    YUAN GUAN,born in 1982,Ph.D,professor,is a member of China Computer Federation.His main research interests include artificial intelligence,big data technology,machine learning and computational intelligence.
  • Supported by:
    This work was supported by the National Natural Science Foundation of China (71774159,61876186,61977061),China Postdoctoral Science Foundation(2018M642358) and Think Tank of Green Safety Management and Policy Science (2018WHCC03)

摘要: 为了提高基于可穿戴设备手势识别的性能,针对单分类器在手势识别时会出现偏向性的问题,提出了基于自适应多分类器融合的手势识别方法(Self-adaptive Multi-classifiers Fusion,SAMCF)。首先,针对统计特征无法表征复杂手势之间类内变异性和相似性的问题,SAMCF使用卷积神经网络(Convolutional Neural Network,CNN)自动提取具有强表征能力的深度特征;然后,采用多个基本分类器对提取的特征向量进行识别,并通过自适应融合算法决策出最优识别结果,解决了单分类器的偏向性问题;最后,基于数据手套采集的数据集,验证了模型的鲁棒性和泛化能力。实验结果表明,SAMCF能够有效地提取手势的深度特征,解决单分类器的偏向性问题,提高了手势识别的效率,增强了手势识别的性能,对字符级手势(美国手语和阿拉伯数字)识别的准确率达到98.23%,较其他算法平均提高了5%;对单词级手势(中国手语)识别的准确率达到97.81%,较其他算法平均提高了4%。

关键词: CNN, 多分类器, 手势识别, 数据手套, 自适应融合算法

Abstract: In order to improve the performance of hand gesture recognition based on wearable devices,a hand gesture recognition method (SAMCF) based on self-adaptive multi-classifiers fusion is proposed to solve the bias of single classifier in hand gesture recognition.First,for statistical features that cannot characterize intra-class variability and similarity between complex gestures,SAMCF uses a convolutional neural network (CNN) to automatically extract depth features with strong representation capabilities.Then,SAMCF uses multiple basic classifiers to recognize the extracted feature vectors,and determines the optimal recognition result through self-adaptive fusion algorithm,which solves the bias of single classifier.After that,the robustness and genera-lization ability of the model are verified by using the data set collected by data glove.The experimental results show that SAMCF can effectively extract the depth features of gesture,solve the bias of single classifier,and improve the efficiency of hand gesture recognition and enhance the performance of hand gesture recognition.The recognition accuracy of character level hand gesture (American Sign Language and Arabic numerals) is 98.23%,which is 5% higher than other algorithms on average;the recognition accuracy of word level gesture (Chinese Sign Language) is 97.81%,which is 4% higher than other algorithm on average.

Key words: Convolutional neural network, Data glove, Hand gesture recognition, Multi-classifiers, Self-adaptive fusion algorithm

中图分类号: 

  • TP391
[1]JIANG D,ZHENG Z J,LI G F,et al.Gesture recognition based on binocular vision[J].Cluster Computing,2019,22(6):13261-13271.
[2]ZHANG X,YANG Z,CHEN T,et al.Cooperative sensing and wearable computing for sequential hand gesture recognition[J].IEEE Sensors,2019,(19):5775-5783.
[3]KUMAR P,ROY P P,DOGRA D P.Independent Bayesian classifier combination based sign language recognition using facial expression[J].Information Sciences,2018,428:30-48.
[4]CHANG G C,PARK J,OH C,et al.A decision tree based real-time hand gesture recognition method using kinect[J].Journal of Korea Multimedia Society,2013,16(12):1393-1402.
[5]SHIFALI S,SHATRUGHAN M,PRASHANT S R,et al.Hand gesture recognition using gaussian threshold and different SVM kernels[C]//Advances in Computing and Data Sciences.2018:138-147.
[6]SAGAYAM K M,HEMANTH D J.A probabilistic model for state sequence analysis in hidden markov model for hand gesture recognition[J].Computational Intelligence,2019,35(1):59-81.
[7]TAVAKOLI M,BENUSSI C,ALHAIS L P,et al.Robust hand gesture recognition with a double channel surface EMG wearable armband and SVM classifier[J].Biomedical Signal Proces-sing and Control,2018:121-130.
[8]ZHANG Z,TIAN Z,ZHANG Y,et al.U-DeepHand:FMCWRadar-Based Unsupervised Hand Gesture Feature Learning Using Deep Convolutional Auto-Encoder Network[J].IEEE Sensors Journal,2019,19(16):6811-6821.
[9]LI G F,TANGH,SUN Y,et al.Hand gesture recognition based on convolution neural network[J].Cluster Computing,2019,22(2):2719-2729.
[10]KRZYSZTOF C,JACEK R,ALICJA K.Gesture recognitionwith the linear optical sensor and recurrent neural networks[J].IEEE Sensors Journal,2018,18(13):5429-5438.
[11]MA C,WANG A,CHEN G,et al.Hand joints-based gesture recognition for noisy dataset using nested interval unscented Kalman filter with LSTM network[J].Visual Computer,2018,34(6/7/8):1053-1063.
[12]NUNEZ J C,CABIDO R,PANTRIGO J J,et al.Convolutional neural networks and long short-term memory for skeleton-based human activity and hand gesture recognition[J].Pattern Recognition,2018,76:80-94.
[13]PAUDYAL P,LEE J,BANERJEE A.DyFAV:Dynamic feature selection and voting for real-time recognition of fingerspelled alphabet using wearables[C]//Proceedings of The 22nd International Conference on Intelligent User Interfaces.2017:457-467.
[14]KITTLER J,HATEF M,DUIN R P W,et al.On combining classifiers[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,1998,20(3):226-239.
[15]YUAN G,WANG Z H,MENG F R,et al.An overview ofhuman activity recognition based on smartphone[J].Sensor Review,2019,39(2):288-306.
[16]WANG X,LI B Z.A novel median filtering forensics based on principal component analysis network[J].International Journal of Electronic Security and Digital Forensics,2019,11(2):145-159.
[17]DERNBACH S,DAS B,KRISHNAN N C,et al.Simple andComplex Activity Recognition through Smart Phones[C]//Intelligent Environments.2012:214-221.
[18]SAINATH T N,VINYALS O,SENIOR A,et al.Convolutional,Long Short-Term Memory,Fully Connected Deep Neural Networks[C]//2015 IEEE International Conference on Acoustics,Speech and Signal Processing(ICASSP).2015:4580-4584.
[19]LI C,XIE C,ZHANG B C.Deep Fisher discriminant learning for mobile hand gesture recognition[J].Pattern Recognition,2018,77:276-288.
[20]ZHANG X,CHEN X,LI Y.A Framework for Hand Gesture Recognition Based on Accelerometer and EMG Sensors[J].IEEE Transactions on Systems Man & Cybernetics Part A Systems & Humans,2011,41(6):1064-1076.
[21]WAHID M F,TAFRESHI R,AL-SOWAIDI M,et al.Subject-independent hand gesture recognition using normalization and machine learning algorithms[J].Journal of Computational Science,2018,27:69-76.
[22]FRANCISCO O,DANIEL R.Deep convolutional and LSTM recurrent neural networks for multimodal wearable activity Re-cognition[J].Sensors,2016,16(1):115.
[23]MA M,CHEN Z,WU J.A Recognition Method of Hand Gesture with CNN-SVM Model[C]//International Conference on Bio-Inspired Computing:Theories and Applications.Singapore:Springer,2016:399-404.
[1] 岳晴, 尹健宇, 王生生.
基于改进CNN的低剂量CT图像肺结节自动检测
Automatic Detection of Pulmonary Nodules in Low-dose CT Images Based on Improved CNN
计算机科学, 2022, 49(6A): 54-59. https://doi.org/10.11896/jsjkx.210400211
[2] 余本功, 张子薇, 王惠灵.
一种融合多层次情感和主题信息的TS-AC-EWM在线商品排序方法
TS-AC-EWM Online Product Ranking Method Based on Multi-level Emotion and Topic Information
计算机科学, 2022, 49(6A): 165-171. https://doi.org/10.11896/jsjkx.210400238
[3] 王杉, 徐楚怡, 师春香, 张瑛.
基于CNN-LSTM的卫星云图云分类方法研究
Study on Cloud Classification Method of Satellite Cloud Images Based on CNN-LSTM
计算机科学, 2022, 49(6A): 675-679. https://doi.org/10.11896/jsjkx.210300177
[4] 祝文韬, 兰先超, 罗唤霖, 岳彬, 汪洋.
改进Faster R-CNN的光学遥感飞机目标检测
Remote Sensing Aircraft Target Detection Based on Improved Faster R-CNN
计算机科学, 2022, 49(6A): 378-383. https://doi.org/10.11896/jsjkx.210300121
[5] 赵征鹏, 李俊钢, 普园媛.
基于卷积神经网络的Retinex低照度图像增强
Low-light Image Enhancement Based on Retinex Theory by Convolutional Neural Network
计算机科学, 2022, 49(6): 199-209. https://doi.org/10.11896/jsjkx.210400092
[6] 黄晓生, 徐静.
基于PCANet的非下采样剪切波域多聚焦图像融合
Multi-focus Image Fusion Method Based on PCANet in NSST Domain
计算机科学, 2021, 48(9): 181-186. https://doi.org/10.11896/jsjkx.200800064
[7] 王炽, 常俊.
基于3D卷积神经网络的CSI跨场景手势识别方法
CSI Cross-domain Gesture Recognition Method Based on 3D Convolutional Neural Network
计算机科学, 2021, 48(8): 322-327. https://doi.org/10.11896/jsjkx.200600122
[8] 刘亮, 蒲浩洋.
基于LSTM的多维度特征手势实时识别
Real-time LSTM-based Multi-dimensional Features Gesture Recognition
计算机科学, 2021, 48(8): 328-333. https://doi.org/10.11896/jsjkx.210300079
[9] 俞建业, 戚湧, 王宝茁.
基于Spark的车联网分布式组合深度学习入侵检测方法
Distributed Combination Deep Learning Intrusion Detection Method for Internet of Vehicles Based on Spark
计算机科学, 2021, 48(6A): 518-523. https://doi.org/10.11896/jsjkx.200700129
[10] 崔雯昊, 蒋慕蓉, 杨磊, 傅鹏铭, 朱凌霄.
结合MCycleGAN与RFCNN实现太阳斑点图高分辨重建
Combining MCycleGAN and RFCNN to Realize High Resolution Reconstruction of Solar Speckle Image
计算机科学, 2021, 48(6A): 38-42. https://doi.org/10.11896/jsjkx.201000160
[11] 刘汉卿, 康晓东, 李博, 张华丽, 冯继超, 韩俊玲.
利用深度学习网络对医学影像分类识别的比较研究
Comparative Study on Classification and Recognition of Medical Images Using Deep Learning Network
计算机科学, 2021, 48(6A): 89-94. https://doi.org/10.11896/jsjkx.201000116
[12] 束鑫, 常锋, 张歆, 杜睿, 余转.
基于BCNN的胎儿颅脑超声横切面识别算法
Transverse Section Recognition Algorithm Based on BCNN for Fetal Craniocerebral Ultrasound
计算机科学, 2021, 48(4): 151-156. https://doi.org/10.11896/jsjkx.200500049
[13] 齐延荣, 周夏冰, 李斌, 周清雷.
基于FPGA的CNN图像识别加速与优化
FPGA-based CNN Image Recognition Acceleration and Optimization
计算机科学, 2021, 48(4): 205-212. https://doi.org/10.11896/jsjkx.200600089
[14] 李冰荣, 皮德常, 候梦如.
基于CNN和LSTM的移动对象目的地预测
Destination Prediction of Moving Objects Based on Convolutional Neural Networks and Long-Short Term Memory
计算机科学, 2021, 48(4): 70-77. https://doi.org/10.11896/jsjkx.200200024
[15] 陈明豪, 祝跃飞, 芦斌, 翟懿, 李玎.
基于Attention-CNN的加密流量应用类型识别
Classification of Application Type of Encrypted Traffic Based on Attention-CNN
计算机科学, 2021, 48(4): 325-332. https://doi.org/10.11896/jsjkx.200900155
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!