基于惯性传感器融合控制算法的聋哑手语识别

doi:10.11896/jsjkx.191200143

计算机科学 ›› 2021, Vol. 48 ›› Issue (2): 231-237.doi: 10.11896/jsjkx.191200143

基于惯性传感器融合控制算法的聋哑手语识别

冉孟元¹, 刘礼¹, 李艳德², 王珊珊¹

1 重庆大学大数据与软件学院重庆400044
2 兰州大学信息科学与工程兰州730000

收稿日期:2019-12-24 修回日期:2020-05-11 出版日期:2021-02-15 发布日期:2021-02-04
通讯作者: 刘礼(dcsliuli@cqu.edu.cn)
作者简介:rmy@cqu.edu.cn

Deaf Sign Language Recognition Based on Inertial Sensor Fusion Control Algorithm

RAN Meng-yuan¹, LIU Li¹, LI Yan-de², WANG Shan-shan¹

1 School of Big Data & Software Engineering,Chongqing University,Chongqing 400044,China
2 School of Information Science & Engineering,Lanzhou University,Lanzhou 730000,China

Received:2019-12-24 Revised:2020-05-11 Online:2021-02-15 Published:2021-02-04
About author:RAN Meng-yuan,born in 1995,postgra-duate,is a student member of China Computer Federation.His main research interests include artificial intelligence and pattern recognition.
LIU Li,born in 1981,Ph.D,associate professor,is a member of China Computer Federation.His main research interests include big data analysis,artificial intelligence and smart wearable technology.

摘要/Abstract

摘要： 聋哑人如何与外界进行有效沟通一直是一个备受关注的难点问题。文中提出了一种基于惯性传感器融合控制算法的手语识别方案,旨在实现高效准确的实时手语识别。该融合控制算法采用反馈控制思想,对两种传统的姿态信息计算方法进行融合,减少了环境对传感器的影响,可以准确获取被测对象在瞬时状态下的姿态信息。该算法通过对自采的聋哑手语数据进行数据融合、数据预处理和特征提取等处理,利用支持向量机、K-近邻法和前馈神经网络分类器自适应模型集成的分类方法进行分类。结果显示,所提传感器融合控制算法有效地得出了实时姿态,该手语识别方案对30种聋哑拼音手语的识别准确率达到96.5%。所提方案将为聋哑人手语识别打下坚实的基础,并为传感器融合控制的相关研究提供参考。

关键词: 传感器融合, 动作捕捉, 惯性传感器, 聋哑手语识别, 姿态航向参考系统

Abstract: How to effectively communicate with the outside world for deaf people has always been a difficult issue that has attracted much attention.This paper proposes a sign language recognition scheme based on inertial sensor fusion control algorithm,which aims to achieve efficient and accurate real-time sign language recognition.This fusion control algorithm uses feedback control ideas to fuse two traditional attitude information calculation methods,reduces the impact of the environment on sensors,and can accurately obtain the attitude information of the measured object in the transient state.The algorithm performs data fusion,data preprocessing and feature extraction on the collected deaf-mute sign language data,and uses an adaptive model integration method composed of support vector machines(SVM),K-nearest neighbors(KNN) and feedforward neural networks(FNN) for classification.The results show that the proposed sensor fusion control algorithm effectively obtains real-time poses.The sign language recognition scheme achieves accurate recognition of 30 deaf-mute pinyin sign languages with a recognition accuracy of 96.5%.The work of this paper will lay a solid foundation for sign language recognition of deaf and dumb people,and provide refe-rences for related research on sensor fusion control.

Key words: Attitude and heading reference system, Deaf sign language recognition, Inertial sensor, Motion capture, Sensor fusion

中图分类号:

TP399

冉孟元, 刘礼, 李艳德, 王珊珊. 基于惯性传感器融合控制算法的聋哑手语识别[J]. 计算机科学, 2021, 48(2): 231-237. https://doi.org/10.11896/jsjkx.191200143

RAN Meng-yuan, LIU Li, LI Yan-de, WANG Shan-shan. Deaf Sign Language Recognition Based on Inertial Sensor Fusion Control Algorithm[J]. Computer Science, 2021, 48(2): 231-237. https://doi.org/10.11896/jsjkx.191200143

参考文献

[1] SHARMA V,KUMAR V,MASAGUPPI S C,et al.Virtual talk for deaf,mute,blind and normal humans[C]//2013 Texas Instruments India Educators' Conference.IEEE,2013:316-320.
[2] BHATNAGAR V S,MAGON R,SRIVASTAVA R,et al.Acost effective Sign Language to voice emulation system[C]//2015 Eighth International Conference on Contemporary Computing (IC3).IEEE,2015:521-525.
[3] SRIRAM N,NITHIYANANDHAM M.A hand gesture recognition based communication system for silent speakers[C]//2013 International Conference on Human Computer Interactions (ICHCI).IEEE,2013:1-5.
[4] PHI L T,NGUYEN H D,BUI T T Q,et al.A glove-based gesture recognition system for Vietnamese sign language[C]//2015 15th International Conference on Control,Automation and Systems (ICCAS).IEEE,2015:1555-1559.
[5] HOQUE M T,RIFAT-UT-TAUWAB M,KABIR M F,et al.Automated Bangla sign language translation system:Prospects,limitations and applications[C]//2016 5th International Confe-rence on Informatics,Electronics and Vision (ICIEV).IEEE,2016:856-862.
[6] ELHAYEK H,NACOUZI J,KASSEM A,et al.Sign to letter translator system using a hand glove[C]//The Third International Conference on e-Technologies and Networks for Development (ICeND2014).IEEE,2014:146-150.
[7] BEDREGAL B R C,DIMURO G P.Interval fuzzy rule-basedhand gesture recognition[C]//12th GAMM-IMACS International Symposium on Scientific Computing,Computer Arithmetic andValidated Numerics (SCAN 2006).IEEE,2006:12.
[8] BORGHETTI M,SARDINI E,SERPELLONI M.Sensorizedglove for measuring hand finger flexion for rehabilitation purposes[J].IEEE Transactions on Instrumentation and Measurement,2013,62(12):3308-3314.
[9] PŁAWIAK P,SONICKI T,NIEDWIECKI M,et al.Hand body language gesture recognition based on signals from specialized glove and machine learning algorithms[J].IEEE Transactions on Industrial Informatics,2016,12(3):1104-1113.
[10] TANYAWIWAT N,THIEMJARUS S.Design of an assistive communication glove using combined sensory channels[C]//2012 Ninth International Conference on Wearable and Implan-table Body Sensor Networks.IEEE,2012:34-39.
[11] VUTINUNTAKASAME S,JAIJONGRAK V,THIEMJARUS S.An assistive body sensor network glove for speech-and hea-ring-impaired disabilities[C]//2011 International Conference on Body Sensor Networks.IEEE,2011:7-12.
[12] KANWAL K,ABDULLAH S,AHMED Y B,et al.AssistiveGlove for Pakistani Sign Language Translation[C]//17th IEEE International Multi Topic Conference 2014.IEEE,2014:173-176.
[13] FU Y F,HO C S.Static finger language recognition for handicapped aphasiacs[C]//Second International Conference on Innovative Computing,Informatio and Control (ICICIC 2007).IEEE,2007:299.
[14] ARIF A,RIZVI S T H,JAWAID I,et al.Techno-Talk:An American Sign Language (ASL) Translator[C]//2016 International Conference on Control,Decision and Information Techno-logies(CoDIT).IEEE,2016:665-670.
[15] WU X,MAO X,CHEN L,et al.Trajectory-based view-invariant hand gesture recognition by fusing shape and orientation[J].IET Computer Vision,2015,9(6):797-805.
[16] WANG J S,CHUANG F C.An accelerometer-based digital pen with a trajectory recognition algorithm for handwritten digit and gesture recognition[J].IEEE Transactions on Industrial Electronics,2011,59(7):2998-3007.
[17] ZHANG X,CHEN X,LI Y,et al.A framework for hand gesture recognition based on accelerometer and EMG sensors[J].IEEE Transactions on Systems,Man,and Cybernetics-Part A:Systems and Humans,2011,41(6):1064-1076.
[18] LU Z,CHEN X,LI Q,et al.A hand gesture recognition framework and wearable gesture-based interaction prototype for mobile devices[J].IEEE transactions on human-machine systems,2014,44(2):293-299.
[19] ZHOU S,FEI F,ZHANG G,et al.2D human gesture tracking and recognition by the fusion of MEMS inertial and vision sensors[J].IEEE Sensors Journal,2013,14(4):1160-1170.
[20] LIU K,CHEN C,JAFARI R,et al.Fusion of inertial and depth sensor data for robust hand gesture recognition[J].IEEE Sensors Journal,2014,14(6):1898-1903.
[21] LU T.A motion control method of intelligent wheelchair based on hand gesture recognition[C]//2013 IEEE 8th Conference on Industrial Electronics and Applications (ICIEA).IEEE,2013:957-962.
[22] ZHU C,SHENG W.Wearable sensor-based hand gesture anddaily activity recognition for robot-assisted living[J].IEEE Transactions on Systems,Man,and Cybernetics-Part A:Systems and Humans,2011,41(3):569-573.
[23] HSU Y L,CHU C L,TSAI Y J,et al.An inertial pen with dynamic time warping recognizer for handwriting and gesture recognition[J].IEEE Sensors Journal,2014,15(1):154-163.
[24] HUSSAIN S M A,RASHID A B M H.User independent hand gesture recognition by accelerated dtw[C]//2012 International Conference on Informatics,Electronics & Vision (ICIEV).IEEE,2012:1033-1037.
[25] XU R,ZHOU S,LI W J.MEMS accelerometer based nonspecific-user hand gesture recognition[J].IEEE Sensors Journal,2011,12(5):1166-1173.
[26] ABID M R,PETRIU E M,AMJADIAN E.Dynamic sign language recognition for smart home interactive application using stochastic linear formal grammar[J].IEEE Transactions on Instrumentation and Measurement,2014,64(3):596-605.
[27] AKL A,FENG C,VALAEE S.A novel accelerometer-basedgesture recognition system[J].IEEE Transactions on Signal Processing,2011,59(12):6197-6205.
[28] ZHANG Z Q.Two-step calibration methods for miniature inertial and magnetic sensor units[J].IEEETransactions on Industrial Electronics,2014,62(6):3714-3723.
[29] CHEN C H,WU J C,CHEN J H.Prediction of flutter derivatives by artificial neural networks[J].Journal of wind enginee-ring and industrial aerodynamics,2008,96(10／11):1925-1937.
[30] ZHANG L,WANG H,LIANG J,et al.Decision support in can-cer base on fuzzy adaptive PSO for feedforward neural network training[C]//2008 International Symposium on Computer Science and Computational Technology.IEEE,2008:220-223.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

基于惯性传感器融合控制算法的聋哑手语识别

Deaf Sign Language Recognition Based on Inertial Sensor Fusion Control Algorithm

PDF (PC)

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 4

Metrics

本文评价

推荐阅读 0

[1]	刘亮, 蒲浩洋. 基于LSTM的多维度特征手势实时识别 Real-time LSTM-based Multi-dimensional Features Gesture Recognition 计算机科学, 2021, 48(8): 328-333. https://doi.org/10.11896/jsjkx.210300079
[2]	吴文华, 宋亚飞, 刘晶. 直觉模糊框架内的证据动态可靠性评估及应用 Dynamic Reliability Evaluation Method of Evidence Based on Intuitionistic Fuzzy Sets and Its Applications 计算机科学, 2018, 45(12): 160-165. https://doi.org/10.11896／j.issn.1002-137X.2018.12.025
[3]	周婧,陈庙红,吴豪杰. 基于惯性导航的平面航迹推算的研究 Research on Plane Dead Reckoning Based on Inertial Navigation System 计算机科学, 2017, 44(Z6): 582-586. https://doi.org/10.11896/j.issn.1002-137X.2017.6A.131
[4]	. 基于异质多传感器融合的网络安全态势感知模型计算机科学, 2008, 35(8): 69-73.