计算机科学 ›› 2023, Vol. 50 ›› Issue (6A): 220300232-5.doi: 10.11896/jsjkx.220300232
王天然, 王琦, 王青山
WANG Tianran, WANG Qi, WANG Qingshan
摘要： 手语是听障人士重要的交流工具,准确识别手语可以减少健全人和听障人士之间的交流障碍。一般深度学习识别模型的性能高度依赖于所采集的数据,这导致模型跨对象泛化能力较差。因此,通过迁移学习的方法设计一种具有跨对象泛化能力的手语手势识别模型。首先,使用特征提取器融合表面肌电流(Surface Electromyography,sEMG)信号和惯性传感器(Inertial Measurement Unit,IMU)信号。然后,提出一种域对抗训练方法,其可以仅依靠源域数据完成特征提取器和域分类器的对抗训练,实现特征提取从源域到目标域的迁移。最后,在手势分类器中利用域不变特征实现手语手势跨对象识别,提高了模型的泛化能力。实验表明,在包含200种手语手势共60000条手语样本数据集上,所提模型可将手语跨对象识别准确率提高到85.1%。
中图分类号:
| [1]World Health Organization.Deafness and hearing loss[EB/OL].(2021-04-01)[2022-03-21].https://www.who.int/news-room/fact-sheets/detail/deafness-and-hearing-loss. [2]QI J Q,JIANG G Z,LI G,et al.Surface EMG Hand GestureRecognition System Based on PCA and GRNN[J].Neural Comput Appl,2020,32(10):6343-6351. [3]KIM M,CHO J,LEE S,et al.IMU Sensor-based Hand GestureRecognition for Human-machine Interfaces[J].Sensors,2019,19(18):3827. [4]JOAO L,MIGUEL S,NUNO M,et al.Hand/Arm Gesture Segmentation by Motion Using IMU and EMG Sensing[C]//International Conference on Flexible Automation and Intelligent Manufacturing.Amsterdam:Elsevier,2017:107-113. [5]WU J,SUN L,JAFARI R.A Wearable System for Recognizing American Sign Language in Real-time Using IMU and Surface EMG Sensors[J].IEEE J.Biomed.Health Informatics,2016,20(5):1281-1290. [6]AÑAZCO E V,HAN S J,KIM K,et al.Hand Gesture Recognition Using Single Patchable Six-axis Inertial Measurement Unit via Recurrent Neural Networks[J].Sensors,2021,21(4):1404. [7]HU Y,WONG Y K,WEI W T,et al.A Novel Attention-based Hybrid CNN-RNN Architecture for sEMG-based Gesture Recognition[J].Plos One,2018,10(13):https://10.1371/journal.pone.0206049. [8]YOSINSKI J,CLUNE J,BENGIO Y,et al.How Transferable are Features in Deep Neural Networks[C]//Annual Conference on Neural Information Processing Systems.Montreal:NIPS,2014:3320-3328. [9]SHIN S,BAEK Y,LEE J H,et al.Korean Sign Language Re-cognition Using EMG and IMU Sensors Based on Group-depen-dent NN Models[C]//IEEE Symposium Series on Computa-tional Intelligence.New York:IEEE,2017:1770-1776. [10]CHEN X,LI Y,HU R,et al.Hand Gesture Recognition Based on Surface Electromyography Using Convolutional Neural Network with Transfer Learning Method[J].IEEE J Biomed Health Informatics,2021,25(4):1292-1304. [11]YANG J,ZOU H,ZHOU Y,et al.Learning Gestures fromWiFi:A Siamese Recurrent Convolutional Architecture[J].IEEE Internet Things J,2019,6(6):10763-10772. [12]KANG P,LI J,FAN B,et al.Wrist-worn Hand Gesture Recognition While Walking via Transfer Learning[J].IEEE J Biomed Health Informatics,2022,26(3):952-961. [13]BEN-DAVID S,BLITZER J,CRAMMER K,et al.Analysis of Representations for Domain Adaptation[C]//Annual Confe-rence on Neural Information Processing Systems.Montreal:NIPS,2006:137-144. [14]ZHOU J,XU W.End-to-end Learning of Semantic Role Labeling Using Recurrent Neural Networks[C]//Annual Meeting of the Association for Computational Linguistics.Stroudsburg:ACL,2015:1127-1137. [15]GOODFELLOW L J,BENGIO Y,COURVILLE A C.Deep Learning[M].Massachusetts:MIT Press,2016:47-49. [16]GOODFELLOW L J,POUGET A J,MIRZA M,et al.Generative Adversarial Nets[C]//Annual Conference on Neural Information Processing Systems.Montreal:NIPS,2014:2672-2680. [17]LI Y,ZHENG W M,ZONG Y,et al.A Bi-hemisphere Domain Adversarial Neural Network Model forEEG Emotion Recognition[J].IEEE Transactions on Affective Computing,2021,12(2):494-504. [18]YAROSLAV G,EVGENIYA U,HANA A,et al.Domain-Ad-versarial Training of Neural Networks[J].J Mach Learn Res,2016,17(59):1-35. [19]PAN S J,TSANG I W,KWOK J S,et al.Domain Adaptation via Transfer Component Analysis[J].IEEE Trans Neural Networks,2011,22(2):199-210. [20]LONG M,WANG J,DING G,et al.Transfer Feature Learning with Joint Distribution Adaptation[C]//IEEE International Conference on Computer Vision.New York:IEEE Press,2013:2200-2207. |
|
||
首页