基于多源数据和逻辑推理的行为识别技术研究

doi:10.11896/jsjkx.210300270

摘要/Abstract

摘要： 利用智能终端设备识别和记录人们日常行为活动对健康监测、残障人士辅助和老年人看护等具有重要意义。已有相关研究大都采用基于机器学习的思路,但都存在着诸如运算资源消耗大、训练数据采集负担重以及不同场景下扩展性低等不足,鉴于此,文中提出了一种基于多源感知和逻辑推理的行为识别技术,通过确定不同肢体之间动作的逻辑关联性,来实现对用户日常生活基础行为的准确刻画,相比已有工作,该技术方案具有运算轻量化、训练成本低及对用户和场景的多样性的扩展能力强等优势,实现了基于上述技术的行为识别系统,并开展了大量实验对系统性能进行评估。结果显示,所提方法对于走、跑、上下楼梯等11种日常行为活动的识别准确率高达90%以上。同时,对比基于机器学习的行为识别方法,所提技术大大减少了用户采集训练数据的量。

关键词: 惯性测量单元, 机器学习, 可穿戴设备, 逻辑推理, 行为识别

Abstract: The use of smart terminal equipment to identify and record people's daily behaviors is of great significance for health monitoring,the disabled assistance and elderly care.Most existing related studies adopt machine learning-based ideas,but there are problems such as high consumption of computing resources and training.Due to the heavy burden of data collection and low scalability in different scenarios,this paper proposes a behavior recognition technology based on multi-source perception and logical reasoning.By determining the logical correlation between the actions of different limbs,the accurate description of basic behaviors of users' daily life is realized.Compared with existing work,this technical solution has the advantages of lightweight calculation,low training cost and strong expansion ability to the diversity of users and scenes.This paper realizes a behavior recognition system based on the above technology.A large number of experiments have been carried out to evaluate the performance of the system.The results show that the proposed method has a recognition accuracy of more than 90% for 11 daily behavior activities such as walking,running,and up and down stairs.At the same time,compared with the behavior recognition method based on machine learning,the proposed technique greatly reduces the amount of training data collected by users.

Key words: Activity recognition, Inertial measurement unit, Logical reasoning, Machine learning, Wearable devices

中图分类号:

TP391

肖治鸿, 韩晔彤, 邹永攀. 基于多源数据和逻辑推理的行为识别技术研究[J]. 计算机科学, 2022, 49(6A): 397-406. https://doi.org/10.11896/jsjkx.210300270

XIAO Zhi-hong, HAN Ye-tong, ZOU Yong-pan. Study on Activity Recognition Based on Multi-source Data and Logical Reasoning[J]. Computer Science, 2022, 49(6A): 397-406. https://doi.org/10.11896/jsjkx.210300270

参考文献

[1] WANG Y,WU K,NI L M.Wifall:Device-free fall detection bywireless networks[J].IEEE Transactions on Mobile Computing,2016,16(2):581-594.
[2] WANG W,LIU A X,SHAHZAD M,et al.Understanding andmodeling of wifi signal based human activity recognition[C]//Proc of the 21st Annual Int Conf on Mobile Computing and Networking.New York:ACM,2015:65-76.
[3] ZOU H,ZHOU Y,YANG J,et al.Poster:WiFi-based Device-Free Human Activity Recognition via Automatic Representation Learning[C]//the 23rd Annual Int Conf.New York:ACM,2017.
[4] FANG B,LANE N D,ZHANG M,et al.BodyScan:Enabling radio-based sensing on wearable devices for contactless activity and vital sign monitoring[C]//Proc. of the 14th Annual Int. Conf. on Mobile Systems,Applications,and Services.New York:ACM,2016:97-110.
[5] ALI K,LIU A X,WANG W,et al.Keystroke recognition using wifi signals[C]//Proc of the 21st Annual Int Conf. on Mobile Computing and Networking.New York:ACM,2015:90-102.
[6] ZHANG L,WANG C,MA M,et al.WiDIGR:Direction-Inde-pendent Gait Recognition System Using Commercial Wi-Fi Devices[J].IEEE Internet of Things Journal,2020,7(2):1178-1191.
[7] PU Q,GUPTA S,GOLLAKOTA S,et al.Whole-home gesture recognition using wireless signals[C]//Proc. of the 19th Annual Int. Conf. on Mobile Computing & Networking.New York:ACM,2013:27-38.
[8] LU H,PAN W,LANE N D,et al.SoundSense:scalable sound sensing for people-centric applications on mobile phones[C]//Proc. of the 7th Int. Conf. on Mobile Systems,Applications,and Services.New York:ACM,2009:165-178.
[9] NI B,WANG G,MOULIN P.Rgbd-hudaact:A color-depthvideo database for human daily activity recognition[C]//2011 IEEE Int. Conf. on Computer Vision Workshops(ICCV workshops).Piscataway,NJ:IEEE,2011:1147-1153.
[10] VOIGT P,BUDDE M,PESCARA E,et al.Feasibility of human activity recognition using wearable depth cameras[C]//Proc. of the 2018 ACM Int. Symp. on Wearable Computers.New York:ACM,2018:92-95.
[11] ROGGEN D,CALATRONI A,ROSSI M,et al.Collecting complex activity datasets in highly rich networked sensor environments[C]//2010 Seventh Int. Conf. on Networked Sensing Systems(INSS).Piscataway,NJ:IEEE,2010:233-240.
[12] TAPIA E M,INTILLE S S,LARSON K.Activity recognition in the home using simple and ubiquitous sensors[C]//Int. Conf. on Pervasive Computing.Berlin:Springer,2004:158-175.
[13] CHING Y T,CHENG C C,HE G W,et al.Full model for sensors placement and activities recognition[C]//Proc. of the 2017 ACM Int. Joint Conf. on Pervasive and Ubiquitous Computing and Proceedings of the 2017 ACM Int. Symp. on Wearable Computers.New York:ACM,2017:17-20.
[14] KWAPISZ J R,WEISS G M,MOORE S A.Activity recognition using cell phone accelerometers[J].ACM SigKDD Explorations Newsletter,2011,12(2):74-82.
[15] ANJUM A,ILYAS M U.Activity recognition using smartphone sensors[C]//2013 IEEE 10th Consumer Communications and Networking Conference(CCNC).Piscataway,NJ:IEEE,2013:914-919.
[16] LEUTHEUSER H,SCHULDHAUS D,ESKOFIER B M.Hie-rarchical,multi-sensor based classification of daily life activities:comparison with state-of-the-art algorithms using a benchmark dataset[J].PloS One,2013,8(10):1-11.
[17] INOUE S,UEDA N,NOHARA Y,et al.Mobile activity recognition for a whole day:recognizing real nursing activities with big dataset[C]//Proc. of the 2015 ACM Int. Joint Conf. on Pervasive and Ubiquitous Computing.New York:ACM,2015:1269-1280.
[18] YAO L,NIE F,SHENG Q Z,et al.Learning from less for better:semi-supervised activity recognition via shared structure discovery[C]//Proc. of the 2016 ACM Int. Joint Conf. on Pervasive and Ubiquitous Computing.New York:ACM,2016:13-24.
[19] BAI L,YEUNG C,EFSTRATIOU C,et al.Motion2Vector:unsupervised learning in human activity recognition using wrist-sensing data[C]//Adjunct Proc. of the 2019 ACM Int. Joint Conf. on Pervasive and Ubiquitous Computing and Proc of the 2019 ACM Int Symp on Wearable Computers.New York:ACM,2019:537-542.
[20] GUO H,CHEN L,PENG L,et al.Wearable sensor based multimodal human activity recognition exploiting the diversity of classifier ensemble[C]//Proc. of the 2016 ACM Int. Joint Conf. on Pervasive and Ubiquitous Computing.New York:ACM,2016:1112-1123.
[21] LIU C,ZHANG L,LIU Z,et al.Lasagna:towards deep hierarchical understanding and searching over mobile sensing data[C]//Proc. of the 22nd Annual Int. Conf. on Mobile Computing and Networking.New York:ACM,2016:334-347.
[22] LAVANIA C,THULASIDASAN S,LAMARCA A,et al.Aweakly supervised activity recognition framework for real-time synthetic biology laboratory assistance[C]//Proc. of the 2016 ACM Int. Joint Conf. on Pervasive and Ubiquitous Computing.New York:ACM,2016:37-48.
[23] ZHENG H,WANG H,BLACK N.Human activity detection in smart home environment with self-adaptive neural networks[C]//2008 IEEE Int. Conf. on Networking,Sensing and Control.Piscataway,NJ:IEEE,2008:1505-1510.
[24] JIANG W,YIN Z.Human activity recognition using wearable sensors by deep convolutional neural networks[C]//Proc. of the 23rd ACM Int. Conf on Multimedia.New York:ACM,2015:1307-1310.
[25] YIN Y,XIE L,GU T,et al.AirContour:Building Contour-based Model for In-Air Writing Gesture Recognition[J].ACM Transa-ctions on Sensor Networks(TOSN),2019,15(4):1-25.
[26] CHEN K,YAO L,ZHANG D,et al.A semisupervised recurrent convolutional attention model for human activity recognition[J].IEEE transactions on neural networks and learning systems,2019,31(5):1747-1756.
[27] ORDÓÑEZ F J,ROGGEN D.Deep convolutional and lstm re-current neural networks for multimodal wearable activity recognition[J].Sensors,2016,16(1):115-140.
[28] FAN X,WANG F,WANG F,et al.When RFID meets deep learning:Exploring cognitive intelligence for activity identification[J].IEEE Wireless Communications,2019,26(3):19-25.
[29] OKITA T,INOUE S.Recognition of multiple overlapping activi-ties using compositional CNN-LSTM model[C]//Proc. of the 2017 ACM Int. Joint Conf. on Pervasive and Ubiquitous Computing and Proc. of the 2017 ACM Int. Symp. on Wearable Computers.New York:ACM,2017:165-168.
[30] LAPUT G,AHUJA K,GOEL M,et al.Ubicoustics:Plug-and-play acoustic activity recognition[C]//Proc. of the 31st Annual ACM Symp. on User Interface Software and Technology.New York:ACM,2018:213-224.
[31] MüNZNER S,SCHMIDT P,REISS A,et al.CNN-based sensor fusion techniques for multimodal human activity recognition[C]//Proc. of the 2017 ACM Int. Symp. on Wearable Computers.New York:ACM,2017:158-165.
[32] ZENG M,GAO H,YU T,et al.Understanding and improving recurrent networks for human activity recognition by continuous attention[C]//Proc. of the 2018 ACM Int. Symp. on Wearable Computers.New York:ACM,2018:56-63.
[33] XIE L,DONG X,WANG W,et al.Meta-activity recognition:A wearable approach for logic cognition-based activity sensing[C]//IEEE INFOCOM 2017-IEEE Conf on Computer Communications.Piscataway,NJ:IEEE,2017:1-9.
[34] RIBONI D,SZTYLER T,CIVITARESE G,et al.Unsupervised recognition of interleaved activities of daily living through ontological and probabilistic reasoning[C]//Proc. of the 2016 ACM Int. Joint Conf. on Pervasive and Ubiquitous Computing.New York:ACM,2016:1-12.
[35] YE J,STEVENSON G,DOBSON S.USMART:An unsuper-vised semantic mining activity recognition technique[J].ACM Transactions on Interactive Intelligent Systems(TiiS),2014,4(4):1-27.
[36] IHIANLE I K,NAEEM U,ISLAM S.Ontology-driven activity recognition from patterns of object use[C]//Proc. of the 2017 ACM Int. Joint Conf. on Pervasive and Ubiquitous Computing and Proc. of the 2017 ACM Int. Symp. on Wearable Computers.New York:ACM,2017:654-657.
[37] CIVITARESE G,PRESOTTO R,BETTINI C.Hybrid data-driven and context-aware activity recognition with mobile devices[C]//Adjunct Proc. of the 2019 ACM Int. Joint Conf. on Pervasive and Ubiquitous Computing and Proc. of the 2019 ACM Int. Symp. on Wearable Computers.New York:ACM,2019:266-267.
[38] RUAN W,SHENG Q Z,YANG L,et al.AudioGest:enablingfine-grained hand gesture detection by decoding echo signal[C]//Proc. of the 2016 ACM Int. Joint Conf. on Pervasive and Ubiquitous Computing.New York:ACM,2016:474-485.
[39] GUPTA S,MORRIS D,PATEL S,et al.Soundwave:using the doppler effect to sense gestures[C]//Proc. of the SIGCHI Conf. on Human Factors in Computing Systems.New York:ACM,2012:1911-1914.
[40] ADIB F,KABELAC Z,KATABI D,et al.3D tracking via body radio reflections[C]//11th {USENIX} Symp on Networked Systems Design and Implementation({NSDI} 14).Berkeley,CA:USENIX Association,2014:317-329.
[41] WANG Y,LIU J,CHEN Y,et al.E-eyes:device-free location-oriented activity identification using fine-grained wifi signatures[C]//Proc. of the 20th Annual Int. Conf. on Mobile Computing and Networking.New York:ACM,2014:617-628.
[42] DING H,SHANGGUAN L,YANG Z,et al.Femo:A platform for free-weight exercise monitoring with rfids[C]//Proc. of the 13th ACM Conf. on Embedded Networked Sensor Systems.New York:ACM,2015:141-154.

相关文章 15

[1]	冷典典, 杜鹏, 陈建廷, 向阳. 面向自动化集装箱码头的AGV行驶时间估计 Automated Container Terminal Oriented Travel Time Estimation of AGV 计算机科学, 2022, 49(9): 208-214. https://doi.org/10.11896/jsjkx.210700028
[2]	宁晗阳, 马苗, 杨波, 刘士昌. 密码学智能化研究进展与分析 Research Progress and Analysis on Intelligent Cryptology 计算机科学, 2022, 49(9): 288-296. https://doi.org/10.11896/jsjkx.220300053
[3]	李瑶, 李涛, 李埼钒, 梁家瑞, Ibegbu Nnamdi JULIAN, 陈俊杰, 郭浩. 基于多尺度的稀疏脑功能超网络构建及多特征融合分类研究 Construction and Multi-feature Fusion Classification Research Based on Multi-scale Sparse Brain Functional Hyper-network 计算机科学, 2022, 49(8): 257-266. https://doi.org/10.11896/jsjkx.210600094
[4]	张光华, 高天娇, 陈振国, 于乃文. 基于N-Gram静态分析技术的恶意软件分类研究 Study on Malware Classification Based on N-Gram Static Analysis Technology 计算机科学, 2022, 49(8): 336-343. https://doi.org/10.11896/jsjkx.210900203
[5]	何强, 尹震宇, 黄敏, 王兴伟, 王源田, 崔硕, 赵勇. 基于大数据的进化网络影响力分析研究综述 Survey of Influence Analysis of Evolutionary Network Based on Big Data 计算机科学, 2022, 49(8): 1-11. https://doi.org/10.11896/jsjkx.210700240
[6]	陈明鑫, 张钧波, 李天瑞. 联邦学习攻防研究综述 Survey on Attacks and Defenses in Federated Learning 计算机科学, 2022, 49(7): 310-323. https://doi.org/10.11896/jsjkx.211000079
[7]	李亚茹, 张宇来, 王佳晨. 面向超参数估计的贝叶斯优化方法综述 Survey on Bayesian Optimization Methods for Hyper-parameter Tuning 计算机科学, 2022, 49(6A): 86-92. https://doi.org/10.11896/jsjkx.210300208
[8]	赵璐, 袁立明, 郝琨. 多示例学习算法综述 Review of Multi-instance Learning Algorithms 计算机科学, 2022, 49(6A): 93-99. https://doi.org/10.11896/jsjkx.210500047
[9]	王飞, 黄涛, 杨晔. 基于Stacking多模型融合的IGBT器件寿命的机器学习预测算法研究 Study on Machine Learning Algorithms for Life Prediction of IGBT Devices Based on Stacking Multi-model Fusion 计算机科学, 2022, 49(6A): 784-789. https://doi.org/10.11896/jsjkx.210400030
[10]	姚烨, 朱怡安, 钱亮, 贾耀, 张黎翔, 刘瑞亮. 一种基于异质模型融合的 Android 终端恶意软件检测方法 Android Malware Detection Method Based on Heterogeneous Model Fusion 计算机科学, 2022, 49(6A): 508-515. https://doi.org/10.11896/jsjkx.210700103
[11]	邵延华, 李文峰, 张晓强, 楚红雨, 饶云波, 陈璐. 基于时空图卷积和注意力模型的航拍暴力行为识别 Aerial Violence Recognition Based on Spatial-Temporal Graph Convolutional Networks and Attention Model 计算机科学, 2022, 49(6): 254-261. https://doi.org/10.11896/jsjkx.210400272
[12]	许杰, 祝玉坤, 邢春晓. 机器学习在金融资产定价中的应用研究综述 Application of Machine Learning in Financial Asset Pricing:A Review 计算机科学, 2022, 49(6): 276-286. https://doi.org/10.11896/jsjkx.210900127
[13]	李野, 陈松灿. 基于物理信息的神经网络:最新进展与展望 Physics-informed Neural Networks:Recent Advances and Prospects 计算机科学, 2022, 49(4): 254-262. https://doi.org/10.11896/jsjkx.210500158
[14]	么晓明, 丁世昌, 赵涛, 黄宏, 罗家德, 傅晓明. 大数据驱动的社会经济地位分析研究综述 Big Data-driven Based Socioeconomic Status Analysis:A Survey 计算机科学, 2022, 49(4): 80-87. https://doi.org/10.11896/jsjkx.211100014
[15]	张潆藜, 马佳利, 刘子昂, 刘新, 周睿. 以太坊Solidity智能合约漏洞检测方法综述 Overview of Vulnerability Detection Methods for Ethereum Solidity Smart Contracts 计算机科学, 2022, 49(3): 52-61. https://doi.org/10.11896/jsjkx.210700004

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed