计算机科学 ›› 2022, Vol. 49 ›› Issue (6A): 397-406.doi: 10.11896/jsjkx.210300270

• 图像处理&多媒体技术 • 上一篇    下一篇


肖治鸿, 韩晔彤, 邹永攀   

  1. 深圳大学计算机科学与软件工程学院 广东 深圳 518060
  • 出版日期:2022-06-10 发布日期:2022-06-08
  • 通讯作者: 邹永攀(yongpan@szu.edu.cn)
  • 作者简介:(xiaozhihong2019@email.szu.edu.cn)

Study on Activity Recognition Based on Multi-source Data and Logical Reasoning

XIAO Zhi-hong, HAN Ye-tong, ZOU Yong-pan   

  1. College of Computer Science and Software Engineering,Shenzhen University,Shenzhen 518060,China
  • Online:2022-06-10 Published:2022-06-08
  • About author:XIAO Zhi-hong,born in 1997,postgra-duate.His main research interests include intelligent perception,mobile computing,human-computer interaction.
    ZOU Yong-pan,born in 1990,Ph.D,assistant professor,is a member of China Computer Federation.His main research interests include intelligent perception,mobile computing,human-computer interaction.

摘要: 利用智能终端设备识别和记录人们日常行为活动对健康监测、残障人士辅助和老年人看护等具有重要意义。已有相关研究大都采用基于机器学习的思路,但都存在着诸如运算资源消耗大、训练数据采集负担重以及不同场景下扩展性低等不足,鉴于此,文中提出了一种基于多源感知和逻辑推理的行为识别技术,通过确定不同肢体之间动作的逻辑关联性,来实现对用户日常生活基础行为的准确刻画,相比已有工作,该技术方案具有运算轻量化、训练成本低及对用户和场景的多样性的扩展能力强等优势,实现了基于上述技术的行为识别系统,并开展了大量实验对系统性能进行评估。结果显示,所提方法对于走、跑、上下楼梯等11种日常行为活动的识别准确率高达90%以上。同时,对比基于机器学习的行为识别方法,所提技术大大减少了用户采集训练数据的量。

关键词: 惯性测量单元, 机器学习, 可穿戴设备, 逻辑推理, 行为识别

Abstract: The use of smart terminal equipment to identify and record people's daily behaviors is of great significance for health monitoring,the disabled assistance and elderly care.Most existing related studies adopt machine learning-based ideas,but there are problems such as high consumption of computing resources and training.Due to the heavy burden of data collection and low scalability in different scenarios,this paper proposes a behavior recognition technology based on multi-source perception and logical reasoning.By determining the logical correlation between the actions of different limbs,the accurate description of basic behaviors of users' daily life is realized.Compared with existing work,this technical solution has the advantages of lightweight calculation,low training cost and strong expansion ability to the diversity of users and scenes.This paper realizes a behavior recognition system based on the above technology.A large number of experiments have been carried out to evaluate the performance of the system.The results show that the proposed method has a recognition accuracy of more than 90% for 11 daily behavior activities such as walking,running,and up and down stairs.At the same time,compared with the behavior recognition method based on machine learning,the proposed technique greatly reduces the amount of training data collected by users.

Key words: Activity recognition, Inertial measurement unit, Logical reasoning, Machine learning, Wearable devices


  • TP391
[1] WANG Y,WU K,NI L M.Wifall:Device-free fall detection bywireless networks[J].IEEE Transactions on Mobile Computing,2016,16(2):581-594.
[2] WANG W,LIU A X,SHAHZAD M,et al.Understanding andmodeling of wifi signal based human activity recognition[C]//Proc of the 21st Annual Int Conf on Mobile Computing and Networking.New York:ACM,2015:65-76.
[3] ZOU H,ZHOU Y,YANG J,et al.Poster:WiFi-based Device-Free Human Activity Recognition via Automatic Representation Learning[C]//the 23rd Annual Int Conf.New York:ACM,2017.
[4] FANG B,LANE N D,ZHANG M,et al.BodyScan:Enabling radio-based sensing on wearable devices for contactless activity and vital sign monitoring[C]//Proc. of the 14th Annual Int. Conf. on Mobile Systems,Applications,and Services.New York:ACM,2016:97-110.
[5] ALI K,LIU A X,WANG W,et al.Keystroke recognition using wifi signals[C]//Proc of the 21st Annual Int Conf. on Mobile Computing and Networking.New York:ACM,2015:90-102.
[6] ZHANG L,WANG C,MA M,et al.WiDIGR:Direction-Inde-pendent Gait Recognition System Using Commercial Wi-Fi Devices[J].IEEE Internet of Things Journal,2020,7(2):1178-1191.
[7] PU Q,GUPTA S,GOLLAKOTA S,et al.Whole-home gesture recognition using wireless signals[C]//Proc. of the 19th Annual Int. Conf. on Mobile Computing & Networking.New York:ACM,2013:27-38.
[8] LU H,PAN W,LANE N D,et al.SoundSense:scalable sound sensing for people-centric applications on mobile phones[C]//Proc. of the 7th Int. Conf. on Mobile Systems,Applications,and Services.New York:ACM,2009:165-178.
[9] NI B,WANG G,MOULIN P.Rgbd-hudaact:A color-depthvideo database for human daily activity recognition[C]//2011 IEEE Int. Conf. on Computer Vision Workshops(ICCV workshops).Piscataway,NJ:IEEE,2011:1147-1153.
[10] VOIGT P,BUDDE M,PESCARA E,et al.Feasibility of human activity recognition using wearable depth cameras[C]//Proc. of the 2018 ACM Int. Symp. on Wearable Computers.New York:ACM,2018:92-95.
[11] ROGGEN D,CALATRONI A,ROSSI M,et al.Collecting complex activity datasets in highly rich networked sensor environments[C]//2010 Seventh Int. Conf. on Networked Sensing Systems(INSS).Piscataway,NJ:IEEE,2010:233-240.
[12] TAPIA E M,INTILLE S S,LARSON K.Activity recognition in the home using simple and ubiquitous sensors[C]//Int. Conf. on Pervasive Computing.Berlin:Springer,2004:158-175.
[13] CHING Y T,CHENG C C,HE G W,et al.Full model for sensors placement and activities recognition[C]//Proc. of the 2017 ACM Int. Joint Conf. on Pervasive and Ubiquitous Computing and Proceedings of the 2017 ACM Int. Symp. on Wearable Computers.New York:ACM,2017:17-20.
[14] KWAPISZ J R,WEISS G M,MOORE S A.Activity recognition using cell phone accelerometers[J].ACM SigKDD Explorations Newsletter,2011,12(2):74-82.
[15] ANJUM A,ILYAS M U.Activity recognition using smartphone sensors[C]//2013 IEEE 10th Consumer Communications and Networking Conference(CCNC).Piscataway,NJ:IEEE,2013:914-919.
[16] LEUTHEUSER H,SCHULDHAUS D,ESKOFIER B M.Hie-rarchical,multi-sensor based classification of daily life activities:comparison with state-of-the-art algorithms using a benchmark dataset[J].PloS One,2013,8(10):1-11.
[17] INOUE S,UEDA N,NOHARA Y,et al.Mobile activity recognition for a whole day:recognizing real nursing activities with big dataset[C]//Proc. of the 2015 ACM Int. Joint Conf. on Pervasive and Ubiquitous Computing.New York:ACM,2015:1269-1280.
[18] YAO L,NIE F,SHENG Q Z,et al.Learning from less for better:semi-supervised activity recognition via shared structure discovery[C]//Proc. of the 2016 ACM Int. Joint Conf. on Pervasive and Ubiquitous Computing.New York:ACM,2016:13-24.
[19] BAI L,YEUNG C,EFSTRATIOU C,et al.Motion2Vector:unsupervised learning in human activity recognition using wrist-sensing data[C]//Adjunct Proc. of the 2019 ACM Int. Joint Conf. on Pervasive and Ubiquitous Computing and Proc of the 2019 ACM Int Symp on Wearable Computers.New York:ACM,2019:537-542.
[20] GUO H,CHEN L,PENG L,et al.Wearable sensor based multimodal human activity recognition exploiting the diversity of classifier ensemble[C]//Proc. of the 2016 ACM Int. Joint Conf. on Pervasive and Ubiquitous Computing.New York:ACM,2016:1112-1123.
[21] LIU C,ZHANG L,LIU Z,et al.Lasagna:towards deep hierarchical understanding and searching over mobile sensing data[C]//Proc. of the 22nd Annual Int. Conf. on Mobile Computing and Networking.New York:ACM,2016:334-347.
[22] LAVANIA C,THULASIDASAN S,LAMARCA A,et al.Aweakly supervised activity recognition framework for real-time synthetic biology laboratory assistance[C]//Proc. of the 2016 ACM Int. Joint Conf. on Pervasive and Ubiquitous Computing.New York:ACM,2016:37-48.
[23] ZHENG H,WANG H,BLACK N.Human activity detection in smart home environment with self-adaptive neural networks[C]//2008 IEEE Int. Conf. on Networking,Sensing and Control.Piscataway,NJ:IEEE,2008:1505-1510.
[24] JIANG W,YIN Z.Human activity recognition using wearable sensors by deep convolutional neural networks[C]//Proc. of the 23rd ACM Int. Conf on Multimedia.New York:ACM,2015:1307-1310.
[25] YIN Y,XIE L,GU T,et al.AirContour:Building Contour-based Model for In-Air Writing Gesture Recognition[J].ACM Transa-ctions on Sensor Networks(TOSN),2019,15(4):1-25.
[26] CHEN K,YAO L,ZHANG D,et al.A semisupervised recurrent convolutional attention model for human activity recognition[J].IEEE transactions on neural networks and learning systems,2019,31(5):1747-1756.
[27] ORDÓÑEZ F J,ROGGEN D.Deep convolutional and lstm re-current neural networks for multimodal wearable activity recognition[J].Sensors,2016,16(1):115-140.
[28] FAN X,WANG F,WANG F,et al.When RFID meets deep learning:Exploring cognitive intelligence for activity identification[J].IEEE Wireless Communications,2019,26(3):19-25.
[29] OKITA T,INOUE S.Recognition of multiple overlapping activi-ties using compositional CNN-LSTM model[C]//Proc. of the 2017 ACM Int. Joint Conf. on Pervasive and Ubiquitous Computing and Proc. of the 2017 ACM Int. Symp. on Wearable Computers.New York:ACM,2017:165-168.
[30] LAPUT G,AHUJA K,GOEL M,et al.Ubicoustics:Plug-and-play acoustic activity recognition[C]//Proc. of the 31st Annual ACM Symp. on User Interface Software and Technology.New York:ACM,2018:213-224.
[31] MüNZNER S,SCHMIDT P,REISS A,et al.CNN-based sensor fusion techniques for multimodal human activity recognition[C]//Proc. of the 2017 ACM Int. Symp. on Wearable Computers.New York:ACM,2017:158-165.
[32] ZENG M,GAO H,YU T,et al.Understanding and improving recurrent networks for human activity recognition by continuous attention[C]//Proc. of the 2018 ACM Int. Symp. on Wearable Computers.New York:ACM,2018:56-63.
[33] XIE L,DONG X,WANG W,et al.Meta-activity recognition:A wearable approach for logic cognition-based activity sensing[C]//IEEE INFOCOM 2017-IEEE Conf on Computer Communications.Piscataway,NJ:IEEE,2017:1-9.
[34] RIBONI D,SZTYLER T,CIVITARESE G,et al.Unsupervised recognition of interleaved activities of daily living through ontological and probabilistic reasoning[C]//Proc. of the 2016 ACM Int. Joint Conf. on Pervasive and Ubiquitous Computing.New York:ACM,2016:1-12.
[35] YE J,STEVENSON G,DOBSON S.USMART:An unsuper-vised semantic mining activity recognition technique[J].ACM Transactions on Interactive Intelligent Systems(TiiS),2014,4(4):1-27.
[36] IHIANLE I K,NAEEM U,ISLAM S.Ontology-driven activity recognition from patterns of object use[C]//Proc. of the 2017 ACM Int. Joint Conf. on Pervasive and Ubiquitous Computing and Proc. of the 2017 ACM Int. Symp. on Wearable Computers.New York:ACM,2017:654-657.
[37] CIVITARESE G,PRESOTTO R,BETTINI C.Hybrid data-driven and context-aware activity recognition with mobile devices[C]//Adjunct Proc. of the 2019 ACM Int. Joint Conf. on Pervasive and Ubiquitous Computing and Proc. of the 2019 ACM Int. Symp. on Wearable Computers.New York:ACM,2019:266-267.
[38] RUAN W,SHENG Q Z,YANG L,et al.AudioGest:enablingfine-grained hand gesture detection by decoding echo signal[C]//Proc. of the 2016 ACM Int. Joint Conf. on Pervasive and Ubiquitous Computing.New York:ACM,2016:474-485.
[39] GUPTA S,MORRIS D,PATEL S,et al.Soundwave:using the doppler effect to sense gestures[C]//Proc. of the SIGCHI Conf. on Human Factors in Computing Systems.New York:ACM,2012:1911-1914.
[40] ADIB F,KABELAC Z,KATABI D,et al.3D tracking via body radio reflections[C]//11th {USENIX} Symp on Networked Systems Design and Implementation({NSDI} 14).Berkeley,CA:USENIX Association,2014:317-329.
[41] WANG Y,LIU J,CHEN Y,et al.E-eyes:device-free location-oriented activity identification using fine-grained wifi signatures[C]//Proc. of the 20th Annual Int. Conf. on Mobile Computing and Networking.New York:ACM,2014:617-628.
[42] DING H,SHANGGUAN L,YANG Z,et al.Femo:A platform for free-weight exercise monitoring with rfids[C]//Proc. of the 13th ACM Conf. on Embedded Networked Sensor Systems.New York:ACM,2015:141-154.
[1] 冷典典, 杜鹏, 陈建廷, 向阳.
Automated Container Terminal Oriented Travel Time Estimation of AGV
计算机科学, 2022, 49(9): 208-214. https://doi.org/10.11896/jsjkx.210700028
[2] 宁晗阳, 马苗, 杨波, 刘士昌.
Research Progress and Analysis on Intelligent Cryptology
计算机科学, 2022, 49(9): 288-296. https://doi.org/10.11896/jsjkx.220300053
[3] 李瑶, 李涛, 李埼钒, 梁家瑞, Ibegbu Nnamdi JULIAN, 陈俊杰, 郭浩.
Construction and Multi-feature Fusion Classification Research Based on Multi-scale Sparse Brain Functional Hyper-network
计算机科学, 2022, 49(8): 257-266. https://doi.org/10.11896/jsjkx.210600094
[4] 张光华, 高天娇, 陈振国, 于乃文.
Study on Malware Classification Based on N-Gram Static Analysis Technology
计算机科学, 2022, 49(8): 336-343. https://doi.org/10.11896/jsjkx.210900203
[5] 何强, 尹震宇, 黄敏, 王兴伟, 王源田, 崔硕, 赵勇.
Survey of Influence Analysis of Evolutionary Network Based on Big Data
计算机科学, 2022, 49(8): 1-11. https://doi.org/10.11896/jsjkx.210700240
[6] 陈明鑫, 张钧波, 李天瑞.
Survey on Attacks and Defenses in Federated Learning
计算机科学, 2022, 49(7): 310-323. https://doi.org/10.11896/jsjkx.211000079
[7] 李亚茹, 张宇来, 王佳晨.
Survey on Bayesian Optimization Methods for Hyper-parameter Tuning
计算机科学, 2022, 49(6A): 86-92. https://doi.org/10.11896/jsjkx.210300208
[8] 赵璐, 袁立明, 郝琨.
Review of Multi-instance Learning Algorithms
计算机科学, 2022, 49(6A): 93-99. https://doi.org/10.11896/jsjkx.210500047
[9] 王飞, 黄涛, 杨晔.
Study on Machine Learning Algorithms for Life Prediction of IGBT Devices Based on Stacking Multi-model Fusion
计算机科学, 2022, 49(6A): 784-789. https://doi.org/10.11896/jsjkx.210400030
[10] 姚烨, 朱怡安, 钱亮, 贾耀, 张黎翔, 刘瑞亮.
一种基于异质模型融合的 Android 终端恶意软件检测方法
Android Malware Detection Method Based on Heterogeneous Model Fusion
计算机科学, 2022, 49(6A): 508-515. https://doi.org/10.11896/jsjkx.210700103
[11] 邵延华, 李文峰, 张晓强, 楚红雨, 饶云波, 陈璐.
Aerial Violence Recognition Based on Spatial-Temporal Graph Convolutional Networks and Attention Model
计算机科学, 2022, 49(6): 254-261. https://doi.org/10.11896/jsjkx.210400272
[12] 许杰, 祝玉坤, 邢春晓.
Application of Machine Learning in Financial Asset Pricing:A Review
计算机科学, 2022, 49(6): 276-286. https://doi.org/10.11896/jsjkx.210900127
[13] 李野, 陈松灿.
Physics-informed Neural Networks:Recent Advances and Prospects
计算机科学, 2022, 49(4): 254-262. https://doi.org/10.11896/jsjkx.210500158
[14] 么晓明, 丁世昌, 赵涛, 黄宏, 罗家德, 傅晓明.
Big Data-driven Based Socioeconomic Status Analysis:A Survey
计算机科学, 2022, 49(4): 80-87. https://doi.org/10.11896/jsjkx.211100014
[15] 张潆藜, 马佳利, 刘子昂, 刘新, 周睿.
Overview of Vulnerability Detection Methods for Ethereum Solidity Smart Contracts
计算机科学, 2022, 49(3): 52-61. https://doi.org/10.11896/jsjkx.210700004
Full text



No Suggested Reading articles found!