计算机科学 ›› 2020, Vol. 47 ›› Issue (10): 1-8.doi: 10.11896/jsjkx.200400092

• 群智感知计算 • 上一篇    下一篇

基于手机传感器的人体活动识别综述

张春祥1, 赵春蕾1, 陈超1, 罗辉2   

  1. 1 天津理工大学计算机科学与工程学院 天津300384
    2 解放军信息工程大学电子技术学院 郑州450002
  • 收稿日期:2020-04-21 修回日期:2020-08-17 接受日期:2016-12-05 出版日期:2020-10-15 发布日期:2020-10-16
  • 通讯作者: 赵春蕾(zcltjut@gmail.com)
  • 作者简介:1449611075@qq.com
  • 基金资助:
    天津市自然科学青年基金(18JCQNJC69900)

Review of Human Activity Recognition Based on Mobile Phone Sensors

ZHANG Chun-xiang1, ZHAO Chun-lei1, CHEN Chao1, LUO Hui2   

  1. 1 School of Computer Science and Engineering,Tianjin University of Technology,Tianjin 300384,China
    2 School of Electronic Technology,PLA Information Engineering University,Zhengzhou 450002,China
  • Received:2020-04-21 Revised:2020-08-17 Accepted:2016-12-05 Online:2020-10-15 Published:2020-10-16
  • About author:ZHANG Chun-xiang,born in 1995,master student,is a member of China Computer Federation.His main research interests include data analysis and cyber security.
    ZHAO Chun-lei,born in 1979,Ph.D,associate professor,is a member of China Computer Federation.Her main research interests include cyber security and so on.
  • Supported by:
    Tianjin Natural Science Youth Fund (18JCQNJC69900)

摘要: 人体活动存在于日常生活的各方面,人体活动识别(HAR)具有广泛的应用价值,并受到广泛关注。随着智能手机的逐步发展,传感器嵌入到手机中使手机更加智能,实现了更加灵活的人机交互。人们一般随身携带智能手机,因此手机传感器信号中有丰富的人体活动信息,通过提取手机传感器的信号便可以识别用户活动。相比基于计算机视觉等方法,基于手机传感器的人体活动识别更能体现人体运动的本质,并且具有成本低、灵活、可移植性强的特点。文中详细阐述了基于手机传感器的人体活动识别的研究现状,并对系统结构和基本原理进行了详细的描述和总结,最后分析了基于手机传感器的人体活动识别目前存在的问题以及未来发展的方向。

关键词: 手机传感器, 人体活动识别, 模式识别, 数据处理

Abstract: All walks of life and daily life are affected by human activities.Human activity recognition (HAR) has a wide range of application,and has been widely concerned.With the gradual development of smart phones,sensors are embedded in the phone to make the phone more intelligent and realize more flexible man-machine interaction.Modern people usually carry smart phones with them,so there is a wealth of information about human activities in the signals of mobile phone sensors.By extracting signals from the phone’s sensors,it is possible to identify users’ activities.Compared with other methods on the strength of computer vision,HAR on account of mobile phone sensors can better reflect the essence of human movements,and has the characteristics of low cost,flexibility and strong portability.In this paper,the current situation of HAR based on mobile phone sensors is described in details,and the system structure and basic principles of the main technologies are described and summarized in details.Finally,the existing problems and future development direction of HAR based on mobile phone sensors are analyzed.

Key words: Mobile phone sensor, Human activity recognition, Pattern recognition, Data processing

中图分类号: 

  • TP391
[1]SAUNDERS J B,IMNAN V T,EBERHART H D.The major determinant in normal and pathological gait[J].Journal of Bone &Joint Surgery-american Volume,1953,35(3):543-558.
[2]MORRIS J R.Accelerometry-A technique for the measurement of human body movement[J].Journal of Biomechanics,1973,6(6):729-736.
[3]KELLOKUMPU V,ZHAO G,PIETIKAINEN M.Human ac-tivity recognition using a dynamictexture based method[C]//British Machine Vision Conference.USA:IEEE,2008,1:1-10.
[4]YOUSAF M H,HABIB H A.Virtual Keyboard:Real-Time Finger Joints Tracking for Keystroke Detection and Recognition[J].Arabian Journal for Science and Engineering,2014,39(2):923-934.
[5]YI W,XINLI J,RONGYU C,et al.Robust Indoor Human Activity Recognition Using Wireless Signals[J].Sensors,2015,15(7):17195-17208.
[6]LI M,MENG Y,LIU J,et al.When CSI Meets PublicWiFi:Inferring Your Mobile Phone Password via WiFi Signals[C]//the 2016 ACM SIGSAC Conference on Computer and Communications Security.New York:ACM,2016:1068-1079.
[7]NANDAKUMAR R,GOLLAKOTA S,WATSON N.Contactless Sleep Apnea Detection onSmartphones[C]//the 13th Annual International Conference.New York:ACM,2015:22-24.
[8]LEUNG X,TANFORD S,JIANG L.Is a picture really worth a thousand words[J].Journal of Hospitality and Tourism Technology,2017,8(1):19-38.
[9]HONG F,WEI M,YOU S,et al.Wavingauthentication:yoursmartphone authenticate you on motion gesture[C]//Proc of ACM Conference on Human Factors in Computing Systems.New York:ACM Press,2015:263-266.
[10]SEN S,GROVER K,SUBBARAJU V,et al.Inferring smart-phone keypress via smartwatch inertial sensing[C]//IEEE International Conference on Pervasive Computing & Communications Workshops.Kona:IEEE,2017:685-690.
[11]WANG Y,WU C,ZHENG K,et al.Improving Reliability:User Authentication onSmartphones Using Keystroke Biometrics[J].IEEE Access,2019,7(7):26218-26228.
[12]AHMADI S S,RASHAD S,ELGAZZAR H.Machine Learning Models for Activity Recognition and Authentication of Smartphone Users[C]//the 2019 IEEE 10th Annual Ubiquitous Computing.New York:Electronics & Mobile Communication Conference (UEMCON),2019:561-567.
[13]DANIEL K,JOAN C,KEVIN C,et al.A multimodalsmartphone sensor system for behaviour measurement and health status inference[J].Information Fusion,2020,53:43-54.
[14]KATHERINE M,SPACHOS P,PLATANIOTIS K N.Smartphone-Based Wellness Assessment Using Mobile Environmental Sensors[J].IEEE Systems Journal,2020,1(1):1-11.
[15]CHEN Z,JIANG C,XIANG S,et al.Smartphone Sensor-Based Human Activity Recognition Using Feature Fusion and Maximum Full a Posteriori[J].IEEE Transactions on Instrumentation and Measurement,2020,69(7):3992-4001.
[16]BERCHTOLD M,BUDDE M,GORDON D,et al.ActiServ:Activity Recognition Service for mobile phones[C]//2010 International Symposium on Wearable Computers (ISWC 2010).Seoul:IEEE Computer Society,2010:1-8.
[17]LI Y.3D gesture recognition based on dynamic template matching algorithm for intelligent wearable devices[J].Chinese Journal on Internet of Things,2019,000(001):97-105.
[18]GONG L,ZHAO Y,CHAOCAN X,et al.Robust Light-weight Magnetic-based Door Event Detection with Smartphones[J].IEEE Transactions on Mobile Computing,2018,18(11):2631-2646.
[19]AHMED A,MAHAMOD I,ROSDIADEE N,et al.On the Best Sensor for Keystrokes Inference Attack on Android[J].Procedia Technology,2013,9(9):989-995.
[20]KWAPISZ J R,WEISS G M,MOORE S A.Activity recognition using cell phone accelerometers[J].Acm Sigkdd Explorations Newsletter,2011,12(2):74-82.
[21]WEISS G M,LOCKHART J W,PULICKAL T,et al.Acti-tracker:A Smartphone-Based Activity Recognition System for Improving Health and Well-Being[C]//IEEE International Conference on Data Science & Advanced Analytics.Montreal:IEEE,2016:682-688.
[22]SHOAIB M,SCHOLTEN H,HAVINGA P J M.Towardsphysical activity recognition using smartphone sensors[C]//IEEE International Conference on Ubiquitous Intelligence & Computing & IEEE International Conference on Autonomic & Trusted Computing.Vietri sul Mere:IEEE Computer Society,2013:80-87.
[23]SHOAIB M,BOSCH S,INCEL O,et al.Fusion of Smartphone Motion Sensors for Physical Activity Recognition[J].Sensors,2014,14(6):10146-10176.
[24]ZHANG Y.A survey on generative adversarial networks[J].Electronic Design Engineering,2018,26(5):34-37,43.
[25]POWELL M J D.A Direct Search Optimization Method That Models the Objective and Constraint Functions by Linear Interpolation[J].Advances in Optimization & Numerical Analysis,1994,275(275):51-67.
[26]GOODFELLOW I J,POUGET-ABADIE J,MIRZA M,et al.Generative AdversarialNetworks[J].Advances in Neural Information Processing Systems,2014,3:2672-2680.
[27]KHAN A M,LEE Y K,LEE S,et al.Accelerometer’s position independent physical activity recognition system for long-term activity monitoring in the elderly[J].Medical & Biological Engineering & Computing,2010,48(12):1271-1279.
[28]KEFER K,HOLZMANN C,FINDLING R D.Evaluating thePlacement of Arm-Worn Devices for Recognizing Variations of Dynamic Hand Gestures[J].Journal of Mobile Multimedia,2017,12(3):225-242.
[29]WANG C,ZHANG J Z,WANG Z,et al.Position-independentactivity recognition model for smartphone based on frequency domain algorithm[C]//International Conference on Computer Science & Network Technology.Dalian:IEEE,2013:396-399.
[30]HENAR M,BERNARDOS A M,JOSUE I.Activity logging using lightweight classification techniques in mobile devices[J].Personal & Ubiquitous Computing,2013(4):675-695.
[31]YANG J.Toward Physical Activity Diary:Motion RecognitionUsing Simple Acceleration Features with Mobile Phones[C]//Proceedings of the 1st international workshop on Interactive multimedia for consumer electronics.Beijing:Association for Computing Machinery,2009:1-10.
[32]CHO S J,CHOI E S,AND BANG W C,et al.Two-stage recognition of raw acceleration signals for 3-D gesture-understanding cell phones[C]//The 10th International Workshop on Frontiers in Handwriting Recognition.Nijme Gen,2006:1-6.
[33]ZHANG Y,MARKOVIC S,SAPIR I,et al.Continuous Functional Activity Monitoring Based on Wearable Tri-axial Accelerometer and Gyroscope[C]//5th International Conference on Pervasive Computing Technologies for Healthcare(PervasiveHealth) and Workshops.Dublin:IEEE,2011:370-373.
[34]KHAN A M,LEE Y K,LEE S Y,et al.Human Activity Recognition via an Accelerometer-Enabled-Smartphone Using Kernel Discriminant Analysis[C]//2010 5th International Conference on Future Information Technology (FutureTech).Busan:IEEE,2010:1-6.
[35]TU B,GU L,TAI R,et al.Gait recognition using adaptivewaveletdenoising and SIFT descriptors [J].Journal of Chinese Computer Systems,2018,39(6):254-258.
[36]BOURKE,ALAN K,et al.Optimum gravity vector and vertical acceleration estimation using a tri-axial accelerometer for falls and normal activities[C]//2011 Annual International Confe-rence of the IEEE Engineering in Medicine and Biology Society.Boston:IEEE,2011:7896-7899.
[37]NOOR M,SALCIC H M Z,WANG K K.Dynamic sliding window method for physical activity recognition using a single tria-xial accelerometer[C]//2015 IEEE 10th Conference on Industrial Electronics and Applications (ICIEA).Auckland:IEEE,2015:102-107.
[38]ORESTI B,JUAN-MANUEL G,MIGUEL D,et al.WindowSize Impact in Human Activity Recognition[J].Sensors,2014,14(4):6474-6499.
[39]YANG J,WANG S,CHEN N,et al.Wearable AccelerometerBased Extendable Activity Recognition System[C]//IEEE International Conference on Robotics & Automation.Anchorage:IEEE,2010:3641-3647.
[40]ARIF M,BILAL M,KATTAN A,et al.Better Physical Activity Classification using Smartphone Acceleration Sensor[J].Journal of Medical Systems,2014,38(9):95-104.
[41]KELA J,KALLIO S.Accelerometer-based gesture control for a design environment[J].Personal and Ubiquitous Computing,2006,10(5):285-299.
[42]HE Z,JIN L.Activity Recognition from Acceleration Data Based on Discrete Consine Transform and SVM[C]//2009 IEEE International Conference on Systems,Man and Cybernetic.San Antonio:IEEE,2009:5041-5044.
[43]WANG Z,OATES T.Imaging time-scrics to improve classification and imputation[J].Computer Science,2015,3(27):3339-3345.
[44]TAPIA E M,INTILLE S,HASKELL W,et al.Real-Time Re-cognition of Physical Activities and Their Intensities Using Wireless Accelerometers and a Heart Rate Monitor[C]//2007 11th IEEE International Symposium on Wearable Computers.Boston:IEEE,2007:37-40.
[45]HE Z,LIU Z,JIN L,et al.Weightlessness feature-a novel feature for single tri-axial accelerometer based activity recognition
[C]//2008 19th International Conference on Pattern Recognition.Tampa:IEEE,2008:1-4.
[46]NATTHAPON P,SURAPA T,EKAWIT N,et al.Analysis of Optimal Sensor Positions for Activity Classification and Application on a Different Data Collection Scenario[J].Sensors,2017,17(4):774-783.
[47]QUINLAN J R.Induction of decision trees[J].Machine Lear-ning,1986,1(1):81-106.
[48]ERMES M,J PÄRKKA,MANTYJARVI J,et al.Detection of Daily Activities and Sports With Wearable Sensors in Controlled and Uncontrolled Conditions[J].IEEE Transactions on Information Technology in Biomedicine,2008,12(1):20-26.
[49]SONG W,ADE C,BROXTERMAN R,et al.Activity recognition in planetary navigation field tests using classification algorithms applied to accelerometer data[C]//2012 Annual International Conference of the IEEE Engineering in Medicine and Bio-logy Society.San Diego:IEEE,2012:1586-1589.
[50]USLU G,ALTUN O,BAYDERE S.A Bayesian approach for indoor human activity monitoring[C]//2011 11th International Conference on Hybrid Intelligent Systems (HIS).Melacca:IEEE,2011:324-327.
[51]TONG L,SONG Q,GE Y,et al.HMM-Based Human Fall Detection and Prediction Method Using Tri-Axial Accelerometer[J].IEEE Sensors Journal,2013(5):1849-1856.
[52]LI A,JI L,WANG S,et al.Physical activity classification using a single triaxial accelerometer based on HMM[C]//IET International Conference on Wireless Sensor Network 2010 (IET-WSN 2010).Beijing:IEEE,2010:155-160.
[53]NAM Y,PARK J W.Child Activity Recognition Based on Coo-perative Fusion Model of aTriaxial Accelerometer and a Barometric Pressure Sensor[J].IEEE Journal of Biomedical and Health Informatics,2013,17(2):420-426.
[54]ZENG M,NGUYEN L T,YU B,et al.Convolutional Neural Networks for Human Activity Recognition using Mobile Sensors[C]//6th International Conference on Mobile Computing,Applications and Services.Austin:IEEE,2014:197-205.
[55]ZHENG Y,LI F,ZHANG L,et al.Human posture detection method based on long short term memory network[J].Journal of Computer Applications,2018,38(6):1568-1574.
[1] 周立鹏, 孟利民, 周磊, 蒋维, 董建平. 基于BP神经网络的摔倒检测算法[J]. 计算机科学, 2020, 47(6A): 242-246.
[2] 王俊茜, 郑文先, 徐勇. 基于测试样本误差重构的协同表示分类方法[J]. 计算机科学, 2020, 47(6): 104-113.
[3] 禹鑫燚, 施甜峰, 唐权瑞, 殷慧武, 欧林林. 面向预测性维护的工业设备管理系统[J]. 计算机科学, 2020, 47(11A): 667-672.
[4] 王绪亮, 聂铁铮, 唐欣然, 黄菊, 李迪, 闫铭森, 刘畅. 流式数据处理的动态自适应缓存策略研究[J]. 计算机科学, 2020, 47(11): 122-127.
[5] 汪洋, 李鹏, 季一木, 樊卫北, 张玉杰, 王汝传, 陈国良. 高性能计算与天文大数据研究综述[J]. 计算机科学, 2020, 47(1): 1-6.
[6] 刘长齐, 邵堃, 霍星, 范冬阳, 檀结庆. 基于加权质量评价函数的K-means图像分割算法[J]. 计算机科学, 2019, 46(6A): 158-160.
[7] 王楠, 孙善武. 基于半监督聚类分析的无人机故障识别[J]. 计算机科学, 2019, 46(6A): 192-195.
[8] 张阳峰, 韦仕鸿, 邓娜娜, 王文瑞. 基于小波降噪的振动传感器数据分析[J]. 计算机科学, 2019, 46(6A): 537-539.
[9] 李宗鑫, 秦勃, 王梦倩. 基于时空关系模型的交通信号灯的实时检测与识别[J]. 计算机科学, 2018, 45(6): 314-319.
[10] 邱赐云, 李礼, 张欢, 吴佳. 大数据时代——从冯·诺依曼到计算存储融合[J]. 计算机科学, 2018, 45(11A): 71-75.
[11] 周宇欢,蒋大伟,龚勇,陈聪. 基于数据随机性特征和极速学习机的加密数据流识别[J]. 计算机科学, 2017, 44(Z6): 380-384.
[12] 任云,程福林,黎洪松. 基于频率敏感三维自组织映射的视差估计算法[J]. 计算机科学, 2017, 44(Z11): 225-227.
[13] 王溪波,葛宏帅,王瑞全,林海. 电梯远程监控系统中高并发通信服务器的设计[J]. 计算机科学, 2017, 44(4): 157-160.
[14] 李伟林,文剑,马文凯. 基于深度神经网络的语音识别系统研究[J]. 计算机科学, 2016, 43(Z11): 45-49.
[15] 王敏光,王喆. 基于类心距离的模糊支持向量数据描述[J]. 计算机科学, 2016, 43(5): 230-233.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] 孙启,金燕,何琨,徐凌轩. 用于求解混合车辆路径问题的混合进化算法[J]. 计算机科学, 2018, 45(4): 76 -82 .
[2] 伍建辉,黄中祥,李武,吴健辉,彭鑫,张生. 城市道路建设时序决策的鲁棒优化[J]. 计算机科学, 2018, 45(4): 89 -93 .
[3] 史雯隽,武继刚,罗裕春. 针对移动云计算任务迁移的快速高效调度算法[J]. 计算机科学, 2018, 45(4): 94 -99 .
[4] 杨羽琦,章国安,金喜龙. 车载自组织网络中基于车辆密度的双簇头路由协议[J]. 计算机科学, 2018, 45(4): 126 -130 .
[5] 夏庆勋,庄毅. 一种基于局部性原理的远程验证机制[J]. 计算机科学, 2018, 45(4): 148 -151 .
[6] 丁舒阳,黎冰,侍洪波. 基于改进的离散PSO算法的FJSP的研究[J]. 计算机科学, 2018, 45(4): 233 -239 .
[7] 侯彦娥,孔云峰,党兰学. 求解多车型校车路径问题的混合集合划分的GRASP算法[J]. 计算机科学, 2018, 45(4): 240 -246 .
[8] 魏芹双,武优西,刘靖宇,朱怀忠. 基于密度约束和间隙约束的对比模式挖掘[J]. 计算机科学, 2018, 45(4): 252 -256 .
[9] 童泽平,李涛,李立杰,任亮. 基于随机需求与产能限制的供应链协同优化研究[J]. 计算机科学, 2018, 45(4): 260 -265 .
[10] 蔡莉,梁宇,朱扬勇,何婧. 数据质量的历史沿革和发展趋势[J]. 计算机科学, 2018, 45(4): 1 -10 .