基于手机传感器的人体活动识别综述

doi:10.11896/jsjkx.200400092

Abstract

Abstract: All walks of life and daily life are affected by human activities.Human activity recognition (HAR) has a wide range of application,and has been widely concerned.With the gradual development of smart phones,sensors are embedded in the phone to make the phone more intelligent and realize more flexible man-machine interaction.Modern people usually carry smart phones with them,so there is a wealth of information about human activities in the signals of mobile phone sensors.By extracting signals from the phone’s sensors,it is possible to identify users’ activities.Compared with other methods on the strength of computer vision,HAR on account of mobile phone sensors can better reflect the essence of human movements,and has the characteristics of low cost,flexibility and strong portability.In this paper,the current situation of HAR based on mobile phone sensors is described in details,and the system structure and basic principles of the main technologies are described and summarized in details.Finally,the existing problems and future development direction of HAR based on mobile phone sensors are analyzed.

Key words: Data processing, Human activity recognition, Mobile phone sensor, Pattern recognition

CLC Number:

TP391

ZHANG Chun-xiang, ZHAO Chun-lei, CHEN Chao, LUO Hui. Review of Human Activity Recognition Based on Mobile Phone Sensors[J].Computer Science, 2020, 47(10): 1-8.

References

[1]SAUNDERS J B,IMNAN V T,EBERHART H D.The major determinant in normal and pathological gait[J].Journal of Bone &Joint Surgery-american Volume,1953,35(3):543-558.
[2]MORRIS J R.Accelerometry-A technique for the measurement of human body movement[J].Journal of Biomechanics,1973,6(6):729-736.
[3]KELLOKUMPU V,ZHAO G,PIETIKAINEN M.Human ac-tivity recognition using a dynamictexture based method[C]//British Machine Vision Conference.USA:IEEE,2008,1:1-10.
[4]YOUSAF M H,HABIB H A.Virtual Keyboard:Real-Time Finger Joints Tracking for Keystroke Detection and Recognition[J].Arabian Journal for Science and Engineering,2014,39(2):923-934.
[5]YI W,XINLI J,RONGYU C,et al.Robust Indoor Human Activity Recognition Using Wireless Signals[J].Sensors,2015,15(7):17195-17208.
[6]LI M,MENG Y,LIU J,et al.When CSI Meets PublicWiFi:Inferring Your Mobile Phone Password via WiFi Signals[C]//the 2016 ACM SIGSAC Conference on Computer and Communications Security.New York:ACM,2016:1068-1079.
[7]NANDAKUMAR R,GOLLAKOTA S,WATSON N.Contactless Sleep Apnea Detection onSmartphones[C]//the 13th Annual International Conference.New York:ACM,2015:22-24.
[8]LEUNG X,TANFORD S,JIANG L.Is a picture really worth a thousand words[J].Journal of Hospitality and Tourism Technology,2017,8(1):19-38.
[9]HONG F,WEI M,YOU S,et al.Wavingauthentication:yoursmartphone authenticate you on motion gesture[C]//Proc of ACM Conference on Human Factors in Computing Systems.New York:ACM Press,2015:263-266.
[10]SEN S,GROVER K,SUBBARAJU V,et al.Inferring smart-phone keypress via smartwatch inertial sensing[C]//IEEE International Conference on Pervasive Computing & Communications Workshops.Kona:IEEE,2017:685-690.
[11]WANG Y,WU C,ZHENG K,et al.Improving Reliability:User Authentication onSmartphones Using Keystroke Biometrics[J].IEEE Access,2019,7(7):26218-26228.
[12]AHMADI S S,RASHAD S,ELGAZZAR H.Machine Learning Models for Activity Recognition and Authentication of Smartphone Users[C]//the 2019 IEEE 10th Annual Ubiquitous Computing.New York:Electronics & Mobile Communication Conference (UEMCON),2019:561-567.
[13]DANIEL K,JOAN C,KEVIN C,et al.A multimodalsmartphone sensor system for behaviour measurement and health status inference[J].Information Fusion,2020,53:43-54.
[14]KATHERINE M,SPACHOS P,PLATANIOTIS K N.Smartphone-Based Wellness Assessment Using Mobile Environmental Sensors[J].IEEE Systems Journal,2020,1(1):1-11.
[15]CHEN Z,JIANG C,XIANG S,et al.Smartphone Sensor-Based Human Activity Recognition Using Feature Fusion and Maximum Full a Posteriori[J].IEEE Transactions on Instrumentation and Measurement,2020,69(7):3992-4001.
[16]BERCHTOLD M,BUDDE M,GORDON D,et al.ActiServ:Activity Recognition Service for mobile phones[C]//2010 International Symposium on Wearable Computers (ISWC 2010).Seoul:IEEE Computer Society,2010:1-8.
[17]LI Y.3D gesture recognition based on dynamic template matching algorithm for intelligent wearable devices[J].Chinese Journal on Internet of Things,2019,000(001):97-105.
[18]GONG L,ZHAO Y,CHAOCAN X,et al.Robust Light-weight Magnetic-based Door Event Detection with Smartphones[J].IEEE Transactions on Mobile Computing,2018,18(11):2631-2646.
[19]AHMED A,MAHAMOD I,ROSDIADEE N,et al.On the Best Sensor for Keystrokes Inference Attack on Android[J].Procedia Technology,2013,9(9):989-995.
[20]KWAPISZ J R,WEISS G M,MOORE S A.Activity recognition using cell phone accelerometers[J].Acm Sigkdd Explorations Newsletter,2011,12(2):74-82.
[21]WEISS G M,LOCKHART J W,PULICKAL T,et al.Acti-tracker:A Smartphone-Based Activity Recognition System for Improving Health and Well-Being[C]//IEEE International Conference on Data Science & Advanced Analytics.Montreal:IEEE,2016:682-688.
[22]SHOAIB M,SCHOLTEN H,HAVINGA P J M.Towardsphysical activity recognition using smartphone sensors[C]//IEEE International Conference on Ubiquitous Intelligence & Computing & IEEE International Conference on Autonomic & Trusted Computing.Vietri sul Mere:IEEE Computer Society,2013:80-87.
[23]SHOAIB M,BOSCH S,INCEL O,et al.Fusion of Smartphone Motion Sensors for Physical Activity Recognition[J].Sensors,2014,14(6):10146-10176.
[24]ZHANG Y.A survey on generative adversarial networks[J].Electronic Design Engineering,2018,26(5):34-37,43.
[25]POWELL M J D.A Direct Search Optimization Method That Models the Objective and Constraint Functions by Linear Interpolation[J].Advances in Optimization & Numerical Analysis,1994,275(275):51-67.
[26]GOODFELLOW I J,POUGET-ABADIE J,MIRZA M,et al.Generative AdversarialNetworks[J].Advances in Neural Information Processing Systems,2014,3:2672-2680.
[27]KHAN A M,LEE Y K,LEE S,et al.Accelerometer’s position independent physical activity recognition system for long-term activity monitoring in the elderly[J].Medical & Biological Engineering & Computing,2010,48(12):1271-1279.
[28]KEFER K,HOLZMANN C,FINDLING R D.Evaluating thePlacement of Arm-Worn Devices for Recognizing Variations of Dynamic Hand Gestures[J].Journal of Mobile Multimedia,2017,12(3):225-242.
[29]WANG C,ZHANG J Z,WANG Z,et al.Position-independentactivity recognition model for smartphone based on frequency domain algorithm[C]//International Conference on Computer Science & Network Technology.Dalian:IEEE,2013:396-399.
[30]HENAR M,BERNARDOS A M,JOSUE I.Activity logging using lightweight classification techniques in mobile devices[J].Personal & Ubiquitous Computing,2013(4):675-695.
[31]YANG J.Toward Physical Activity Diary:Motion RecognitionUsing Simple Acceleration Features with Mobile Phones[C]//Proceedings of the 1st international workshop on Interactive multimedia for consumer electronics.Beijing:Association for Computing Machinery,2009:1-10.
[32]CHO S J,CHOI E S,AND BANG W C,et al.Two-stage recognition of raw acceleration signals for 3-D gesture-understanding cell phones[C]//The 10th International Workshop on Frontiers in Handwriting Recognition.Nijme Gen,2006:1-6.
[33]ZHANG Y,MARKOVIC S,SAPIR I,et al.Continuous Functional Activity Monitoring Based on Wearable Tri-axial Accelerometer and Gyroscope[C]//5th International Conference on Pervasive Computing Technologies for Healthcare(PervasiveHealth) and Workshops.Dublin:IEEE,2011:370-373.
[34]KHAN A M,LEE Y K,LEE S Y,et al.Human Activity Recognition via an Accelerometer-Enabled-Smartphone Using Kernel Discriminant Analysis[C]//2010 5th International Conference on Future Information Technology (FutureTech).Busan:IEEE,2010:1-6.
[35]TU B,GU L,TAI R,et al.Gait recognition using adaptivewaveletdenoising and SIFT descriptors [J].Journal of Chinese Computer Systems,2018,39(6):254-258.
[36]BOURKE,ALAN K,et al.Optimum gravity vector and vertical acceleration estimation using a tri-axial accelerometer for falls and normal activities[C]//2011 Annual International Confe-rence of the IEEE Engineering in Medicine and Biology Society.Boston:IEEE,2011:7896-7899.
[37]NOOR M,SALCIC H M Z,WANG K K.Dynamic sliding window method for physical activity recognition using a single tria-xial accelerometer[C]//2015 IEEE 10th Conference on Industrial Electronics and Applications (ICIEA).Auckland:IEEE,2015:102-107.
[38]ORESTI B,JUAN-MANUEL G,MIGUEL D,et al.WindowSize Impact in Human Activity Recognition[J].Sensors,2014,14(4):6474-6499.
[39]YANG J,WANG S,CHEN N,et al.Wearable AccelerometerBased Extendable Activity Recognition System[C]//IEEE International Conference on Robotics & Automation.Anchorage:IEEE,2010:3641-3647.
[40]ARIF M,BILAL M,KATTAN A,et al.Better Physical Activity Classification using Smartphone Acceleration Sensor[J].Journal of Medical Systems,2014,38(9):95-104.
[41]KELA J,KALLIO S.Accelerometer-based gesture control for a design environment[J].Personal and Ubiquitous Computing,2006,10(5):285-299.
[42]HE Z,JIN L.Activity Recognition from Acceleration Data Based on Discrete Consine Transform and SVM[C]//2009 IEEE International Conference on Systems,Man and Cybernetic.San Antonio:IEEE,2009:5041-5044.
[43]WANG Z,OATES T.Imaging time-scrics to improve classification and imputation[J].Computer Science,2015,3(27):3339-3345.
[44]TAPIA E M,INTILLE S,HASKELL W,et al.Real-Time Re-cognition of Physical Activities and Their Intensities Using Wireless Accelerometers and a Heart Rate Monitor[C]//2007 11th IEEE International Symposium on Wearable Computers.Boston:IEEE,2007:37-40.
[45]HE Z,LIU Z,JIN L,et al.Weightlessness feature-a novel feature for single tri-axial accelerometer based activity recognition
[C]//2008 19th International Conference on Pattern Recognition.Tampa:IEEE,2008:1-4.
[46]NATTHAPON P,SURAPA T,EKAWIT N,et al.Analysis of Optimal Sensor Positions for Activity Classification and Application on a Different Data Collection Scenario[J].Sensors,2017,17(4):774-783.
[47]QUINLAN J R.Induction of decision trees[J].Machine Lear-ning,1986,1(1):81-106.
[48]ERMES M,J PÄRKKA,MANTYJARVI J,et al.Detection of Daily Activities and Sports With Wearable Sensors in Controlled and Uncontrolled Conditions[J].IEEE Transactions on Information Technology in Biomedicine,2008,12(1):20-26.
[49]SONG W,ADE C,BROXTERMAN R,et al.Activity recognition in planetary navigation field tests using classification algorithms applied to accelerometer data[C]//2012 Annual International Conference of the IEEE Engineering in Medicine and Bio-logy Society.San Diego:IEEE,2012:1586-1589.
[50]USLU G,ALTUN O,BAYDERE S.A Bayesian approach for indoor human activity monitoring[C]//2011 11th International Conference on Hybrid Intelligent Systems (HIS).Melacca:IEEE,2011:324-327.
[51]TONG L,SONG Q,GE Y,et al.HMM-Based Human Fall Detection and Prediction Method Using Tri-Axial Accelerometer[J].IEEE Sensors Journal,2013(5):1849-1856.
[52]LI A,JI L,WANG S,et al.Physical activity classification using a single triaxial accelerometer based on HMM[C]//IET International Conference on Wireless Sensor Network 2010 (IET-WSN 2010).Beijing:IEEE,2010:155-160.
[53]NAM Y,PARK J W.Child Activity Recognition Based on Coo-perative Fusion Model of aTriaxial Accelerometer and a Barometric Pressure Sensor[J].IEEE Journal of Biomedical and Health Informatics,2013,17(2):420-426.
[54]ZENG M,NGUYEN L T,YU B,et al.Convolutional Neural Networks for Human Activity Recognition using Mobile Sensors[C]//6th International Conference on Mobile Computing,Applications and Services.Austin:IEEE,2014:197-205.
[55]ZHENG Y,LI F,ZHANG L,et al.Human posture detection method based on long short term memory network[J].Journal of Computer Applications,2018,38(6):1568-1574.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 0

No Suggested Reading articles found!

Review of Human Activity Recognition Based on Mobile Phone Sensors

PDF (PC)

Abstract

Cite this article

share this article

References

Related Articles 15

Metrics

Comments

Recommended 0

[1]	XING Yun-bing, LONG Guang-yu, HU Chun-yu, HU Li-sha. Human Activity Recognition Method Based on Class Increment SVM [J]. Computer Science, 2022, 49(5): 78-83.
[2]	SHI Jian, MO Jun. Optimization of GHTSOM Model by Data Corrosion [J]. Computer Science, 2021, 48(6A): 664-667.
[3]	SUN Wen-yun, JIN Zhong, ZHAO Hai-tao, CHEN Chang-sheng. Cross-domain Few-shot Face Spoofing Detection Method Based on Deep Feature Augmentation [J]. Computer Science, 2021, 48(2): 330-336.
[4]	JIAO Dong-lai, WANG Hao-xiang, LYU Hai-yang, XU Ke. Road Surface Object Detection from Mobile Phone Based Sensor Trajectories [J]. Computer Science, 2021, 48(11A): 283-289.
[5]	WANG Xin-ping, XIA Chun-ming, YAN Jian-jun. Sign Language Recognition Based on Image-interpreted Mechanomyography and Convolution Neural Network [J]. Computer Science, 2021, 48(11): 242-249.
[6]	ZHOU Li-peng, MENG Li-min, ZHOU Lei, JIANG Wei and DONG Jian-ping. Fall Detection Algorithm Based on BP Neural Network [J]. Computer Science, 2020, 47(6A): 242-246.
[7]	WANG Jun-qian, ZHENG Wen-xian, XU Yong. Novel Image Classification Based on Test Sample Error Reconstruction Collaborative Representation [J]. Computer Science, 2020, 47(6): 104-113.
[8]	YU Xin-yi, SHI Tian-feng, TANG Quan-rui, YIN Hui-wu, OU Lin-lin. Industrial Equipment Management System for Predictive Maintenance [J]. Computer Science, 2020, 47(11A): 667-672.
[9]	WANG Xu-liang, NIE Tie-zheng, TANG Xin-ran, HUANG Ju, LI Di, YAN Ming-sen, LIU Chang. Study on Dynamic Adaptive Caching Strategy for Streaming Data Processing [J]. Computer Science, 2020, 47(11): 122-127.
[10]	WANG Yang, LI Peng, JI Yi-mu, FAN Wei-bei, ZHANG Yu-jie, WANG Ru-chuan, CHEN Guo-liang. High Performance Computing and Astronomical Data:A Survey [J]. Computer Science, 2020, 47(1): 1-6.
[11]	LIU Chang-qi, SHAO Kun, HUO Xing, FAN Dong-yang, TAN Jie-qing. K-means Image Segmentation Algorithm Based on Weighted Quality Evaluation Function [J]. Computer Science, 2019, 46(6A): 158-160.
[12]	WANG Nan, SUN Shan-wu. UAV Fault Recognition Based on Semi-supervised Clustering [J]. Computer Science, 2019, 46(6A): 192-195.
[13]	ZHANG Yang-feng, WEI Shi-hong, DENG Na-na, WANG Wen-rui. Vibration Sensor Data Analysis Based on Wavelet Denoising [J]. Computer Science, 2019, 46(6A): 537-539.
[14]	LI Zong-xin, QIN Bo, WANG Meng-qian. Real-time Detection and Recognition of Traffic Light Based on Time-Space Model [J]. Computer Science, 2018, 45(6): 314-319.
[15]	QIU Ci-yun, LI Li, ZHANG Huan, WU Jia. Age of Big Data:from Von Neumann to Computing Storage Fusion [J]. Computer Science, 2018, 45(11A): 71-75.