Computer Science ›› 2024, Vol. 51 ›› Issue (4): 291-298.doi: 10.11896/jsjkx.230300158

• Artificial Intelligence • Previous Articles     Next Articles

Study on Open Set Activity Recognition Technology Based on Wearable Devices

WANG Jiahao1, YAN Hang1, HU Xin1, ZHAO Dexin2   

  1. 1 School of Information and Software Engineering,University of Electronic Science and Technology,Chengdu 610051,China
    2 National Innovation Institute of Defense Technology,Academy of Military Sciences,Beijing 100071,China
  • Received:2023-03-20 Revised:2023-06-30 Online:2024-04-15 Published:2024-04-10
  • Supported by:
    UESTC-ZHIXIAOJING Joint Research Center of Smart Home(H04W210180),Neijiang Technology Incubation and Transformation Funds(2021KJFH004) and Science and Technology Program of Sichuan Province,China(2022YFG0212,2021YFG0024).

Abstract: With the popularity of wearable devices such as smart watches and bracelets,using them for human activity recognition and decoding human behavior is of great significance for health monitoring,daily behavior analysis,smart home and other applications.However,traditional action recognition algorithms have problems such as difficult feature extraction and low recognition accuracy,and are all based on the close set assumption,that is,all training data and test data come from the same label space,while most of the real world is open.In the open-set scene,unknown label samples may be sent to the model during the test phase,resulting in incorrect classification.This paper proposes a multi-channel adaptive convolutional network(MCACN) for human acti-vity recognition.For the problem that the traditional CNN network feature extraction is limited to a small range,the adaptive convolution module can use convolution kernels of different sizes to extract features of different time spans,automatically calculate the weights and sum them up.In addition,the multi-channel structure of MCACN enables each sensor data to be processed separately to obtain feature details that can distinguish similar actions.Finally,this paper designs a label-based multivariate variational autoencoder,and proposes MCACN-VAE for open set recognition.The model can identify unknown classes by calculating recons-truction loss,focusing on known class actions,and improving the robustness of the model.Experimental results show that in the closed set experiment,the MCACN model can effectively recognize the actions,and the accuracy of the recognition of seven daily actions has reached more than 91%,the overall accuracy has reached 95%.In the open set experiment,the overall recognition accuracy of MCACN-VAE for known categories has reached more than 89% at different degrees of openness,and the recognition accuracy of unknown action segments has also remained above 75%.It proves that the proposed model can effectively reject unknown classes and identify known classes.

Key words: Wearable devices, Activity recognition, Adaptive convolution, Open set recognition

CLC Number: 

  • TP181
[1]SINGH S P,SHARMA M K,LAY-EKUAKILLE A,et al.Deep ConvLSTM with self-attention for human activity decoding using wearable sensors[J].IEEE Sensors Journal,2020,21(6):8575-8582.
[2]LECUN Y,BOTTOU L,BENGIO Y,et al.Gradient-basedlearning applied to document recognition[J].Proceedings of the IEEE,1998,86(11):2278-2324.
[3]MIKOLOV T,KARAFIÁT M,BURGET L,et al.Recurrent neural network based language model[C]//Interspeech.2010:1045-1048.
[4]KINGMA D P,WELLING M.Auto-encoding variational bayes[C]//Proceedings of the International Conference on Learning Representations(ICLR).Banff,Canada:ICLR,2014.
[5]PARKKA J,ERMES M,KORPIPAA P,et al.Activity classification using realistic data from wearable sensors[J].IEEE Transactions on Information Technology in Biomedicine,2006,10(1):119-128.
[6]KWAPISZ J R,WEISS G M,MOORE S A.Activity recognition using cell phone accelerometers[J].ACM SigKDD Explorations Newsletter,2011,12(2):74-82.
[7]LI J.Research on Motion Capture Algorithm Based on 3D Static Model [D].Beijing:North China University of Technology,2019.
[8]MOTTAGHI A,SORYANI M,SEIFI H.Action recognition in freestyle wrestling using silhouette-skeleton features[J].Engineering Science and Technology,an International Journal,2020,23(4):921-930.
[9]LAPTEV I,MARSZALEK M,SCHMID C,et al.Learning rea-listic human actions from movies[C]//2008 IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2008:1-8.
[10]MAHBUB U,IMTIAZ H,AHAD M A R.An optical flowbased approach for action recognition[C]//14th International Conference on Computerand Information Technology(ICCIT 2011).IEEE,2011:646-651.
[11]LIU Z,ZHANG C,TIAN Y.3D-based deep convolutional neuralnetwork for action recognition with depth sequences[J].Image and Vision Computing,2016,55:93-100.
[12]DU Y,WANG W,WANG L.Hierarchical recurrent neural network for skeleton based action recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2015:1110-1118.
[13]XIA K,HUANG J,WANG H.LSTM-CNN architecture for human activity recognition[J].IEEE Access,2020,8:56855-56866.
[14]SENYUREK V Y,IMTIAZ M H,BELSARE P,et al.A CNN-LSTM neural network for recognition of puffing in smoking episodes using wearable sensors[J].Biomedical Engineering Letters,2020,10(2):195-203.
[15]ORDÓÑEZ F J,ROGGEN D.Deep convolutional and lstm recurrent neural networks for multimodal wearable activity recognition[J].Sensors,2016,16(1):115.
[16]AGAC S,SHOAIB M,INCEL O D.Context-aware and dynamically adaptable activity recognition with smart watches:A case study on smoking[J].Computers & Electrical Engineering,2021,90:106949.
[17]PHILLIPS P J,GROTHER P,MICHEALS R.Evaluation me-thods in face recognition[M]//Handbook of Face Recognition.2011:551-574.
[18]SCHEIRER W J,DE REZENDE ROCHA A,SAPKOTA A,et al.Toward open set recognition[J].IEEE Transactionson Pattern Analysis and Machine Intelligence,2012,35(7):1757-1772.
[19]DONG H,FU Y,SIGAL L,et al.Learning to separate domains in generalized zero-shot and open set learning:a probabilistic perspective[J].arXiv.1810.07368,2018.
[20]NEIRA M A C,JÚNIOR P R M,ROCHA A,et al.Data-fusion techniques for open-set recognition problems[J].IEEE Access,2018,6:21242-21265.
[21]YANG H M,ZHANG X Y,YIN F,et al.Convolutional prototype network for open set recognition[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2020,44(5):2358-2370.
[22]VASWANI A,SHAZEER N,PARMAR N,et al.Attention isall you need[J].Advances in Neural Information Processing Systems,2017,30:5998-6008.
[23]BENDALE A,BOULT T E.Towards open set deep networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:1563-1572.
[24]OZA P,PATEL V M.C2ae:Class conditioned auto-encoder for open-set recognition[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:2307-2316.
[1] XIAO Zhi-hong, HAN Ye-tong, ZOU Yong-pan. Study on Activity Recognition Based on Multi-source Data and Logical Reasoning [J]. Computer Science, 2022, 49(6A): 397-406.
[2] XING Yun-bing, LONG Guang-yu, HU Chun-yu, HU Li-sha. Human Activity Recognition Method Based on Class Increment SVM [J]. Computer Science, 2022, 49(5): 78-83.
[3] ZHANG Chun-xiang, ZHAO Chun-lei, CHEN Chao, LUO Hui. Review of Human Activity Recognition Based on Mobile Phone Sensors [J]. Computer Science, 2020, 47(10): 1-8.
[4] FAN Min, WANG Xiao-feng, MENG Xiao-feng. Study on Patient-adaptive Algorithm for ECG Classification Based on Wearable Devices [J]. Computer Science, 2019, 46(12): 292-297.
[5] WANG Zhong-min, ZHANG Shuang and HE Yan. Selective Ensemble Learning Human Activity Recognition Model Based on Diversity Measurement Cluster [J]. Computer Science, 2018, 45(1): 307-312.
[6] SONG Xiang-fa, ZHANG Yan-feng and ZHENG Feng-bin. Activity Recognition from Depth Image Sequences Based on L2,1-norm Sparse Feature Selection and Super Normal Vector [J]. Computer Science, 2017, 44(2): 306-308.
[7] HUAN Ruo-hong and CHEN Yue. Method of Human Activity Recognition Based on Feature Enhancement and Decision Fusion [J]. Computer Science, 2016, 43(Z11): 151-155.
[8] WANG Zhong-min, CAO Hong-jiang and FAN Lin. Method on Human Activity Recognition Based on Convolutional Neural Networks [J]. Computer Science, 2016, 43(Z11): 56-58.
[9] GUO Bin, CHEN Hui-hui, LI Wen-peng, YU Zhi-wen, JIANG Jia-jun and WANG Wen-hui. Memory Computing:Concept,Characteristics and Research Advances [J]. Computer Science, 2016, 43(9): 32-38.
[10] WANG Zhong-min, WANG Ke and HE Yan. Human Motion Activity Recognition Model Based on Multi-classifier Fusion [J]. Computer Science, 2016, 43(12): 297-301.
[11] YANG Lu-lu, CHEN Jian-xin, ZHOU Liang and WEI Xin. Real-time Analysis of Prisoner’s Abnormal Behavior Based on Wireless Body Area Network [J]. Computer Science, 2015, 42(3): 47-50.
[12] HOU Cang-jian,CHEN Ling,LV Ming-qi and CHEN Gen-cai. Acceleration-based Activity Recognition Independent of Device Orientation and Placement [J]. Computer Science, 2014, 41(10): 76-79.
[13] WANG Zhong-wei and SUN Guang-zhong. LiveData—A Data Collecting System Based on Sensors in Smart Phones [J]. Computer Science, 2014, 41(10): 27-30.
Full text



No Suggested Reading articles found!