Computer Science

Computer Science ›› 2021, Vol. 48 ›› Issue (4): 138-143.doi: 10.11896/jsjkx.200300042

• Computer Graphics & Multimedia • Previous Articles     Next Articles

Multi-person Activity Recognition Based on Bone Keypoints Detection

LI Meng-he, XU Hong-ji, SHI Lei-xin, ZHAO Wen-jie, LI Juan   

  1. School of Information Science and Engineering,Shandong University,Qingdao,Shandong 266237,China
  • Received:2020-06-24 Revised:2020-08-02 Online:2021-04-15 Published:2021-04-09
  • About author:LI Meng-he,born in 1994,postgra-duate.Her main research interests include computer vision and artificial intelligence.(limenghe0309@163.com)
    XU Hong-ji,born in 1976,Ph.D,asso-ciate professor.His main research intere-sts include wireless communications,ubiquitous computing,intelligent perception,blind signal processing and artificial intelligence.
  • Supported by:
    National Key Research and Development Program of China(2018YFC0831001),National Natural Science Foundation of China(61771292) and the 13th Five-Year Plan on Education Science of Shandong Province(YZ2019070).

PDF (PC)

2511

Abstract: Human activity recognition(HAR) technology is a research hotspot in the field of computer vision,but there are still many technical difficulties in the research of multi-person HAR.The problem of the inaccurate judgment of the number of people and the difficulty of feature extraction in multi-person activity recognition may lead to the low accuracy.A multi-person activity recognition system based on bone keypoints detection is proposed in this paper,which combines the extraction of bone points with the action recognition.Firstly,the image frame is extracted from the original video.Secondly,the OpenPose algorithm is used to obtain keypoints data of the human skeleton to detect the number of people in the image and mark activity information.At last,human posture features are extracted according to characteristics of skeleton points.Meanwhile,in order to accurately describe the relationship between posture features,a feature description method based on frame window matrix is proposed.Finally,a support vector machine(SVM) is used as a classifier to complete multi-person activity recognition.10 types of daily typical activities from UT-Interaction and HMDB51 datasets are taken as test objects,and experimental results prove that the proposed method can effectively extract keypoints of multiple human bones in the image.Its average recognition accuracy of 10 activities is 86.25%,which is higher than other compared methods.

Key words: OpenPose algorithm, Posture feature extraction, Skeleton keypoints extraction, SVM Classifier

CLC Number: 

  • TP391
[1]LIU A A,SU Y T,JIA P P,et al.Multiple/Single-View HumanAction Recognition via Part-Induced Multitask Structural Learning [J].IEEE Transactions on Cybernetics,2015,45(6):1194-1208.
[2]GONG W.Design and Implementation of Student Learning Behavior Recognition System Based on Skeleton Keypoint Detection [D].Changchun:Jilin University,2019.
[3]DAWAR N,KEHTARNAVAZ N.Action Detection and Recognition inContinuous Action Stream by Deep Learning-Based Sensing Fusion [J].IEEE Sensors Journal,2018,18(23):9660-9668.
[4]CHENG J,LIU H J,WANG F,et al.Silhouette Analysis for Human Action Recognition Based on Supervised Temporal T-SNE and Incremental Learning [J].IEEE Transactions on Image Processing,2015,24(10):3203-3217.
[5]LIU A A,XU N,NIE W Z,et al.Multi-Domain and Multi-Task Learning for Human Action Recognition[J].IEEE Transactions on Image Processing,2019,28(2):853-867.
[6]SUN J F,XU H J,ZHOU Y M,et al.Human Actions Recognition Using Improved MHI and 2-D Gabor Filter Based on Energy Blocks [C]//2018 International Conference on Artificial Intelligence:Technologies and Applications(ICAITA2018).Chengdu:Atlantis Press,2018:1-4.
[7]TU Z G,LU H Y,ZHANG D J,et al.Action-Stage Emphasized Spatiotemporal VLAD for Video Action Recognition [J].IEEE Transactions on Image Processing,2019,28(6):2799-2812.
[8]BAGAUTDINOV T,ALAHI A,FLEURET F,et al.SocialScene Understanding:End-to-End Multi-Person Action Localization and Collective Activity Recognition [C]//2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR 2017).Hawaii:IEEE,2017:3425-3434.
[9]ZHOU Q Q,ZHONG B N.Deep Alignment Network BasedMulti-Person Tracking with Occlusion and Motion Reasoning [J].IEEE Transactions on Multimedia.2019,21(5):1183-1194.
[10]LI M P,ZHOU Z M,et al.Multi-Person Pose Estimation Using Bounding Box Constraint and LSTM [J].IEEE Transactions on Multimedia.2019,21(10):2653-2263.
[11]LIN L,WANG Y F,et al.Multi-Person Pose Estimation Using Aurous Convolution [J].Electronics Letters.2019,55(9):533-535.
[12]CHEN X,YANG G K.Multi-Person Pose Estimation withLIMB Detection Heatmaps [C]//2018 IEEE International Conference on Image Processing(ICIP 2018).Athens:IEEE,2018:4078-4082.
[13]ANDRILUKA M,ROTH S,SCHIELE B.Pictorial StructuresRevisited:People Detection and Articulated Pose Estimation [C]//2009 IEEE Conference on Computer Vision and Pattern Recognition(CVPR 2009).Miami,FL:IEEE,2009:1014-1021.
[14]CAO Z,SIMON T,WEI S E,et al.Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields [C]//2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR 2017).Hawaii:IEEE,2017:1302-1310.
[15]CAO Z,HIDALGO G,SIMON T.OpenPose:Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields [C]//2019 IEEE Conference on Computer Vision and Pattern Recognition(CVPR 2019).Hawaii:IEEE,2019:1-14.
[16]RYOO M S,AGGARWAL J K.Spatio Temporal Relationship Match:Video Structure Comparison for Recognition of Complex Human Activities [C]//2009 IEEE International Conference on Computer Vision(CVPR 2009).2009:1593-1600.
[17]YAN S J,XIONG Y J,LIN D H.Spatial Temporal Graph Con-volutional Networks for Skeleton Based Action Recognition [C]//2018 IEEE Conference on Computer Vision and Pattern Recognition(CVPR 2018).Salt Lake City:IEEE,2018:7444-7452.
[18]CARREIRAL J,ZISSENRMAN A.Quo Vadis:Action Recognition? A New Model and the Kinetics Dataset [C]//2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR 2017).Hawaii:IEEE,2017:4724-4733.
[19]CHOUTAS V,WEINZAEPFEL P,REVAUD J.Potion:Pose-Motion Representation for Action Recognition [C]//2018 Conference on Computer Vision and Pattern Recognition(CVPR 2018).Salt Lake City:IEEE,2018:7024-7033.
[20]WANG L,KONIUSZ P,HUYNH D Q.Hallucinating IDT Descriptors and I3D Optical Feature for Action Recognition with CNNs [C]//2019 IEEE International Conference on Computer Vision(ICCV 2019).Seoul:IEEE,2019:1-12.
[1] ZHAO Cheng, CHEN Jun-xin, YAO Ming-hai. XSS Attack Detection Technology Based on SVM Classifier [J]. Computer Science, 2018, 45(11A): 356-360.
[2] LI Kun-lun, ZHANG Ya-xin, LIU Li-li and GENG Xue-fei. Palmprint Recognition Based on Improved PCA and SVM [J]. Computer Science, 2015, 42(Z11): 146-150.
[3] SHEN Xuan-jing, LI Meng-zhen, LV Ying-da and CHEN Hai-peng. Blind Identification Algorithm of Photorealistic Computer Graphics Based on Local Binary Count [J]. Computer Science, 2015, 42(6): 135-138.
[4] LIU Chun-li and ZHANG Gong. Grain Classification Based on Edge Feature [J]. Computer Science, 2013, 40(7): 280-282.
[5] ZHANG Yong,XUE Zhi-mao. Face Detection System Design Based on Two Classifiers [J]. Computer Science, 2010, 37(4): 293-.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.