Computer Science ›› 2018, Vol. 45 ›› Issue (8): 22-27.doi: 10.11896/j.issn.1002-137X.2018.08.005

• ChinaMM 2017 • Previous Articles     Next Articles

Human Action Recognition Framework with RGB-D Features Fusion

MAO Xia1, WANG Lan1, LI Jian-jun1,2   

  1. School of Electronic and Information Engineering,Beihang University,Beijing 100191,China1
    School of Information Engineering,Inner Mongolia University of Science and Technology,Baotou,Inner Mongolia 014010,China2
  • Received:2017-10-24 Online:2018-08-29 Published:2018-08-29

Abstract: Human action recognition is an important research direction in the field of computer vision and pattern recognition.The complexity of human behavior and the variety of action performing make behavior recognition still as a challenging subject.With the new generation of sensing technology,RGB-D cameras can simultaneously record RGB images,depth images,and extract skeleton information from depth images in real time.How to take advantages of above information has become the new hotspot and breakthrough point of behavior recognition research.This paper presented a new feature extraction method based on Gaussian weighted pyramid histograms of orientation gradients for RGB images,and built an action recognition framework fusing multiple features.The feature extraction method and the framework proposed in this paper were researched on three databases:UTKinect-Action3D,MSR-Action 3D and Florence 3D Actions.The results indicate that the proposed action recognition framework achieves the accuracy of 97.5%,93.1%,91.7% respectively.It shows the effectiveness of the proposed action recognition framework.

Key words: Action recognition, Feature fusion, Gaussian weighted, Histogram of orientation gradients, Sparse representation classifier

CLC Number: 

  • TP391
[1]XIA L,CHEN C C,Aggarwal J K.View Invariant Human Action Recognition Using Histograms of 3D Joints[C]∥Procee-dings of IEEE Conference on Computer Vision and Pattern Re-cognition.Providence:IEEE Press,2012:20-27.
[2]DALAL N,TRIGGS B.Histograms of Oriented Gradients for Human Detection[C]∥Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.San Diego:IEEE Press,2005:886-893.
[3]BOSCH A,ZISSERMAN A,MUMOZ X.Representing Shapewith a Spatial Pyramid Kernel[C]∥Proceedings of the Sixth ACM International Conference on Image and Video Retrieval.New York:ACM,2007:401-408.
[4]LUVIZON D C,TABIA H,PICARD D.Learning FeaturesCombination for Human Recognition from Skeleton Sequences[J/OL].(2017-02-02) [2017-07-06].
[5]YE J,LI K,QI G,et al.Temporal Order-Preserving DynamicQuantization for Human Action Recognition from Multimodal Sensor Streams[C]∥Proceedings of the Annual ACM International Conference on Multimedia Retrieval.Shanghai:ACM,2015:99-106.
[6]ZHANG H L,ZHONG P,HE J L,et al.Combining Depth-Ske-leton Feature with Sparse Coding for Action Recognition[J].Neurocomputing,2016,230(C):417-426.
[7]AHARON M,ELAD M,BRUCKSTEIN A.K-SVD:An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation[J].IEEE Transactions on Signal Processing,2006,54(11):4311-4322.
[8]JEGOU H,DOUZE M,SCHMID C,et al.Aggregating LocalDescriptors into a Compact Image Representation[C]∥Proceedings of the 23th IEEE Conference on Computer Vision and Pattern Recognition.California:IEEE Press,2010:3304-3311.
[9]WRIGHT J,YANG A Y,GANESH A,et al.Robust FaceRe-cognition via Sparse Representation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2009,31(2):210-227.
[10]LI W,ZHANG Z,LIU Z.Action Recognition Based on a Bag of 3D Points[C]∥Proceedings of the 23th IEEE Conference on Computer Vision and Pattern Recognition.San Francisco:IEEE Press,2010:9-14.
[11]SEUDEBARU L,VARANO V,BERRETTI S,et al.Recogni-zing Actions from Depth Cameras as Weakly Aligned Multi-part Bag-of-Poses[C]∥Proceedings of the 26th IEEE Conference on Computer Vision and Pattern Recognition.Portland:IEEE Press,2013:479-485.
[12]WANG J,LIU Z,WU Y,et al.Mining Actionlet Ensemble for Action Recognition with Depth Cameras[C]∥Proceedings of the 25th IEEE Conference on Computer Vision and Pattern Re-cognition.Providence:IEEE Press,2012:1290-1297.
[13]VEMULAPALLI R,ARRATE F,CHELLAPPA R,et al.Human Action Recognition by Representing 3D Skeletons as Points in a Lie Group[C]∥Proceedings of the 27th IEEE Conference on Computer Vision and Pattern Recognition.Columbus:IEEE Press,2014:588-595.
[14]DEVANNE M,WANNOUS H,BERRETTI S,et al.3-D Human Action Recognition by Shape Analysis of Motion Trajectories on Riemannian Manifold[J].IEEE Transactions on Systems,Man,and Cybernetics,2015,45(7):1340-1352.
[15]YIN F,JIAO L C,SHANG F,et al.Sparse Regularization Discriminant Analysis for Face Recognition[J].Neurocomputing,2014,128(5):341-362.
[1] JIN Kun, CHEN Shao-chang. Status and Development of Gait Recognition [J]. Computer Science, 2019, 46(6A): 30-34.
[2] WU Fan, LI Shou-shan, ZHOU Guo-dong. Movie Review Professionalism Classification Using LSTM and Features Fusion [J]. Computer Science, 2019, 46(6A): 74-79.
[3] WANG Xiao, ZOU Ze-wei, LI Bo-bo, WANG Jing. Target Detection in Colorful Imaging Sonar Based on Multi-feature Fusion [J]. Computer Science, 2019, 46(6A): 177-181.
[4] HOU Yuan-yuan, HE Ru-han, LI Min, CHEN Jia. Clothing Image Retrieval Method Combining Convolutional Neural Network Multi-layerFeature Fusion and K-Means Clustering [J]. Computer Science, 2019, 46(6A): 215-221.
[5] LIU Ze-kang, SUN Hua-zhi, MA Chun-mei, JIANG Li-fen. Vehicle Recognition Model Based on Multi-feature Combination inConvolutional Neural Network [J]. Computer Science, 2019, 46(6A): 254-258.
[6] DU Chen-xi, YAN Yun-yang, LIU Yi-an, GAO Shang-bing. Video Fire Detection Method Based on YOLOv2 [J]. Computer Science, 2019, 46(6): 301-304.
[7] WEI Yu-hui, WANG Yong-jun, WANG Guo-dong, LIU Hong-min, WANG Jing. Mismatch Elimination Algorithm Based on Point Line Feature Fusion [J]. Computer Science, 2019, 46(2): 286-293.
[8] LI Yun-bo, TANG Si-qi, ZHOU Xing-yu, PAN Zhi-song. Crowd Counting Method via Scalable Modularized Convolutional Neural Network [J]. Computer Science, 2018, 45(8): 17-21,40.
[9] ZHANG Yu-xue,TANG Zhen-min ,QIAN Bin ,XU Wei. Pavement Crack Detection Based on Sparse Representation and Multi-feature Fusion [J]. Computer Science, 2018, 45(7): 271-277.
[10] GUO Yu, HAO Xiao-yan, ZHANG Xing-zhong. Multi-feature Fusion Mean-Shift Tracking Algorithm Based on Prediction [J]. Computer Science, 2018, 45(6A): 171-173, 205.
[11] WANG Xue-qiao,QI Hua-shan, YUAN Jia-zheng, LIANG Ai-hua, SUN Li-hong. Face Recognition Using 2D Gabor Feature and 3D NP-3DHOG Feature [J]. Computer Science, 2018, 45(6A): 247-251.
[12] CHEN Rong, LI Peng, HUANG Yong. Moving Shadow Removal Algorithm Based on Multi-feature Fusion [J]. Computer Science, 2018, 45(6): 291-295.
[13] WANG Meng-di, ZHANG You-mei and CHANG Fa-liang. Text Localization Based on Edge Detection and Features Fusion in Natural Scene [J]. Computer Science, 2017, 44(9): 300-303, 314.
[14] JING Chen-yong, ZHAN Yong-zhao and JIANG Zhen. Research on Action Recognition Algorithm Based on Hybrid Cooperative Training [J]. Computer Science, 2017, 44(7): 275-278.
[15] JI Chong, WANG Sheng and LU Jian-feng. Human Action Recognition Based on Fisher Discrimination Dictionary Learning [J]. Computer Science, 2017, 44(7): 270-274.
Full text



[1] . [J]. Computer Science, 2018, 1(1): 1 .
[2] LEI Li-hui and WANG Jing. Parallelization of LTL Model Checking Based on Possibility Measure[J]. Computer Science, 2018, 45(4): 71 -75, 88 .
[3] XIA Qing-xun and ZHUANG Yi. Remote Attestation Mechanism Based on Locality Principle[J]. Computer Science, 2018, 45(4): 148 -151, 162 .
[4] LI Bai-shen, LI Ling-zhi, SUN Yong and ZHU Yan-qin. Intranet Defense Algorithm Based on Pseudo Boosting Decision Tree[J]. Computer Science, 2018, 45(4): 157 -162 .
[5] WANG Huan, ZHANG Yun-feng and ZHANG Yan. Rapid Decision Method for Repairing Sequence Based on CFDs[J]. Computer Science, 2018, 45(3): 311 -316 .
[6] SUN Qi, JIN Yan, HE Kun and XU Ling-xuan. Hybrid Evolutionary Algorithm for Solving Mixed Capacitated General Routing Problem[J]. Computer Science, 2018, 45(4): 76 -82 .
[7] ZHANG Jia-nan and XIAO Ming-yu. Approximation Algorithm for Weighted Mixed Domination Problem[J]. Computer Science, 2018, 45(4): 83 -88 .
[8] WU Jian-hui, HUANG Zhong-xiang, LI Wu, WU Jian-hui, PENG Xin and ZHANG Sheng. Robustness Optimization of Sequence Decision in Urban Road Construction[J]. Computer Science, 2018, 45(4): 89 -93 .
[9] LIU Qin. Study on Data Quality Based on Constraint in Computer Forensics[J]. Computer Science, 2018, 45(4): 169 -172 .
[10] ZHONG Fei and YANG Bin. License Plate Detection Based on Principal Component Analysis Network[J]. Computer Science, 2018, 45(3): 268 -273 .