计算机科学 ›› 2018, Vol. 45 ›› Issue (8): 22-27.doi: 10.11896/j.issn.1002-137X.2018.08.005

• 2017 中国多媒体大会 • 上一篇    下一篇

一种基于RGB-D特征融合的人体行为识别框架

毛峡1, 王岚1, 李建军1,2   

  1. 北京航空航天大学电子信息工程学院 北京1001911
    内蒙古科技大学信息工程学院 内蒙古 包头0140102
  • 收稿日期:2017-10-24 出版日期:2018-08-29 发布日期:2018-08-29
  • 作者简介:毛 峡(1952-),女,博士,教授,博士生导师,主要研究方向为模式识别与人工智能,E-mail:moukyou@buaa.edu.cn(通信作者); 王 岚(1992-),女,硕士生,主要研究方向为人体行为识别; 李建军(1977-),男,博士生,主要研究方向为行为识别。
  • 基金资助:
    本文受国家自然科学基金项目(61603013)资助。

Human Action Recognition Framework with RGB-D Features Fusion

MAO Xia1, WANG Lan1, LI Jian-jun1,2   

  1. School of Electronic and Information Engineering,Beihang University,Beijing 100191,China1
    School of Information Engineering,Inner Mongolia University of Science and Technology,Baotou,Inner Mongolia 014010,China2
  • Received:2017-10-24 Online:2018-08-29 Published:2018-08-29

摘要: 人体行为识别是计算机视觉和模式识别领域内一个重要的研究方向。人体行为的复杂性和不同人执行同一动作的差异性,使得行为识别仍然是一个具有挑战性的课题。采用新一代传感技术的RGB-D相机能够同时记录RGB图像和深度图像,并能够实时提取骨骼点信息。充分利用以上信息,成为行为识别领域的研究热点和突破点。文中提出了一种新的基于高斯加权金字塔式梯度方向直方图的RGB图像特征提取方法,并构建了一种多模特征融合的行为识别框架。在UTKinect-Action3D,MSR-Action 3D和Florence 3D Actions 3个数据库上对本研究所提特征和框架进行实验,结果表明,所提框架在3个行为数据库上的识别正确率分别达到了97.5%,93.1%,91.7%,从而证明了该行为识别框架的有效性。

关键词: 高斯加权, 特征融合, 梯度直方图, 稀疏表示分类器, 行为识别

Abstract: Human action recognition is an important research direction in the field of computer vision and pattern recognition.The complexity of human behavior and the variety of action performing make behavior recognition still as a challenging subject.With the new generation of sensing technology,RGB-D cameras can simultaneously record RGB images,depth images,and extract skeleton information from depth images in real time.How to take advantages of above information has become the new hotspot and breakthrough point of behavior recognition research.This paper presented a new feature extraction method based on Gaussian weighted pyramid histograms of orientation gradients for RGB images,and built an action recognition framework fusing multiple features.The feature extraction method and the framework proposed in this paper were researched on three databases:UTKinect-Action3D,MSR-Action 3D and Florence 3D Actions.The results indicate that the proposed action recognition framework achieves the accuracy of 97.5%,93.1%,91.7% respectively.It shows the effectiveness of the proposed action recognition framework.

Key words: Action recognition, Feature fusion, Gaussian weighted, Histogram of orientation gradients, Sparse representation classifier

中图分类号: 

  • TP391
[1]XIA L,CHEN C C,Aggarwal J K.View Invariant Human Action Recognition Using Histograms of 3D Joints[C]∥Procee-dings of IEEE Conference on Computer Vision and Pattern Re-cognition.Providence:IEEE Press,2012:20-27.
[2]DALAL N,TRIGGS B.Histograms of Oriented Gradients for Human Detection[C]∥Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.San Diego:IEEE Press,2005:886-893.
[3]BOSCH A,ZISSERMAN A,MUMOZ X.Representing Shapewith a Spatial Pyramid Kernel[C]∥Proceedings of the Sixth ACM International Conference on Image and Video Retrieval.New York:ACM,2007:401-408.
[4]LUVIZON D C,TABIA H,PICARD D.Learning FeaturesCombination for Human Recognition from Skeleton Sequences[J/OL].(2017-02-02) [2017-07-06].http://dx.doi.org/10.1016/j.patrec.2017.02.001.
[5]YE J,LI K,QI G,et al.Temporal Order-Preserving DynamicQuantization for Human Action Recognition from Multimodal Sensor Streams[C]∥Proceedings of the Annual ACM International Conference on Multimedia Retrieval.Shanghai:ACM,2015:99-106.
[6]ZHANG H L,ZHONG P,HE J L,et al.Combining Depth-Ske-leton Feature with Sparse Coding for Action Recognition[J].Neurocomputing,2016,230(C):417-426.
[7]AHARON M,ELAD M,BRUCKSTEIN A.K-SVD:An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation[J].IEEE Transactions on Signal Processing,2006,54(11):4311-4322.
[8]JEGOU H,DOUZE M,SCHMID C,et al.Aggregating LocalDescriptors into a Compact Image Representation[C]∥Proceedings of the 23th IEEE Conference on Computer Vision and Pattern Recognition.California:IEEE Press,2010:3304-3311.
[9]WRIGHT J,YANG A Y,GANESH A,et al.Robust FaceRe-cognition via Sparse Representation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2009,31(2):210-227.
[10]LI W,ZHANG Z,LIU Z.Action Recognition Based on a Bag of 3D Points[C]∥Proceedings of the 23th IEEE Conference on Computer Vision and Pattern Recognition.San Francisco:IEEE Press,2010:9-14.
[11]SEUDEBARU L,VARANO V,BERRETTI S,et al.Recogni-zing Actions from Depth Cameras as Weakly Aligned Multi-part Bag-of-Poses[C]∥Proceedings of the 26th IEEE Conference on Computer Vision and Pattern Recognition.Portland:IEEE Press,2013:479-485.
[12]WANG J,LIU Z,WU Y,et al.Mining Actionlet Ensemble for Action Recognition with Depth Cameras[C]∥Proceedings of the 25th IEEE Conference on Computer Vision and Pattern Re-cognition.Providence:IEEE Press,2012:1290-1297.
[13]VEMULAPALLI R,ARRATE F,CHELLAPPA R,et al.Human Action Recognition by Representing 3D Skeletons as Points in a Lie Group[C]∥Proceedings of the 27th IEEE Conference on Computer Vision and Pattern Recognition.Columbus:IEEE Press,2014:588-595.
[14]DEVANNE M,WANNOUS H,BERRETTI S,et al.3-D Human Action Recognition by Shape Analysis of Motion Trajectories on Riemannian Manifold[J].IEEE Transactions on Systems,Man,and Cybernetics,2015,45(7):1340-1352.
[15]YIN F,JIAO L C,SHANG F,et al.Sparse Regularization Discriminant Analysis for Face Recognition[J].Neurocomputing,2014,128(5):341-362.
[1] 张颖涛, 张杰, 张睿, 张文强.
全局信息引导的真实图像风格迁移
Photorealistic Style Transfer Guided by Global Information
计算机科学, 2022, 49(7): 100-105. https://doi.org/10.11896/jsjkx.210600036
[2] 程成, 降爱莲.
基于多路径特征提取的实时语义分割方法
Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction
计算机科学, 2022, 49(7): 120-126. https://doi.org/10.11896/jsjkx.210500157
[3] 肖治鸿, 韩晔彤, 邹永攀.
基于多源数据和逻辑推理的行为识别技术研究
Study on Activity Recognition Based on Multi-source Data and Logical Reasoning
计算机科学, 2022, 49(6A): 397-406. https://doi.org/10.11896/jsjkx.210300270
[4] 陈永平, 朱建清, 谢懿, 吴含笑, 曾焕强.
基于外接圆半径差损失的实时安全帽检测算法
Real-time Helmet Detection Algorithm Based on Circumcircle Radius Difference Loss
计算机科学, 2022, 49(6A): 424-428. https://doi.org/10.11896/jsjkx.220100252
[5] 孙洁琪, 李亚峰, 张文博, 刘鹏辉.
基于离散小波变换的双域特征融合深度卷积神经网络
Dual-field Feature Fusion Deep Convolutional Neural Network Based on Discrete Wavelet Transformation
计算机科学, 2022, 49(6A): 434-440. https://doi.org/10.11896/jsjkx.210900199
[6] 郁舒昊, 周辉, 叶春杨, 王太正.
SDFA:基于多特征融合的船舶轨迹聚类方法研究
SDFA:Study on Ship Trajectory Clustering Method Based on Multi-feature Fusion
计算机科学, 2022, 49(6A): 256-260. https://doi.org/10.11896/jsjkx.211100253
[7] 杨玥, 冯涛, 梁虹, 杨扬.
融合交叉注意力机制的图像任意风格迁移
Image Arbitrary Style Transfer via Criss-cross Attention
计算机科学, 2022, 49(6A): 345-352. https://doi.org/10.11896/jsjkx.210700236
[8] 蓝凌翔, 池明旻.
基于特征注意力融合网络的遥感变化检测研究
Remote Sensing Change Detection Based on Feature Fusion and Attention Network
计算机科学, 2022, 49(6): 193-198. https://doi.org/10.11896/jsjkx.210500058
[9] 邵延华, 李文峰, 张晓强, 楚红雨, 饶云波, 陈璐.
基于时空图卷积和注意力模型的航拍暴力行为识别
Aerial Violence Recognition Based on Spatial-Temporal Graph Convolutional Networks and Attention Model
计算机科学, 2022, 49(6): 254-261. https://doi.org/10.11896/jsjkx.210400272
[10] 李发光, 伊力哈木·亚尔买买提.
基于改进CenterNet的航拍绝缘子缺陷实时检测模型
Real-time Detection Model of Insulator Defect Based on Improved CenterNet
计算机科学, 2022, 49(5): 84-91. https://doi.org/10.11896/jsjkx.210400142
[11] 董奇达, 王喆, 吴松洋.
结合注意力机制与几何信息的特征融合框架
Feature Fusion Framework Combining Attention Mechanism and Geometric Information
计算机科学, 2022, 49(5): 129-134. https://doi.org/10.11896/jsjkx.210300180
[12] 李鹏祖, 李瑶, Ibegbu Nnamdi JULIAN, 孙超, 郭浩, 陈俊杰.
基于多特征融合的重叠组套索脑功能超网络构建及分类
Construction and Classification of Brain Function Hypernetwork Based on Overlapping Group Lasso with Multi-feature Fusion
计算机科学, 2022, 49(5): 206-211. https://doi.org/10.11896/jsjkx.210300049
[13] 范新南, 赵忠鑫, 严炜, 严锡君, 史朋飞.
结合注意力机制的多尺度特征融合图像去雾算法
Multi-scale Feature Fusion Image Dehazing Algorithm Combined with Attention Mechanism
计算机科学, 2022, 49(5): 50-57. https://doi.org/10.11896/jsjkx.210400093
[14] 许华杰, 秦远卓, 杨洋.
基于多级特征融合与注意力模块的场景识别方法
Scene Recognition Method Based on Multi-level Feature Fusion and Attention Module
计算机科学, 2022, 49(4): 209-214. https://doi.org/10.11896/jsjkx.210100135
[15] 高心悦, 田汉民.
基于改进U-Net网络的液滴分割方法
Droplet Segmentation Method Based on Improved U-Net Network
计算机科学, 2022, 49(4): 227-232. https://doi.org/10.11896/jsjkx.210300193
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!