面向行为识别的人体空间协同运动结构特征表示与融合

doi:10.13700/j.bh.1001-5965.2019.0373

北京航空航天大学学报 ›› 2019, Vol. 45 ›› Issue (12): 2495-2505.doi: 10.13700/j.bh.1001-5965.2019.0373

面向行为识别的人体空间协同运动结构特征表示与融合

莫宇剑¹, 侯振杰¹, 常兴治², 梁久祯¹, 陈宸³, 宦娟¹

1. 常州大学信息科学与工程学院, 常州 213164;
2. 常州信息职业技术学院智能制造工业云开放实验室, 常州 213164;
3. 北卡罗来纳大学夏洛特分校电气与计算机工程系, 夏洛特市 28223

收稿日期:2019-07-09 出版日期:2019-12-20 发布日期:2019-12-31
通讯作者: 侯振杰 E-mail:houzj@cczu.edu.cn
作者简介:莫宇剑男,硕士研究生。主要研究方向:机器学习、行为识别;侯振杰男,博士,教授,硕士生导师。主要研究方向:机器学习、人工智能、图像处理。
基金资助:
国家自然科学基金（61063021，61803050）；江苏省物联网移动互联技术工程重点实验室开放课题（JSWLW-2017-013）

Structural feature representation and fusion of behavior recognition oriented human spatial cooperative motion

MO Yujian¹, HOU Zhenjie¹, CHANG Xingzhi², LIANG Jiuzhen¹, CHEN Chen³, HUAN Juan¹

1. School of Information Science and Engineering, Changzhou University, Changzhou 213164, China;
2. Open Lab of Industrial Cloud for Intelligent Manufacturing, Changzhou College of Information Technology, Changzhou 213164, China;
3. Department of Electrical and Computer Engineering, University of North Carolina at Charlotte, Charlotte 28223, USA

Received:2019-07-09 Online:2019-12-20 Published:2019-12-31
Supported by:
National Natural Science Foundation of China (61063021,61803050); Jiangsu Province Networking and Mobile Internet Technology Engineering Key Laboratory Open Research Fund Project (JSWLW-2017-013)

摘要/Abstract

摘要： 针对人体执行动作时不同身体部位之间的协同关系，提出了基于人体空间协同运动结构特征的行为识别方法。首先度量人体不同部位对完成动作的贡献度，并将不同部位的贡献度转变为协同运动结构特征模型。然后利用模型无监督、自适应地对不同身体部位的运动特征进行约束。在此基础上借鉴跨媒体检索方法JFSSL对不同模态的特征进行特征选择与多模态特征融合。实验表明，所提方法在自建的行为数据库上明显提高了开放测试的识别率，且计算过程简便，易于实现。

关键词: 贡献度度量, 协同运动, 结构特征, 特征选择, 多模态融合

Abstract: In view of the synergistic relationship among different parts of the body when human body performs actions, a behavior recognition method based on human body spatial cooperative motion structural features is proposed. Firstly, the contribution of different parts of the human body to the completion of the action is measured, and the contribution of different parts of the human body is transformed into a structural feature model of cooperative motion. Then, the model is used to constrain the motion characteristics of different parts of the body self-adaptively without supervision. On this basis, feature selection and multi-modal feature fusion are carried out using JFSSL, a cross-media retrieval method. The experiments show that the recognition rate of the open test is obviously improved by the proposed method on the self-built behavior database. At the same time, the calculation process of the method is simple and easy to implement.

Key words: measure of contribution, cooperative motion, structural features, feature selection, multi-modal fusion

中图分类号:

TP391

莫宇剑, 侯振杰, 常兴治, 梁久祯, 陈宸, 宦娟. 面向行为识别的人体空间协同运动结构特征表示与融合[J]. 北京航空航天大学学报, 2019, 45(12): 2495-2505.

MO Yujian, HOU Zhenjie, CHANG Xingzhi, LIANG Jiuzhen, CHEN Chen, HUAN Juan. Structural feature representation and fusion of behavior recognition oriented human spatial cooperative motion[J]. JOURNAL OF BEIJING UNIVERSITY OF AERONAUTICS AND A, 2019, 45(12): 2495-2505.

参考文献

[1] LUVIZON D C,PICARD D,TABIA H.2D/3D pose estimation and action recognition using multitask deep learning[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR).Piscataway,NJ:IEEE Press,2018:5137-5146.
[2] FABIEN B,CHRISTIAN W,JULIEN M,et al.Glimpse clouds:Human activity recognition from unstructured feature points[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR).Piscataway,NJ:IEEE Press,2018:469-478.
[3] HOU R,CHEN C,MUBARAK S.Tube convolutional neural network (T-CNN) for action detection in videos[C]//Proceedings of the IEEE International Conference on Computer Vision(ICCV).Piscataway,NJ:IEEE Press,2017:5822-5831.
[4] CHEN C,JAFARI R,KEHTARNAVAZ N.Action recognition from depth sequences using depth motion maps-based local binary patterns[C]//IEEE Winter Conference on Applications of Computer Vision.Piscataway,NJ:IEEE Perss,2015:1092-1099.
[5] XIA L,CHEN C,AGGARWAL J K.View invariant human action recognition using histograms of 3d joints[C]//2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops(CVPRW).Piscataway,NJ:IEEE Press,2012:20-27.
[6] HUSSEIN M E,TORKI M,GOWAYYED M A.Human action recognition using a temporal hierarchy of covariance descriptors on 3d joint locations[C]//23rd International Joint Conference on Artificial Intelligence.Palo Alto:AAAI Press,2013:2466-2472.
[7] 苏本跃,蒋京,汤庆丰,等.基于函数型数据分析方法的人体动态行为识别[J].自动化学报,2017,43(5):866-876.SU B Y,JIANG J,TANG Q F,at al.Human dynamic action recognition based on functional data analysis[J].Acta Automatica Sinica,2017,43(5):866-876(in Chinese).
[8] GRAVINA R,ALINIA P,GHASEMZADEH H,et al.Multi-sensor fusion in body sensor networks:State-of-the-art and research challenges[J].Information Fusion,2017,35:68-80.
[9] CHEN C,LIU K,KEHTARNAVAZ N.Real-time human action recogniti on based on depth motion maps[J].Journal of Real-Time Image Processing,2016,12(1):155-163.
[10] CHEN C,JAFARI R,KEHTARNAVAZ N.Improving human action recognition using fusion of depth camera and inertial sensors[J].IEEE Transactions on Human-Machine Systems,2015,45(1):51-61.
[11] HAGHIGHAT M,ABDEL-MOTTALEB M,ALHALABI W.Fully automatic face normalization and single sample face recognition in unconstrained environments[J].Expert Systems with Applications,2016,47:23-34.
[12] HAGHIGHAT M,ABDEL-MOTTALEB M,ALHALABI W.Discriminant correlation analysis:Real-time feature level fusion for multimodal biometric recognition[J].IEEE Transactions on Information Forensics and Security,2016,11(9):1984-1996.
[13] 唐超,王文剑,李伟,等.基于多学习器协同训练模型的人体行为识别方法[J].软件学报,2015,26(11):2939-2950.TANG C,WANG W J,LI W,et al.Multi-learner co-training model for human actioin recognition[J].Journal of Software,2015,26(11):2939-2950(in Chinese).
[14] SI C Y,JING Y,WANG W,et al.Skeleton-based action recognition with spatial reasoning and temporal stack learning[C]//Proceedings of the European Conference on Computer Vision (ECCV).Berlin:Springer,2018:103-118.
[15] LIU M Y,MENG F Y,CHEN C,et al.Joint dynamic pose image and space time reversal for human action recognition from videos[C]//33rd AAAI Conference on Artificial Intelligence.Palo Alto:AAAI Press,2019:8762-8769.
[16] 邓诗卓,王波涛,杨传贵,等.CNN多位置穿戴式传感器人体活动识别[J].软件学报,2019,30(3):718-737.DENG S Z,WANG B T,YANG C G,et al.Convolutional neural networks for human activity recognition using multi-location wearable sensors[J].Journal of Software,2019,30(3):718-737(in Chinese).
[17] CHEN C,JAFARI R,KEHTARNAVAZ N.A real-time human action recognition system using depth and inertial sensor fusion[J].IEEE Sensors Journal,2016,16(3):773-781.
[18] 许艳,侯振杰,梁久祯,等.权重融合深度图像与骨骼关键帧的行为识别[J].计算机辅助设计与图形学学报,2018,30(7):139-146.XU Y,HOU Z J,LIANG J Z,et al.Action recognition using weighted fusion of depth images and skeleton's key frames[J].Journal of Computer-Aided Design & Computer Graphics,2018,30(7):139-146(in Chinese).
[19] WANG K,HE R,WANG L,et al.Joint feature selection and subspace learning for cross-modal retrieval[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2016,38(10):2010-2023.
[20] NIE F,HUANG H,CAI X,et al.Efficient and robust feature selection via joint l_2,1-norms minimization[C]//Advances in Neural Information Processing Systems(NIPS).New York:Curran Associates,2010:1813-1821.
[21] HE R,TIAN T N,WANG L,et al.L_2,1 regularized correntropy for robust feature selection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR).Piscataway,NJ:IEEE Press,2012:2504-2511.
[22] OFLI F,CHAUDHRY R,KURILLO G,et al.Berkeley MHAD:A comprehensive multimodal human action database[C]//2013 IEEE Workshop on Applications of Computer Vision (WACV).Piscataway,NJ:IEEE Press,2013:53-60.
[23] CHEN C,JAFARI R,KEHTARNAVAZ N.UTD-MHAD:A multimodal dataset for human action recognition utilizing a depth camera and a wearable inertial sensor[C]//2015 IEEE International Conference on Image Processing (ICIP).Piscataway,NJ:IEEE Press,2015:168-172.
[24] CHEN C,ZHANG B G,HOU Z J,et al.Action recognition from depth sequences using weighted fusion of 2D and 3D auto-correlation of gradients features[J].Multimedia Tools and Applications,2017,76(3):4651-4669.

面向行为识别的人体空间协同运动结构特征表示与融合

Structural feature representation and fusion of behavior recognition oriented human spatial cooperative motion

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 4

编辑推荐

Metrics

本文评价

[1]	魏乐松, 陈俊豪, 牛玉贞. 基于边缘和结构的无参考屏幕内容图像质量评估[J]. 北京航空航天大学学报, 2019, 45(12): 2449-2455.
[2]	湛颖, 高妍, 谢凌云. 中国国画艺术美感特征分析与分类[J]. 北京航空航天大学学报, 2019, 45(12): 2514-2522.
[3]	张旭, 蒋建国, 洪日昌, 杜跃. 基于朴素贝叶斯K近邻的快速图像分类算法[J]. 北京航空航天大学学报, 2015, 41(2): 302-310.
[4]	李国和, 赵沁平. 信息系统的一种分块特征选取方法[J]. 北京航空航天大学学报, 2003, 29(3): 255-259.