| Citation: | MO Yujian, HOU Zhenjie, CHANG Xingzhi, et al. Structural feature representation and fusion of behavior recognition oriented human spatial cooperative motion[J]. Journal of Beijing University of Aeronautics and Astronautics, 2019, 45(12): 2495-2505. doi: 10.13700/j.bh.1001-5965.2019.0373(in Chinese)
|
In view of the synergistic relationship among different parts of the body when human body performs actions, a behavior recognition method based on human body spatial cooperative motion structural features is proposed. Firstly, the contribution of different parts of the human body to the completion of the action is measured, and the contribution of different parts of the human body is transformed into a structural feature model of cooperative motion. Then, the model is used to constrain the motion characteristics of different parts of the body self-adaptively without supervision. On this basis, feature selection and multi-modal feature fusion are carried out using JFSSL, a cross-media retrieval method. The experiments show that the recognition rate of the open test is obviously improved by the proposed method on the self-built behavior database. At the same time, the calculation process of the method is simple and easy to implement.
| [1] |
LUVIZON D C, PICARD D, TABIA H.2D/3D pose estimation and action recognition using multitask deep learning[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR).Piscataway, NJ: IEEE Press, 2018: 5137-5146.
|
| [2] |
FABIEN B, CHRISTIAN W, JULIEN M, et al.Glimpse clouds: Human activity recognition from unstructured feature points[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR).Piscataway, NJ: IEEE Press, 2018: 469-478.
|
| [3] |
HOU R, CHEN C, MUBARAK S.Tube convolutional neural network (T-CNN) for action detection in videos[C]//Proceedings of the IEEE International Conference on Computer Vision(ICCV).Piscataway, NJ: IEEE Press, 2017: 5822-5831.
|
| [4] |
CHEN C, JAFARI R, KEHTARNAVAZ N.Action recognition from depth sequences using depth motion maps-based local binary patterns[C]//IEEE Winter Conference on Applications of Computer Vision.Piscataway, NJ: IEEE Perss, 2015: 1092-1099.
|
| [5] |
XIA L, CHEN C, AGGARWAL J K.View invariant human action recognition using histograms of 3d joints[C]//2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops(CVPRW).Piscataway, NJ: IEEE Press, 2012: 20-27.
|
| [6] |
HUSSEIN M E, TORKI M, GOWAYYED M A.Human action recognition using a temporal hierarchy of covariance descriptors on 3d joint locations[C]//23rd International Joint Conference on Artificial Intelligence.Palo Alto: AAAI Press, 2013: 2466-2472.
|
| [7] |
苏本跃, 蒋京, 汤庆丰, 等.基于函数型数据分析方法的人体动态行为识别[J].自动化学报, 2017, 43(5):866-876.
SU B Y, JIANG J, TANG Q F, at al.Human dynamic action recognition based on functional data analysis[J]. Acta Automatica Sinica, 2017, 43(5):866-876(in Chinese).
|
| [8] |
GRAVINA R, ALINIA P, GHASEMZADEH H, et al.Multi-sensor fusion in body sensor networks:State-of-the-art and research challenges[J]. Information Fusion, 2017, 35:68-80. doi: 10.1016/j.inffus.2016.09.005
|
| [9] |
CHEN C, LIU K, KEHTARNAVAZ N.Real-time human action recogniti on based on depth motion maps[J]. Journal of Real-Time Image Processing, 2016, 12(1):155-163. doi: 10.1007/s11554-013-0370-1
|
| [10] |
CHEN C, JAFARI R, KEHTARNAVAZ N.Improving human action recognition using fusion of depth camera and inertial sensors[J]. IEEE Transactions on Human-Machine Systems, 2015, 45(1):51-61. doi: 10.1109/THMS.2014.2362520
|
| [11] |
HAGHIGHAT M, ABDEL-MOTTALEB M, ALHALABI W.Fully automatic face normalization and single sample face recognition in unconstrained environments[J]. Expert Systems with Applications, 2016, 47:23-34. doi: 10.1016/j.eswa.2015.10.047
|
| [12] |
HAGHIGHAT M, ABDEL-MOTTALEB M, ALHALABI W.Discriminant correlation analysis:Real-time feature level fusion for multimodal biometric recognition[J]. IEEE Transactions on Information Forensics and Security, 2016, 11(9):1984-1996. doi: 10.1109/TIFS.2016.2569061
|
| [13] |
唐超, 王文剑, 李伟, 等.基于多学习器协同训练模型的人体行为识别方法[J].软件学报, 2015, 26(11):2939-2950.
TANG C, WANG W J, LI W, et al.Multi-learner co-training model for human actioin recognition[J]. Journal of Software, 2015, 26(11):2939-2950(in Chinese).
|
| [14] |
SI C Y, JING Y, WANG W, et al.Skeleton-based action recognition with spatial reasoning and temporal stack learning[C]//Proceedings of the European Conference on Computer Vision (ECCV).Berlin: Springer, 2018: 103-118.
|
| [15] |
LIU M Y, MENG F Y, CHEN C, et al.Joint dynamic pose image and space time reversal for human action recognition from videos[C]//33rd AAAI Conference on Artificial Intelligence.Palo Alto: AAAI Press, 2019: 8762-8769.
|
| [16] |
邓诗卓, 王波涛, 杨传贵, 等.CNN多位置穿戴式传感器人体活动识别[J].软件学报, 2019, 30(3):718-737.
DENG S Z, WANG B T, YANG C G, et al.Convolutional neural networks for human activity recognition using multi-location wearable sensors[J]. Journal of Software, 2019, 30(3):718-737(in Chinese).
|
| [17] |
CHEN C, JAFARI R, KEHTARNAVAZ N.A real-time human action recognition system using depth and inertial sensor fusion[J]. IEEE Sensors Journal, 2016, 16(3):773-781. doi: 10.1109/JSEN.2015.2487358
|
| [18] |
许艳, 侯振杰, 梁久祯, 等.权重融合深度图像与骨骼关键帧的行为识别[J].计算机辅助设计与图形学学报, 2018, 30(7):139-146.
XU Y, HOU Z J, LIANG J Z, et al.Action recognition using weighted fusion of depth images and skeleton's key frames[J]. Journal of Computer-Aided Design & Computer Graphics, 2018, 30(7):139-146(in Chinese).
|
| [19] |
WANG K, HE R, WANG L, et al.Joint feature selection and subspace learning for cross-modal retrieval[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(10):2010-2023. doi: 10.1109/TPAMI.2015.2505311
|
| [20] |
NIE F, HUANG H, CAI X, et al.Efficient and robust feature selection via joint ℓ2, 1-norms minimization[C]//Advances in Neural Information Processing Systems(NIPS).New York: Curran Associates, 2010: 1813-1821.
|
| [21] |
HE R, TIAN T N, WANG L, et al.L2, 1 regularized correntropy for robust feature selection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR).Piscataway, NJ: IEEE Press, 2012: 2504-2511.
|
| [22] |
OFLI F, CHAUDHRY R, KURILLO G, et al.Berkeley MHAD: A comprehensive multimodal human action database[C]//2013 IEEE Workshop on Applications of Computer Vision (WACV).Piscataway, NJ: IEEE Press, 2013: 53-60.
|
| [23] |
CHEN C, JAFARI R, KEHTARNAVAZ N.UTD-MHAD: A multimodal dataset for human action recognition utilizing a depth camera and a wearable inertial sensor[C]//2015 IEEE International Conference on Image Processing (ICIP).Piscataway, NJ: IEEE Press, 2015: 168-172.
|
| [24] |
CHEN C, ZHANG B G, HOU Z J, et al.Action recognition from depth sequences using weighted fusion of 2D and 3D auto-correlation of gradients features[J]. Multimedia Tools and Applications, 2017, 76(3):4651-4669. doi: 10.1007/s11042-016-3284-7
|
| [1] | GAO Hongxin, ZHAO Shougen, ZHU Jialin, YU Yihao, ZHANG Zhen. Research on Measurement Methods for the Critical Bending Radius of Flexible Solar Cells[J]. Journal of Beijing University of Aeronautics and Astronautics. doi: 10.13700/j.bh.1001-5965.2024.0801 |
| [2] | HU YaoWei, SHANG YunBin, HE Xi, YIN JinTao, JIANG ShengJu, LEI JuanMian. Wind Tunnel Experiment and Numerical Simulation on the Magnus Effect of Rotating Wing-body Combination[J]. Journal of Beijing University of Aeronautics and Astronautics. doi: 10.13700/j.bh.1001-5965.2024.0830 |
| [3] | TAN T X，TENG Y，WANG C Y. Research on pre-curved spiral wound pneumatic soft gripper[J]. Journal of Beijing University of Aeronautics and Astronautics，2025，51（2）：616-624 （in Chinese）. doi: 10.13700/j.bh.1001-5965.2023.0010. |
| [4] | ZHANG Y M，DAI Y T，WEI R K，et al. Experiment on dynamic response alleviation of a wing with variable-camber flexible trailing edge[J]. Journal of Beijing University of Aeronautics and Astronautics，2024，50（10）：3239-3249 （in Chinese）. doi: 10.13700/j.bh.1001-5965.2022.0761. |
| [5] | LIU S Y，YANG H L，ZHANG Z G，et al. Vibration control of flexible spacecraft with output constraints and external disturbances[J]. Journal of Beijing University of Aeronautics and Astronautics，2024，50（5）：1560-1567 （in Chinese）. doi: 10.13700/j.bh.1001-5965.2022.0622. |
| [6] | FAN X，CHENG Z H，LI S X，et al. Effect of critical eccentricity on forming accuracy of tubes in 3D free bending process[J]. Journal of Beijing University of Aeronautics and Astronautics，2024，50（1）：208-215 （in Chinese）. doi: 10.13700/j.bh.1001-5965.2022.0221. |
| [7] | WANG Z，PAN Y H，ZHAO R，et al. Effect of small inclination angle on heat transfer performance of Ω-shaped bending heat pipe[J]. Journal of Beijing University of Aeronautics and Astronautics，2024，50（7）：2314-2321 （in Chinese）. doi: 10.13700/j.bh.1001-5965.2022.0603. |
| [8] | SHAO Z Z，ZHENG K，DONG S，et al. Design of double bending rotary ultrasonic elliptical vibration machining system[J]. Journal of Beijing University of Aeronautics and Astronautics，2024，50（9）：2912-2918 （in Chinese）. doi: 10.13700/j.bh.1001-5965.2022.0712. |
| [9] | XIA J W，LIU Z K，ZHU X F，et al. A coordinated rendezvous method for unmanned surface vehicle swarms based on multi-agent reinforcement learning[J]. Journal of Beijing University of Aeronautics and Astronautics，2023，49（12）：3365-3376 （in Chinese）. doi: 10.13700/j.bh.1001-5965.2022.0088. |
| [10] | LIU Ren-di, JIANG Ju, ZHANG Zhe, LIU Xiang. Direct lift control technology of carrier aircraft landing based on reinforcement learning[J]. Journal of Beijing University of Aeronautics and Astronautics. doi: 10.13700/j.bh.1001-5965.2023.0403 |
| [11] | BAI Jing-bo, CHEN Yu, XIE Shi-yu, DAI Xin-wei. Design Method for Modulation Strategy of a Single-Inductor Multi-Port Converter Based on Reinforcement Learning[J]. Journal of Beijing University of Aeronautics and Astronautics. doi: 10.13700/j.bh.1001-5965.2023.0302 |
| [12] | LU: Chao, LI Mu-chen, OU Jia-jun. UAV hybrid path planning based on hierarchical deep reinforcement learning[J]. Journal of Beijing University of Aeronautics and Astronautics. doi: 10.13700/j.bh.1001-5965.2023.0550 |
| [13] | ZHANG Hao, ZHU Jian-wen, LI Xiao-ping, BAO Wei-min. Deep reinforcement learning intelligent guidance for intercepting high maneuvering targets[J]. Journal of Beijing University of Aeronautics and Astronautics. doi: 10.13700/j.bh.1001-5965.2023.0375 |
| [14] | ZHANG J L，YANG X X，DENG X L，et al. Altitude control of stratospheric aerostat based on deep reinforcement learning[J]. Journal of Beijing University of Aeronautics and Astronautics，2023，49（8）：2062-2070 （in Chinese）. doi: 10.13700/j.bh.1001-5965.2021.0622. |
| [15] | ZHU Jiazheng, WANG Cong, LI Xinkai, DONG Yingchao, ZHANG Hongli. A deep reinforcement learning based discrete state transition algorithm for fuzzy flexible job shop scheduling[J]. Journal of Beijing University of Aeronautics and Astronautics. doi: 10.13700/j.bh.1001-5965.2023.0211 |
| [16] | SUN D，GAO D，ZHENG J H，et al. UAV reinforcement learning control algorithm with demonstrations[J]. Journal of Beijing University of Aeronautics and Astronautics，2023，49（6）：1424-1433 （in Chinese）. doi: 10.13700/j.bh.1001-5965.2021.0466. |
| [17] | GONG K Q，WEI H K，LI J W，et al. Trajectory optimization algorithm of skipping missile based on deep reinforcement learning[J]. Journal of Beijing University of Aeronautics and Astronautics，2023，49（6）：1383-1393 （in Chinese）. doi: 10.13700/j.bh.1001-5965.2021.0436. |
| [18] | HE Z P，ZHOU J X，XIN J，et al. Endwall profiling of turbine blade hub with rim seal[J]. Journal of Beijing University of Aeronautics and Astronautics，2023，49（10）：2596-2607 （in Chinese）. doi: 10.13700/j.bh.1001-5965.2021.0728. |
| [19] | CHENG Xiao-quan, CAI Mo-quan, WANG Song-wei. Study progress of gap sealing structure for aircraft movable wing[J]. Journal of Beijing University of Aeronautics and Astronautics. doi: 10.13700/j.bh.1001-5965.2023.0397 |
| [20] | ZHANG Qingsong, LI Dongqi, YANG Juan. Effect of vibration on cyclic and thermal runaway characteristics of lithium ion batteries[J]. Journal of Beijing University of Aeronautics and Astronautics. doi: 10.13700/j.bh.1001-5965.2023.0267 |
Figures(8) / Tables(6)
Copyright © Journal of Beijing University of Aeronautics and Astronautics
Address: Editorial Department of Journal of Beijing University of Aeronautics and Astronautics, 37 Xueyuan Road, Haidian District, Beijing Post Code: 100191 Email: jbuaa@buaa.edu.cn
Supported by:
Beijing Renhe Information Technology Co., Ltd.
Cao Zhipeng, Liu Bo, Ding Weiet al. Stator setting angles adjustment on performance improvement of axial-centrifugal compressor[J]. Journal of Beijing University of Aeronautics and Astronautics, 2007, 33(08): 878-881. (in Chinese)
DownLoad:
