-
摘要:
随着手势动作识别技术在人机交互、生活娱乐及医疗服务等应用领域的逐步深入, 其对非接触、微光条件下的稳健测量与识别能力提出更高要求。针对该问题, 研究了一种基于线性调频连续波(LFMCW)雷达距离-多普勒(RD)信息和卷积神经网络(CNN)的典型手势动作识别方法。首先, 对于LFMCW雷达回波, 通过去斜、快时间域快速傅里叶变换和相干积累, 获取手势目标的二维RD像数据; 其次, 以RD像幅度矩阵作为CNN输入样本, 利用2层卷积与池化处理构建特征空间, 从而通过全连接与softmax分类器实现对手势动作的有效识别; 最后, 在此基础上, 采用24 GHz工业雷达传感器设计手势测量实验系统, 形成关于4种典型手势动作的LFMCW雷达回波数据库。实验结果表明, 将24 GHz LFMCW雷达回波RD处理与CNN结合能够实现对典型手势动作的有效识别。
-
关键词:
- 手势动作识别 /
- 线性调频连续波(LFMCW)雷达 /
- 距离-多普勒(RD) /
- 卷积神经网络(CNN) /
- softmax分类器
Abstract:With the widespread use of hand gesture recognition technique, capabilities of robust measurement and classification in non-contact and all-day conditions are much desired in its applications, such as human-computer interaction, life entertainment and medical service.According to this requirement, the paper introduces a hand gesture recognition method based on linear frequency modulated continuous wave (LFMCW) radar range-Doppler (RD) information and convolutional neural network (CNN).Firstly, for LFMCW radar echoes from hand gestures, dechirping, fast Fourier transform in fast-time domain and coherent integration are applied to produce the two-dimensional RD images of hand gesture.Next, they are used as the input data of CNN, and the feature space is constructed with the process of two-layer convolution and pooling.Finally, the effective hand gesture recognition is achieved by full connection and softmax classifier.On this basis, a 24 GHz industrial radar sensor is used to design the experimental system for hand gesture measurement, and a dataset of four typical hand gestures is also generated with the LFMCW waveform.The experimental results show that the proposed method based on RD information and CNN is applicable to general radar sensors at 24 GHz and could achieve effective recognition of typical hand gestures.
-
表 1 基于测试数据的手势识别准确率
Table 1. Testing accuracy of hand gesture recognition
方法 准确率 全局准确率 向后运动 向前运动 旋转运动 静止 DTW 0.816 7 0.913 3 0.240 0 1.000 0 0.740 0 本文 0.932 0 0.920 0 0.816 0 0.844 0 0.878 0 -
[1] 张诗潮. 基于Kinect的手语教学系统设计研究[D]. 上海: 华东师范大学, 2014.ZHANG S C. Design research in Kinect based sign-language teaching system[D]. Shanghai: East China Normal University, 2014(in Chinese). [2] 孙凯, 严潇然, 谢荣平.基于手势识别的智能家居人机交互系统设计[J].工业控制计算机, 2014, 27(4):54-56. http://www.cqvip.com/QK/92874X/201404/49581541.htmlSUN K, YAN X R, XIE R P.Design of man-machine interaction system in smart home[J].Industrial Control Computer, 2014, 27(4):54-56(in Chinese). http://www.cqvip.com/QK/92874X/201404/49581541.html [3] 万华根, 肖海英, 邹松.面向新一代大众游戏的手势交互技术[J].计算机辅助设计与图形学学报, 2011, 23(7):1159-1165. http://www.cqvip.com/QK/97390X/201107/38413315.htmlWAN H G, XIAO H Y, ZOU S.Hand gesture interaction for next-generation public games[J].Journal of Comouter-Aided Design & Computer Graphics, 2011, 23(7):1159-1165(in Chinese). http://www.cqvip.com/QK/97390X/201107/38413315.html [4] OLIVITO R S, STUMPO P, SURACE L.Glove-Talk:A neural network interface between a data-glove and a speech synthesizer[J].IEEE Transactions on Neural Networks, 1993, 4(1):2-8. doi: 10.1109/72.182690 [5] STURMAN D J, ZELYZER D.A survey of glove-based input[J].IEEE Computer Graphics & Applications, 1994, 14(1):30-39. http://ieeexplore.ieee.org/iel1/38/6422/00250916.pdf?arnumber=250916 [6] QUAM D L. Gesture recognition with a DataGlove[C]//Proceedings of Aerospace and Electronics Conference. Piscataway, NJ: IEEE Press, 1990, 2: 755-760. [7] WANG C, CANNON D J. A virtual end-effector pointing system in point-and-direct robotics for inspection of surface flaws using a neural network based skeleton transform[C]//Proceedings of IEEE International Conference on Robotics and Automation. Piscataway, NJ: IEEE Press, 1993, 3: 784-789. [8] BAUDEL T, BEAUDOUIN-LAFON M.Charade:Remote control of objects using free-hand gestures[J].Communications of the ACM, 1993, 36(7):28-35. doi: 10.1145/159544.159562 [9] TRAVER V J, LATORRE-CARMONA P, SALVADOR-BALAGUER E, et al.Three-dimensional integral imaging for gesture recognition under occlusions[J].IEEE Signal Processing Le-tters, 2017, 24(2):171-175. doi: 10.1109/LSP.2016.2643691 [10] SIMONYAN K, ZISSERMAN A.Two-stream convolutional networks for action recognition in videos[J].Advances in Neural Information Processing Systems, 2014, 1(4):568-576. http://export.arxiv.org/abs/1704.00389 [11] WANG C, LIU Z, ZHAO J. Hand gesture recognition based on canonical formed superpixel earth mover's distance[C]//Proceedings of IEEE International Conference on Multimedia and Expo. Piscataway, NJ: IEEE Press, 2016: 1-6. [12] FAN T, MA C, GU Z, et al.Wireless hand gesture recognition based on continuous-wave Doppler radar sensors[J].IEEE Transactions on Microwave Theory & Techniques, 2016, 64(11):4012-4020. http://adsabs.harvard.edu/abs/2016ITMTT..64.4012F [13] KIM Y, TOOMAJIAN B.Hand gesture recognition using micro-Doppler signatures with convolutional neural network[J].IEEE Access, 2016, 4:7125-7130. doi: 10.1109/ACCESS.2016.2617282 [14] REN N, QUAN X, CHO S H.Algorithm for gesture recognition using an IR-UWB radar sensor[J].Journal of Computer & Communications, 2016, 4(3):95-100. http://www.i-scholar.in/index.php/JCC/article/view/95624 [15] LIEN J, GILLIAN N, KARAGOZLER M E, et al.Soli:Ubiquitous gesture sensing with millimeter wave radar[J].ACM Transactions on Graphics, 2016, 35(4):1-19. https://dl.acm.org/ft_gateway.cfm?id=2925953&ftid=1806870 [16] MOLCHANOV P, GUPTA S, KIM K, et al. Multi-sensor system for driver's hand-gesture recognition[C]//Proceedings of IEEE International Conference and Workshops on Automatic Face and Gesture Recognition. Piscataway, NJ: IEEE Press, 2015: 1-8. [17] MALYSA G, WANG D, NETSCH L, et al. Hidden Markov model-based gesture recognition with FMCW radar[C]//Proceedings of IEEE Global Conference on Signal and Information Processing. Piscataway, NJ: IEEE Press, 2016: 1017-1021. [18] Infineon Technologies AG. 24 GHz chipset-family for industrial applications[EB/OL]. [2017-06-08]. http://www.infineon.com/dgdl?folderId=db3a304344baa87a0144d47968a72df7&fileId=db3a304344baa87a0144d47b9a5e2df9. [19] EYE R M, DOMINGUEZ G, ESCALERA S. Feature weighting in dynamic time warping for gesture recognition in depth data[C]//Proceedings of IEEE International Conference on Computer Vision Workshops. Piscataway, NJ: IEEE Press, 2011: 1182-1188.