留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于卷积神经网络的手势动作雷达识别方法

王俊 郑彤 雷鹏 张原 樵明朗

王俊, 郑彤, 雷鹏, 等 . 基于卷积神经网络的手势动作雷达识别方法[J]. 北京航空航天大学学报, 2018, 44(6): 1117-1123. doi: 10.13700/j.bh.1001-5965.2017.0397
引用本文: 王俊, 郑彤, 雷鹏, 等 . 基于卷积神经网络的手势动作雷达识别方法[J]. 北京航空航天大学学报, 2018, 44(6): 1117-1123. doi: 10.13700/j.bh.1001-5965.2017.0397
WANG Jun, ZHENG Tong, LEI Peng, et al. Hand gesture recognition method by radar based on convolutional neural network[J]. Journal of Beijing University of Aeronautics and Astronautics, 2018, 44(6): 1117-1123. doi: 10.13700/j.bh.1001-5965.2017.0397(in Chinese)
Citation: WANG Jun, ZHENG Tong, LEI Peng, et al. Hand gesture recognition method by radar based on convolutional neural network[J]. Journal of Beijing University of Aeronautics and Astronautics, 2018, 44(6): 1117-1123. doi: 10.13700/j.bh.1001-5965.2017.0397(in Chinese)

基于卷积神经网络的手势动作雷达识别方法

doi: 10.13700/j.bh.1001-5965.2017.0397
基金项目: 

国家自然科学基金 61501011

国家自然科学基金 61671035

详细信息
    作者简介:

    王俊  男, 博士, 教授, 博士生导师。主要研究方向:雷达信号处理、实时信号处理

    郑彤  女, 博士研究生。主要研究方向:信号处理、模式识别

    雷鹏  男, 博士, 讲师。主要研究方向:信号处理、模式识别

    通讯作者:

    雷鹏,E-mail:peng.lei@buaa.edu.cn

  • 中图分类号: TN951;TN959.5;TP183

Hand gesture recognition method by radar based on convolutional neural network

Funds: 

National Natural Science Foundation of China 61501011

National Natural Science Foundation of China 61671035

More Information
  • 摘要:

    随着手势动作识别技术在人机交互、生活娱乐及医疗服务等应用领域的逐步深入, 其对非接触、微光条件下的稳健测量与识别能力提出更高要求。针对该问题, 研究了一种基于线性调频连续波(LFMCW)雷达距离-多普勒(RD)信息和卷积神经网络(CNN)的典型手势动作识别方法。首先, 对于LFMCW雷达回波, 通过去斜、快时间域快速傅里叶变换和相干积累, 获取手势目标的二维RD像数据; 其次, 以RD像幅度矩阵作为CNN输入样本, 利用2层卷积与池化处理构建特征空间, 从而通过全连接与softmax分类器实现对手势动作的有效识别; 最后, 在此基础上, 采用24 GHz工业雷达传感器设计手势测量实验系统, 形成关于4种典型手势动作的LFMCW雷达回波数据库。实验结果表明, 将24 GHz LFMCW雷达回波RD处理与CNN结合能够实现对典型手势动作的有效识别。

     

  • 图 1  LFMCW雷达测距示意图

    Figure 1.  Schematic of range measurement by LFMCW radar

    图 2  用于手势动作识别的24 GHz LFMCW雷达实验系统结构

    Figure 2.  Block diagram of 24 GHz LFMCW radar experimental system for hand gesture recognition

    图 3  手势向前动作时的雷达回波中频信号

    Figure 3.  Radar intermediate-frequency echoes of hand pushing

    图 4  基于快时间FFT的手势目标径向距离

    Figure 4.  Radial distance of gesture target based on FFT in fast-time domain

    图 5  4种手势动作的RD测量结果

    Figure 5.  RD measurement results of four hand gestures

    图 6  运动轨迹模板

    Figure 6.  Templates of moving tracks

    图 7  基于RD像的CNN架构示意图

    Figure 7.  Schematic diagram of CNN architecture used in RD image

    图 8  训练错误率曲线

    Figure 8.  Error rates in training

    表  1  基于测试数据的手势识别准确率

    Table  1.   Testing accuracy of hand gesture recognition

    方法 准确率 全局准确率
    向后运动 向前运动 旋转运动 静止
    DTW 0.816 7 0.913 3 0.240 0 1.000 0 0.740 0
    本文 0.932 0 0.920 0 0.816 0 0.844 0 0.878 0
    下载: 导出CSV
  • [1] 张诗潮. 基于Kinect的手语教学系统设计研究[D]. 上海: 华东师范大学, 2014.

    ZHANG S C. Design research in Kinect based sign-language teaching system[D]. Shanghai: East China Normal University, 2014(in Chinese).
    [2] 孙凯, 严潇然, 谢荣平.基于手势识别的智能家居人机交互系统设计[J].工业控制计算机, 2014, 27(4):54-56. http://www.cqvip.com/QK/92874X/201404/49581541.html

    SUN K, YAN X R, XIE R P.Design of man-machine interaction system in smart home[J].Industrial Control Computer, 2014, 27(4):54-56(in Chinese). http://www.cqvip.com/QK/92874X/201404/49581541.html
    [3] 万华根, 肖海英, 邹松.面向新一代大众游戏的手势交互技术[J].计算机辅助设计与图形学学报, 2011, 23(7):1159-1165. http://www.cqvip.com/QK/97390X/201107/38413315.html

    WAN H G, XIAO H Y, ZOU S.Hand gesture interaction for next-generation public games[J].Journal of Comouter-Aided Design & Computer Graphics, 2011, 23(7):1159-1165(in Chinese). http://www.cqvip.com/QK/97390X/201107/38413315.html
    [4] OLIVITO R S, STUMPO P, SURACE L.Glove-Talk:A neural network interface between a data-glove and a speech synthesizer[J].IEEE Transactions on Neural Networks, 1993, 4(1):2-8. doi: 10.1109/72.182690
    [5] STURMAN D J, ZELYZER D.A survey of glove-based input[J].IEEE Computer Graphics & Applications, 1994, 14(1):30-39. http://ieeexplore.ieee.org/iel1/38/6422/00250916.pdf?arnumber=250916
    [6] QUAM D L. Gesture recognition with a DataGlove[C]//Proceedings of Aerospace and Electronics Conference. Piscataway, NJ: IEEE Press, 1990, 2: 755-760.
    [7] WANG C, CANNON D J. A virtual end-effector pointing system in point-and-direct robotics for inspection of surface flaws using a neural network based skeleton transform[C]//Proceedings of IEEE International Conference on Robotics and Automation. Piscataway, NJ: IEEE Press, 1993, 3: 784-789.
    [8] BAUDEL T, BEAUDOUIN-LAFON M.Charade:Remote control of objects using free-hand gestures[J].Communications of the ACM, 1993, 36(7):28-35. doi: 10.1145/159544.159562
    [9] TRAVER V J, LATORRE-CARMONA P, SALVADOR-BALAGUER E, et al.Three-dimensional integral imaging for gesture recognition under occlusions[J].IEEE Signal Processing Le-tters, 2017, 24(2):171-175. doi: 10.1109/LSP.2016.2643691
    [10] SIMONYAN K, ZISSERMAN A.Two-stream convolutional networks for action recognition in videos[J].Advances in Neural Information Processing Systems, 2014, 1(4):568-576. http://export.arxiv.org/abs/1704.00389
    [11] WANG C, LIU Z, ZHAO J. Hand gesture recognition based on canonical formed superpixel earth mover's distance[C]//Proceedings of IEEE International Conference on Multimedia and Expo. Piscataway, NJ: IEEE Press, 2016: 1-6.
    [12] FAN T, MA C, GU Z, et al.Wireless hand gesture recognition based on continuous-wave Doppler radar sensors[J].IEEE Transactions on Microwave Theory & Techniques, 2016, 64(11):4012-4020. http://adsabs.harvard.edu/abs/2016ITMTT..64.4012F
    [13] KIM Y, TOOMAJIAN B.Hand gesture recognition using micro-Doppler signatures with convolutional neural network[J].IEEE Access, 2016, 4:7125-7130. doi: 10.1109/ACCESS.2016.2617282
    [14] REN N, QUAN X, CHO S H.Algorithm for gesture recognition using an IR-UWB radar sensor[J].Journal of Computer & Communications, 2016, 4(3):95-100. http://www.i-scholar.in/index.php/JCC/article/view/95624
    [15] LIEN J, GILLIAN N, KARAGOZLER M E, et al.Soli:Ubiquitous gesture sensing with millimeter wave radar[J].ACM Transactions on Graphics, 2016, 35(4):1-19. https://dl.acm.org/ft_gateway.cfm?id=2925953&ftid=1806870
    [16] MOLCHANOV P, GUPTA S, KIM K, et al. Multi-sensor system for driver's hand-gesture recognition[C]//Proceedings of IEEE International Conference and Workshops on Automatic Face and Gesture Recognition. Piscataway, NJ: IEEE Press, 2015: 1-8.
    [17] MALYSA G, WANG D, NETSCH L, et al. Hidden Markov model-based gesture recognition with FMCW radar[C]//Proceedings of IEEE Global Conference on Signal and Information Processing. Piscataway, NJ: IEEE Press, 2016: 1017-1021.
    [18] Infineon Technologies AG. 24 GHz chipset-family for industrial applications[EB/OL]. [2017-06-08]. http://www.infineon.com/dgdl?folderId=db3a304344baa87a0144d47968a72df7&fileId=db3a304344baa87a0144d47b9a5e2df9.
    [19] EYE R M, DOMINGUEZ G, ESCALERA S. Feature weighting in dynamic time warping for gesture recognition in depth data[C]//Proceedings of IEEE International Conference on Computer Vision Workshops. Piscataway, NJ: IEEE Press, 2011: 1182-1188.
  • 加载中
图(8) / 表(1)
计量
  • 文章访问数:  715
  • HTML全文浏览量:  10
  • PDF下载量:  602
  • 被引次数: 0
出版历程
  • 收稿日期:  2017-06-12
  • 录用日期:  2017-06-30
  • 刊出日期:  2018-06-20

目录

    /

    返回文章
    返回
    常见问答