Detection and localization of concealed forbidden objects on human body based on complementary advantages of PMMWI and Ⅵ
-
摘要:
根据公共场所人体安检的性能要求和技术需求，将被动毫米波成像（PMMWI）的可透视成像性能优势与可见光成像（Ⅵ）的细节高分辨性能优势相结合，提出一种基于PMMWI与Ⅵ优势互补的人体隐蔽违禁物检测与定位算法。首先，提出一种基于低层特征融合的改进U-Net以增强深度神经网络（DNN）对PMMWI中弱小目标轮廓的敏感度，提高PMMWI中人体轮廓和隐蔽违禁物的分割精度，并同时实现Ⅵ中人体轮廓的像素级分割；然后，在PMMWI和Ⅵ中的人体轮廓分割基础上，通过基于人体轮廓的尺度变换与滑动适配实现PMMWI人体轮廓和Ⅵ人体轮廓的良好配准，根据配准结果实现单帧图像中人体隐蔽违禁物的高效检测；最后，通过序列图像检测结果的对比融合与优化决策给出隐蔽违禁物的定位结果。一系列综合实验与对比分析结果，验证了提出的人体隐蔽违禁物检测与定位算法的性能优势。
-
关键词:
- 毫米波安检 /
- 被动毫米波成像(PMMWI) /
- 人体轮廓分割 /
- 深度学习 /
- 深度神经网络(DNN) /
- 隐蔽违禁物检测与定位
Abstract:According to the performance requirements and technology demands of human security check in public place, combining the performance advantages of perspective imaging of passive millimeter wave imaging (PMMWI) and high discriminability in image details of visible imaging (Ⅵ), an approach for detection and localization of concealed forbidden objects on human body is presented in this paper based on the complementary advantages of PMMWI and Ⅵ. Firstly, an improved U-Net based on feature fusion in low layers is presented to enhance the sensitivity of deep neural networks (DNN) to the contour of dim small targets, and improve the accuracy of segmentation of human contours and concealed forbidden objects. Meanwhile, the pixel-level segmentation of human contours in Ⅵ is also implemented. Then, the contours of human body in PMMWI are registered with some corresponding ones in Ⅵ by scale transform and sliding fit, so the concealed forbidden objects on human body can be detected from a single frame with a high precision. Finally, the concealed forbidden objects are localized by contrasting fusion and optimizing decision according to detection results with sequence images. A series of comprehensive experiments and comparative analysis results validate the good performance of the proposed detection and localization algorithm of concealed forbidden objects on human body towards security check of public places.
-
图 1 主/被动毫米波成像安检设备示例
Figure 1. Illustration for AMMWI and PMMWI security check equipment
图 2 主/被动毫米波成像结合互补的安检方案
Figure 2. Security check scheme based on combination and complementation of AMMWI and PMMWI
图 4 基于PMMWI和Ⅵ优势互补的检测与定位方案
Figure 4. Detection and localization scheme based on complementary advantage of PMMWI and Ⅵ
图 5 基于PMMWI和Ⅵ优势互补的隐蔽违禁物检测与定位系统的组织结构与核心模块
Figure 5. Organization structure and core modules of concealed forbidden object detection and localization system based on complementary advantage of PMMWI and Ⅵ
图 8 基于低层特征融合的LFF-UNet网络结构
Figure 8. LFF-UNet architecture based on low-layer feature fusion
图 10 PMMWI和Ⅵ样本数据集与人工标注图像
Figure 10. Dataset and manually annotated image of PMMWI and Ⅵ samples
图 11 PMMWI中人体轮廓分割DNN模型性能进化曲线
Figure 11. Performance evolution curves of human contour segmentation DNN models in PMMWI
图 12 基于LFF-UNet的PMMWI人体轮廓分割算法
Figure 12. Segmentation algorithm of human contour in PMMWI based on LFF-UNet architecture
图 13 不同DNN模型PMMWI人体轮廓分割对比
Figure 13. Comparison of human contour segmentation in PMMWI among different DNN models
图 14 不同DNN模型分割结果IoU对比
Figure 14. Comparison of IoU of segmentation result among different DNN models
图 15 不同距离下的Ⅵ和PMMWI人体轮廓分割对比
Figure 15. Comparison of segmentation of human contours between Ⅵ and PMMWI in different distances
图 16 PMMWI与Ⅵ人体轮廓尺度变换与滑动适配
Figure 16. Scale transform and sliding fit of human contour in PMMWI and Ⅵ
图 18 基于人体轮廓配准的隐蔽违禁物检测与定位
Figure 18. Detection and localization of concealed forbidden object based on human contour registration
图 19 人体隐蔽违禁物检测与定位结果
Figure 19. Results of detection and localization of concealed forbidden objects on human body
图 20 多帧人体隐蔽违禁物检测结果对比
Figure 20. Comparison of multi-frame detection results of concealed forbidden objects on human body
图 21 基于检测结果对比融合的复合定位
Figure 21. Composite localization based on contrasting fusion of detection results
图 22 算法与模型的综合集成与优化
Figure 22. Comprehensive integration and optimization of algorithms and models
图 23 远距离人体隐蔽违禁物检测
Figure 23. Detection of concealed forbidden objectson human body from far distance
图 24 近距离人体隐蔽违禁物检测
Figure 24. Detection of concealed forbidden objectson human body from close distance
图 25 人体隐蔽违禁物定位误差对比
Figure 25. Comparison of localization error of concealed forbidden objects on human body
图 26 人体隐蔽违禁物定位可视化结果
Figure 26. Visualization of localization result for concealed forbidden objects on human body
图 27 U-Net与LFF3-UNet网络跨越连接层特征图可视化对比
Figure 27. Comparison of feature map visualization at skip-connection layer between U-Net and LFF3-UNet
表 1 人体隐蔽违禁物检测IoU对比
Table 1. Comparison of IoU for detection of concealed forbidden objects on human body
检测方式 a b c d e 远距离检测 0.642 0.859 0.782 0.490 0.648 近距离检测 0.786 0.810 0.362 0.584 0.853 
下载: 导出CSV
-
[1] 王任肩, 王宇光, 陈志福, 等.大型活动中的安检排爆工作及技术保障研究[J].警察技术, 2018(5):19-22. doi: 10.3969/j.issn.1009-9875.2018.05.004WANG R J, WANG Y G, CHEN Z F, et al.Research on exploder clearing work and technical support in security inspection of large activities[J].Police Technology, 2018(5):19-22(in Chinese). doi: 10.3969/j.issn.1009-9875.2018.05.004 [2] 费鹏, 方维海, 温鑫, 等.用于人员安检的主动毫米波成像技术现状与展望[J].微波学报, 2015, 31(2):91-96. http://d.old.wanfangdata.com.cn/Periodical/wbxb201502020FEI P, FANG W H, WEN X, et al.State of the art and future prospect of the active millimeter wave imaging technique for personnel screening[J].Journal of Microwaves, 2015, 31(2):91-96(in Chinese). http://d.old.wanfangdata.com.cn/Periodical/wbxb201502020 [3] GROSSMAN E N, MILLER A J.Active millimeter-wave imaging for concealed weapons detection[C]//Proceedings of Conference on Passive Millimeter-Wave Imaging Technology Ⅵ and Radar International Society for Optics and Photonics.Bellingham: SPIE, 2003, 5077: 62-70. [4] ANDREWS D A, HARMER S W, BOWRING N J, et al.Active millimeter wave sensor for standoff concealed threat detection[J].IEEE Sensors Journal, 2013, 13(12):4948-4954. doi: 10.1109/JSEN.2013.2273487 [5] DU K, ZHANG L, CHEN W, et al.Concealed objects detection based on FWT in active millimeter-wave images[C]//Proceedings of 7th International Conference on Electronics and Information Engineering.Bellingham: SPIE, 2017, 10322: 103221O. [6] REN S, HE K, GIRSHICK R, et al.Faster R-CNN:Towards real-time object detection with region proposal networks[J].IEEE Transaction on Pattern Analysis and Machine Intelligence, 2017, 39(6):1137-1149. doi: 10.1109/TPAMI.2016.2577031 [7] 骆尚, 吴晓峰, 杨明辉, 等.基于卷积神经网络的毫米波图像人体隐匿物检测[J].复旦学报(自然科学版), 2018, 57(4):442-452. http://d.old.wanfangdata.com.cn/Periodical/fdxb201804005LUO S, WU X F, YANG M H, et al.Convolutional-neural-network-based human concealed object detection for millimeter wave images[J].Journal of Fudan University(Natural Science), 2018, 57(4):442-452(in Chinese). http://d.old.wanfangdata.com.cn/Periodical/fdxb201804005 [8] SINCLAIR G N, ANDERTON R N, APPLEBY R.Passive millimeter-wave concealed weapon detection[C]//Proceedings of Enabling Technologies for Law Enforcement and Security.Bellingham: SPIE, 2001, 4232: 142-152. [9] YEOM S, LEE D, SON J, et al.Real-time outdoor concealed-object detection with passive millimeter wave imaging[J].Optics Express, 2011, 19(3):2530-2536. doi: 10.1364/OE.19.002530 [10] YEOM S, LEE D, JANG Y, et al.Real-time concealed-object detection and recognition with passive millimeter wave imaging[J].Optics Express, 2012, 20(9):9371-9381. doi: 10.1364/OE.20.009371 [11] 周健, 叶金晶, 孙谦晨, 等.主动毫米波成像性别识别算法研究[J].红外, 2018, 39(9):34-40. doi: 10.3969/j.issn.1672-8785.2018.09.006ZHOU J, YE J J, SUN Q C, et al.Study of a gender identification algorithm for active millimeter-wave imaging[J].Infrared, 2018, 39(9):34-40(in Chinese). doi: 10.3969/j.issn.1672-8785.2018.09.006 [12] SHELHAMER E, LONG J, DARRELL T.Fully convolutional networks for semantic segmentation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4):640-651. doi: 10.1109/TPAMI.2016.2572683 [13] BADRINARAYANAN V, KENDALL A, CIPOLLA R.SegNet:A deep convolutional encoder-decoder architecture for image segmentation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12):2481-2495. doi: 10.1109/TPAMI.2016.2644615 [14] RONNEBERGER O, FISCHER P, BROX T.U-Net: Convolutional networks for biomedical image segmentation[C]//Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention.Berlin: Springer, 2015: 234-241. [15] 桑伟, 岳胜利.毫米波成像技术在人体安全检查领域的应用[J].中国安防, 2013(4):83-87. doi: 10.3969/j.issn.1673-7873.2013.04.018SANG W, YUE S L.Application of millimeter wave imaging technology on the area of human body security inspection[J].China Security & Protection, 2013(4):83-87(in Chinese). doi: 10.3969/j.issn.1673-7873.2013.04.018 [16] IGLOVIKOV V, SHVETS A.TernausNet: U-Net with VGG11 encoder pre-trained on ImageNet for image segmentation[D].(2018-01-17)[2018-12-20].https://arxiv.org/abs/1801.05746. https://arxiv.org/abs/1801.05746 [17] LIN M, CHEN Q, YAN S C.Network in network[D].(2014-03-04)[2018-12-20].https://arxiv.org/abs/1312.4400. [18] SZEGEDY C, LIU W, JIA Y, et al.Going deeper with convolutions[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Piscataway, NJ: IEEE Press, 2015: 1-9. [19] SZEGEDY C, VANHOUCKE V, IOFFE S, et al.Rethinking the inception architecture for computer vision[D].(2015-12-11)[2018-12-20].https://arxiv.org/abs/1512.00567v2. [20] HE K M, ZHANG X Y, REN S Q, et al.Deep residual learning for image recognition[D].(2015-12-10)[2018-12-20].https://arxiv.org//abs/1512.03385. [21] XIE S, GIRSHICK R, DOLLÁR P, et al.Aggregated residual transformations for deep neural networks[D].(2017-04-11)[2018-12-20].https://arxiv.org/abs/1611.05431. [22] KINGMA D P, BA J L.Adam: A method for stochastic optimization[D].(2017-01-30)[2018-12-20].https://arxiv.org/abs/1412.6980. [23] ZEILER M D, FERGUS R.Visualizing and understanding convolutional networks[C]//Proceedings of European Conference on Computer Vision.Berlin: Springer, 2014: 818-833. -
下载:
点击查看大图
计量
- 文章访问数: 860
- HTML全文浏览量: 134
- PDF下载量: 516
- 被引次数: 0
