基于视频帧间运动估计的无人机图像车辆检测

doi:10.13700/j.bh.1001-5965.2019.0279

北京航空航天大学学报 ›› 2020, Vol. 46 ›› Issue (3): 634-642.doi: 10.13700/j.bh.1001-5965.2019.0279

基于视频帧间运动估计的无人机图像车辆检测

陈映雪¹, 丁文锐², 李红光², 王蒙¹, 王旭³

1. 北京航空航天大学电子信息工程学院, 北京 100083;
2. 北京航空航天大学无人系统研究院, 北京 100083;
3. 合一智芯科技有限公司, 北京 100083

收稿日期:2019-06-03 发布日期:2020-03-28
通讯作者: 李红光 E-mail:lihongguang@buaa.edu.cn
作者简介:陈映雪,女,硕士研究生。主要研究方向:遥感图像目标检测算法及应用;丁文锐,女,博士,研究员,博士生导师。主要研究方向:多源图像信息处理、视觉目标检测与跟踪;李红光,男,博士,高级工程师,硕士生导师。主要研究方向:无人系统光学图像智能处理及边缘计算应用。
基金资助:
国防基础科研计划（JCKY2017601C006）；武汉大学测绘遥感信息工程国家重点实验室开放基金（17E01）

Vehicle detection in UAV image based on video interframe motion estimation

CHEN Yingxue¹, DING Wenrui², LI Hongguang², WANG Meng¹, WANG Xu³

1. School of Electronic and Information Engineering, Beihang University, Beijing 100083, China;
2. Institute of Unmanned System, Beihang University, Beijing 100083, China;
3. Heyintelligence Technology Limited Company, Beijing 100083, China

Received:2019-06-03 Published:2020-03-28
Supported by:
National Defense Basic Scientific Research Program of China (JCKY2017601C006); Open Fund of State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University (17E01)

摘要/Abstract

摘要： 基于人工智能（AI）芯片搭建轻量化深度神经网络，可以在无人机（UAV）机载端实现视频中车辆目标的自动检测，具有重要的应用前景。为此，提出了一种针对无人机图像车辆目标的检测方法，并在AI芯片上进行部署与测试。方法具体包括：结合无人机图像中车辆目标的尺寸范围，对MobileNet-SSD网络进行裁剪，构建轻量化单帧图像检测器；为解决小目标特性在轻量网络框架下引发的检测性能下降问题，引入帧间运动矢量估计，根据相邻帧信息辅助预测当前帧丢失目标的位置范围，并利用检测结果进行修正，实现丢失目标的再召回。通过对多个数据集进行融合与自动补充标注，搭建了一个高质量的无人机图像车辆目标数据集；同时将方法在基于RK3399芯片计算的嵌入式开发平台上进行实验验证，结果表明：搭建的网络能够显著减少存储资源占用，具有轻量化的特点；同时相比于单帧检测法，引入视频帧间运动估计方法可以有效提高检测精度，并在AI芯片上实现125.3 ms/帧的检测速度。

关键词: 无人机(UAV), 目标检测, 轻量化神经网络, 人工智能(AI)芯片, 运动估计

Abstract: The lightweight neural network embedded on artificial intelligence (AI) chips can realize the onboard automatic detection of vehicle objects in unmanned aerial vehicle (UAV) videos, which is important in practical applications. In this paper, a vehicle object detection algorithm in UAV videos is proposed, and then deployed and tested on AI chips. For the proposed detection algorithm, firstly, the MobileNet-SSD network is clipped based on the range of vehicle objects' size in UAV images to construct a lightweight single-frame object detector. Secondly, the interframe motion estimation was introduced to improve the poor detection performance which is usually caused by small object characteristics and lightweight network. Thirdly, the position range of missing objects in the current frame is predicted according to the information of adjacent frames. Finally, the predicted position is corrected by detection results, and the recall of lost objects is realized. Additionally, a high-quality UAV image vehicle dataset was built by fusion and automatic supplementary annotation of multiple datasets. The proposed algorithm is verified on the embedded development platform based on RK3399 chip. The results show that the network with the proposed algorithm can significantly reduce the occupation of storage resources with the lightweight characteristics. Compared to the traditional single-image detection algorithm, the proposed algorithm can effectively improve the detection accuracy. Moreover, detection speed can be as low as 125.3 ms per frame on the AI chip.

Key words: unmanned aerial vehicle (UAV), object detection, lightweight neural network, artificial intelligence (AI) chip, motion estimation

中图分类号:

陈映雪, 丁文锐, 李红光, 王蒙, 王旭. 基于视频帧间运动估计的无人机图像车辆检测[J]. 北京航空航天大学学报, 2020, 46(3): 634-642.

CHEN Yingxue, DING Wenrui, LI Hongguang, WANG Meng, WANG Xu. Vehicle detection in UAV image based on video interframe motion estimation[J]. Journal of Beijing University of Aeronautics and Astronautics, 2020, 46(3): 634-642.

参考文献

[1] 张可,杨灿坤,周春平,等.无人机视频图像运动目标检测算法综述[J].液晶与显示,2019,34(1):98-109.ZHANG K,YANG C K,ZHOU C P,et al.Review of moving target detection algorithms for UAV video images[J].Chinese Journal of Liquid Crystals and Displays,2019,34(1):98-109(in Chinese).
[2] GIRSHICK R,DONAHUE J,DARRELL T,et al.Rich feature hierarchies for accurate object detection and semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE Press,2014:580-587.
[3] GIRSHICK R.Fast R-CNN[C]//Proceedings of the IEEE International Conference on Computer Vision.Piscataway,NJ:IEEE Press,2015:1440-1448.
[4] REN S,HE K,GIRSHICK R,et al.Faster R-CNN:Towards real-time object detection with region proposal networks[C]//Advances in Neural Information Processing Systems.Red Hook,NY:Curran Associates,2015:91-99.
[5] DAI J,LI Y,HE K,et al.R-FCN:Object detection via region-based fully convolutional networks[C]//Advances in Neural Information Processing Systems.Red Hook,NY:Curran Associates,2016:379-387.
[6] REDMON J,DIVVALA S,GIRSHICK R,et al.You only look once:Unified,real-time object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE Press,2016:779-788.
[7] LIU W,ANGUELOV D,ERHAN D,et al.SSD:Single shot multibox detector[C]//Proceedings of the European Conference on Computer Vision.Berlin:Springer,2016:21-37.
[8] IANDOLA F N,HAN S,MOSKEWICZ M W,et al.SqueezeNet:AlexNet-level accuracy with 50x fewer parameters and <0.5 MB model size[EB/OL].(2016-11-04)[2019-06-01] https://arxiv.org/abs/1602.07360.
[9] ZHANG X,ZHOU X,LIN M,et al.Shufflenet:An extremely efficient convolutional neural network for mobile devices[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE Press,2018:6848-6856.
[10] HOWARD A G,ZHU M,CHEN B,et al.MobileNet:Efficient convolutional neural networks for mobile vision applications[EB/OL].(2017-04-17)[2019-06-01] https://arxiv.org/abs/1704.04861.
[11] 顾帅.Android移动平台下基于深度学习的目标检测技术研究[D].西安:西安电子科技大学,2018.GU S.Deep learning based object detection technology research under android mobile platform[D].Xi'an:Xidian University,2018(in Chinese).
[12] 吴广伟.基于移动终端的轻量级卷积神经网络研究与实现[D].西安:西安电子科技大学,2018.WU G W.Research and implementation of lightweight convolution neural network based on mobile terminal[D].Xi'an:Xidian University,2018(in Chinese).
[13] ZHU X,WANG Y,DAI J,et al.Flow-guided feature aggregation for video object detection[C]//Proceedings of the IEEE International Conference on Computer Vision.Piscataway,NJ:IEEE Press,2017:408-417.
[14] WANG S,ZHOU Y,YAN J,et al.Fully motion-aware network for video object detection[C]//Proceedings of the European Conference on Computer Vision.Berlin:Springer,2018:542-557.
[15] HAN W,KHORRAMI P,PAINE T L,et al.Seq-NMS for video object detection[EB/OL].(2016-08-22)[2019-06-01].https://arxiv.org/abs/1602.08465.
[16] KANG K,LI H,YAN J,et al.T-CNN:Tubelets with convolutional neural networks for object detection from videos[J].IEEE Transactions on Circuits and Systems for Video Technology,2018,28(10):2896-2907.
[17] SHUAI H,LIU Q,ZHANG K,et al.Cascaded regional spatio-temporal feature-routing networks for video object detection[J].IEEE Access,2018,6:3096-3106.
[18] 余启明.基于背景减法和帧差法的运动目标检测算法研究[D].赣州:江西理工大学,2013.YU Q M.Moving object detection research based on background subtraction and frame difference[D].Ganzhou:Jiangxi University of Science and Technology,2013(in Chinese).
[19] GUNNAR F.Two-frame motion estimation based on polynomial expansion[C]//Scandinavian Conference on Image Analysis.Berlin:Brev Publishing,2003:363-370.
[20] DU D,QI Y,YU H,et al.The unmanned aerial vehicle benchmark:Object detection and tracking[C]//Proceedings of the European Conference on Computer Vision.Berlin:Springer,2018:370-386.
[21] ZHU P,WEN L,DU D,et al.VisDrone-VDT2018:The vision meets drone video detection and tracking challenge results[C]//Proceedings of the European Conference on Computer Vision.Berlin:Springer,2018:496-518.
[22] ZHU P,WEN L,DU D,et al.Visdrone-DET2018:The vision meets drone object detection in image challenge results[C]//European Conference on Computer Vision.Berlin:Springer,2018:437-468.

基于视频帧间运动估计的无人机图像车辆检测

Vehicle detection in UAV image based on video interframe motion estimation

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	何吕龙, 张佳强, 侯岳奇, 梁晓龙, 柏鹏. 有向通信拓扑和时延条件下的无人机集群时变编队控制[J]. 北京航空航天大学学报, 2020, 46(2): 314-323.
[2]	戴月领, 刘莉, 张晓辉. 燃料电池无人机动力系统半实物仿真[J]. 北京航空航天大学学报, 2020, 46(2): 439-446.
[3]	张鹏, 张嘉峰, 刘涛. 基于相干度优化的极化顺轨干涉SAR慢小目标CFAR检测[J]. 北京航空航天大学学报, 2019, 45(3): 575-587.
[4]	赵蓬辉, 孟春宁, 常胜江. 基于异步卷积分解与分流结构的单阶段检测器[J]. 北京航空航天大学学报, 2019, 45(10): 2089-2098.
[5]	郭琳, 秦世引. 遥感图像飞机目标高效搜检深度学习优化算法[J]. 北京航空航天大学学报, 2019, 45(1): 159-173.
[6]	林翊钧, 吴凤鸽, 赵军锁. 基于图像分割和密度聚类的遥感动目标分块提取[J]. 北京航空航天大学学报, 2018, 44(12): 2510-2520.
[7]	郑毓轩, 李云松, 师艳子, 曲家慧, 谢卫莹. 基于FPGA的高光谱异常目标检测RXD算法加速方案[J]. 北京航空航天大学学报, 2018, 44(12): 2556-2567.
[8]	张帅, 李学仁, 张鹏, 张建业, 白晗, 赵晓林. 基于人机合作的无人机实时航迹规划[J]. 北京航空航天大学学报, 2017, 43(4): 814-822.
[9]	张文秋, 丁文锐, 刘春辉. 一种无人机数据链信道选择和功率控制方法[J]. 北京航空航天大学学报, 2017, 43(3): 583-591.
[10]	周绍磊, 康宇航, 史贤俊, 戴邵武, 周超. 基于RQPSO-DMPC的多无人机编队自主重构控制方法[J]. 北京航空航天大学学报, 2017, 43(10): 1960-1971.
[11]	方旭, 刘金琨. 四旋翼无人机动态面控制[J]. 北京航空航天大学学报, 2016, 42(8): 1777-1784.
[12]	吴俊成, 周锐, 董卓宁, 车军. 基于诱导航线的多无人机编队飞行控制方法[J]. 北京航空航天大学学报, 2016, 42(7): 1518-1525.
[13]	何仁珂, 魏瑞轩, 张启瑞, 许卓凡. 基于拟态电势能的飞行器航路规划方法[J]. 北京航空航天大学学报, 2016, 42(7): 1543-1549.
[14]	祁晓明, 魏瑞轩, 周凯. 面向多威胁的无人机智能目标跟随策略设计[J]. 北京航空航天大学学报, 2016, 42(4): 780-788.
[15]	张帅, 李学仁, 张建业, 张鹏, 李博, 赵晓林. 基于动态步长的无人机三维实时航迹规划[J]. 北京航空航天大学学报, 2016, 42(12): 2745-2754.