留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于倒置残差注意力的无人机航拍图像小目标检测

刘树东 刘业辉 孙叶美 李懿霏 王娇

刘树东,刘业辉,孙叶美,等. 基于倒置残差注意力的无人机航拍图像小目标检测[J]. 北京航空航天大学学报,2023,49(3):514-524 doi: 10.13700/j.bh.1001-5965.2022.0362
引用本文: 刘树东,刘业辉,孙叶美,等. 基于倒置残差注意力的无人机航拍图像小目标检测[J]. 北京航空航天大学学报,2023,49(3):514-524 doi: 10.13700/j.bh.1001-5965.2022.0362
LIU S D,LIU Y H,SUN Y M,et al. Small object detection in UAV aerial images based on inverted residual attention[J]. Journal of Beijing University of Aeronautics and Astronautics,2023,49(3):514-524 (in Chinese) doi: 10.13700/j.bh.1001-5965.2022.0362
Citation: LIU S D,LIU Y H,SUN Y M,et al. Small object detection in UAV aerial images based on inverted residual attention[J]. Journal of Beijing University of Aeronautics and Astronautics,2023,49(3):514-524 (in Chinese) doi: 10.13700/j.bh.1001-5965.2022.0362

基于倒置残差注意力的无人机航拍图像小目标检测

doi: 10.13700/j.bh.1001-5965.2022.0362
基金项目: 天津市科技计划(20YDTPJC01310)
详细信息
    作者简介:

    刘树东,等:基于倒置残差注意力的无人机航拍图像小目标检测

    通讯作者:

    E-mail:wangjiaoq@163.com

  • 中图分类号: TP391.4

Small object detection in UAV aerial images based on inverted residual attention

Funds: Science and Technology Program of Tianjin (20YDTPJC01310)
More Information
  • 摘要:

    针对无人机航拍图像背景复杂、小尺寸目标较多等问题,提出了一种基于倒置残差注意力的无人机航拍图像小目标检测算法。在主干网络部分嵌入倒置残差模块与倒置残差注意力模块,利用低维向高维的特征信息映射,获得丰富的小目标空间信息和深层语义信息,提升小目标的检测精度;在特征融合部分设计多尺度特征融合模块,融合浅层空间信息和深层语义信息,并生成4个不同感受野的检测头,提升模型对小尺寸目标的识别能力,减少小目标的漏检;设计马赛克混合数据增强方法,建立数据之间的线性关系,增加图像背景复杂度,提升算法的鲁棒性。在VisDrone数据集上的实验结果表明:所提模型的平均精度均值比DSHNet模型提升了1.2%,有效改善了无人机航拍图像小目标漏检、误检的问题。

     

  • 图 1  基于倒置残差注意力的无人机航拍图像小目标检测模型结构

    Figure 1.  Structure of small object detection in UAV aerial image based on inverted residual attention

    图 2  IRC3模块

    Figure 2.  IRC3 module

    图 3  IRAC3模块

    Figure 3.  IRAC3 module

    图 4  深度可分离卷积模块

    Figure 4.  Depthwise separable convolution module

    图 5  ECA-Net模块

    Figure 5.  ECA-Net module

    图 6  无人机航拍图像

    Figure 6.  UAV aerial image

    图 7  目标分布图像

    Figure 7.  Object distribution image

    图 8  融合增强方法过程

    Figure 8.  Fusion enhancement method process

    图 9  不同模型检测结果

    Figure 9.  Detection results of different models

    图 10  三种模型检测结果

    Figure 10.  Detection results of three models

    图 11  本文模型在不同背景下的检测结果

    Figure 11.  Detection results of the proposed model under different backgrounds

    表  1  不同模型的客观指标对比

    Table  1.   Comparison of objective indicators of different models

    模型mAP/%mAP0.5/%mAP0.75/%参数量/106检测速度/FPS
    YOLOv5x23.435.725.183.241.3
    模型 124.638.626.286.728.7
    模型 225.439.727.185.528.7
    模型 326.841.428.869.325.6
    模型 427.442.429.072.523.4
    下载: 导出CSV

    表  2  不同算法的检测结果对比

    Table  2.   Comparison of detection results of different algorithms

    算法backbonemAP/%AP/%
    PedestrianPersonBicycleCarVanTruckTricycleAwning-tricycleBusMotor
    RetinaNetR5013.913.07.91.445.519.911.56.34.217.811.8
    Faster R-CNNX10122.421.315.57.952.029.520.514.78.932.121.6
    Cascade R-CNNR5023.222.214.87.654.631.521.614.88.634.921.4
    Faster R-CNN+MMFR5022.621.615.39.651.528.520.415.97.533.721.6
    Faster R-CNN+SimCalR5020.018.713.85.751.028.416.413.65.927.019.4
    Faster R-CNN +BGSR5023.021.816.08.151.831.119.815.08.436.121.5
    RetinaNet+DSHNetR5016.114.18.91.348.224.814.28.86.021.613.1
    Faster R-CNN+DSHNetR5024.622.516.510.152.832.622.117.58.839.523.7
    Faster R-CNN+DSHNetX10125.823.316.711.453.733.123.819.511.140.025.5
    Cascade R-CNN+DSHNetR5026.223.216.111.255.533.525.219.110.043.025.1
    本文模型CSPDarknet5327.428.96.09.560.136.234.516.717.247.218.0
    下载: 导出CSV

    表  3  不同算法的平均精度均值与检测速度结果对比

    Table  3.   Comparison of average accuracy and detection speed of different algorithms

    算法backbonemAP/%检测速度/FPS
    RetinaNet+DSHNetR5016.119
    Faster R-CNN+DSHNetR5024.622.5
    Cascade R-CNN+DSHNetR5026.215
    本文模型CSPDarknet5327.423.4
    下载: 导出CSV
  • [1] WU X, LI W, HONG D, et al. Deep learning for UAV-based object detection and tracking: A survey[EB/OL]. (2021-10-25)[2022-05-01].https://arxiv.org/abs/2110.12638.
    [2] GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2014: 580-587.
    [3] GIRSHICK R. Fast R-CNN[C]//Proceedings of the IEEE International Conference on Computer Vision. Piscataway: IEEE Press, 2015: 1440-1448.
    [4] REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: Towards real-time object detection with region proposal networks[C]//Proceedings of the 28th International Conference on Neural Information Processing Systems. Piscataway: IEEE Press, 2015: 91-99.
    [5] HE K, GKIOXARI G, DOLLÁR P, et al. Mask R-CNN[C]//Proceedings of the IEEE International Conference on Computer Vision. Piscataway: IEEE Press, 2017: 2961-2969.
    [6] CAI Z, VASCONCELOS N. Cascade R-CNN: Delving into high quality object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2018: 6154-6162.
    [7] REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: Unified, real time object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2016: 779-788.
    [8] REDMON J, FARHADI A. YOLO9000: Better, faster, stronger[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2017: 7263-7271.
    [9] REDMON J, FARHADI A. YOLOv3: An incremental improvement[EB/OL]. (2018-04-08)[2022-05-01].https://arxiv.org/abs/1804.02767.
    [10] BOCHKOVSKIY A, WANG C Y, LIAO H Y M. YOLOv4: Optimal speed and accuracy of object detection[EB/OL]. (2020-04-23)[2022-05-01].https://arxiv.org/abs/2004. 10934?sid=NDAqzT.
    [11] LIU W, ANGUELOV D, ERHAN D, et al. SSD: Single shot multibox detector[C]//Proceedings of the European Conference on Computer Vision. Berlin: Springer, 2016: 21-37.
    [12] LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[C]//Proceedings of the IEEE International Conference on Computer Vision. Piscataway: IEEE Press, 2017: 2980-2988.
    [13] LIU M, WANG X, ZHOU A, et al. UAV-YOLO: Small object detection on unmanned aerial vehicle perspective[J]. Sensors, 2020, 20(8): 2238. doi: 10.3390/s20082238
    [14] LIANG X, ZHANG J, ZHUO L, et al. Small object detection in unmanned aerial vehicle images using feature fusion and scaling-based single shot detector with spatial context analysis[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2019, 30(6): 1758-1770.
    [15] ZHANG P, ZHONG Y, LI X. SlimYOLOv3: Narrower, faster and better for real-time UAV applications[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE Press, 2019: 37-45.
    [16] 裴伟, 许晏铭, 朱永英, 等. 改进的 SSD 航拍目标检测方法[J]. 软件学报, 2019, 30(3): 738-758.

    PEI W, XU Y M, ZHU Y Y, et al. The target detection method of aerial photography images with improved SSD[J]. Journal of Software, 2019, 30(3): 738-758(in Chinese).
    [17] 刘婷婷, 苗华, 李琳, 等. 融合场景上下文的轻量级目标检测网络[J]. 激光与光电子学进展, 2021, 58(20): 127-135.

    LIU T T, MIAO H, LI L, et al. Lightweight target detection network integrating scene context[J]. Laser & Optoelectronics Progress, 2021, 58(20): 127-135(in Chinese).
    [18] HOWARD A, SANDLER M, CHEN B, et al. Searching for mobileNetV3[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE Press, 2019: 1314-1324.
    [19] TAN M, LE Q. EfficientNetV2: Smaller models and faster training[EB/OL](2021-06-23)[2022-05-01].https://arxiv.org/abs/2104.00298v2.
    [20] 刘艳菊, 王秋霁, 赵开峰, 等. 基于卷积神经网络的热轧钢条表面实时缺陷检测[J]. 仪器仪表学报, 2021, 42(12): 211-219. doi: 10.19650/j.cnki.cjsi.J2108078

    LIU Y J, WANG Q J, ZHAO K F, et al. Real-time defect detection of hot rolling steel bar based on convolution neural network[J]. Chinese Journal of Scientific Instrument, 2021, 42(12): 211-219(in Chinese). doi: 10.19650/j.cnki.cjsi.J2108078
    [21] 周中, 张俊杰, 龚琛杰, 等. 基于深度语义分割的隧道渗漏水智能识别[J]. 岩石力学与工程学报, 2022, 41(10): 2082-2093. doi: 10.13722/j.cnki.jrme.2022.0016

    ZHOU Z, ZHANG J J, GONG C J, et al. Automatic identification of tunnel leakage based on deep semantic segmentation[J]. Chinese Journal of Rock Mechanics and Engineering, 2022, 41(10): 2082-2093(in Chinese). doi: 10.13722/j.cnki.jrme.2022.0016
    [22] WANG Q, WU B, ZHU P, et al. ECA-Net: Efficient channel attention for deep convolutional neural networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2020: 11534-11542.
    [23] HOWARD A G, ZHU M, CHEN B, et al. MobileNets: Efficient convolutional neural networks for mobile vision applications[EB/OL]. (2017-04-17)[2022-05-01].https://arxiv.org/abs/1704.04861.
    [24] HUANG G, SUN Y, LIU Z, et al. Deep networks with stochastic depth[C]//Proceedings of the European Conference on Computer Vision. Berlin: Springer, 2016: 646-661.
    [25] ZHANG H, CISSE M, DAUPHIN Y N, et al. Mixup: Beyond empirical risk minimization[EB/OL]. (2018-04-27)[2022-05-01].https://arxiv.org/abs/1710.09412.
    [26] ZHANG X, IZQUIERDO E, CHANDRAMOULI K. Dense and small object detection in UAV vision based on cascade network[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE Press, 2019: 118-126.
    [27] WANG T, LI Y, KANG B, et al. The devil is in classification: A simple framework for long-tail instance segmentation[C]//Proceedings of the European Conference on Computer Vision. Berlin: Springer, 2020: 728-744.
    [28] LI Y, WANG T, KANG B, et al. Overcoming classifier imbalance for long-tail object detection with balanced group softmax[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2020: 10991-11000.
    [29] YU W P, YANG T J N, CHEN C. Towards resolving the challenge of long-tail distribution in UAV images for object detection[C]//Proceedings of the 2021 IEEE Winter Conference on Applications of Computer Vision. Piscataway: IEEE Press, 2021: 3258-3267.
  • 加载中
图(11) / 表(3)
计量
  • 文章访问数:  519
  • HTML全文浏览量:  145
  • PDF下载量:  124
  • 被引次数: 0
出版历程
  • 收稿日期:  2022-05-16
  • 录用日期:  2022-08-19
  • 网络出版日期:  2022-10-18
  • 整期出版日期:  2023-03-30

目录

    /

    返回文章
    返回
    常见问答