A DINO remote sensing target detection algorithm combining efficient hybrid encoder and structural reparameterization
-
摘要:
应用于遥感图像检测的DINO算法最近受到广泛关注。遥感目标检测任务要求算法具备细粒度特征提取和广阔的空间搜索能力，但DINO算法的多层编解码结构会极大地增加算法的空间和运算复杂度，也不利于快速推理。基于此，提出改进DINO算法，从并行的大核卷积结构更适合遥感图像处理的事实出发，设计单层的高效混合编码器结构，提高了DINO算法的模型参数效率；在高效混合编码器结构内部，重新设计高效混合编码器，该编码器基于高效的卷积运算，采用结构参数化技术，使模型推理的参数量和计算量得到一定程度上的降低，进而降低了DINO算法的推理延迟。在NWPU和DOTA数据集上实验表明，改进DINO算法平均精度均值(mAP)相较原算法有1.8%、3.8%的微弱提升。值得关注的是，改进DINO算法大幅度降低了模型参数量和计算复杂度。相较目前主流算法，改进DINO算法在保持较好精度的情况下，参数量、计算复杂度和推理延迟方面取得较好的效果。
Abstract:The DINO algorithm, tailored for remote sensing image detection, has garnered significant traction in recent research circles. Remote sensing object detection mandates algorithms to excel in both fine-grained feature extraction and large-scale spatial search capabilities. However, an improved DINO algorithm is proposed, the inherent multi-layer encoder-decoder architecture of the vanilla DINO algorithm incurs substantial spatial and computational overhead, posing notable impediments to real-time inference performance. To address these limitations, this study capitalizes on the inherent suitability of parallel large-kernel convolution structures for remote sensing image processing, proposing a single-layer efficient hybrid encoder architecture to enhance the parameter efficiency of the DINO algorithm framework. Within this novel encoder structure, we redesign the core module based on high-efficiency convolution operations and integrate structural parameterization techniques. This design strategically reduces both the number of trainable parameters and floating-point operations during inference, thereby effectively mitigating the latency bottleneck of the original DINO algorithm. Experimental evaluations on the NWPU VHR and DOTA datasets demonstrate that the improved DINO algorithm achieves marginal yet consistent performance gains, with mean average precision (mAP) improvements of 1.8% and 3.8% respectively compared to the baseline. Most notably, the proposed modifications yield substantial reductions in model size and computational complexity. When benchmarked against state-of-the-art remote sensing detection algorithms, the improved DINO algorithm maintains competitive detection accuracy while outperforming counterparts in terms of parameter efficiency, computational cost, and inference speed.
-
Key words:
- DINO algorithm /
- structural reparameterization /
- gradient path /
- hybrid coder /
- remote sensing image
-
图 4 不同网络架构的热力图映射可视化结果
Figure 4. Visualization results of heat map mapping of different network architectures
图 6 改进的DINO算法整体结构示意图
Figure 6. Overall structure diagram of the improved DINO algorithm
图 7 本文改进DINO算法检测结果可视化
Figure 7. Visualization of detection results of the improved DINO algorithm
图 8 改进DINO算法与其他主流算法结果对比
Figure 8. The improved DINO algorithm is compared with other mainstream algorithms
图 9 改进DINO算法与DINO算法、RT-DETR算法的检测结果对比
Figure 9. Comparison of detection results of the improved DINO algorithm with DINO and RT-DETR algorithms
表 1 各算法性能比较
Table 1. Performance comparison of each algorithm
网络架构 算法 参数量 浮点计算量 延迟/ms mAP50/% mAP50-95/% mAP50*/% mAP50-95*/% CNN Faster RCNN[24] 137.1×106 370×109 60.2 88.80 58.9 Yolox-l[3] 54.2×106 155.6×109 14.6 95.60 60.2 Yolov7[4] 37.6×106 106.5×109 15.8 96.50 62.3 Yolov8m[9] 25.9×106 79.1×109 14.2 94.20 61.5 82.5 59.0 Transformer DETR[8] 42.8×106 89.2×109 125.0 89.37 56.7 DINO[7] 41.2×106 240×109 42.6 94.70 60.2 80.4 57.1 RT-DETR[8] 29.3×106 105.2×109 22.0 95.30 63.2 82.0 58.4 改进DINO算法 21.3×106 64.8×109 13.5 96.30 64.0 83.7 59.7 注：*表示在DOTA数据集的测试结果。 
下载: 导出CSV
表 2 改进DINO算法的消融实验
Table 2. Ablation experiment of the improved DINO algorithm
架构 参数量 浮点计算量 延迟/ms mAP50/% mAP50*/% mAP50-95*/% DINO(HGNet) 41.2×106 240×109 42.6 94.7 80.4 57.1 DINO(HGNet)+ EHE(C2F) 29.3×106 105.2×109 22 95.3 82.3 59.0 DINO(HGNet)+REPEHE 21.3×106 64.8×109 13.5 96.3 83.7 59.7 注：*表示在DOTA数据集的测试结果。
下载: 导出CSV
表 3 由主流卷积模块构成的混合编码器在NWPU数据集的性能
Table 3. Performance comparison of hybrid encoders composed of mainstream convolution modules on NWPU
网络架构 参数量 浮点计算量 延迟/ms mAP50/% DINO(HGNet) 41.2×106 240×109 42.6 94.7 DETR(HGNet)+EHE (ELAN) 31.9×106 118.3×109 15.2 93.8 DETR(HGNet)+ EHE(C3) 32.3×106 121.8×109 16.2 94.1 DETR+EHE+(C2F) 29.3×106 105.2×109 22.0 95.3 DETR(HGNet)+REPEHE 21.3×106 64.8×109 13.5 96.3
下载: 导出CSV
-
[1] 成科扬, 荣兰, 蒋森林, 等. 面向目标检测的双驱自适应遥感图像超分重建方法[J]. 北京航空航天大学学报, 2022, 48(8): 1343-1352.Cheng K Y, Rong L, Jiang S L, et al. Double drive adaptive super-resolution reconstruction method of remote sensing images for object detection[J]. Journal of Beijing University of Aeronautics and Astronautics, 2022, 48(8): 1343-1352(in Chinese). [2] Dai X Y, Chen Y P, Xiao B, et al. Dynamic head: unifying object detection heads with attentions[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2021: 7369-7378. [3] Ge Z, Liu S T, Wang F, et al. YOLOX: exceeding YOLO series in 2021[EB/OL]. (2021-07-18)[2024-04-20]. https://arxiv.org/abs/2107.08430. [4] Wang C Y, Bochkovskiy A, Liao H M. YOLOv7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2023: 7464-7475. [5] Chen K, Pang J M, Wang J Q, et al. Hybrid task cascade for instance segmentation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2020: 4969-4978. [6] Carion N, Massa F, Synnaeve G, et al. End-to-end object detection with transformers[C]//Proceedings of the Computer Vision-ECCV. Berlin: Springer, 2020: 213-229. [7] Zhang H, Li F, Liu S L, et al. DINO: DETR with improved DeNoising anchor boxes for end-to-end object detection[EB/OL]. (2022-03-07)[2024-04-20]. https://arxiv.org/abs/2203.03605. [8] Zhao Y A, Lv W Y, Xu S L, et al. DETRs beat YOLOs on real-time object detection[EB/OL]. (2023-04-17)[2024-04-20]. https://arxiv.org/abs/2304.08069. [9] Muhammad Yaseen M. What is YOLOv8: an in-depth exploration of the internal features of the next-generation object detector[EB/OL]. (2024-08-28)[2024-08-30]. https://arxiv.org/abs/2408.15857. [10] Zhu X Z, Su W J, Lu L W, et al. Deformable DETR: deformable transformers for end-to-end object detection[EB/OL]. (2020-10-08)[2024-04-20]. https://arxiv.org/abs/2010.04159. [11] Chen X K, Wei F Y, Zeng G, et al. Conditional DETR V2: efficient detection transformer with box queries[EB/OL]. (2022-07-18)[2024-04-21]. https://arxiv.org/abs/2207.08914. [12] Chen Q, Chen X K, Wang J, et al. Group DETR: fast DETR training with group-wise one-to-many assignment[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE Press, 2024: 6610-6619. [13] Liu S L, Li F, Zhang H, et al. DAB-DETR: dynamic anchor boxes are better queries for DETR[EB/OL]. (2022-01-28)[2024-04-21]. https://arxiv.org/abs/2201.12329. [14] Ranftl R, Bochkovskiy A, Koltun V. Vision transformers for dense prediction[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE Press, 2022: 12159-12168. [15] Zheng S X, Lu J C, Zhao H S, et al. Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2021: 6877-6886. [16] Liu Z, Mao H Z, Wu C Y, et al. A ConvNet for the 2020s[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2022: 11966-11976. [17] Ding X H, Zhang X Y, Han J G, et al. Scaling up your kernels to 31x31: revisiting large kernel design in CNNs[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2022: 11963-11975. [18] Hou Q B, Lu C Z, Cheng M M, et al. Conv2Former: a simple transformer-style ConvNet for visual recognition[EB/OL]. (2022-11-22)[2024-04-21]. https://arxiv.org/abs/2211.11943. [19] Ding X H, Zhang X Y, Ma N N, et al. RepVGG: making VGG-style ConvNets great again[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2021: 13728-13737. [20] Hatamizadeh A, Heinrich G, Yin H X, et al. FasterViT: fast vision transformers with hierarchical attention[EB/OL]. (2023-06-09)[2024-04-21]. https://arxiv.org/abs/2306.06189. [21] Ding X H, Zhang Y Y, Ge Y X, et al. UniRepLKNet: a universal perception large-kernel ConvNet for audio, video, point cloud, time-series and image recognition[EB/OL]. (2023-11-27)[2024-04-22]. https://arxiv.org/abs/2311.15599. [22] Wang C Y, Yeh I H, Liao H M. YOLOv9: learning what you want to learn using programmable gradient information[EB/OL]. (2024-02-21)[2024-04-22]. https://arxiv.org/abs/2402.13616. [23] Chen J R, Kao S H, He H, et al. Run, don’t walk: chasing higher FLOPS for faster neural networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2023: 12021-12031. [24] Girshick R. Fast R-CNN[C]//Proceedings of the IEEE International Conference on Computer Vision. Piscataway: IEEE Press, 2016: 1440-1448. [25] Wu W T, Liu H, Li L L, et al. Application of local fully convolutional neural network combined with YOLOv5 algorithm in small target detection of remote sensing image[J]. PLOS One, 2021, 16(10): e0259283. -
下载:
点击查看大图
计量
- 文章访问数: 400
- HTML全文浏览量: 230
- PDF下载量: 18
- 被引次数: 0
