面向目标检测的双驱自适应遥感图像超分重建方法

成科扬; 荣兰; 蒋森林; 詹永照

doi:10.13700/j.bh.1001-5965.2021.0517

面向目标检测的双驱自适应遥感图像超分重建方法

doi: 10.13700/j.bh.1001-5965.2021.0517

成科扬^{1, 2, 3, ,},
荣兰¹,
蒋森林¹,
詹永照¹

1.
江苏大学计算机科学与通信工程学院，镇江 212013
2.
江苏省大数据泛在感知与智能农业应用工程研究中心，镇江 212013
3.
镇江昭远智能科技有限公司，镇江 212013

基金项目:

国家自然科学基金 61972183

江苏省科技项目 BE2022781

镇江市“金山英才”高层次领军人才培养计划培养对象科研项目

详细信息

通讯作者:
成科扬, E-mail: kycheng@ujs.edu.cn

中图分类号: V221⁺.3;TB553
计量
- 文章访问数: 435
- HTML全文浏览量: 183
- PDF下载量: 114
- 被引次数: 0
出版历程
- 收稿日期: 2021-09-06
- 录用日期: 2021-09-17
- 网络出版日期: 2021-10-29
- 整期出版日期: 2022-08-20

Double drive adaptive super-resolution reconstruction method of remote sensing images for object detection

1.
School of Computer Science and Communication Engineering, Jiangsu University, Zhenjiang 212013, China
2.
Jiangsu Province Big Data Ubiquitous Perception and Intelligent Agricultural Application Engineering Research Center, Zhenjiang 212013, China
3.
Zhenjiang Zhaoyuan Intelligent Technology Co., Ltd., Zhenjiang 212013, China

Funds:

National Natural Science Foundation of China 61972183

Jiangsu Science and Technology Project BE2022781

Zhenjiang Jinshan High-level Leading Talents Training Plan Scientific Research Project

More Information

Corresponding author: CHENG Keyang, E-mail: kycheng@ujs.edu.cn

摘要

摘要:
有光学遥感图像超分重建方法主要是生成视觉上令人满意的图像，并未考虑后续目标检测任务的特殊性，不能有效地应用到目标检测中。基于此，提出了面向目标检测的双驱动自适应多尺度光学遥感图像超分重建方法，将超分重建网络和目标检测网络结合起来，进行联合优化。针对光学遥感图像的特点设计了自适应多尺度遥感图像超分重建网络，集成选择性内核网络和自适应特征门控单元来特征提取和融合，重建出初步遥感图像。通过提出的双驱动模块，将特征先验驱动损失和任务驱动损失传到超分重建网络中，提高目标检测的性能。在UCAS-AOD和NWPU VHR-10数据集上进行实验，并与5种主流方法进行比较，所提方法的峰值信噪比和平均准确率相较于FDSR方法分别提高了1.86 dB和3.73%。实验结果表明，所提方法和光学遥感图像目标检测结合可以取得更好的效果，综合性能更佳。
- 遥感图像超分重建 /
- 目标检测 /
- 多尺度 /
- 特征先验驱动 /
- 任务驱动
Abstract:
The existing optical remote sensing image super-resolution reconstruction method is mainly to generate visually satisfactory images, and does not take into account the particularity of the subsequent target detection task, so it cannot be effectively applied to target detection. Therefore, a double drive adaptive multi-scale optical remote sensing image super-resolution reconstruction method for target detection is proposed. The super-resolution reconstruction network and target detection network are combined for joint optimization. According to the characteristics of optical remote sensing image, an adaptive multi-scale remote sensing image super-partition reconstruction network is designed. The selective kernel network and adaptive gating unit are integrated to extract and fuse features, and the primary remote sensing image is reconstructed. Through the double drive module, and task driven will feature a priori driver loses to the above points in the network, on the one hand, improve the performance of target detection. The proposed method was tested on UCAS-AOD and NWPU VHR-10 datasets, and compared with the five mainstream algorithms, peak signal-to-noise ratio and average accuracy improved by 1.86 dB and 3.73%, respectively, compared with the FDSR algorithm. Experimental results show that compared with other methods, the combination of the proposed algorithm and optical remote sensing image target detection can achieve better results and the comprehensive performance is the best.
- super-resolution reconstruction of remote sensing images /
- object detection /
- multi-scale /
- features priori driven /
- task driven

HTML全文

图 1 整体网络结构

Figure 1. Overall network structure

下载: 全尺寸图片幻灯片

图 2 自适应多尺度特征提取块结构

Figure 2. Adaptive multi-scale feature extraction block structure

下载: 全尺寸图片幻灯片

图 3 自适应特征门控单元结构

Figure 3. Adaptive feature gating structure

下载: 全尺寸图片幻灯片

图 4 不同λ₁和λ₂下本文方法的检测性能

Figure 4. Proposed method detects performance under different λ₁ and λ₂

下载: 全尺寸图片幻灯片

图 5 检测效果对比示例

Figure 5. Example of detection effect comparison

下载: 全尺寸图片幻灯片

表 1 消融实验

Table 1. Ablation test

实验	PSRN/dB	mAP/%
FR-CNN(HR)		70.43
FR-CNN(LR)		40.32
AMNN+FR-CNN (无联合训练)	27.75	55.34
AMNN+FR-CNN+TD (联合训练)	28.06	65.33
AMNN+FR-CNN+TD+FPD (联合训练)	28.79	69.13

下载: 导出CSV

表 2 不同方法在UCAS-AOD数据集飞机类上的实验效果比较

Table 2. Experimental results compared with different methods on UCAS-AOD dataset

方法	PSNR/dB	AP/%	AP_0.5/%	AP_0.75/%	AP_S/%	AP_M/%	AP_L/%
原始图像		47.6	59.2	41.1	21.5	48.5	58.7
Bicubicu^[19]	25.89	22.14	37.9	23.01	6.71	23.46	38.15
MSRN^[20]	28.15	23.45	44.5	24.78	8.83	25.67	43.63
TDSR^[10]	27.34	26.34	46.98	27.78	9.04	28.84	45.96
AMFFN^[7]	28.56	24.78	44.54	25.01	8.92	25.65	43.98
FDSR^[21]	27.56	26.33	46.78	27.66	9.15	29.03	45.97
本文方法	28.97	44.89	55.02	37.32	20.3	38.45	57.87
注：黑体数据表示最优结果。

下载: 导出CSV

表 3 不同方法在UCAS-AOD数据集上的实验效果比较

Table 3. Experimental results compared with different methods on UCAS-AOD dataset

方法	PSNR/dB	mAP/%
Bicubicu^[19]	25.95	48.85
MSRN^[20]	27.55	58.97
AMFFN^[7]	27.56	59.23
TDSR^[10]	26.43	62.96
FDSR^[21]	26.65	63.98
本文方法	28.75	69.67
注：黑体数据表示最优结果。

下载: 导出CSV

表 4 不同方法在NWPU VHR-10数据集上的实验效果比较

Table 4. Experimental results compared with different methods on NWPU VHR-10 dataset

方法	PSNR/dB	mAP/%
Bicubicu^[19]	24.86	47.56
MSRN^[20]	27.01	57.98
AMFFN^[7]	27.12	58.32
TDSR^[10]	26.82	62.97
FDSR^[21]	26.96	63.84
本文方法	28.58	68.61
注：黑体数据表示最优结果。

下载: 导出CSV

参考文献(21)

[1]	TSAGKATAKIS G, AIDINI A, FOTIADOU K, et al. Survey of deep-learning approaches for remote sensing observation enhancement[J]. Sensors, 2019, 19(18): 3929. doi: 10.3390/s19183929
[2]	LEI S, SHI Z W, ZOU Z X. Super-resolution for remote sensing images via local-global combined network[J]. IEEE Geoscience and Remote Sensing Letters, 2017, 14(8): 1243-1247. doi: 10.1109/LGRS.2017.2704122
[3]	JIANG K, WANG Z, YI P, et al. Deep distillation recursive network for remote sensing imagery super-resolution[J]. Remote Sensing, 2018, 10(11): 1700. doi: 10.3390/rs10111700
[4]	XU W J, XU G L, WANG Y, et al. High quality remote sensing image super-resolution using deep memory connected network[C]//IEEE International Geoscience and Remote Sensing Symposium. Piscataway: IEEE Press, 2018: 8889-8892.
[5]	GU J, SUN X, ZHANG Y, et al. Deep residual squeeze and excitation network for remote sensing image super-resolution[J]. Remote Sensing, 2019, 11(15): 1817. doi: 10.3390/rs11151817
[6]	LU T, WANG J M, ZHANG Y D, et al. Satellite image super-resolution via multi-scale residual deep neural network[J]. Remote Sensing, 2019, 11(13): 1588. doi: 10.3390/rs11131588
[7]	WANG X Y, WU Y D, MING Y, et al. Remote sensing imagery super resolution based on adaptive multi-scale feature fusion network[J]. Sensors, 2020, 20(4): 1142. doi: 10.3390/s20041142
[8]	KOESTER E, SAHIN C S. A comparison of super-resolution and nearest neighbors interpolation applied to object detection on satellite data[EB/OL]. (2019-07-08)[2021-09-01]. https://arxiv.org/abs/1907.05283.
[9]	CAO L J, WANG C, LI J. Vehicle detection from highway satellite images via transfer learning[J]. Information Sciences, 2016, 366: 177-187. doi: 10.1016/j.ins.2016.01.004
[10]	HARIS M, SHAKHNAROVICH G, UKITA N. Task-driven super resolution: Object detection in low-resolution images[C]//International Conference on Neural Information Processing. Berlin: Springer, 2021: 387-395.
[11]	HARIS M, SHAKHNAROVICH G, UKITA N. Deep back-projection networks for super-resolution[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2018: 1664-1673.
[12]	RABBI J, RAY N, SCHUBERT M, et al. Small-object detection in remote sensing images with end-to-end edge-enhanced GAN and object detector network[EB/OL]. (2020-04-28)[2021-09-01]. https://arxiv.org/abs/2003.09085.
[13]	JI H, GAO Z, MEI T C, et al. Vehicle detection in remote sensing images leveraging on simultaneous super-resolution[J]. IEEE Geoscience and Remote Sensing Letters, 2020, 17(4): 676-680. doi: 10.1109/LGRS.2019.2930308
[14]	LI X, WANG W H, HU X L, et al. Selective kernel networks[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2019: 510-519.
[15]	HE K, GKIOXARI G, DOLLÁR P, et al. Mask R-CNN[C]//Proceedings of the IEEE International Conference on Computer Vision. Piscataway: IEEE Press, 2017: 2961-2969.
[16]	REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149. doi: 10.1109/TPAMI.2016.2577031
[17]	ZHU H G, CHEN X G, DAI W Q, et al. Orientation robust object detection in aerial images using deep convolutional neural network[C]//2015 IEEE International Conference on Image Processing. Piscataway: IEEE Press, 2015: 3735-3739.
[18]	CHENG G, ZHOU P C, HAN J W. Learning rotation-invariant convolutional neural networks for object detection in VHR optical remote sensing images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2016, 54(12): 7405-7415. doi: 10.1109/TGRS.2016.2601622
[19]	KEYS R. Cubic convolution interpolation for digital image processing[J]. IEEE Transactions on Acoustics, Speech, and Signal Processing, 1981, 29(6): 1153-1160. doi: 10.1109/TASSP.1981.1163711
[20]	LI J, FANG F, MEI K, et al. Multi-scale residual network for image super-resolution[C]//Proceedings of the European Conference on Computer Vision (ECCV). Berlin: Springer, 2018: 517-532.
[21]	WANG B, LU T, ZHANG Y D. Feature-driven super-resolution for object detection[C]//2020 5th International Conference on Control, Robotics and Cybernetics (CRC). Piscataway: IEEE Press, 2020: 211-215.