一种FPGA实现的复杂背景红外小目标检测网络

周海; 侯晴宇; 卞春江; 冯水春; 刘一腾

doi:10.13700/j.bh.1001-5965.2021.0221

一种FPGA实现的复杂背景红外小目标检测网络

doi: 10.13700/j.bh.1001-5965.2021.0221

周海^{1, 2, 3},
侯晴宇^4, ,,
卞春江^{1, 2, 3},
冯水春^{1, 2, 3},
刘一腾^{1, 2, 3}

1.
中国科学院国家空间科学中心，北京 100190
2.
中国科学院复杂航天系统电子信息技术重点实验室，北京 100190
3.
中国科学院大学，北京 100049
4.
哈尔滨工业大学航天学院，哈尔滨 150001

基金项目: 中国科学院青年创新促进会资助项目(E0293401)

详细信息

通讯作者:
E-mail： houqingyu@126.com

中图分类号: V249.32⁺6；TP391；TN215
计量
- 文章访问数: 44
- HTML全文浏览量: 7
- PDF下载量: 16
- 被引次数: 0
出版历程
- 收稿日期: 2021-04-29
- 录用日期: 2021-06-06
- 刊出日期: 2021-08-09

An infrared small target detection network under various complex backgrounds realized on FPGA

ZHOU Hai^{1, 2, 3},
HOU Qingyu^{4
, ,},
BIAN Chunjiang^{1, 2, 3},
FENG Shuichun^{1, 2, 3},
LIU Yiteng^{1, 2, 3}

1.
National Space Science Center，Chinese Academy of Sciences，Beijing 100190，China
2.
Laboratory of Electronic and Information Technology for Space Systems，Chinese Academy of Sciences，Beijing 100190，China
3.
University of Chinese Academy of Sciences，Beijing 100049，China
4.
School of Astronautics，Harbin Institute of Technology，Harbin 150001，China

Funds: Youth Innovation Promotion Association CAS (E0293401)

More Information

Corresponding author: E-mail：houqingyu@126.com

摘要

摘要:
红外（IR）小目标检测算法具有检测率高、虚警率低、实时性好等优点在红外遥感领域有重要的应用价值。由于复杂背景下小目标对比度低和信噪比（SNR）低，传统红外小目标检测算法难以保证检测性能。在强鲁棒性的红外小目标检测网络（RISTDnet）基础上，面向更为多样的目标结构特征和更高的实时处理性能要求，提出一种增强型红外小目标检测网络（EISTDnet）与其基于现场可编程逻辑门阵列（FPGA）高性能并行处理的算法。EISTDnet构造了手工特征算法与卷积神经网络相结合的多尺度小目标特征提取框架，采用多级展开思路对卷积核尺寸进行归一化设计，并通过数据深度复用和多维循环并行展开有效提高推理阶段实时处理性能。实验结果表明：采用单片FPGA实现的EISTDnet能够快速实时检测复杂背景下不同大小、低信噪比的小目标,与现有5种算法相比在10⁻³低虚警率下平均检测率提升49.5%，与RISTDnet相比，在实时处理速度提高1.33倍的优势下，对低信噪比条状小目标检测率提升29.4%，所提算法具有更好的有效性和鲁棒性。
- 卷积神经网络 /
- 红外小目标 /
- 目标检测 /
- 现场可编程逻辑门阵列 /
- 实时
Abstract:
The infrared (IR) small target detection algorithm with high detection rate, low false alarm rate and good real-time performance has important application value in the field of IR remote sensing. Traditional IR small targets detection algorithms cannot guarantee the detection performance due to the low contrast and low signal-to-noise ratio (SNR) of small targets under various complex backgrounds. Based on robust infrared small target detection network (RISTDnet) proposed, for more diverse target structure characteristics and higher real-time processing performance requirements, an enhanced infrared small target detection network (EISTDnet) and its field programmable logic gate array (FPGA) based high-performance parallel processing method are proposed. In EISTDnet, a multi-scale small target feature extraction framework that combines manual feature methods and convolutional neural networks is constructed, the size of the convolution kernel is normalized by the idea of multi-level expansion, and real-time processing performance in the inference stage is effectively improved through deep data reuse and multi-dimensional loop parallel unfolding. Experimental results show that the EISTDnet realized on a single FPGA can quickly detect small targets with different sizes and low SNR in various complex backgrounds in real time. Compared with the existing 5 algorithms, the average detection rate is increased by 49.5% with a low false alarm rate of 10⁻³. Compared with RISTDnet, the real-time processing speed is increased by 1.33 times, and the detection rate of low SNR strip small targets is increased by 29.4%. EISTDnet has better effectiveness and robustness.
- convolutional neural network /
- infrared small target /
- target detection /
- field programmable logic gate array /
- real time

HTML全文

图 1 低信噪比条状小目标示意图^[17]

Figure 1. Small strip target with low SNR^[17]

下载: 全尺寸图片幻灯片

图 2 EISTDnet定权特征提取卷积核

Figure 2. EISTDnet feature extraction convolution kernel with fixed weight

下载: 全尺寸图片幻灯片

图 3 卷积核尺寸多级串联优化

Figure 3. Multi-stage series optimization of convolution kernel size

下载: 全尺寸图片幻灯片

图 4 EISTDnet网络结构

Figure 4. Network structure of EISTDnet

下载: 全尺寸图片幻灯片

图 5 特征拼接子网络细节

Figure 5. Detail of feature connection sub-network

下载: 全尺寸图片幻灯片

图 6 特征映射层细节

Figure 6. The detail of feature mapping layer

下载: 全尺寸图片幻灯片

图 7 多尺度定权特征提取数据复用关系示意图

Figure 7. Multi-scale feature extraction data reuse relationship with fixed weight

下载: 全尺寸图片幻灯片

图 8 EISTDnet新增定权特征提取数据复用关系示意图

Figure 8. New feature extraction data reuse relationship with fixed weight of EISTDnet

下载: 全尺寸图片幻灯片

图 9 定权特征提取模块FPGA实现框图

Figure 9. FPGA implementation block diagram of fixed-weight feature extraction module

下载: 全尺寸图片幻灯片

图 10 卷积运算4层循环示意图

Figure 10. Four-layer loop diagram of convolution operation

下载: 全尺寸图片幻灯片

图 11 循环一维展开示意图

Figure 11. One-dimensional unfolding loop diagram

下载: 全尺寸图片幻灯片

图 12 EISTDnet二维循环展开示意图

Figure 12. Two-dimensional loop unfolding diagram of EISTDnet

下载: 全尺寸图片幻灯片

图 13 多级缓存示意图

Figure 13. Multi-level cache diagram

下载: 全尺寸图片幻灯片

图 14 循环计算顺序示意图

Figure 14. Diagram of cyclic calculation sequence

下载: 全尺寸图片幻灯片

图 15 变权卷积网络实时高性能并行处理架构示意图

Figure 15. Real-time high-performance parallel processing architecture of variable-weight convolutional networks

下载: 全尺寸图片幻灯片

图 16 测试序列典型帧示意图

Figure 16. Diagram of test sequence typical frames

下载: 全尺寸图片幻灯片

图 17 高性能FPGA处理板

Figure 17. Self-developed image processing board

下载: 全尺寸图片幻灯片

图 18 自测试验证系统

Figure 18. Self-test verification system

下载: 全尺寸图片幻灯片

图 19 不同算法检测结果对比

Figure 19. Detection results of different methods

下载: 全尺寸图片幻灯片

图 20 不同算法ROC曲线对比

Figure 20. ROC curves of different methods

下载: 全尺寸图片幻灯片

图 21 不同信噪比下EISTDnet与RISTDnet条状弱小目标检测率曲线

Figure 21. Stripe dim targets detection rate curves of different SNR between EISTDnet and RISTDnet

下载: 全尺寸图片幻灯片

图 22 EISTDnet网络在FPGA内处理流水线示意图

Figure 22. Schematic diagram of EISTDnet network processing pipeline in FPGA

下载: 全尺寸图片幻灯片

图 23 EISTDnet 的FPGA实现仿真波形

Figure 23. FPGA simulation waveform of EISTDnet

下载: 全尺寸图片幻灯片

图 24 RISTDnet 的FPGA实现仿真波形

Figure 24. FPGA simulation waveform of RISTDnet

下载: 全尺寸图片幻灯片

表 1 EISTDnet网络参数

Table 1. EISTDnet network parameters

层编号	卷积核数量	卷积核尺寸/步长	输出特征图尺寸
特征提取			512×640
Conv1.1	24	3×3/1	512×640
Conv1.2	24	3×3/1	512×640
Conv1.3	24	3×3/1	512×640
Conv1.4	24	3×3/1	512×640
Conv1.5	32	3×3/1	512×640
Pool1		2×2/2	256×320
Conv2.1	32	3×3/1	256×320
Conv2.2	32	3×3/1	256×320
Conv2.3	64	3×3/1	256×320
Pool2		2×2/2	128×160
Conv3.1	64	3×3/1	128×160
Conv3.2	128	3×3/1	128×160
Pool3		2×2/2	64×80
Conv4	256	3×3/1	64×80
Conv5	64	3×3/1	64×80

下载: 导出CSV

表 2 EISTDnet与RISTDnet运算操作数比对

Table 2. Comparison of operands between EISTDnet and RISTDnet

网络改进阶段	运算操作数
RISTDnet	69999411200
EISTDnet强化多尺度特征	90460323840
EISTDnet卷积网络轻量化	33919795200

下载: 导出CSV

表 3 图像读取与运算操作比对

Table 3. Image reading and operation comparison

算法	像素读取次数	运算操作数
优化前	607846400	629473280
优化后	39649280	97648640
降低比例/%	6.52	15.51

下载: 导出CSV

表 4 测试序列典型帧与数量

Table 4. Typical frames and number of test sequences

项目	测试序列 1	测试序列 2	测试序列 3
典型帧	图16(a)	图16(b)	图16(c)
总帧数	1412	1845	1660

下载: 导出CSV

表 5 EISTDnet与RISTDnet目标检测性能对比

Table 5. Comparison of target detection performance between EISTDnet and RISTDnet %

项目	EISTDnet	RISTDnet
测试序列1	66.97	63.81
测试序列2	50.99	48.65
测试序列3	52.76	50.25

下载: 导出CSV

表 6 EISTDnet与RISTDnet条状弱小目标检测性能对比

Table 6. Comparison of stripe dim target detection performance between EISTDnet and RISTDnet

项目	EISTDnet 目标数/检出数	RISTDnet 目标数/检出数
测试序列1	50/31	50/15
测试序列2	50/24	50/12
测试序列3	50/27	50/14
平均检测率/%	56.7	27.3

下载: 导出CSV

表 7 FPGA资源利用率

Table 7. FPGA resource utilization

资源	BRAM	DSP硬核	触发器	查找表
使用资源	2746	3258	755915	360141
可利用资源	2940	3600	866400	433200
利用率/%	93	91	87	83

下载: 导出CSV

参考文献(17)

[1]	CHEN C L P, LI H, WEI Y, et al. A local contrast method for small infrared target detection[J]. IEEE Transactions on Geoscience and Remote Sensing, 2014, 52(1): 574-581. doi: 10.1109/TGRS.2013.2242477
[2]	HAN J, MA Y, ZHOU B, et al. A robust infrared small target detection algorithm based on human visual system[J]. IEEE Geoscience and Remote Sensing Letters, 2014, 11(12): 2168-2172. doi: 10.1109/LGRS.2014.2323236
[3]	WEI Y, YOU X, LI H. Multiscale patch-based contrast measure for small infrared target detection[J]. Pattern Recognition, 2016, 58: 216-226. doi: 10.1016/j.patcog.2016.04.002
[4]	HAN J, LIANG K, ZHOU B, et al. Infrared small target detection utilizing the multiscale relative local contrast measure[J]. IEEE Geoscience and Remote Sensing Letters, 2018, 15(4): 612-616. doi: 10.1109/LGRS.2018.2790909
[5]	YAO S, CHANG Y, QIN X. A coarse-to-fine method for infrared small target detection[J]. IEEE Geoscience and Remote Sensing Letters, 2019, 16(2): 256-260. doi: 10.1109/LGRS.2018.2872166
[6]	HAN J, LIU S, QIN G, et al. A local contrast method combined with adaptive background estimation for infrared small target detection[J]. IEEE Geoscience and Remote Sensing Letters, 2019, 16(9): 1442-1446. doi: 10.1109/LGRS.2019.2898893
[7]	GAO C, MENG D, YANG Y, et al. Infrared patch-image model for small target detection in a single image[J]. IEEE Transactions on Image Processing, 2013, 22(12): 4996-5009. doi: 10.1109/TIP.2013.2281420
[8]	DAI Y, WU Y, SONG Y, et al. Non-negative infrared patch-image model: Robust target-background separation via partial sum minimization of singular values[J]. Infrared Physics and Technology, 2017, 81: 182-194. doi: 10.1016/j.infrared.2017.01.009
[9]	DAI Y, WU Y. Reweighted infrared patch-tensor model with both nonlocal and local priors for single-frame small target detection[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2017, 10(8): 3752-3767. doi: 10.1109/JSTARS.2017.2700023
[10]	ZHANG L, PENG L, ZHANG T, et al. Infrared small target detection via non-convex rank approximation minimization joint l2, 1 norm[J]. Remote Sensing, 2018, 10(11): 1821. doi: 10.3390/rs10111821
[11]	ZHOU F, WU Y, DAI Y, et al. Detection of small target using schatten 1/2 quasi-norm regularization with reweighted sparse enhancement in complex infrared scenes[J]. Remote Sensing, 2019, 11(17): 2058. doi: 10.3390/rs11172058
[12]	REN S, HE K, GIRSHICK R, et al. Faster r-cnn: Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149. doi: 10.1109/TPAMI.2016.2577031
[13]	REDMON J, FARHADI A. Yolo9000: Better, faster, stronger[C]//In 30th IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2014: 6517–6525.
[14]	JOSEPH R, ALI F. Yolov3: An incremental improvement[EB/OL]. (2018-04-08)[2021-04-21]. http: //arxiv. org/abs/1804.02767.
[15]	LIU W, ANGUELOV D, ERHAN D, et al. SSD: Single shot multibox detector[C]//in 32th IEEE Conference on Computer Vision and Pattern Recognition. Amsterdam: University of Amsterdam, 2016.
[16]	HOU Q, WANG Z, TAN F, et al. RISTDnet: Robust infrared small target detection network[EB/OL]. (2020-12-27) [2021-04-20]. https://doi.org/10.1109/LGRS.2021.3050828.
[17]	李岩. 基于"高分五号"卫星红外影像的舰船尾迹特征分析[J]. 航天返回与遥感, 2020, 41(5): 106-113. LI Y.The ship wake characterization study based on GF-5 infrared images[J]. Spacecraft Recovery & Remote Sensing,2020,41(5):106-113(in Chinese).