基于混合网络的异源遥感图像变化检测

周圆; 李祥瑞; 杨晶

doi:10.13700/j.bh.1001-5965.2020.0455

基于混合网络的异源遥感图像变化检测

doi: 10.13700/j.bh.1001-5965.2020.0455

周圆^1, ,,
李祥瑞²,
杨晶¹

1.
天津大学电气自动化与信息工程学院, 天津 300072
2.
天津大学国际工程师工程学院, 天津 300072

详细信息

作者简介:
周圆女，博士，教授，博士生导师。主要研究方向：机器学习、计算机视觉、3D图像/视频编码与传输、无线传感器网络、网络视频通信等

通讯作者:
周圆, E-mail: zhouyuan@tju.edu.cn

中图分类号: TP751.1
计量
- 文章访问数: 564
- HTML全文浏览量: 193
- PDF下载量: 128
- 被引次数: 0
出版历程
- 收稿日期: 2020-08-24
- 录用日期: 2020-09-25
- 网络出版日期: 2021-03-20

Heterogeneous remote sensing image change detection based on hybrid network

1.
School of Electrical and Information Engineering, Tianjin University, Tianjin 300072, China
2.
Tianjin International Engineering Institute, Tianjin University, Tianjin 300072, China

More Information

Corresponding author: ZHOU Yuan, E-mail: zhouyuan@tju.edu.cn

摘要

摘要:
为了更加准确地进行异源遥感图像的变化检测任务，提出了一种基于混合网络的异源遥感图像变化检测算法。利用伪孪生网络提取异源遥感图像块间空间维度的变化特征，利用早期融合网络提取异源遥感图像块间光谱维度的变化特征，将2支网络提取的特征进行融合，并将融合后的变化特征输入到sigmoid层进行二分类检测。同时，在伪孪生网络中加入对比损失函数，通过优化对比损失函数，使得在特征空间中，未变化图像对的空间特征差异更小，变化图像对的空间特征差异更大，以提升网络的区分能力和收敛速度。
- 变化检测 /
- 异源 /
- 伪孪生网络 /
- 早期融合网络 /
- 对比损失函数
Abstract:
In order to more quickly and accurately perform the change detection task of heterogeneous remote sensing images, this paper presents a heterogeneous remote sensing image change detection algorithm based on a hybrid network. The algorithm uses a pseudo-siamese network to extract change features between the heterogeneous image blocks in spatial dimension, and uses an early fusion network to extract change features between the heterogeneous image blocks in spectral dimension. The features extracted from the two networks are fused and the fused features are input to the sigmoid layer for binary classification to determine whether the feature has changed. In addition, the contrast loss function is added to the pseudo-siamese network, so that in the features space, the spatial features of the unchanged image pair are closer, and the spatial features of the changed image pair are farther away, which is conducive to improving the network's distinction ability and convergence speed.
- change detection /
- heterogeneous /
- pseudo-siamese network /
- early fusion network /
- contrast loss function

HTML全文

图 1 基于混合网络的异源遥感图像变化检测

Figure 1. Change detection of heterogeneous remote sensing image based on hybrid network

下载: 全尺寸图片幻灯片

图 2 变化检测任务中使用LSTM模型

Figure 2. LSTM model used in change detection task

下载: 全尺寸图片幻灯片

图 3 2D卷积和3D卷积操作的比较

Figure 3. Comparison of 2D convolution and 3D

下载: 全尺寸图片幻灯片

图 4 Bastrop复杂火灾数据集

Figure 4. Bastrop complex fire dataset

下载: 全尺寸图片幻灯片

图 5 Onera变化检测数据集

Figure 5. Onera change detection dataset

下载: 全尺寸图片幻灯片

图 6 HCNN的网络配置示意图

Figure 6. Schematic diagram of HCNN network configuration

下载: 全尺寸图片幻灯片

图 7 Bastrop复杂火灾数据集的变化检测结果数据分析(Kappa系数)

Figure 7. Change detection result data analysis of Bastrop complex fire dataset (Kappa coefficient)

下载: 全尺寸图片幻灯片

图 8 L5T1 vs. L5T2实验中的变化检测结果

Figure 8. Results of change detection in L5T1 vs. L5T2 experiment

下载: 全尺寸图片幻灯片

图 9 L5T1 vs. ALIT2实验中的变化检测结果

Figure 9. Results of change detection in L5T1 vs. ALIT2 experiment

下载: 全尺寸图片幻灯片

图 10 L5T1 vs. L8T2实验中的变化检测结果

Figure 10. Results of change detection in L5T1 vs. L8T2 experiment

下载: 全尺寸图片幻灯片

图 11 在香港图像对上使用不同算法获得的变化检测结果

Figure 11. Change detection results on Hong Kong image pair obtained by different algorithms

下载: 全尺寸图片幻灯片

图 12 在孟买图像对上使用不同算法获得的变化检测结果

Figure 12. Change detection results on Mumbai image pair obtained by different algorithms

下载: 全尺寸图片幻灯片

图 13 PS网络与HCNN网络是否添加对比损失函数的二元交叉熵损失随着网络训练的变化曲线

Figure 13. Binary cross entropy loss curves of PS and HCNN networks with and without contrast loss function binary in network training

下载: 全尺寸图片幻灯片

表 1 Bastrop复杂火灾数据集的变化检测结果

Table 1. Change detection results on Bastrop complex fire dataset

算法	OA/%
算法	L5T1 vs.L5T2	L5T1 vs.ALIT2	L5T1 vs.L8T2
MAD+CVA	87.233	79.991	77.075
KCCA+CVA	92.614	90.175	91.671
DCCA+CVA	95.556	94.676	93.943
DCCAE+CVA	96.598	95.713	94.308
EF	97.358	97.115	94.314
PS	98.382	98.136	94.526
HCNN	99.286	98.764	95.581

下载: 导出CSV

表 2 在香港图像对上使用不同算法获得的变化检测定量结果

Table 2. Quantitative results of change detection obtained by different algorithms on Hong Kong image pairs

算法	OA/%	AA/%	Kappa系数
S	96.542	68.511	0.424
PS	97.900	76.090	0.631
EF	97.134	74.943	0.598
HCNN	97.617	81.200	0.656

下载: 导出CSV

表 3 在孟买图像对上使用不同算法获得的变化检测定量结果

Table 3. Quantitative results of change detection obtained by different algorithms on Mumbai image pairs

算法	OA/%	AA/%	Kappa系数
S	95.928	69.293	0.457
PS	96.325	80.217	0.635
EF	95.449	80.599	0.628
HCNN	97.293	82.583	0.673

下载: 导出CSV

表 4 L5T1 vs.L8T2实验中EF网络与HCNN网络使用2D卷积和3D卷积对检测精度的影响

Table 4. Effect of EF and HCNN networks with 2D convolution and 3D convolution on detection accuracy in L5T1 vs. L8T2 experiment

算法	OA/%	Kappa系数
EF(2D卷积)	94.314	0.711
EF(3D卷积)	94.938	0.723
HCNN(2D卷积)	95.258	0.731
HCNN(3D卷积)	95.581	0.759

下载: 导出CSV

表 5 L5T1 vs.ALIT2实验中PS网络与HCNN网络是否添加对比损失函数对检测精度的影响

Table 5. Effect of PS and HCNN networks with and without contrast loss function on detection accuracy in L5T1 vs. ALIT2 experiment

算法	OA/%	Kappa系数
PS(未添加对比损失函数)	98.136	0.903
HCNN(未添加对比损失函数)	98.66	0.932
PS(添加对比损失函数)	98.356	0.917
HCNN(添加对比损失函数)	98.764	0.943

下载: 导出CSV

参考文献(24)

[1]	ALBERGA V. Similarity measures of remotely sensed multi-sensor images for change detection applications[J]. Remote Sensing, 2009, 1(3): 122-143. doi: 10.3390/rs1030122
[2]	FERNANDEZ-PRIETO D, MARCONCINI M. A novel partially supervised approach to targeted change detection[J]. IEEE Transactions on Geoscience and Remote Sensing, 2011, 49(12): 5016-5038. doi: 10.1109/TGRS.2011.2154336
[3]	LAI P L, FYFE C. Kernel and nonlinear canonical correlation analysis[J]. International Journal of Neural Systems, 2000, 10(5): 365-377. doi: 10.1142/S012906570000034X
[4]	VOLPI M, DE MORSIER F, CAMPS-VALLS G, et al. Multi-sensor change detection based on nonlinear canonical correlations[C]//2013 IEEE International Geoscience and Remote Sensing Symposium. Piscataway: IEEE Press, 2013: 1944-1947.
[5]	ZHU Z, WOODCOCK C E. Object-based cloud and cloud shadow detection in Landsat imagery[J]. Remote Sensing of Environment, 2012, 118: 83-94. doi: 10.1016/j.rse.2011.10.028
[6]	JIAN M, LAM K, DONG J, et al. Visual-patch-attention-aware saliency detection[J]. IEEE Transactions on Systems, Man, and Cybernetics, 2015, 45(8): 1575-1586. http://europepmc.org/abstract/med/25291809
[7]	JIAN M, QI Q, DONG J, et al. Integrating QDWD with pattern distinctness and local contrast for underwater saliency detection[J]. Journal of Visual Communication and Image Representation, 2018, 53: 31-41. doi: 10.1016/j.jvcir.2018.03.008
[8]	JIAN M, ZHANG W, YU H, et al. Saliency detection based on directional patches extraction and principal local color contrast[J]. Journal of Visual Communication and Image Representation, 2018, 57: 1-11. doi: 10.1016/j.jvcir.2018.10.008
[9]	MERCIER G, MOSER G, SERPICO S B. Conditional copulas for change detection in heterogeneous remote sensing images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2008, 46(5): 1428-1441. doi: 10.1109/TGRS.2008.916476
[10]	GAO F, LIU X, DONG J, et al. Change detection in SAR images based on deep Semi-NMF and SVD networks[J]. Remote Sensing, 2017, 9(5): 435. doi: 10.3390/rs9050435
[11]	MOU L, BRUZZONE L, ZHU X X. Learning spectral-spatial-temporal features via a recurrent convolutional neural network for change detection in multispectral imagery[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(2): 924-935. doi: 10.1109/TGRS.2018.2863224
[12]	ZHANG P, GONG M, SU L, et al. Change detection based on deep feature representation and mapping transformation for multi-spatial-resolution remote sensing images[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2016, 116: 24-41. doi: 10.1016/j.isprsjprs.2016.02.013
[13]	ZHANG Z, VOSSELMAN G, GERKE M, et al. Change detection between multimodal remote sensing data using Siamese CNN[EB/OL]. (2018-07-25)[2020-08-01]. https://arxiv.org/abs/1807.09562.
[14]	LIU J, GONG M, QIN K, et al. A deep convolutional coupling network for change detection based on heterogeneous optical and radar images[J]. IEEE Transactions on Neural Networks and Learning Systems, 2016, 29(3): 545-559. http://ieeexplore.ieee.org/document/7795259/citations
[15]	HUGHES L H, SCHMITT M, MOU L, et al. Identifying corresponding patches in SAR and optical images with a pseudo-Siamese CNN[J]. IEEE Geoscience and Remote Sensing Letters, 2018, 15(5): 784-788. doi: 10.1109/LGRS.2018.2799232
[16]	NIU X, GONG M, ZHAN T, et al. A conditional adversarial network for change detection in heterogeneous images[J]. IEEE Geoscience and Remote Sensing Letters, 2018, 16(1): 45-49. http://it.ckcest.cn/portal.php?mod=viewaid=3338420
[17]	HOCHREITER S, SCHMIDHUBER J. Long short-term memory[J]. Neural Computation, 1997, 9(8): 1735-1780. doi: 10.1162/neco.1997.9.8.1735
[18]	LI Y, ZHANG H, SHEN Q. Spectral-spatial classification of hyperspectral imagery with 3D convolutional neural network[J]. Remote Sensing, 2017, 9(1): 67. doi: 10.3390/rs9010067
[19]	CHOPRA S, HADSELL R, LECUN Y. Learning a similarity metric discriminatively, with application to face verification[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2005, 1: 539-546.
[20]	DAUDT R C, LE SAUX B, BOULCH A, et al. Urban change detection for multispectral earth observation using convolutional neural networks[C]//2018 IEEE International Geoscience and Remote Sensing Symposium. Piscataway: IEEE Press, 2018: 2115-2118.
[21]	VOLPI M, CAMPS-VALLS G, TUIA D. Spectral alignment of multi-temporal cross-sensor images with automated kernel canonical correlation analysis[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2015, 107: 50-63. doi: 10.1016/j.isprsjprs.2015.02.005
[22]	ANDREW G, ARORA R, BILMES J, et al. Deep canonical correlation analysis[C]//International Conference on International Conference on Machine Learning, 2013, 28: 1247-1255.
[23]	WANG W, ARORA R, LIVESCU K, et al. On deep multi-view representation learning[C]//International Conference on Machine Learning, 2015: 1083-1092.
[24]	ZAGORUYKO S, KOMODAKIS N. Learning to compare image patches via convolutional neural networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2015: 4353-4361.