-
摘要:
遥感图像融合的目的是融合高光谱分辨率、低空间分辨率的多光谱（MS）图像和高空间分辨率、低光谱分辨率的全色（PAN）图像，得到高光谱分辨率与高空间分辨率的融合图像。遥感图像的注入模型中如何确定注入细节及注入系数是该技术研究的关键。针对注入细节优化，先通过模拟MS传感器的特性来定义一种多尺度高斯滤波器，再用该滤波器卷积PAN图像以提取细节，得到与MS图像高度相关的细节。针对注入系数优化，综合考虑光谱信息与细节信息提出一种自适应的注入量系数。为更好地保留边缘信息，提出一种新的边缘保持权重矩阵，实现光谱信息与空间的双保真。将优化后的注入系数与注入细节相乘注入到上采样后的MS图像中，得到融合结果。对所提方法进行性能分析，并在各卫星数据集上进行大量测试，与一些先进的遥感图像融合方法进行对比，实验结果表明，所提方法在主观与综合客观指标上都能达到最优。
Abstract:The purpose of remote sensing image fusion is to fuse high-spectral-resolution low-spatial-resolution multispectral (MS) images and high-spatial-resolution low-spectral-resolution panchromatic (PAN) images, so as to obtain the fusion images with high spectral resolution and high spatial resolution. How to determine the injection details and injection coefficients in the injection model is the key to image fusion research. For detail optimization, a multi-scale Gaussian filter is defined by simulating the characteristics of MS sensor, and then the filter is used to convolve with PAN image to extract the details and obtain the details highly related to MS image. In order to optimize the injection coefficient, an adaptive injection coefficient is proposed based on spectral information and detail information. To better preserve the edge information, a new edge preserving weight matrix is proposed to achieve the dual fidelity of spectral information and space. Finally, the optimized injection coefficient is multiplied by details and injected into the up-sampled MS image to obtain the final fusion result. Performance analysis of the method proposed in this paper has been carried out and a large number of tests have been conducted in each satellite dataset. The experimental results show that, compared with the most advanced methods, the proposed method performs best in both subjective and comprehensive objective assessment.
-
图 3 优化过程平均客观评价指标对比
Figure 3. Average objective evaluation indicator comparison of optimization steps
图 4 边缘保持方法改进前后平均客观评价指标对比
Figure 4. Comparison of average objective evaluation indicators before and after improvement of edge preserving method
图 5 QuickBird数据集遥感图像融合结果
Figure 5. Fusion results of remote sensing images from QuickBird dataset
图 6 WorldView-2数据集遥感图像融合结果
Figure 6. Fusion results of remote sensing images from WorldView-2 dataset
图 7 pleiades数据集遥感图像融合结果
Figure 7. Fusion results of remote sensing images from pleiades dataset
图 9 QuickBird数据集真实图像融合结果
Figure 9. Fusion results of real images from QuickBird dataset
表 1 图 5对应的遥感图像融合结果客观评价指标
Table 1. Objective evaluation indicators of remote sensing image fusion results in Fig. 5
方法 CC(1) UIQI(1) RASE(0) RMSE(0) SAM(0) ERGAS(0) AWLP 0.954 3 0.842 5 24.337 7 19.664 9 4.835 5 7.141 5 BFLP 0.961 5 0.848 1 21.473 9 17.350 9 2.944 6 6.763 3 CBD 0.974 0 0.861 9 18.325 4 14.806 9 5.320 4 5.191 5 IMG 0.946 8 0.842 5 26.540 4 21.444 7 3.506 1 8.677 0 MMMT 0.962 9 0.844 2 21.438 4 17.322 2 5.585 2 5.609 1 ASIMP 0.977 3 0.862 4 17.356 5 14.024 1 5.107 4 4.722 1 本文 0.982 9 0.884 3 14.008 4 11.318 8 2.951 5 3.964 6 
下载: 导出CSV
表 2 图 6对应的遥感图像融合结果客观评价指标
Table 2. Objective evaluation indicators of remote sensing image fusion results in Fig. 6
方法 CC(1) UIQI(1) RASE(0) RMSE(0) SAM(0) ERGAS(0) AWLP 0.908 7 0.846 3 23.071 4 25.239 6 5.531 1 5.629 8 BFLP 0.919 0 0.850 9 20.849 9 22.809 3 5.185 0 5.137 5 CBD 0.948 3 0.855 1 17.116 3 18.724 8 5.660 9 4.210 8 IMG 0.916 7 0.855 8 20.678 2 22.621 5 5.179 1 5.177 6 MMMT 0.936 8 0.840 1 17.873 8 19.553 6 6.018 0 4.468 8 ASIMP 0.957 6 0.861 9 14.912 1 16.313 6 4.966 9 3.712 6 本文 0.957 9 0.862 4 14.592 5 15.963 8 4.775 3.639 7
下载: 导出CSV
表 3 图 7对应的遥感图像融合结果客观评价指标
Table 3. Objective evaluation indicators of remote sensing image fusion results in Fig. 7
方法 CC(1) UIQI(1) RASE(0) RMSE(0) SAM(0) ERGAS(0) AWLP 0.922 7 0.907 3 19.585 9 20.150 3 4.060 0 4.847 1 BFLP 0.926 7 0.903 4 21.395 9 22.012 5 3.726 1 5.906 0 CBD 0.907 5 0.892 4 21.936 6 22.568 8 4.207 2 5.509 9 IMG 0.927 8 0.909 1 19.256 1 19.811 0 3.222 4 4.840 1 MMMT 0.923 2 0.909 2 17.370 1 17.870 6 4.153 9 4.347 1 ASIMP 0.939 0 0.918 1 16.884 7 17.371 3 3.573 3 4.251 1 本文 0.940 5 0.913 3 15.644 5 16.095 3 3.170 0 3.934 0
下载: 导出CSV
表 4 80组遥感图像仿真实验的平均客观评价指标
Table 4. Average objective evaluation indicators of simulation experiments from 80 groups of remote sensing images
方法 CC(1) UIQI(1) RASE(0) RMSE(0) SAM(0) ERGAS(0) AWLP 0.910 5 0.865 7 21.373 0 14.326 7 3.718 4 5.261 6 BFLP 0.905 4 0.827 9 29.250 6 18.965 0 3.900 0 6.973 7 CBD 0.903 3 0.858 1 21.788 5 14.848 6 4.289 0 5.579 6 IMG 0.904 9 0.853 8 22.586 3 15.437 3 3.394 0 5.667 3 MMMT 0.914 7 0.873 3 18.287 9 12.496 2 3.841 8 4.679 7 ASIMP 0.917 4 0.871 7 19.916 1 13.270 7 3.845 1 5.072 7 本文 0.921 5 0.881 5 18.244 7 12.333 0 3.303 5 4.638 0 注：“——”表示平均值。
下载: 导出CSV
方法 Dλ(0) Ds(0) QNR(1) AWLP 0.040 3 0.052 8 0.909 1 BFLP 0.039 4 0.055 8 0.906 9 CBD 0.039 6 0.051 3 0.911 1 IMG 0.050 5 0.056 2 0.896 1 MMMT 0.043 9 0.071 0 0.888 2 ASIMP 0.035 5 0.048 1 0.918 1 本文 0.028 1 0.049 0 0.924 3
下载: 导出CSV
方法 Dλ(0) Ds(0) QNR(1) AWLP 0.022 0 0.040 9 0.938 1 BFLP 0.013 9 0.042 4 0.944 3 CBD 0.026 0 0.043 9 0.931 2 IMG 0.020 4 0.046 6 0.934 0 MMMT 0.019 6 0.043 4 0.937 8 ASIMP 0.019 4 0.040 1 0.941 2 本文 0.011 7 0.038 8 0.949 9
下载: 导出CSV
表 7 50组真实图像的平均定量评价结果
Table 7. Average quantitative evaluation results of 50 groups of real images
方法 
QNR(1) AWLP 0.018 4 0.043 1 0.939 4 BFLP 0.018 0 0.042 1 0.940 8 CBD 0.020 9 0.043 0 0.937 1 IMG 0.024 5 0.048 4 0.928 4 MMMT 0.018 6 0.047 3 0.935 1 ASIMP 0.024 9 0.047 8 0.930 6 本文 0.013 8 0.040 8 0.946 0 注：“——”表示平均值。
下载: 导出CSV
-
[1] GHASSEMIAN H.A review of remote sensing image fusion methods[J]. Information Fusion, 2016, 32:75-89. doi: 10.1016/j.inffus.2016.03.003 [2] GEMINE V, PAOLO A, ROCCO R, et al.Pansharpening based on deconvolution for multiband filter estimation[J].IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(1):540-552. doi: 10.1109/TGRS.2018.2858288 [3] MALPICA J A.Hue adjustment to IHS pan-sharpened IKONOS imagery for vegetation enhancement[J].IEEE Geoscience and Remote Sensing Letters, 2007, 4(1):27-31. doi: 10.1109/LGRS.2006.883523 [4] PSJR C, SIDES S C, ANDERSON J A.Comparison of three different methods to merge multiresolution and multispectral data:Landsat TM and SPOT panchromatic[J].Photogrammetric Engineering & Remote Sensing, 1991, 57(3):265-303. http://cn.bing.com/academic/profile?id=32f78ce8535f3ce1bb1f202d3705a7b3&encoded=0&v=paper_preview&mkt=zh-cn [5] LABEN C A, BROWER B V.Process for enhancing the spatial resolution of multispectral imagery using pan-sharpening: U.S.6011875 A[P].2000-01-04. [6] MENG X C, SHEN H F, LI H F, et al.Review of the pansharpening methods for remote sensing images based on the idea of meta-analysis:Practical discussion and challenges[J].Information Fusion, 2019, 46:102-113. doi: 10.1016/j.inffus.2018.05.006 [7] LI S, KWOK J T, WANG Y.Using the discrete wavelet frame transform to merge Landsat TM and SPOT panchromatic images[J].Information Fusion, 2002, 3(1):17-23. doi: 10.1016/S1566-2535(01)00037-9 [8] NUNEZ J, OTAZU X, FORS O, et al.Multiresolution-based image fusion with additive wavelet decomposition[J].IEEE Transactions on Geoscience and Remote Sensing, 1999, 37(3):1204-1211. doi: 10.1109/36.763274 [9] CUNHA A L D, ZHOU J, DO M N.The nonsubsampled contourlet transform:Theory, design, and applications[J].IEEE Transactions on Image Processing, 2006, 15(10):3089-3101. doi: 10.1109/TIP.2006.877507 [10] YIN M, LIU W, ZHAO X, et al.A novel image fusion algorithm based on nonsubsampled shearlet transform[J].Optik-International Journal for Light and Electron Optics, 2014, 125(10):2274-2282. doi: 10.1016/j.ijleo.2013.10.064 [11] YANG Y, WAN W, HUANG S Y, et al.Remote sensing image fusion based on adaptive HIS and multiscale guided filter[J].IEEE Access, 2016, 4:4573-4582. doi: 10.1109/ACCESS.2016.2599403 [12] YANG Y, WU L, HUANG S Y, et al.Compensation details-based injection model for remote sensing image fusion[J].IEEE Geoscience and Remote Sensing Letters, 2018, 15(5):734-738. doi: 10.1109/LGRS.2018.2810219 [13] YANG Y, WU L, HUANG S Y, et al.Remote sensing image fusion based on adaptively weighted joint detail injection[J].IEEE Access, 2018, 6:6849-6864. doi: 10.1109/ACCESS.2018.2791574 [14] GIUSEPPE M, DAVIDEC, LUISA V, et al.Pansharpening by convolutional neural networks[J].Remote Sensing, 2016, 8(7):594. doi: 10.3390/rs8070594 [15] AIAZZI B, ALPARONE L, BARONTI S, et al.MTF-tailored multiscale fusion of high-resolution MS and pan imagery[J].Photogrammetric Engineering and Remote Sensing, 2006, 72(5):591-596. doi: 10.14358/PERS.72.5.591 [16] VIVONE G, RESTAINO R, CHANUSSOT J.Full scale regression-based injection coefficients for panchromatic sharpening[J].IEEE Transactions on Image Processing, 2018, 27(7):3418-3431. doi: 10.1109/TIP.2018.2819501 [17] YIN H, LI S.Pansharpening with multiscale normalized nonlocal means filter:A two-step approach[J].IEEE Transactions on Geoscience and Remote Sensing, 2015, 53(10):5734-5745. doi: 10.1109/TGRS.2015.2429691 [18] VIVONE G, ALPARONE L, CHANUSSOT J, et al.A critical comparison among pansharpening algorithms[J].IEEE Transactions on Geoscience and Remote Sensing, 2015, 53(5):2565-2586. doi: 10.1109/TGRS.2014.2361734 [19] RAHMANI S, STRAIT M, MERKURJEV D, et al.An adaptive IHS pan-sharpening method[J].IEEE Geoscience and Remote Sensing Letters, 2010, 7(4):746-750. doi: 10.1109/LGRS.2010.2046715 [20] LEUNG Y, LIU J, ZHANG J.An improved adaptive intensity-hue-saturation method for the fusion of remote sensing images[J].IEEE Geoscience and Remote Sensing Letters, 2013, 11(5):985-989. http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=fc1cc43de2c9b14f8fbdc2db892dc1e9 [21] HE K, SUN J, TANG X.Guided image filtering[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(6):1397-1409. doi: 10.1109/TPAMI.2012.213 [22] 杨勇, 阙越, 黄淑英, 等.多视觉特征和引导滤波的鲁棒多聚焦图像融合[J].计算机辅助设计与图形学学报, 2017, 29(7):1324-1333. doi: 10.3969/j.issn.1003-9775.2017.07.019YANG Y, QUE Y, HUANG S Y, et al.Robust multifocus image fusion via multiple visual features and guided filtering[J].Journal of Computer-Aided Design & Computer Graphics, 2017, 29(7):1324-1333(in Chinese). doi: 10.3969/j.issn.1003-9775.2017.07.019 [23] 敬忠良, 肖刚.图像融合:理论与应用[M].北京:高等教育出版社, 2007:160-172.JING Z L, XIAO G.Image fusion:Principle and application[M].Beijing:Higher Education Press, 2007:160-172(in Chinese). [24] YANG Y, WANG W G, HUANG S H, et al.Anovel pan-sharpening framework based on matting model and multiscale transform[J].Remote Sensing, 2017, 9(4):391-750. doi: 10.3390/rs9040391 [25] YANG Y, WU L, HUANG S Y, et al.Pansharpening for multiband images with adaptive spectral-intensity modulation[J].IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2018, 11(9):3196-3208. doi: 10.1109/JSTARS.2018.2849011 [26] WALD L, RANCHIN T, MANGOLINI M.Fusion of satellite images of different spatial resolutions:Assessing the quality of resulting images[J].Photogrammetric Engineering and Remote Sensing, 1997, 63(6):691-699. http://cn.bing.com/academic/profile?id=72dfcaea4793b5939cd13da01141c6b7&encoded=0&v=paper_preview&mkt=zh-cn [27] OTAZU X, GONZÁLEZ-AUDÍCANA M, FORS O, et al.Introduction of sensor spectral response into image fusion methods.Application to wavelet-based methods[J].IEEE Transactions on Geoscience and Remote Sensing, 2005, 43(10):2376-2385. doi: 10.1109/TGRS.2005.856106 [28] KAPLAN N H, ERER I.Bilateral filtering-based enhanced pansharpening of multispectral satellite images[J].IEEE Geoscience and Remote Sensing Letters, 2014, 11(11):1941-1945. doi: 10.1109/LGRS.2014.2314389 [29] WANG Z, BOVIK A C.A universal image quality index[J].IEEE Signal Processing Letters, 2002, 9(3):81-84. doi: 10.1109/97.995823 [30] YANG Y, TONG S, HUANG S Y, et al.Multifocus image fusion based on NSCT and focused area detection[J].IEEE Sensors Journal, 2015, 15(5):2824-2838. http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=c9013ea70ae3b0bbd28431b1a11d9f02 [31] RANCHIN T, WALD L.Fusion of high spatial and spectral resolution images:The ARSIS concept and its implementation[J].Photogrammetric Engineering and Remote Sensing, 2000, 66(1):49-61. http://cn.bing.com/academic/profile?id=a562d5815c79f8c211ee34e00625c010&encoded=0&v=paper_preview&mkt=zh-cn [32] ALPARONE L, WALD L, CHANUSSOT J, et al.Comparison of pansharpening algorithms:Outcome of the 2006 GRSS data fusion contest[J].IEEE Transactions on Geoscience and Remote Sensing, 2007, 45(10):3012-3021. doi: 10.1109/TGRS.2007.904923 [33] WALD L.Quality of high resolution synthesized images: Is there a simple criterion [C]//Proceedings of the Third Conference on Fusion of Earth Data, 2000: 99-103. [34] ALPARONE L, AIAZZI B, BARONTI S, et al.Multispectral and panchromatic data fusion assessment without reference[J].Photogrammetric Engineering and Remote Sensing, 2008, 74(2):193-200. doi: 10.14358/PERS.74.2.193 -
下载:
点击查看大图
计量
- 文章访问数: 794
- HTML全文浏览量: 53
- PDF下载量: 314
- 被引次数: 0
