-
摘要:
遥感图像镶嵌是遥感图像解译的一项重要研究内容，然而受成像时间、角度及地物纹理的影响，镶嵌图像经常会有颜色不一致、地物结构错位等情况。针对遥感图像镶嵌缝两侧出现的颜色差异、地物结构错位等质量问题，设计了双分支网络进行遥感图像镶嵌质量的盲评价，2个分支网络分别用于镶嵌缝两侧颜色差异评价和结构错位评价，综合2个网络的输出实现遥感图像镶嵌质量的综合评价。由于获得图像的质量真值需要耗费大量的人力物力，为了减少训练卷积网络所需要的数据量，提出了一种基于两阶段训练的弱监督学习策略，第1阶段在仿真的镶嵌数据集上以颜色差异量和结构错位量为客观真值对网络进行预训练，学习与质量评价有关的先验知识，第2阶段在有主观质量真值的数据集上进行微调。在建立的带有质量真值的仿真数据集和真实数据集上的实验结果表明：所提方法能够有效评价遥感图像镶嵌的质量，性能优于对比方法。
Abstract:A remote sensing image mosaic is an important research content of remote sensing image interpretation. However, affected by the imaging time, angle, and object texture, mosaic images often suffer inconsistent colors and structure dislocation. Aiming at the above quality problems, a double-branch network is designed to perform the blind assessment of the remote sensing image mosaic quality. The branch network are used to assess the color difference and structural dislocation respectively. Finally, the output of the branch networks is integrated to realize the comprehensive assessment. A weakly supervised learning technique based on two-stage training is proposed to lower the quantity of images in network training because it requires a lot of labor and material resources to determine the true score of the image quality. Firstly, to gain the previous knowledge associated with quality assessment, the network is initially pre-trained on the simulated mosaic dataset, which uses color change and structural dislocation as the objective quality score. Secondly, fine-tuning is performed on the dataset with subjective scores. Secondly, fine-tuning is performed on the dataset with subjective score. The experiment results on the established simulation dataset and authentic dataset show that the proposed method can effectively assess the quality of remote sensing image mosaic and outperforms the comparison algorithms.
-
图 3 实验中使用的一幅具有结构错位和颜色差异的真实遥感镶嵌图像
Figure 3. A real remote sensing mosaic image with structural dislocation and color difference used in experiment
图 6 不同方法在仿真数据集上的散点图和线性回归结果
Figure 6. Scatter plot and linear regression results of different algorithms on simulation dataset
图 7 不同方法在真实数据集上的散点图和线性回归结果
Figure 7. Scatter plot and linear regression results of different algorithms on authentic dataset
表 1 在带有主观质量真值的仿真数据集上的评估结果
Table 1. Evaluation results on simulation dataset with subjective quality score
SSROCC PPLCC 颜色差异 结构错位 混合失真 颜色差异 结构错位 混合失真 两阶段训练策略 0.8143 0.7635 0.8624 0.8236 0.8261 0.9038 ImageNet 0.7656 0.3326 0.7604 0.7524 0.3542 0.7643 
下载: 导出CSV
表 2 在带有主观质量真值的真实数据集上的评估结果
Table 2. Evaluation results on authentic dataset with subjective quality score
SSROCC PPLCC 颜色差异 结构错位 混合失真 颜色差异 结构错位 混合失真 两阶段训练策略 0.603 6 0.601 8 0.613 2 0.886 2 0.890 1 0.905 6 ImageNet 0.574 6 0.544 1 0.574 5 0.876 3 0.887 2 0.870 1
下载: 导出CSV
表 3 不同方法在仿真数据集上的性能对比
Table 3. Performance comparison of different algorithms on simulation dataset
SSROCC PPLCC 颜色差异 结构错位 混合失真 颜色差异 结构错位 混合失真 NIQE 0.006 5 −0.054 2 0.196 5 0.356 4 0.228 4 0.435 6 BRISQUE 0.024 5 0.170 5 0.203 9 0.223 1 0.275 9 0.304 3 SSEQ 0.034 1 0.695 6 0.745 8 0.214 2 0.805 4 0.844 3 SPIQA 0.674 2 0.375 9 0.680 1 0.687 2 0.386 5 0.725 6 本文 0.814 3 0.763 5 0.862 4 0.823 6 0.826 1 0.903 8
下载: 导出CSV
表 4 不同方法在真实数据集上的性能对比
Table 4. Performance comparison of different algorithms on authentic dataset
SSROCC PPLCC 颜色差异 结构错位 混合失真 颜色差异 结构错位 混合失真 NIQE −0.003 2 0.022 5 0.002 3 0.112 5 0.083 4 0.082 1 BRISQUE 0.323 1 0.336 9 0.325 0 0.796 4 0.785 6 0.798 5 SSEQ 0.348 9 0.354 2 0.324 1 0.774 5 0.764 4 0.778 9 SPIQA 0.435 6 0.375 9 0.436 4 0.846 9 0.822 1 0.852 3 本文 0.603 6 0.601 8 0.613 2 0.886 2 0.890 1 0.905 6
下载: 导出CSV
-
[1] 周清华, 潘俊, 李德仁. 遥感图像镶嵌接缝线自动生成方法综述[J]. 国土资源遥感, 2013, 25(2): 1-7.ZHOU Q H, PAN J, LI D R. Overview of automatic generation of mosaicking seamlines for remote sensing images[J]. Remote Sensing for Land & Resources, 2013, 25(2): 1-7(in Chinese). [2] 秦曼君. 多光谱遥感图像去雾算法研究[D]. 北京: 北京航空航天大学, 2018: 7-13.QIN M J. Research on the fog removal algorithm of multispectral remote sensing image[D]. Beijing: Beihang University, 2018: 7-13(in Chinese). [3] 徐冬宇. 高光谱遥感图像质量评价算法研究[D]. 杭州: 浙江大学, 2019: 5-10.XU D Y. Research on quality evaluation algorithm of hyperspectral remote sensing image[D]. Hangzhou: Zhejiang University, 2019: 5-10(in Chinese). [4] 徐冬宇, 厉小润, 赵辽英, 等. 基于多模型融合的高光谱图像质量评价[J]. 激光与光电子学进展, 2019, 56(2): 92-101.XU D Y, LI X R, ZHAO L Y, et al. Hyperspectral image quality evaluation based on multi-model fusion[J]. Laser & Optoelectronics Progress, 2019, 56(2): 92-101(in Chinese). [5] YU S J, LI T S, XU X Y, et al. NRQQA: A no-reference quantitative quality assessment method for stitched images[C]//Proceedings of the ACM Multimedia Asia. New York: ACM, 2019: 1-6. [6] ZHU S D, ZHANG Y Z, TAO L, et al. A novel method for quality assessment of image stitching based on the Gabor filtering[C]//2018 IEEE International Conference on Information and Automation (ICIA). Piscataway: IEEE Press, 2019: 1605-1610. [7] LING S Y, CHEUNG G, LE CALLET P. No-reference quality assessment for stitched panoramic images using convolutional sparse coding and compound feature selection[C]//2018 IEEE International Conference on Multimedia and Expo (ICME). Piscataway: IEEE Press, 2018: 1-6. [8] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6): 84-90. doi: 10.1145/3065386 [9] DENG J, DONG W, SOCHER R, et al. ImageNet: A large-scale hierarchical image database[C]//2009 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2009: 248-255. [10] KANG L, YE P, LI Y, et al. Convolutional neural networks for no-reference image quality assessment[C]//2014 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2014: 1733-1740. [11] BOSSE S, MANIRY D, MÜLLER K R, et al. Deep neural networks for no-reference and full-reference image quality assessment[J]. IEEE Transactions on Image Processing, 2018, 27(1): 206-219. [12] ZHU H C, LI L D, WU J J, et al. MetaIQA: Deep meta-learning for no-reference image quality assessment[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2020: 14131-14140. [13] HOU J W, LIN W S, ZHAO B Q. Content-dependency reduction with multi-task learning in blind stitched panoramic image quality assessment[C]//2020 IEEE International Conference on Image Processing (ICIP). Piscataway: IEEE Press, 2020: 3463-3467. [14] LIN H H, HOSU V, SAUPE D. DeepFL-IQA: Weak supervision for deep IQA feature learning[EB/OL]. (2021-01-20)[2021-11-01]. https//arxiv.org/abs/2001.08113. [15] LIU X L, VAN DE WEIJER J, BAGDANOV A D. RankIQA: Learning from rankings for No-reference image quality assessment[C]//2017 IEEE International Conference on Computer Vision (ICCV). Piscataway: IEEE Press, 2017: 1040-1049. [16] KIM J, ZHENG H, GHADIYARAM D, et al. Deep convolutional neural models for picture-quality prediction: Challenges and solutions to data-driven image quality assessment[J]. IEEE Signal Processing Magazine, 2017, 34(6): 130-141. doi: 10.1109/MSP.2017.2736018 [17] GHADIYARAM D, BOVIK A C. Massive online crowdsourced study of subjective and objective picture quality[J]. IEEE Transactions on Image Processing, 2016, 25(1): 372-387. [18] CHENG G, HAN J W, LU X Q. Remote sensing image scene classification: Benchmark and state of the art[J]. Proceedings of the IEEE, 2017, 105(10): 1865-1883. doi: 10.1109/JPROC.2017.2675998 [19] HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2016: 770-778. [20] KINGMA D P, BA J. Adam: A method for stochastic optimization[EB/OL]. (2017-01-30)[2021-11-01]. https://arxiv.org/abs/1412.6980. [21] MITTAL A, SOUNDARARAJAN R, BOVIK A C. Making a “completely blind” image quality analyzer[J]. IEEE Signal Processing Letters, 2013, 20(3): 209-212. doi: 10.1109/LSP.2012.2227726 [22] MITTAL A, MOORTHY A K, BOVIK A C. No-reference image quality assessment in the spatial domain[J]. IEEE Transactions on Image Processing, 2012, 21(12): 4695-4708. doi: 10.1109/TIP.2012.2214050 [23] LIU L X, LIU B, HUANG H, et al. No-reference image quality assessment based on spatial and spectral entropies[J]. Signal Processing:Image Communication, 2014, 29(8): 856-863. doi: 10.1016/j.image.2014.06.006 -
下载:
点击查看大图
计量
- 文章访问数: 150
- HTML全文浏览量: 44
- PDF下载量: 9
- 被引次数: 0
