留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

遥感图像语义分割的空间增强注意力U型网络

宝音图 刘伟 李润生 李钦 胡庆

宝音图,刘伟,李润生,等. 遥感图像语义分割的空间增强注意力U型网络[J]. 北京航空航天大学学报,2023,49(7):1828-1837 doi: 10.13700/j.bh.1001-5965.2021.0544
引用本文: 宝音图,刘伟,李润生,等. 遥感图像语义分割的空间增强注意力U型网络[J]. 北京航空航天大学学报,2023,49(7):1828-1837 doi: 10.13700/j.bh.1001-5965.2021.0544
BAO Y T,LIU W,LI R S,et al. Semantic segmentation of remote sensing images based on U-shaped network combined with spatial enhance attention[J]. Journal of Beijing University of Aeronautics and Astronautics,2023,49(7):1828-1837 (in Chinese) doi: 10.13700/j.bh.1001-5965.2021.0544
Citation: BAO Y T,LIU W,LI R S,et al. Semantic segmentation of remote sensing images based on U-shaped network combined with spatial enhance attention[J]. Journal of Beijing University of Aeronautics and Astronautics,2023,49(7):1828-1837 (in Chinese) doi: 10.13700/j.bh.1001-5965.2021.0544

遥感图像语义分割的空间增强注意力U型网络

doi: 10.13700/j.bh.1001-5965.2021.0544
基金项目: 国家自然科学基金(41901378)
详细信息
    通讯作者:

    E-mail:greatliuliu@163.com

  • 中图分类号: TP751.1;V19

Semantic segmentation of remote sensing images based on U-shaped network combined with spatial enhance attention

Funds: National Natural Science Foundation of China (41901378)
More Information
  • 摘要:

    针对基于深度学习的语义分割模型在解析遥感图像时,小尺寸目标和目标边界存在分割不准确的问题,提出一种U型网络模型SGE-Unet。该模型通过优化网络结构加强模型的特征提取能力;融合空间组增强注意力,提升模型对上下文语义信息的解析能力;采用中值频率平衡交叉熵损失函数抑制类别分布不均衡的影响。在2个数据集上进行实验,SGE-Unet的整体准确率、平均交并比、$\overline F _{1} $分数和Kappa系数均高于主流模型,Vaihingen数据集中小尺寸目标车的交并比和F1分数分别为0.719和0.901,比次优模型提升了16%和11%,实验结果表明所提模型能更精准地分割小尺寸目标及目标边界。

     

  • 图 1  SGE-Unet结构

    Figure 1.  Structure of SGE-Unet

    图 2  拼接痕迹示意图

    Figure 2.  Seam after splicing

    图 3  拼接策略

    Figure 3.  Splicing method

    图 4  SGE-Unet全局分割结果

    Figure 4.  Global segmentation results of SGE-Unet

    图 5  Vaihingen数据集上的特征热图及局部分割结果

    Figure 5.  Feature heat map and local segmentation results on Vaihingen dataset

    图 6  Potsdam数据集上的特征热图及局部分割结果

    Figure 6.  Feature heat map and local segmentation results on Potsdam dataset

    表  1  数据集分配

    Table  1.   Allocation of dataset

    类别VaihingenPotsdam
    训练集1, 3, 11, 13, 15, 17, 21, 26, 28, 32, 34, 372_12, 3_10, 3_11, 3_12, 4_11, 4_12, 5_10, 5_12, 6_7,
    6_8, 6_9, 6_10, 6_12, 7_7, 7_9, 7_10, 7_11, 7_12
    验证集5, 7, 23, 302_11, 4_10, 5_11, 7_8
    测试集2, 4, 6, 8, 10, 12, 14, 16, 20, 22, 24, 27, 29, 31, 33, 35, 382_10, 2_13, 2_14, 3_13, 3_14, 4_13, 4_14, 4_15,
    5_13, 5_14, 5_15, 6_13, 6_14, 6_15, 7_13
    下载: 导出CSV

    表  2  Vaihingen数据集上的语义分割结果

    Table  2.   Semantic segmentation result on Vaihingen dataset

    模型IOUF1OAmIoU$\overline F _{1} $Kappa
    不透水表面建筑低植被不透水表面建筑低植被
    FCN[5]0.6690.7610.5060.6970.6340.8010.8640.6870.8220.7680.8230.6530.7880.783
    SegNet[21]0.7530.8050.6560.7050.4580.8590.8920.7920.8270.6290.8450.6750.8000.791
    DeepLabV3[26]0.8220.9120.7110.7700.5680.9020.9540.8310.8700.7240.8580.7570.8560.809
    SCAttNetV2[27]0.8040.8230.6670.6710.5440.8910.9030.8010.8030.7050.8550.7020.8210.788
    UNet++[17]0.7830.8780.6140.7160.6190.9520.9520.8170.9310.8090.8540.7220.8920.808
    SGE-Unet0.8030.8900.7060.8300.7190.9350.9660.9340.9410.9010.8660.7900.9350.824
    下载: 导出CSV

    表  3  Potsdam数据集上的语义分割结果

    Table  3.   Semantic segmentation result on Potsdam dataset

    模型IOUF1OAmIoU$\overline F _{1} $Kappa
    不透水表面建筑低植被不透水表面建筑低植被
    FCN[5]0.7760.7990.7200.6700.7970.8740.8890.8370.8200.8870.8080.7520.8610.781
    SegNet[21]0.8540.9120.7860.7570.8810.9210.9540.8800.8620.9370.8110.8380.9110.782
    DeepLabV3[26]0.8820.9440.7890.7450.8710.9380.9710.8820.8540.9310.8350.8460.9150.799
    SCAttNetV2[27]0.8180.8880.7070.6630.8030.9010.9410.8290.7970.8910.8790.7760.8720.805
    UNet++[17]0.8640.9230.7810.7670.8650.9410.9740.8950.8830.9330.8720.8400.9250.828
    SGE-Unet0.8790.9400.8150.7950.8910.9550.9820.9050.9570.9690.8830.8640.9540.843
    下载: 导出CSV

    表  4  Vaihingen数据集上的消融实验

    Table  4.   Ablation experiments result on Vaihingen dataset

    模型IOUF1OAmIoU$\overline F _{1} $Kappa
    不透水表面建筑低植被不透水表面建筑低植被
    UNet++0.7830.8770.6130.7160.6190.9730.9520.8170.9300.8090.8530.7220.8960.808
    UNet++&EfficientNet0.8150.9110.6270.7210.6550.9760.9670.8230.9370.8320.8660.7460.9070.825
    UNet++&EfficientNet&SGE0.8130.9010.6260.7220.7140.9710.9630.8260.9380.8940.8670.7550.9180.824
    下载: 导出CSV

    表  5  Potsdam数据集上的消融实验

    Table  5.   Ablation experiments result on Potsdam dataset

    模型IOUF1OAmIoU$\overline F _{1} $Kappa
    不透水表面建筑低植被不透水表面建筑低植被
    UNet++0.8110.9050.6950.6660.8040.9650.9740.8950.8830.9310.87207760.9300.828
    UNet++&EfficientNet0.8290.9180.7110.6910.8100.9600.9860.8920.9090.9430.8790.7920.9380.839
    UNet++&EfficientNet&SGE0.8260.9150.7090.6940.8230.9550.9820.8980.9130.9700.8830.7930.9440.843
    下载: 导出CSV

    表  6  各模型的参数量和计算复杂度对比

    Table  6.   Comparison of parameters and computational complexity of different modes

    模型参数量/MB计算复杂度
    GFLOPS
    OA$\overline F _{1} $
    VaihingenPotsdamVaihingenPotsdam
    FCN[5] 21.30 19.20 0.823 0.808 0.621 0.757
    SegNet[21] 18.82 117.74 0.845 0.811 0.675 0.838
    DeepLabV3[26] 26.01 109.34 0.858 0.835 0.757 0.846
    UNet++[17] 26.79 73.77 0.854 0.872 0.722 0.840
    SGE-Unet 20.01 39.13 0.866 0.883 0.790 0.864
    下载: 导出CSV
  • [1] YUAN X H, SHI J F, GU L C. A review of deep learning methods for semantic segmentation of remote sensing imagery[J]. Expert Systems with Applications, 2021, 169: 114417. doi: 10.1016/j.eswa.2020.114417
    [2] XING S, XIE Q, WANG M. Semantic segmentation for remote sensing images based on adaptive feature selection network[J]. IEEE Geoscience and Remote Sensing Letters, 2022, 19: 8006705.
    [3] 蒋晨琛, 霍宏涛, 冯琦. 一种基于PCA的面向对象多尺度分割优化算法[J]. 北京航空航天大学学报, 2020, 46(6): 1192-1203.

    JIANG C C, HUO H T, FENG Q. An object-oriented multi-scale segmentation optimization algorithm based on PCA[J]. Journal of Beijing University of Aeronautics and Astronautics, 2020, 46(6): 1192-1203(in Chinese).
    [4] KAMPFFMEYER M, SALBERG A B, JENSSEN R. Semantic segmentation of small objects and modeling of uncertainty in urban remote sensing images using deep convolutional neural networks[C]// IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Piscataway: IEEE Press, 2016: 680-688.
    [5] SHELHAMER E, LONG J, DARRELL T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4): 640-651. doi: 10.1109/TPAMI.2016.2572683
    [6] GUO R, LIU J B, LI N, et al. Pixel-wise classification method for high resolution remote sensing imagery using deep neural networks[J]. ISPRS International Journal of Geo-Information, 2018, 7(3): 110. doi: 10.3390/ijgi7030110
    [7] LI R, DUAN C X, ZHENG S Y, et al. MACU-Net for semantic segmentation of fine-resolution remotely sensed images[J]. IEEE Geoscience and Remote Sensing Letters, 2022, 19: 8007205.
    [8] ALAM M, WANG J F, CONG G P, et al. Convolutional neural network for the semantic segmentation of remote sensing images[J]. Mobile Networks and Applications, 2021, 26(1): 200-215. doi: 10.1007/s11036-020-01703-3
    [9] RONNEBERGER O, FISCHER P, BROX T. U-Net: Convolutional networks for biomedical image segmentation[C]// International Conference on Medical Image Computing and Computer-Assisted Intervention. Berlin: Springer, 2015: 234-241.
    [10] 张小娟, 汪西莉. 完全残差连接与多尺度特征融合遥感图像分割[J]. 遥感学报, 2020, 24(9): 1120-1133.

    ZHANG X J, WANG X L. Image segmentation models of remote sensing using full residual connection and multiscale feature fusion[J]. Journal of Remote Sensing, 2020, 24(9): 1120-1133(in Chinese).
    [11] FENG Y, DIAO W, SUN X, et al. NPALoss: Neighboring pixel affinity loss for semantic segmentation in high-resolution aerial imagery[J]. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2020, V-2-2020(I-3): 475-482. doi: 10.5194/isprs-annals-V-2-2020-475-2020
    [12] 肖春姣, 李宇, 张洪群, 等. 深度融合网结合条件随机场的遥感图像语义分割[J]. 遥感学报, 2020, 24(3): 254-264. doi: 10.11834/jrs.20208298

    XIAO C J, LI Y, ZHANG H Q, et al. Semantic segmentation of remote sensing image based on deep fusion networks and conditional random field[J]. Journal of Remote Sensing, 2020, 24(3): 254-264(in Chinese). doi: 10.11834/jrs.20208298
    [13] 翟鹏博, 杨浩, 宋婷婷, 等. 结合注意力机制的双路径语义分割[J]. 中国图象图形学报, 2020, 25(8): 1627-1636. doi: 10.11834/jig.190533

    ZHAI P B, YANG H, SONG T T, et al. Two-path semantic segmentation algorithm combining attention mechanism[J]. Journal of Image and Graphics, 2020, 25(8): 1627-1636(in Chinese). doi: 10.11834/jig.190533
    [14] 杨军, 于茜子. 结合Atrous卷积的FuseNet变体网络高分遥感影响语义分割[J]. 武汉大学学报 (信息科学版), 2022, 47(7): 1071-1080. doi: 10.13203/j.whugis20200305

    YANG J, YU X Z. Semantic segmentation of high-resolution remote sensing images based on improved FuseNet combined with the Atrous convolution[J]. Geomatics and Information Science of Wuhan University, 2022, 47(7): 1071-1080(in Chinese). doi: 10.13203/j.whugis20200305
    [15] WANG X Y, CUI Z Y, CAO Z J, et al. Dense docked ship detection via spatial group-wise enhance attention in SAR images[C]// IEEE International Geoscience and Remote Sensing Symposium. Piscataway: IEEE Press, 2021: 1244-1247.
    [16] TAN M X, LE Q V. EfficientNet: Rethinking model scaling for convolutional neural networks[C]// IEEE International Conference on Machine Learning (ICML).Piscataway: IEEE press, 2019: 6105-6114.
    [17] ZHOU Z W, SIDDIQUEE M M R, TAJBAKHSH N, et al. UNet++: A nested U-net architecture for medical image segmentation[C]// International Workshop on Deep Learning in Medical Image Analysis, International Workshop on Multimodal Learning for Clinical Decision Support. Berlin: Springer, 2018: 3-11.
    [18] 李道纪, 郭海涛, 卢俊, 等. 遥感影像地物分类多注意力融和U型网络法[J]. 测绘学报, 2020, 49(8): 1051-1064. doi: 10.11947/j.AGCS.2020.20190407

    LI D J, GUA H T, LU J, et al. A remote sensing image classification procedure based on multilevel attention fusion U-Net[J]. Acta Geodaetica et Cartographica Sinica, 2020, 49(8): 1051-1064(in Chinese). doi: 10.11947/j.AGCS.2020.20190407
    [19] 言有三. 深度学习之图像识别: 核心技术与案例实战[M]. 北京: 机械工业出版社, 2019: 231-232.

    YAN Y S. Image recognition by deep learning: Core technologies and practices[M]. Beijing: China Machine Press, 2019: 231-232(in Chinese).
    [20] ROTTENSTEINER F, SOHN G, JUNG J, et al. The ISPRS benchmark on urban object classification and 3d building reconstruction[J]. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2012(I-3): 293-298.
    [21] BADRINARAYANAN V, KENDALL A, CIPOLLA R. SegNet: A deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12): 2481-2495. doi: 10.1109/TPAMI.2016.2644615
    [22] CHAI D F, NEWSAM S, HUANG J F. Aerial image semantic segmentation using DCNN predicted distance maps[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 161: 309-322. doi: 10.1016/j.isprsjprs.2020.01.023
    [23] XU Z Y, SU C, ZHANG X C. A semantic segmentation method with category boundary for land use and land cover (LULC) mapping of very-high resolution (VHR) remote sensing image[J]. International Journal of Remote Sensing, 2021, 42(8): 3146-3165. doi: 10.1080/01431161.2020.1871100
    [24] 胡伟, 高博川, 黄振航, 等. 树形结构卷积神经网络优化的城区遥感图像语义分割[J]. 中国图象图形学报, 2020, 25(5): 1043-1052. doi: 10.11834/jig.190324

    HU W, GAO B C, HUANG Z H, et al. Semantic segmentation of urban remote sensing image based on optimized tree structure convolutional neural network[J]. Journal of Image and Graphics, 2020, 25(5): 1043-1052(in Chinese). doi: 10.11834/jig.190324
    [25] KINGMA D, BA J. Adam: A method for stochastic optimization[C]// International Conference on Learning Representations (ICLR). [S.1.]: ICLR, 2015.
    [26] CHEN L C, PAPANDREOU G, KOKKINOS I, et al. DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(4): 834-848. doi: 10.1109/TPAMI.2017.2699184
    [27] LI H F, QIU K J, CHEN L, et al. SCAttNet: Semantic segmentation network with spatial and channel attention mechanism for high-resolution remote sensing images[J]. IEEE Geoscience and Remote Sensing Letters, 2021, 18(5): 905-909.
  • 加载中
图(6) / 表(6)
计量
  • 文章访问数:  292
  • HTML全文浏览量:  80
  • PDF下载量:  26
  • 被引次数: 0
出版历程
  • 收稿日期:  2021-09-10
  • 录用日期:  2022-02-25
  • 网络出版日期:  2022-03-18
  • 整期出版日期:  2023-07-31

目录

    /

    返回文章
    返回
    常见问答