北京航空航天大学学报 ›› 2021, Vol. 47 ›› Issue (3): 665-674.doi: 10.13700/j.bh.1001-5965.2020.0450

• 论文 • 上一篇    

一种结合全局和局部相似性的小样本分割方法

刘宇轩, 孟凡满, 李宏亮, 杨嘉莹, 吴庆波, 许林峰   

  1. 电子科技大学 信息与通信工程学院, 成都 610000
  • 收稿日期:2020-08-24 发布日期:2021-04-08
  • 通讯作者: 孟凡满 E-mail:fmmeng@uestc.edu.cn
  • 作者简介:刘宇轩,男,硕士研究生。主要研究方向:计算机视觉与图像分割;孟凡满,男,博士,副教授,博士生导师。主要研究方向:智能图像分析、深度学习。
  • 基金资助:
    国家自然科学基金(61871087);四川省科技厅自然科学基金(2018JY0141)

A few shot segmentation method combining global and local similarity

LIU Yuxuan, MENG Fanman, LI Hongliang, YANG Jiaying, WU Qingbo, XU Linfeng   

  1. School of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu 610000, China
  • Received:2020-08-24 Published:2021-04-08
  • Supported by:
    National Natural Science Foundation of China (61871087); Natural Science Foundation of Sichuan Science and Technology Department (2018JY0141)

摘要: 针对小样本分割中如何提取支持图像和查询图像共性信息的问题,提出一种新的小样本分割模型,同时结合了全局相似性和局部相似性,实现了更具泛化能力的小样本分割。具体地,根据支持图像和查询图像全局特征和局部特征之间的相似性,提出了一种新型注意力谱生成器,进而实现查询图像的注意力谱生成和区域分割。所提注意力谱生成器包含2个级联模块:全局引导器和局部引导器。在全局引导器中,提出了一种新的基于指数函数的全局相似性度量,对查询图像特征和支持图像的全局特征进行关系建模,输出前景增强的查询图像特征。在局部引导器中,通过引入局部关系矩阵对支持图像特征和查询图像特征之间的局部相似性进行建模,得到与类别无关的注意力谱。在Pascal-5i数据集上做了大量的实验,在1-shot设定下mIoU达到了59.9%,5-shot设定下mIoU达到了61.9%,均优于现有方法。

关键词: 小样本语义分割, 全局相似性测度, 局部相似性测度, 知识迁移, 度量学习

Abstract: Few shot segmentation aims at segmenting objects of novel classes with few annotated images, whose key is to extract the general information between support and query images. The existing methods utilize global features or local features to obtain the general information and validate their effectiveness. However, these two kinds of features' similarity are considered separately in the existing methods, while their mutual effect is ignored. This paper proposes a novel few shot segmentation model using both global and local similarity to achieve more generalizable few shot segmentation. Specifically, an attention generator is proposed to build the attention map of query images based on the relationship between support and query images. The proposed attention generator consists of two cascaded modules: global guider and local guider. In global guider, a novel exponential function based global similarity metric is proposed to model the relationship between query images and support images with respect to global feature, and output foreground-enhanced query image features. As for local guider, a local relationship matrix is introduced to model the local similarity between support and query image features, and obtain the class-agnostic attention map. Comprehensive experiments are performed on Pascal-5i idataset. The proposed method achieves a mean IoU value of 59.9% under 1-shot setting, and a mean IoU of 61.9% under 5-shot setting, which outperforms many state-of-the-art methods.

Key words: few shot semantic segmentation, global similarity measure, local similarity measure, knowledge transferring, metric learning

中图分类号: 


版权所有 © 《北京航空航天大学学报》编辑部
通讯地址:北京市海淀区学院路37号 北京航空航天大学学报编辑部 邮编:100191 E-mail:jbuaa@buaa.edu.cn
本系统由北京玛格泰克科技发展有限公司设计开发