留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于自注意力语义分割的航空发动机孔探图像检测

曹斯言 刘君强 宋高腾 左洪福

曹斯言,刘君强,宋高腾,等. 基于自注意力语义分割的航空发动机孔探图像检测[J]. 北京航空航天大学学报,2023,49(6):1504-1515 doi: 10.13700/j.bh.1001-5965.2021.0448
引用本文: 曹斯言,刘君强,宋高腾,等. 基于自注意力语义分割的航空发动机孔探图像检测[J]. 北京航空航天大学学报,2023,49(6):1504-1515 doi: 10.13700/j.bh.1001-5965.2021.0448
CAO S Y,LIU J Q,SONG G T,et al. Borehole image detection of aero-engine based on self-attention semantic segmentation model[J]. Journal of Beijing University of Aeronautics and Astronautics,2023,49(6):1504-1515 (in Chinese) doi: 10.13700/j.bh.1001-5965.2021.0448
Citation: CAO S Y,LIU J Q,SONG G T,et al. Borehole image detection of aero-engine based on self-attention semantic segmentation model[J]. Journal of Beijing University of Aeronautics and Astronautics,2023,49(6):1504-1515 (in Chinese) doi: 10.13700/j.bh.1001-5965.2021.0448

基于自注意力语义分割的航空发动机孔探图像检测

doi: 10.13700/j.bh.1001-5965.2021.0448
基金项目: 国家自然科学基金(U1533128,U1933202);中央高校基本科研业务费专项资金(NS2020050)
详细信息
    作者简介:

    曹斯言 男,硕士研究生。主要研究方向:航空发动机故障诊断与寿命预测

    刘君强 男,博士,副教授,硕士生导师。主要研究方向:航空发动机健康管理,机场运行与管理

    通讯作者:

    E-mail:liujunqiang@nuaa.edu.cn

  • 中图分类号: V263.6;TP391.41

Borehole image detection of aero-engine based on self-attention semantic segmentation model

Funds: National Natural Science Foundation of China (U1533128,U1933202); Research on key technology of insitu minimally invasive intelligent maintenance for civil aviation engine (NS2020050)
More Information
  • 摘要:

    针对传统语义分割模型对于航空发动机孔探图像内损伤的检测存在小尺度或高相似度损伤易被漏检误判的问题,提出了一种基于自注意力语义分割(SA-SS)模型的航空发动机孔探图像检测方法。基于语义分割模型DeepLabv3+的总体架构,采用轻量级MobileNetV2替代原始的Xception作为主干特征提取网络,利用扩张—提取—压缩的结构进行特征提取,以减少模型计算量。基于多层级联结构,改进原始DeepLabv3+的空洞空间金字塔池化结构,使特征图保有更丰富的特征信息。在模型内融合一种自注意力机制,建立全局像素的内部相关性,加强对细节特征的注意力。改进原始DeepLabv3+的解码层,将多尺度空间融合方法引入低层特征提取,融合多个跃层特征。实验结果表明:与传统DeepLabv3+、SegNet-ResNet等方法相比,SA-SS模型的平均交并比和平均像素精确度最大分别提升了4.10%和3.92%,训练时间和平均检测速度最大分别改善了24.43%和5.11 帧/s。

     

  • 图 1  SA-SS模型总体结构

    Figure 1.  Overall structure of SA-SS model

    图 2  MobileNetV2网络结构

    Figure 2.  Network structure of MobileNetV2

    图 3  自注意力机制结构

    Figure 3.  Structure of self-attention

    图 4  两种形式空洞卷积

    Figure 4.  Multi-layer cascaded atrous convolution

    图 5  不同空洞率组合下mASPP的感受野

    Figure 5.  Receptive field of mASPP under different atrous rate combinations

    图 6  4类典型损伤

    Figure 6.  Four types of typical faults

    图 7  打标后的图像

    Figure 7.  Marked images

    图 8  模型训练损失值变化

    Figure 8.  Change of training loss

    图 9  4种模型对不同损伤类型的检测性能比较

    Figure 9.  Comparison of performance of four models for different types of faults

    图 10  各模型可视化结果

    Figure 10.  Visualization results of each method

    图 11  训练时间及平均检测速度对比

    Figure 11.  Comparison of training time and average test speed

    表  1  MobileNetV2网络参数

    Table  1.   Parameters of MobileNetV2

    输入操作tpqs
    2242× 3conv2d3212
    1122× 32Block11611
    1122× 16Block62422
    562× 24Block63232
    282× 32Block66442
    142× 64Block69631
    142× 96Block616032
    72× 160Block632011
    72× 320conv2d128011
    下载: 导出CSV

    表  2  数据集信息

    Table  2.   Dataset information

    标签类型数量/张尺寸/(像素×像素)掩膜颜色
    background背景
    burn烧蚀631513×513
    coat剥蚀656513×513绿
    crack裂纹639513×513
    material掉块574513×513
    下载: 导出CSV

    表  3  4种模型平均性能

    Table  3.   Average performance of four models %

    模型测试集
    MIoU
    测试集
    mPA
    验证集
    MIoU
    验证集
    mPA
    A80.1588.4779.7089.54
    B82.4590.2582.1192.76
    C83.5591.7683.7292.59
    D84.2592.3985.1494.69
    下载: 导出CSV
  • [1] 陈果, 汤洋. 基于孔探图像纹理特征的航空发动机损伤识别方法[J]. 仪器仪表学报, 2008, 29(8): 1709-1713.

    CHEN G, TANG Y. Aero-engine interior damage recognition based on texture features of borescope image[J]. Chinese Journal of Scientific Instrument, 2008, 29(8): 1709-1713(in Chinese).
    [2] 樊玮, 李晨炫, 邢艳, 等. 航空发动机损伤图像的二分类到多分类递进式检测网络[J]. 计算机应用, 2021, 41(8): 2352-2357.

    FAN W, LI C X, XING Y, et al. Two-class to multi-class progressive detection network for aero-engine damage images[J]. Journal of Computer Applications, 2021, 41(8): 2352-2357(in Chinese).
    [3] 黄鹏, 郑淇, 梁超. 图像分割方法综述[J]. 武汉大学学报(理学版), 2020, 66(6): 519-531.

    HUANG P, ZHENG Q, LIANG C. Overview of image segmentation methods[J]. Journal of Wuhan University(Natural Science Edition), 2020, 66(6): 519-531(in Chinese).
    [4] 杨宇. 数字图像处理在航空发动机孔探检测技术中的应用[D]. 沈阳: 东北大学, 2011: 8-15.

    YANG Y. Application of digital image processing in the borescope inspection technology of aeroengine[D]. Shenyang: Northeastern University, 2011: 8-15(in Chinese).
    [5] 张勇, 刘冠军, 邱静. 基于图像自动测量的航空发动机故障检测技术研究[J]. 机械科学与技术, 2008, 27(2): 176-179. doi: 10.3321/j.issn:1003-8728.2008.02.009

    ZHANG Y, LIU G J, QIU J. Aeroengine’s fault detection technology based on image automatic measurement[J]. Mechanical Science and Technology for Aerospace Engineering, 2008, 27(2): 176-179(in Chinese). doi: 10.3321/j.issn:1003-8728.2008.02.009
    [6] 张维亮. 航空发动机叶片损伤图像快速识别技术研究[D]. 沈阳: 沈阳航空航天大学, 2014: 18-23.

    ZHANG W L. Research on technologies of aeroengine blades damage image fast recognition[D]. Shenyang: Shenyang Aerospace University, 2014: 18-23(in Chinese).
    [7] REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149. doi: 10.1109/TPAMI.2016.2577031
    [8] 张新峰, 郭宇桐, 蔡轶珩, 等. 基于DCNN和全连接CRF的舌图像分割算法[J]. 北京航空航天大学学报, 2019, 45(12): 2364-2374.

    ZHANG X F, GUO Y T, CAI Y H, et al. Tongue image segmentation algorithm based on deep convolutional neural network and fully conditional random fields[J]. Journal of Beijing University of Aeronautics and Astronautics, 2019, 45(12): 2364-2374(in Chinese).
    [9] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB/OL]. (2019-04-10)[2021-08-14].https://arxiv.org/1409.1556.
    [10] 李浩. 基于图像识别的航空发动机叶片裂纹检测研究[D]. 成都: 电子科技大学, 2019: 17-23.

    LI H. Research on the blade crack detection of aero-engine based on image recognition[D]. Chengdu: University of Electronic Science and Technology of China, 2019: 17-23(in Chinese).
    [11] KIM Y H, LEE J R. Videoscope-based inspection of turbofan engine blades using convolutional neural networks and image processing[J]. Structural Health Monitoring, 2019, 18(5-6): 2020-2039. doi: 10.1177/1475921719830328
    [12] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[EB/OL]. (2017-06-12)[2021-08-14]. https://arxiv.org/abs/1706.03762v5.
    [13] 陈永, 陈锦, 陶美风. 多尺度特征和注意力融合的生成对抗壁画修复[J]. 北京航空航天大学学报, 2023, 49(2): 254-264.

    CHEN Y, CHEN J, TAO M F. Mural inpainting generative adversarial networks based on multi-scale feature and attention fusion[J]. Journal of Beijing University of Aeronautics and Astronautics, 2023, 49(2): 254-264(in Chinese).
    [14] NIU R G, SUN X, TIAN Y, et al. HMANet: Hybrid multiple attention network for semantic segmentation in aerial images[EB/OL]. (2020-03-25)[2021-08-14].https://arxiv.org/abs/2001.02870.
    [15] LI Z L, YUAN L M, XU H X, et al. Deep multi-instance learning with induced self-attention for medical image classification[C]//2020 IEEE International Conference on Bioinformatics and Biomedicine(BIBM). Piscataway: IEEE Press, 2021: 446-450.
    [16] SANDLER M, HOWARD A, ZHU M L, et al. MobileNetV2: Inverted residuals and linear bottlenecks[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2018: 4510-4520.
    [17] 刘梦竹. 基于自注意力机制的图像语义分割算法研究[D]. 大连: 大连理工大学, 2020: 20.

    LIU M Z. The research on image semantic segmentation algorithm based on self-attention mechanism[D]. Dalian: Dalian University of Technology, 2020: 20(in Chinese).
    [18] 蒲磊, 冯新喜, 侯志强, 等. 基于级联注意力机制的孪生网络视觉跟踪算法[J]. 北京航空航天大学学报, 2020, 46(12): 2302-2310.

    PU L, FENG X X, HOU Z Q, et al. Siamese network visual tracking algorithm based on cascaded attention mechanism[J]. Journal of Beijing University of Aeronautics and Astronautics, 2020, 46(12): 2302-2310(in Chinese).
    [19] YU F, KOLTUN V. Multi-scale context aggregation by dilated convolutions[EB/OL]. (2016-04-30)[2021-08-14]. https://arxiv.org/abs/1511.07122.
    [20] 李春虹, 卢宇. 基于深度可分离卷积的人脸表情识别[J]. 计算机工程与设计, 2021, 42(5): 1448-1454.

    LI C H, LU Y. Facial expression recognition based on depthwise separable convolution[J]. Computer Engineering and Design, 2021, 42(5): 1448-1454(in Chinese).
    [21] 杨波, 陶青川, 董沛君. 改进Deeplab v3+网络的手术器械分割方法[J]. 计算机工程与应用, 2021, 57(7): 222-227. doi: 10.3778/j.issn.1002-8331.2001-0064

    YANG B, TAO Q C, DONG P J. Surgical instrument segmentation method based on improved Deeplab v3+ network[J]. Computer Engineering and Applications, 2021, 57(7): 222-227(in Chinese). doi: 10.3778/j.issn.1002-8331.2001-0064
    [22] HOFFMAN J, WANG D Q, YU F, et al. FCNs in the wild: Pixel-level adversarial and constraint-based adaptation[EB/OL]. (2016-12-08)[2021-08-14].https://arxiv.org/abs/1612.02649v1.
    [23] SI Y F, GONG D W, GUO Y, et al. An advanced spectral-spatial classification framework for hyperspectral imagery based on DeepLab v3+[J]. Applied Sciences, 2021, 11(12): 5703. doi: 10.3390/app11125703
    [24] IBRAHIM M S, VAHDAT A, RANJBAR M, et al. Semi-supervised semantic image segmentation with self-correcting networks[C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2020: 12712-12722.
  • 加载中
图(11) / 表(3)
计量
  • 文章访问数:  297
  • HTML全文浏览量:  67
  • PDF下载量:  36
  • 被引次数: 0
出版历程
  • 收稿日期:  2021-08-09
  • 录用日期:  2021-10-29
  • 网络出版日期:  2021-11-18
  • 整期出版日期:  2023-06-30

目录

    /

    返回文章
    返回
    常见问答