Lightweight fault diagnosis of rolling bearings based on improved linear attention Transformer
-
摘要:
针对基于 Transformer 滚动轴承故障诊断算法计算复杂度随输入时间窗口呈二次增长，导致模型推理实时性能下降的问题，提出一种基于改进线性注意力 Transformer 轻量化故障诊断模型。提出改进线性注意力使用交换点积计算次序策略，降低二次计算复杂度；提出改进的线性注意力特征恢复模块，通过特征偏置映射函数代替 Softmax 全局映射，缓解使用全局接受域的计算开销，该偏置函数具有高效的特征聚焦机制，通过强化相似特征间的联系并削弱不相似特征间的耦合，展现出显著的抗噪声干扰特性；使用特征多样性恢复模块，近似获得原始自注意力全局激活后的性能，恢复对长期依赖关系的建模能力。在来自西安交通大学和渥太华大学的3个机械故障数据集上进行实验，相比 CLFormer、ConvFormer-NSE、MCSwin-T、MobileNet、MobileNet-V2、ResNet18、MK-ResCNN 这 7 种典型模型，结果表明，所提模型在准确率和实时性方面优于对比模型，同时在高噪声环境下有良好的鲁棒性。可视化特征偏置函数模块的权重信息，建立所提模型与预测结果的可解释关系。通过消融实验验证了所提模块（特征偏置函数，特征多样性恢复模块）的有效性。
Abstract:The Transformer-based rolling bearing fault diagnosis algorithms have a quadratic increase in computational complexity with the input time window, leading to a decrease in the real-time performance of the model inference. To address this problem, a lightweight Transformer fault diagnosis model based on improved linear attention is proposed. Firstly, the improved linear attention is proposed to reduce the quadratic computational complexity, which uses the strategy of changing the computation order of the dot product. Secondly, by substituting the suggested feature bias mapping function for the Softmax global mapping function, the enhanced linear attention feature recovery block is suggested, which reduces the computational burden of utilizing the global acceptance field. At the same time, the bias function has an efficient feature focusing mechanism, which demonstrates significant anti-noise interference properties by strengthening the connection between similar features and weakening the coupling between dissimilar features. Then, the feature diversity recovery block is used to approximate the performance of the original self-attention after global activation and to recover the modeling ability for long-term temporal dependencies. Experiments are conducted on three mechanical failure datasets from Xi'an Jiaotong University and the University of Ottawa. Compared with seven typical models, namely CLFormer, ConvFormer-NSE, MCSwin-T, MobileNet, MobileNet-V2, ResNet18 and MK-ResCNN, the results show that the proposed model outperforms the above models in terms of accuracy and real time performance, and has good robustness in heavy noise environments at the same time. To create a comprehensible connection between the suggested approach and the prediction outcomes, visualize the feature bias mapping function's weight information. Finally, the effectiveness of the proposed modules (feature bias mapping function, feature diversity recovery module) is verified by ablation experiments.
-
Key words:
- fault diagnosis /
- heavy noise /
- lightweight /
- linear attention /
- feature diversity
-
图 2 Transformer和线性自注意力计算复杂度对比
Figure 2. Comparison of computational complexity of Transformer and linear self-attention
图 5 不同模型在Gearbox数据集上的平均准确率和复杂度比较结果
Figure 5. Comparative results of average accuracy and complexity of different models on Gearbox dataset
图 6 不同模型在Gearbox数据集上的平均损失和准确率比较结果
Figure 6. Comparative results of average loss and accuracy of different models on Gearbox dataset
图 7 不同模型在Spur gear数据集上的准确率比较结果
Figure 7. Comparative results of accuracy of different models on Spur gear dataset
图 9 不同模型在轴承数据集上的特征评价指标比较结果
Figure 9. Comparative results of feature evaluation index of different models on Bearing dataset
图 10 有/无$ {f}_{\text{fb}}(\boldsymbol{x}) $的qkv权重对比
Figure 10. Comparison of qkv weights with/without $ {f}_{\text{fb}}(\boldsymbol{x}) $
图 11 有/无$ {f}_{\text{fb}}(\boldsymbol{x}) $的注意力权重对比
Figure 11. Comparison of attention weights with/without $ {f}_{\text{fb}}(\boldsymbol{x}) $
图 12 综合准确率与特征评价指标对比结果
Figure 12. Comparative results of comprehensive accuracy and feature evaluation index
图 15 各模块在轴承数据集上的准确率对比结果
Figure 15. Comparative results of accuracy of each block on Bearing dataset
图 16 不同$ \alpha $在轴承数据集上的准确率对比结果
Figure 16. Comparative results of accuracy of different $ \alpha $ on Bearing dataset
表 1 本文模型各层网络结构参数
Table 1. Structural parameters of network at each layer of the proposed model
模型结构 模型参数 尺寸变换/像素×像素×像素 一维平均池化层 s=2 $ 32\times 1\times 1024\rightarrow 32\times 1\times 512 $ 卷积层 dout=64,k=15,s=2 $ 32\times 64\times 256\rightarrow 32\times 64\times 256 $ 轻量化特征提取层 din=dout=64 $ 32\times 1\times 512\rightarrow 32\times 64\times 256 $ 一维平均池化层 N=1 $ 32\times 64\times 256\rightarrow 32\times 64\times 1 $ 全连接层 dout=c $ 32\times 64\times 1\rightarrow 32\times {c} $ 
下载: 导出CSV
表 2 Gearbox数据集
Table 2. Gearbox dataset
故障编码 故障模式 1 滚珠故障 2 内滚道故障 3 混合故障（滚珠+内滚道+外滚道） 4 外滚道故障 5 断齿故障 6 缺齿故障 7 齿根裂纹故障 8 齿面磨损故障 9 正常状态
下载: 导出CSV
表 3 不同模型在Gearbox数据集上的准确率和复杂度比较结果
Table 3. Comparative results of accuracy and complexity of different models on Gearbox dataset
模型 准确率/% 参数量 浮点运算量/106次 运行时间/s $ \lambda $=0 $ \lambda $=0.2 $ \lambda $=0.4 $ \lambda $=0.6 $ \lambda $=0.8 $ \lambda $=1 训练时间 推理时间 本文模型 99.88 97.81 84.66 73.22 67.40 62.13 6×103 2.869 36.15 0.31 CLFormer[41] 90.23 84.96 71.91 63.82 56.99 54.24 5×103 0.144 203.08 0.54 ConvFormer-NSE[31] 82.68 76.91 70.80 63.18 53.25 51.47 2.45×105 6.270 259.34 0.66 MCSwin-T[42] 99.92 97.39 82.17 70.88 65.06 60.52 1.937×106 227.003 258.30 0.61 MobileNet[43] 99.47 89.01 74.61 67.59 58.21 54.99 3.186×106 333.620 201.75 0.85 MobileNet-V2[44] 98.53 89.87 75.96 69.59 60.18 55.03 2.192×106 96.955 324.04 0.76 ResNet18[45] 99.63 94.56 80.95 70.67 61.41 54.51 3.854×106 175.920 171.80 0.55 MK-ResCNN[46] 99.79 96.13 80.88 69.46 61.65 57.12 2.117×106 83.893 189.90 0.53
下载: 导出CSV
表 4 不同模型在Gearbox数据集上的特征评价指标比较结果
Table 4. Comparative results of feature evaluation index of different models on Gearbox dataset
模型 J $ \lambda $=0 $ \lambda $=0.2 $ \lambda $=0.4 $ \lambda $=0.6 $ \lambda $=0.8 $ \lambda $=1 本文模型 2.78 2.35 1.68 1.53 1.21 1.12 CLFormer[41] 2.34 1.98 1.22 1.06 1.00 0.82 ConvFormer-NSE[31] 3.70 3.08 2.00 1.31 0.83 0.73 MCSwin-T[42] 3.62 2.78 1.94 1.32 1.09 0.99 MobileNet[43] 2.96 2.36 1.45 1.06 0.64 0.59 MobileNet-V2[44] 2.72 2.31 1.52 1.28 0.91 0.73 ResNet18[45] 2.86 2.30 1.61 1.26 0.91 0.78 MK-ResCNN[46] 2.76 2.32 1.51 1.02 0.85 0.70
下载: 导出CSV
表 6 Spur gear数据集转速
Table 6. Spur gear dataset rotational speed
数据集 源域转速/（r·min−1） 目标域转速/（r·min−1） D1 900 1200 D2 1200 900
下载: 导出CSV
表 7 不同模型在Spur gear数据集上的准确率和复杂度比较结果
Table 7. Comparative results of accuracy and complexity of different models on Spur gear dataset
模型 D1-准确率/% D2-准确率/% 参数量 浮点运算量/106次 运行时间/s $ \lambda $=0.2 $ \lambda $=0.4 $ \lambda $=0.6 $ \lambda $=0.2 $ \lambda $=0.4 $ \lambda $=0.6 训练时间 推理时间 本文模型 86.25 77.45 69.14 88.81 79.85 70.65 2.5×104 13.032 31.86 0.92 CLFormer[41] 71.79 62.17 54.61 71.12 60.13 53.12 5×103 0.246 102.50 1.14 ConvFormer-NSE[31] 73.50 62.32 56.45 70.77 56.80 54.29 2.45×105 6.760 129.28 1.13 MCSwin-T[42] 84.45 75.17 65.58 84.01 74.93 64.39 1.946×106 231.959 132.08 1.10 MobileNet[43] 69.43 58.96 53.09 73.75 59.95 53.55 3.183×106 334.599 92.79 1.26 MobileNet-V2[44] 74.93 61.66 56.95 71.25 59.32 54.13 2.188×106 97.441 155.67 1.21 ResNet18[45] 83.82 71.09 61.29 80.95 63.69 58.71 3.856×106 178.212 85.32 0.99 MK-ResCNN[46] 79.88 66.49 57.68 80.26 64.94 57.10 2.119×106 86.184 97.26 0.99
下载: 导出CSV
表 8 不同模型在Spur gear数据集上的特征评价指标比较结果
Table 8. Comparative results of feature evaluation index of different models on Spur gear dataset
模型 D1-J D2-J $ \lambda $=0 $ \lambda $=0.2 $ \lambda $=0.4 $ \lambda $=0.6 $ \lambda $=0.8 $ \lambda $=1 $ \lambda $=0 $ \lambda $=0.2 $ \lambda $=0.4 $ \lambda $=0.6 $ \lambda $=0.8 $ \lambda $=1 本文模型 2.79 1.91 1.40 1.20 0.94 0.84 3.15 1.95 1.35 1.06 0.88 0.79 CLFormer[41] 2.92 1.29 0.93 0.75 0.65 0.63 3.89 1.19 0.87 0.70 0.66 0.59 ConvFormer-NSE[31] 3.89 1.45 0.97 0.78 0.69 0.65 3.60 1.27 0.88 0.73 0.70 0.62 MCSwin-T[42] 2.80 2.06 1.37 1.15 0.87 0.71 3.40 1.87 1.31 0.99 0.78 0.71 MobileNet[43] 2.64 0.92 0.62 0.48 0.32 0.31 2.59 1.06 0.63 0.59 0.44 0.38 MobileNet-V2[44] 2.65 1.17 0.83 0.72 0.64 0.58 3.29 1.13 0.81 0.67 0.60 0.59 ResNet18[45] 2.53 1.50 1.12 0.85 0.72 0.64 2.79 1.40 0.93 0.83 0.72 0.60 MK-ResCNN[46] 2.61 1.38 0.96 0.79 0.62 0.62 3.08 1.43 0.98 0.79 0.67 0.61
下载: 导出CSV
表 10 不同模型在轴承数据集上的准确率和复杂度比较结果
Table 10. Comparative results of accuracy and complexity of different models on Bearing dataset
模型 准确率/% 参数量 浮点运算量/106次 运行时间/s $ \lambda $=0 $ \lambda $=0.2 $ \lambda $=0.4 $ \lambda $=0.6 $ \lambda $=0.8 $ \lambda $=1 训练时间 推理时间 本文模型 96.45 95.43 88.79 78.52 70.13 64.35 4×103 1.852 24.07 0.11 CLFormer[41] 91.57 84.91 74.87 68.59 63.96 59.84 5×103 0.100 132.25 0.21 ConvFormer-NSE[31] 95.12 69.36 59.21 58.87 58.43 56.33 2×105 6.220 166.06 0.25 MCSwin-T[42] 97.79 75.60 65.77 62.08 59.12 58.40 1.936×106 226.507 173.94 0.25 MobileNet[43] 96.33 65.49 62.99 60.08 58.92 57.47 3.182×106 333.518 133.07 0.34 MobileNet-V2[44] 96.24 80.81 66.72 62.91 58.37 55.67 2.187×106 96.901 203.35 0.29 ResNet18[45] 97.61 82.35 69.11 66.36 65.13 64.25 3.851×106 175.689 114.69 0.21 MK-ResCNN[46] 98.44 84.87 72.35 68.81 64.09 61.23 2.114×106 83.661 131.69 0.21
下载: 导出CSV
表 11 不同模型在轴承数据集上的特征评价指标比较结果
Table 11. Comparative results of feature evaluation index of different models on Bearing dataset
方法 J $ \lambda $=0 $ \lambda $=0.2 $ \lambda $=0.4 $ \lambda $=0.6 $ \lambda $=0.8 $ \lambda $=1 本文模型 1.64 1.61 1.37 1.10 0.91 0.77 CLFormer[41] 1.82 1.60 1.32 1.08 0.93 0.75 ConvFormer-NSE[31] 4.06 1.47 0.91 0.78 0.73 0.64 MCSwin-T[42] 2.07 1.20 0.75 0.57 0.53 0.45 MobileNet[43] 1.96 1.02 0.66 0.56 0.53 0.51 MobileNet-V2[44] 1.94 1.49 0.92 0.81 0.60 0.56 ResNet18[45] 1.83 1.32 0.89 0.84 0.70 0.65 MK-ResCNN[46] 1.93 1.31 0.95 0.74 0.61 0.50
下载: 导出CSV
表 12 各模块在轴承数据集上的准确率对比结果
Table 12. Comparative results of accuracy of each block on Bearing dataset
不同模块 准确率/% 参数量 浮点运算量/106次 运行时间/s $ \lambda $=0 $ \lambda $=0.2 $ \lambda $=0.4 $ \lambda $=0.6 $ \lambda $=0.8 $ \lambda $=1 训练时间 推理时间 $ {f}_{\text{fb}}(\boldsymbol{x}) $+DC 96.45 95.43 88.79 78.52 70.13 64.35 4×103 1.852 24.07 0.11 Softmax+DC 96.17 95.20 87.73 77.03 67.80 61.40 4×103 1.852 25.41 0.12 Softmax 95.83 94.65 87.13 76.56 67.10 60.87 4×103 1.852 24.12 0.12 $ {f}_{\text{fb}}(\boldsymbol{x}) $ 95.42 93.25 86.07 76.10 66.70 60.10 4×103 1.852 22.31 0.11
下载: 导出CSV
表 13 不同$ \alpha $在轴承数据集上的准确率对比结果
Table 13. Comparative results of accuracy of different $ \alpha $ on Bearing dataset
$ \alpha $ 准确率/% $ \lambda $=0 $ \lambda $=0.2 $ \lambda $=0.4 $ \lambda $=0.6 $ \lambda $=0.8 $ \lambda $=1 2 95.20 94.00 86.20 75.93 70.01 63.53 3 96.45 95.43 88.79 78.52 70.13 64.35 4 96.36 95.28 86.25 77.60 68.29 62.07 5 96.22 95.19 87.40 78.16 69.00 63.39 6 95.87 94.80 86.25 77.48 70.02 63.33
下载: 导出CSV
-
[1] Zhou G H, Zhao E K, Yang R H, et al. A new complex system fault detection method based on belief rule base for unreliable interval values[J]. Measurement Science and Technology, 2023, 34(8): 084007. [2] 邵海东, 林健, 闵志闪, 等. 分布外样本干扰下基于改进半监督原型网络的齿轮箱跨域故障诊断[J]. 机械工程学报, 2024, 60(4): 212-221.Shao H D, Lin J, Min Z S, et al. Improved semi-supervised prototype network for cross-domain fault diagnosis of gearbox under out-of-distribution interference samples[J]. Journal of Mechanical Engineering, 2024, 60(4): 212-221(in Chinese). [3] 邵海东, 李伟, 刘翊, 等. 时变转速下基于双阈值注意力生成对抗网络和小样本的转子-轴承系统故障诊断[J]. 机械工程学报, 2023, 59(12): 215-224.Shao H D, Li W, Liu Y, et al. Fault diagnosis of rotor-bearing system under time-varying speeds by using dual-threshold attention-embedded GAN and small samples[J]. Journal of Mechanical Engineering, 2023, 59(12): 215-224(in Chinese). [4] Gao Z W, Cecati C, Ding S X. A survey of fault diagnosis and fault-tolerant techniques: part I: fault diagnosis with model-based and signal-based approaches[J]. IEEE Transactions on Industrial Electronics, 2015, 62(6): 3757-3767. [5] Zhu L H, Liu J F, Liu J, et al. Frequent pattern extraction based on data and prior knowledge fusion in gas turbine combustion system[C]//Proceedings of the Prognostics and System Health Management Conference. Piscataway: IEEE Press, 2017: 1-8. [6] Kumar A, Parkash C, Vashishtha G, et al. State-space modeling and novel entropy-based health indicator for dynamic degradation monitoring of rolling element bearing[J]. Reliability Engineering & System Safety, 2022, 221: 108356. [7] Chen Y Z, Zhao X D, Xiang H C, et al. A sequential model-based approach for gas turbine performance diagnostics[J]. Energy, 2021, 220: 119657. [8] Zhang L W, Lin J, Liu B, et al. A review on deep learning applications in prognostics and health management[J]. IEEE Access, 2019, 7: 162415-162438. [9] 邵海东, 肖一鸣, 颜深. 仿真数据驱动的改进无监督域适应轴承故障诊断[J]. 机械工程学报, 2023, 59(3): 76-85.Shao H D, Xiao Y M, Yan S. Simulation data-driven enhanced unsupervised domain adaptation for bearing fault diagnosis[J]. Journal of Mechanical Engineering, 2023, 59(3): 76-85 (in Chinese). [10] 曹洁, 尹浩楠, 雷晓刚, 等. 基于领域自适应的变工况轴承故障诊断[J]. 北京航空航天大学学报, 2024, 50(8): 2382-2390.Cao J, Yin H N, Lei X G, et al. Bearing fault diagnosis in variable working conditions based on domain adaptation[J]. Journal of Beijing University of Aeronautics and Astronautics, 2024, 50(8): 2382-2390(in Chinese). [11] 邵海东, 陈星恺, 曹鸿儒, 等. 内对抗指导无监督多域适配网络的轴承协同故障诊断[J]. 中国科学: 技术科学, 2023, 53(7): 1229-1240.Shao H D, Chen X K, Cao H R, et al. Internal adversarial guided unsupervised multi-domain adaptation network for collaborative fault diagnosis of bearing[J]. Scientia Sinica (Technologica), 2023, 53(7): 1229-1240(in Chinese). [12] Zhou H Y, Ying Y L, Li J C, et al. Long-short term memory and gas path analysis based gas turbine fault diagnosis and prognosis[J]. Advances in Mechanical Engineering, 2021, 13(8): 1-12. [13] Bai M L, Liu J F, Ma Y J, et al. Long short-term memory network-based normal pattern group for fault detection of three-shaft marine gas turbine[J]. Energies, 2020, 14(1): 13. [14] Li X G, Jiang X X, Wang Q, et al. Multi-perspective deep transfer learning model: a promising tool for bearing intelligent fault diagnosis under varying working conditions[J]. Knowledge-Based Systems, 2022, 243: 108443. [15] Han S Y, Shao H D, Huo Z Q, et al. End-to-end chiller fault diagnosis using fused attention mechanism and dynamic cross-entropy under imbalanced datasets[J]. Building and Environment, 2022, 212: 108821. [16] Fang H R, Deng J, Chen D S, et al. You can get smaller: a lightweight self-activation convolution unit modified by transformer for fault diagnosis[J]. Advanced Engineering Informatics, 2023, 55: 101890. [17] Han K, Wang Y H, Chen H T, et al. A survey on vision transformer[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(1): 87-110. [18] Xiao Y M, Shao H D, Feng M J, et al. Towards trustworthy rotating machinery fault diagnosis via attention uncertainty in transformer[J]. Journal of Manufacturing Systems, 2023, 70: 186-201. [19] Li Y S, Zhou Z, Sun C, et al. Variational attention-based interpretable transformer network for rotary machine fault diagnosis[J]. IEEE Transactions on Neural Networks and Learning Systems, 2024, 35(5): 6180-6193. [20] Chen Z H, Chen J L, Liu S, et al. Multi-channel Calibrated Transformer with Shifted Windows for few-shot fault diagnosis under sharp speed variation[J]. ISA Transactions, 2022, 131: 501-515. [21] Ding Y F, Jia M P, Miao Q H, et al. A novel time-frequency Transformer based on self-attention mechanism and its application in fault diagnosis of rolling bearings[J]. Mechanical Systems and Signal Processing, 2022, 168: 108616. [22] Liu W, Chen W, Zhang Z H. A novel fault diagnosis approach for rolling bearing based on high-order synchrosqueezing transform and detrended fluctuation analysis[J]. IEEE Access, 2020, 8: 12533-12541. [23] Wu R, Liu C, Han T, et al. A planetary gearbox fault diagnosis method based on time-series imaging feature fusion and a transformer model[J]. Measurement Science and Technology, 2023, 34(2): 024006. [24] Ding Y F, Jia M P. Convolutional transformer: an enhanced attention mechanism architecture for remaining useful life estimation of bearings[J]. IEEE Transactions on Instrumentation and Measurement, 2022, 71: 3515010. [25] Han S Y, Shao H D, Cheng J S, et al. Convformer-NSE: a novel end-to-end gearbox fault diagnosis framework under heavy noise using joint global and local information[J]. IEEE/ASME Transactions on Mechatronics, 2023, 28(1): 340-349. [26] 赵洪利, 杨佳强. 基于融合卷积Transformer的航空发动机故障诊断[J]. 北京航空航天大学学报, 2025, 51(4): 1117-1126.Zhao H L, Yang J Q. Aero-engine fault diagnosis based on fusion convolutional Transformer[J]. Journal of Beijing University of Aeronautics and Astronautics, 2025, 51(4): 1117-1126(in Chinese). [27] Choromanski K, Likhosherstov V, Dohan D, et al. Rethinking attention with performers[EB/OL]. (2020-09-30)[2023-10-20]. https://arxiv.org/abs/2009.14794. [28] Bolya D, Fu C Y, Dai X L, et al. Hydra attention: efficient attention with many heads[EB/OL]. (2022-09-15)[2026-04-15]. https://arxiv.org/abs/2209.07484. [29] Wang R, Zhang S J, Liu S Q, et al. A bearing fault diagnosis method for high-noise and unbalanced dataset[J]. Smart and Resilient Transportation, 2023, 5(1): 28-45. [30] Zaheer M, Guruganesh G, Dubey A, et al. Big bird: transformers for longer sequences[EB/OL]. (2020-07-28)[2026-04-15]. https://arxiv.org/abs/2007.14062. [31] Beltagy I, Peters M E, Cohan A. Longformer: the long-document transformer[EB/OL]. (2020-04-10)[2026-04-15]. https://arxiv.org/abs/2004.05150. [32] Katharopoulos A, Vyas A, Pappas N, et al. Transformers are RNNs: fast autoregressive transformers with linear attention[EB/OL]. (2020-06-29)[2026-04-15]. https://arxiv.org/abs/2006.16236. [33] Shen Z R, Zhang M Y, Zhao H Y, et al. Efficient attention: attention with linear complexities[EB/OL]. (2018-12-04)[2026-04-15]. https://arxiv.org/abs/1812.01243. [34] Lu J C, Yao J H, Zhang J G, et al. SOFT: softmax-free transformer with linear complexity[EB/OL]. (2021-10-22)[2026-04-15]. https://arxiv.org/abs/2110.11945. [35] Tian C W, Zheng M H, Zuo W M, et al. A cross Transformer for image denoising[J]. Information Fusion, 2024, 102: 102043. [36] Shao R, Shi Z, Yi J, et al. On the adversarial robustness of vision Transformers[EB/OL]. (2021-03-29)[2026-04-15]. https://arxiv.org/abs/2103.15670. [37] Guo Y, Stutz D, Schiele B. Robustifying token attention for vision transformers[EB/OL]. (2023-03-20)[2026-04-15]. https://arxiv.org/abs/2303.11126. [38] Wang S N, Li B Z, Khabsa M, et al. Linformer: self-attention with linear complexity[EB/OL]. (2020-06-08)[2026-04-15]. https://arxiv.org/abs/2006.04768. [39] Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. New York: ACM, 2017: 6000-6010. [40] Xiong Y Y, Zeng Z P, Chakraborty R, et al. Nyströmformer: a Nyström-based algorithm for approximating self-attention[EB/OL]. (2021-02-07)[2026-04-15]. https://arxiv.org/abs/2102.03902. [41] Fang H R, Deng J, Bai Y X, et al. CLFormer: a lightweight transformer based on convolutional embedding and linear self-attention with strong robustness for bearing fault diagnosis under limited sample conditions[J]. IEEE Transactions on Instrumentation and Measurement, 2022, 71: 3504608. [42] Dong X Y, Bao J M, Chen D D, et al. CSWin transformer: a general vision transformer backbone with cross-shaped windows[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2022: 12114-12124. [43] Howard A G, Zhu M L, Chen B, et al. MobileNets: efficient convolutional neural networks for mobile vision applications[EB/OL]. (2017-04-17)[2026-04-15]. https://arxiv.org/abs/1704.04861. [44] Sandler M, Howard A, Zhu M L, et al. MobileNetV2: inverted residuals and linear bottlenecks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2018: 4510-4520. [45] He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2016: 770-778. [46] Liu R N, Wang F, Yang B Y, et al. Multiscale kernel based residual convolutional neural network for motor fault diagnosis under nonstationary conditions[J]. IEEE Transactions on Industrial Informatics, 2020, 16(6): 3797-3806. [47] Zhao X L, Yao J Y, Deng W X, et al. Intelligent fault diagnosis of gearbox under variable working conditions with adaptive intraclass and interclass convolutional neural network[J]. IEEE Transactions on Neural Networks and Learning Systems, 2023, 34(9): 6339-6353. [48] Li T F, Zhou Z, Li S N, et al. The emerging graph neural networks for intelligent fault diagnostics and prognostics: a guideline and a benchmark study[J]. Mechanical Systems and Signal Processing, 2022, 168: 108653. [49] Yan S, Shao H D, Wang J, et al. LiConvFormer: a lightweight fault diagnosis framework using separable multiscale convolution and broadcast self-attention[J]. Expert Systems with Applications, 2024, 237: 121338. [50] Huang H, Baddour N. Bearing vibration data collected under time-varying rotational speed conditions[J]. Data in Brief, 2018, 21: 1745-1749. -
下载:
点击查看大图
计量
- 文章访问数: 545
- HTML全文浏览量: 151
- PDF下载量: 43
- 被引次数: 0
