面向无人机平台的CNN-Transformer混合模型训练后量化方法

赵呈功; 胡纪远; 杨星宇; 陈磊; 姜宏旭

doi:10.13700/j.bh.1001-5965.2025.0411

面向无人机平台的CNN-Transformer混合模型训练后量化方法

doi: 10.13700/j.bh.1001-5965.2025.0411

赵呈功^{1, 2},
胡纪远^{1, 3},
杨星宇⁴,
陈磊⁵,
姜宏旭^3, ,

1.
北京航空航天大学杭州创新研究院，杭州 310056
2.
北京航空航天大学软件学院，北京 100191
3.
北京航空航天大学计算机学院，北京 100191
4.
内蒙古大学计算机学院（软件学院），呼和浩特 010021
5.
杭州萧山国际机场飞行区中心，杭州 311241

详细信息

通讯作者:
E-mail：jianghx@buaa.edu.cn

中图分类号: V221⁺.3；TB553
计量
- 文章访问数: 250
- HTML全文浏览量: 93
- PDF下载量: 13
- 被引次数: 0
出版历程
- 收稿日期: 2025-06-27
- 录用日期: 2025-09-25
- 网络出版日期: 2025-09-30
- 整期出版日期: 2026-09-01

A post-training quantization method for CNN-Transformer hybrid models on UAV platforms

ZHAO Chenggong^{1, 2},
HU Jiyuan^{1, 3},
YANG Xingyu⁴,
CHEN Lei⁵,
JIANG Hongxu^{3
, ,}

1.
Hangzhou Innovation Institute of Beihang University，Hangzhou 310056，China
2.
College of Software，Beihang University，Beijing 100191，China
3.
College of Computer Science，Beihang University，Beijing 100191，China
4.
College of Computer Science (College of Software)，Inner Mongolia University，Hohhot 010021，China
5.
Hangzhou Xiaoshan International Airport，Hangzhou 311241，China

More Information

Corresponding author: E-mail：jianghx@buaa.edu.cn

摘要

摘要:
近年，无人机在各类低空生产、服务场景作用愈发重要。由于边缘计算平台的存储、计算资源受限，视觉模型的边端侧运行性能短板成为算法应用于以上场景的技术瓶颈。针对卷积神经网络(CNN)-Transformer混合模型激活值长尾分布导致量化精度严重退化问题，提出高效训练后量化方法。该方法通过激活值噪声补偿(ANC)和自适应难度迁移抑制异常值对量化精度的影响，在提升模型量化精度同时提高其推理速度。实验结果表明：所提方法在典型模型的8 bit量化场景下，精度损失小于1%，量化后推理速度最高提升超过200%。所提方法显著提升了边端侧设备的视觉模型推理性能，为低空无人机平台上的模型部署提供了技术支撑。
- 训练后量化 /
- 模型压缩 /
- 推理加速 /
- 异常值感知 /
- 边缘计算
Abstract:
In recent years, drones have played increasingly important roles in various low-altitude production and service scenarios. Due to the limited storage and computing resources of edge computing platforms, the deployment performance of vision models becomes a bottleneck for various algorithm applications. The long-tail distribution of activation values in convolutional neural network (CNN)-Transformer hybrid models causes substantial quantization accuracy deterioration, an efficient post-training quantization method is proposed, which this study addresses by suggesting an effective post-training quantization technique. This method uses activation noise compensation (ANC) and adaptive difficulty migration to suppress the impact of outliers on quantization accuracy and improve inference efficiency. According to experimental results, the proposed method achieves less than 1% accuracy loss on typical models under 8-bit quantization, and it improves inference time after quantization by at most 200%. In summary, the proposed method significantly enhances vision model inference performance on edge devices and supports model deployment on low-altitude drone platforms.
- post-training quantization /
- model compression /
- inference acceleration /
- outlier-aware /
- edge computing

HTML全文

图 1 ANC/ADM的优化过程

Figure 1. Optimization process of ANC/ADM

下载: 全尺寸图片幻灯片

图 2 比较量化前后各模型的推理速度

Figure 2. Comparison of inference speed before and after quantization across different models

下载: 全尺寸图片幻灯片

图 3 ANC和ADM在MobileViT模型和MobileViTv2模型上的消融实验结果

Figure 3. Ablation results of ANC and ADM on MobileViT and MobileViTv2 model

下载: 全尺寸图片幻灯片

图 4 Efficientformer中间和最后阶段全连接层卷积的激活值分布变化

Figure 4. Change in activation value distribution of fully connected layer convolutions in middle and final stages of Efficientformer

下载: 全尺寸图片幻灯片

表 1 MobileViT模型量化后精度

Table 1. Accuracy of MobileViT after quantization

方法	权重/激活值	精度Top-1/%
方法	权重/激活值	MobileViT-xxs	MobileViT-xs	MobileViT-s
全精度	32/32	69.00	74.80	78.40
FQ-ViT^[18]	8/8	66.46	68.28	77.67
PTQ4ViT^[19]	8/8	37.75	65.52	68.19
RepQ-ViT^[24]	8/8	1.85	41.96	50.01
Q-HyViT^[22]	8/8	67.20	73.89	77.72
HyQ^[23]	8/8	68.15	73.99	77.93
EasyQuant^[25]	8/8	36.13	73.16	74.21
本文OAQ方法	8/8	68.65	74.35	78.09

下载: 导出CSV

表 2 MobileViTv2模型量化后精度

Table 2. Accuracy of MobileViTv2 after quantization

方法	权重/激活值	精度Top-1/%
方法	权重/激活值	MobileViTv2-50	MobileViTv2-75	MobileViTv2-100
全精度	32/32	70.20	75.60	78.10
FQ-ViT^[18]	8/8	67.66	69.56	77.15
PTQ4ViT^[19]	8/8	39.39	65.54	51.02
RepQ-ViT^[24]	8/8	26.60	55.52	40.85
Q-HyViT^[22]	8/8	69.89	75.29	77.63
HyQ^[23]	8/8	69.16	74.47	76.63
EasyQuant^[25]	8/8	66.80	62.91	69.34
本文OAQ方法	8/8	69.97	75.31	77.86

下载: 导出CSV

表 3 EfficientFormer模型量化后精度

Table 3. Accuracy of EfficientFormer after quantization

方法	权重/激活值	Top-1精度/%
方法	权重/激活值	EfficientFormer-L1	EfficientFormer-L3	EfficientFormer-L7
全精度	32/32	80.20	82.40	83.30
FQ-ViT^[18]	8/8	66.63	81.85	82.47
PTQ4ViT^[19]	8/8	79.14	80.31	81.56
RepQ-ViT^[24]	8/8	79.67	81.98	82.47
HyQ^[23]	8/8	78.55	82.26	82.66
EasyQuant^[25]	8/8	78.42	81.03	82.27
本文OAQ方法	8/8	80.09	82.14	83.15

下载: 导出CSV

参考文献(27)

[1]	Dai Z, Liu H, Le Q V, et al. Coatnet: marrying convolution and attention for all data sizes[J]. Advances in neural information processing systems, 2021, 34: 3965-3977.
[2]	KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6): 84-90.
[3]	Zeiler M D, Fergus R. Visualizing and understanding convolutional networks[C]//Proceedings of the Computer Vision-ECCV. Berlin: Springer, 2014: 818-833.
[4]	Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition[EB/OL]. (2014-09-04)[2025-06-20]. https://arxiv.org/abs/1409.1556.
[5]	Shafiq M, Gu Z Q. Deep residual learning for image recognition: a survey[J]. Applied Sciences, 2022, 12(18): 8972.
[6]	Liu Z, Lin Y T, Cao Y, et al. Swin transformer: hierarchical vision transformer using shifted windows[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE Press, 2022: 9992-10002.
[7]	Dosovitskiy A, Beyer L, Kolesnikov A, et al. An image is worth 16x16 words: transformers for image recognition at scale[EB/OL]. (2020-12-22)[2025-06-20]. https://arxiv.org/abs/2010.11929.
[8]	Touvron H, Cord M, Douze M, et al. Training data-efficient image transformers & distillation through attention[EB/OL]. (2020-12-23)[2025-06-20]. https://arxiv.org/abs/2012.12877.
[9]	Selva J, Johansen A S, Escalera S, et al. Video transformers: a survey[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(11): 12922-12943.
[10]	Hashiguchi R, Tamaki T. Vision transformer with cross-attention by temporal shift for efficient action recognition[EB/OL]. (2022-04-01)[2025-06-21]. https://arxiv.org/abs/2204.00452.
[11]	Manzari O N, Ahmadabadi H, Kashiani H, et al. MedViT: a robust vision transformer for generalized medical image classification[J]. Computers in Biology and Medicine, 2023, 157: 106791.
[12]	Mehta S, Rastegari M. MobileViT: light-weight, general-purpose, and mobile-friendly vision transformer[EB/OL]. (2021-10-05)[2025-06-21]. https://arxiv.org/abs/2110.02178.
[13]	Maaz M, Shaker A, Cholakkal H, et al. EdgeNeXT: efficiently amalgamated CNN-transformer architecture forMobile vision applications[C]//Proceedings of the Computer Vision-ECCV Workshops. Berilin: Springer, 2023: 3-20.
[14]	Evangelidis G, Hu J, Li Y Y, et al. EfficientFormer: vision transformers at MobileNet speed[C]//Proceedings of the Advances in Neural Information Processing Systems 35. San Diego: Neural Information Processing Systems Foundation, Inc., 2022: 12934-12949.
[15]	Zhang D Q, Yang J L, Ye D, et al. LQ-nets: learned quantization for highly accurate and compact deep neural networks[C]//Proceedings of the Computer Vision-ECCV. Berlin: Springer, 2018: 373-390.
[16]	Dong P Y, Lu L, Lyu C, et al. PackQViT: faster sub-8-bit vision transformers via full and packed quantization on the mobile[C]// Proceedings of the Advances in Neural Information Processing Systems 36. San Diego: Neural Information Processing Systems Foundation, Inc., 2023: 9015-9028.
[17]	Liu Z H, Wang Y H, Han K, et al. Post-training quantization for vision transformer[EB/OL]. (2021-06-27)[2025-06-21]. https://arxiv.org/abs/2106.14156.
[18]	Lin Y, Zhang T Y, Sun P Q, et al. FQ-ViT: post-training quantization for fully quantized vision transformer[EB/OL]. (2021-11-27)[2025-06-21]. https://arxiv.org/abs/2111.13824.
[19]	Yuan Z H, Xue C H, Chen Y Q, et al. PTQ4ViT: post-training quantization for vision transformers with twin uniform quantization[C]//Proceedings of the Computer Vision-ECCV. Berlin: Springer, 2022: 191-207.
[20]	Jacob B, Kligys S, Chen B, et al. Quantization and training of neural networks for efficient integer-arithmetic-only inference[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2018: 2704-2713.
[21]	Nagel M, Ali Amjad R, Van Baalen M, et al. Up or down adaptive rounding for post-training quantization[C]//Proceedings of the 37th International Conference on Machine Learning. New York: ACM, 2020: 7197-7206.
[22]	Lee J, Kwon Y, Park S, et al. Q-HyViT: post-training quantization of hybrid vision transformers with bridge block reconstruction for IoT systems[J]. IEEE Internet of Things Journal, 2024, 11(22): 36384-36396.
[23]	Kim N J, Lee J, Kim H. HyQ: hardware-friendly post-training quantization for CNN-transformer hybrid networks[C]//Proceedings of the 33th International Joint Conference on Artificial Intelligence . New York: ACM, 2024: 4291-4299.
[24]	Mehta S, Rastegari M. Separable self-attention for mobile vision transformers[EB/OL]. (2022-06-06)[2025-06-21]. https://arxiv.org/abs/2206.02680.
[25]	Li Z K, Xiao J R, Yang L W, et al. RepQ-ViT: scale reparameterization for post-training quantization of vision transformers[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE Press, 2024: 17181-17190.
[26]	Wu D, Tang Q, Zhao Y L, et al. EasyQuant: post-training quantization via scale optimization[EB/OL]. (2020-06-30)[2025-06-21]. https://arxiv.org/abs/2006.16669.
[27]	Chen J M, Chao Y H, Wang Y J, et al. QuantTune: optimizing model quantization with adaptive outlier-driven fine tuning[EB/OL]. (2024-03-11)[2025-06-21]. https://arxiv.org/abs/2403.06497.