面向不规则结构的低空感知模型剪枝算法

杨子; 庄连生

doi:10.13700/j.bh.1001-5965.2025.0313

面向不规则结构的低空感知模型剪枝算法

doi: 10.13700/j.bh.1001-5965.2025.0313

杨子,
庄连生^,

中国科学技术大学网络空间安全学院，合肥 230026

详细信息

通讯作者:
E-mail：yangzi@mail.ustc.edu.cn

中图分类号: V221⁺.3；TB553
计量
- 文章访问数: 293
- HTML全文浏览量: 211
- PDF下载量: 5
- 被引次数: 0
出版历程
- 收稿日期: 2025-05-19
- 录用日期: 2025-05-30
- 网络出版日期: 2025-06-05
- 整期出版日期: 2026-09-01

Low-altitude perception models pruning algorithm with irregular structures

YANG Zi,
ZHUANG Liansheng^,

School of Cyber Science and Technology，University of Science and Technology of China，Hefei 230026，China

More Information

Corresponding author: E-mail：yangzi@mail.ustc.edu.cn

摘要

摘要:
深度学习在低空智能感知中性能优越，但终端计算资源有限，难以部署大型神经网络。剪枝技术通过删除冗余参数降低复杂度，提升实时性，但现有算法多针对规则结构模型，难以处理神经架构搜索(NAS)得到的不规则结构。提出一种基于依赖图的不规则结构剪枝方案，以优化模型效率。通过计算图的结构解析技术，自动识别低空感知模型中特有的多层级连接模式；构建高效的参数分组模块，精确锁定可完整移除的最小功能单元；采用跨层对比的全局优化策略，实现对同构子结构的统一评估与剪枝。相较于现有算法，所提算法可对不规则网络架构实施全局精准压缩，特别适用于需要处理复杂空间关系的低空感知场景。实验结果表明：所提不规则结构剪枝算法在NAS-Bench101模型及ResNet系列网络均能有效识别出多分支连接、残差连接、拼接等复杂结构，并完成分组剪枝，在CIFAR-10数据集以50%的压缩比例对模型轻量化，识别准确率下降不超过1%。
- 低空感知 /
- 卷积网络 /
- 轻量化模型 /
- 模型剪枝 /
- 不规则网络架构
Abstract:
Large neural networks are challenging to implement on terminal devices due to restricted processing resources, notwithstanding deep learning’s superior performance in low-altitude intelligent sensing. Pruning techniques reduce model complexity by eliminating redundant parameters, thereby improving real-time performance. However, existing methods primarily target regularly structured models and struggle to handle irregular structures generated by neural architecture search (NAS). To address this, a dependency graph-based pruning scheme for irregular structures is proposed to optimize model efficiency. The method first employs structural parsing techniques of computational graphs to automatically identify unique multi-level connection patterns in low-altitude perception models. Subsequently, it constructs an efficient parameter grouping module to precisely locate minimum functional units that can be completely removed. Finally, a cross-layer comparison global optimization strategy is adopted to implement unified evaluation and pruning of isomorphic substructures. This solution overcomes the technical constraints of structural adaptability when compared to current approaches, allowing global precision compression for irregular network architectures. This feature makes it especially appropriate for low-altitude perception scenarios that call for intricate spatial relationship processing. Experimental results demonstrate that the proposed irregular structure pruning method can effectively identify and perform group pruning on complex structures, including multi-branch connections, residual connections, and concatenations, in both NAS-Bench101 models and ResNet series networks. When implementing 50% compression rate network lightweighting on the CIFAR-10 dataset, the recognition accuracy drop does not exceed 1%.
- low-altitude perception /
- convolutional networks /
- lightweight models /
- model pruning /
- irregular network architectures

HTML全文

图 1 面向不规则结构的模型剪枝算法框架

Figure 1. Frame of models pruning algorithm with irregular structures

下载: 全尺寸图片幻灯片

图 2 层内依赖模式示意

Figure 2. Illustration of inter-layer dependency patterns

下载: 全尺寸图片幻灯片

图 3 依赖关系图构建

Figure 3. Construction of a dependency relationship graph

下载: 全尺寸图片幻灯片

图 4 不同结构中具有相互依赖性的分组参数

Figure 4. Grouped parameters with dependency in different structures

下载: 全尺寸图片幻灯片

图 5 无分支网络依赖模式示意

Figure 5. Dependency patterns diagram in branchless networks

下载: 全尺寸图片幻灯片

图 6 残差网络依赖模式示意

Figure 6. Dependency patterns diagram in residual networks

下载: 全尺寸图片幻灯片

图 7 拼接依赖模式示意

Figure 7. Dependency pattern diagram with concatenation

下载: 全尺寸图片幻灯片

图 8 神经网络中的同构子结构

Figure 8. Isomorphic substructures in a neural network

下载: 全尺寸图片幻灯片

图 9 分组矩阵可视化

Figure 9. Group matrix visualization

下载: 全尺寸图片幻灯片

表 1 在CIFAR-10数据集上的识别准确率对比(ResNet56)

Table 1. Comparison of recognition accuracy rate on CIFAR-10 dataset (ResNet56)

算法	参数量	压缩比例/%	识别准确率/%
不压缩	2.352×10⁷	0	94.90
L₁-norm	1.151×10⁷	51.06	93.37
L₂-norm	1.151×10⁷	51.06	93.86
FPGM	1.151×10⁷	51.06	93.64
Taylor	1.151×10⁷	51.06	93.80
本文	1.151×10⁷	51.06	94.93

下载: 导出CSV

表 2 在CIFAR-100数据集上的识别准确率对比(ResNet101)

Table 2. Comparison of recognition accuracy rate on CIFAR-100 dataset (ResNet101)

算法	参数量	压缩比例/%	识别准确率/%
不压缩	4.270×10⁷	0	78.30
L₁-norm	2.094×10⁷	50.96	74.75
L₂-norm	2.094×10⁷	50.96	75.59
FPGM	2.094×10⁷	50.96	74.45
Taylor	2.094×10⁷	50.96	75.42
本文	2.094×10⁷	50.96	76.68

下载: 导出CSV

表 3 在CIFAR-10数据集上的识别准确率对比(NAS)

Table 3. Comparison of recognition accuracy rate on CIFAR-10 dataset (NAS)

算法	参数量	压缩比例/%	识别准确率/%
不压缩	4.30×10⁶	0	96.97
L₁-norm	2.14×10⁶	50.24	95.81
L₂-norm	2.14×10⁶	50.24	95.50
FPGM	2.14×10⁶	50.24	95.61
Taylor	2.14×10⁶	50.24	95.77
本文	2.14×10⁶	50.24	95.90

下载: 导出CSV

表 4 不同压缩比例下的性能对比

Table 4. Performance comparison at different compression ratios

压缩比例/%	参数量	浮点运算量/次	识别准确率/%
10	2.108×10⁷	5.717×10¹⁰	95.10
20	1.856×10⁷	5.021×10¹⁰	95.14
30	1.635×10⁷	4.433×10¹⁰	94.92
40	1.394×10⁷	3.786×10¹⁰	95.01
50	1.151×10⁷	3.119×10¹⁰	94.93
60	9.210×10⁷	2.511×10¹⁰	94.61
70	7.460×10⁶	2.036×10¹⁰	94.57
80	4.550×10⁶	1.241×10¹⁰	94.03

下载: 导出CSV

表 5 不同分层裁剪策略的效果对比

Table 5. Comparison of the effectiveness of different hierarchical cutting strategies

网络架构	分组策略	压缩比例/%	识别准确率/%
ResNet56	全局稀疏	10	92.91
		30	91.05
		50	88.93
	均匀稀疏	10	92.59
		30	91.91
		50	90.84
NAS-Bench101	全局稀疏	10	95.81
		30	92.66
		50	91.05
	均匀稀疏	10	96.29
		30	96.10
		50	95.61

下载: 导出CSV

参考文献(28)

[1]	He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2016: 770-778.
[2]	Wang H, Wang H, Liu Y J, et al. Real-time obstacle perception method for UAVs with an RGB-D camera in low-light environments[J]. Signal, Image and Video Processing, 2025, 19(3): 256.
[3]	El Ghazouali S, Mhirit Y, Oukhrid A, et al. FusionVision: a comprehensive approach of 3D object reconstruction and segmentation from RGB-D cameras using YOLO and fast segment anything[J]. Sensors, 2024, 24(9): 2889.
[4]	Saha A, Dhara B C. 3D LiDAR-based obstacle detection and tracking for autonomous navigation in dynamic environments[J]. International Journal of Intelligent Robotics and Applications, 2024, 8(1): 39-60.
[5]	Wang C H, Wang Y C, Xu M Z, et al. Stepwise goal-driven networks for trajectory prediction[J]. IEEE Robotics and Automation Letters, 2022, 7(2): 2716-2723.
[6]	Li R C, Katsigiannis S, Kim T K, et al. BP-SGCN: behavioral pseudo-label informed sparse graph convolution network for pedestrian and heterogeneous trajectory prediction[J]. IEEE Transactions on Neural Networks and Learning Systems, 2025, 36(8): 14566-14580.
[7]	Xin G P, Chu D F, Lu L P, et al. Multiagent trajectory prediction with difficulty-guided feature enhancement network[J]. IEEE Robotics and Automation Letters, 2025, 10(3): 2582-2589.
[8]	Cai Y J, Ge L H, Liu J, et al. Exploiting spatial-temporal relationships for 3D pose estimation via graph convolutional networks[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE Press, 2020: 2272-2281.
[9]	Cao Z, Simon T, Wei S H, et al. Realtime multi-person 2D pose estimation using part affinity fields[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2017: 1302-1310.
[10]	Mcnally W, Vats K, Wong A, et al. Rethinking keypoint representations: modeling keypoints and poses as objects for multi-person human pose estimation[C]//Proceedings of the Computer Vision-ECCV 2022. Berlin: Springer, 2022: 37-54.
[11]	姜晓勇, 李忠义, 黄朗月, 等. 神经网络剪枝技术研究综述[J]. 应用科学学报, 2022, 40(5): 838-849. Jiang X Y, Li Z Y, Huang L Y, et al. Review of neural network pruning techniques[J]. Journal of Applied Sciences, 2022, 40(5): 838-849(in Chinese).
[12]	Lecun Y, Denker J S, Solla S A. Optimal brain damage[J]. Advances in Neural Information Processing Systems, 1989, 2: 598-605.
[13]	Han S, Mao H Z, Dally W J. Deep compression: compressing deep neural network with pruning, trained quantization and Huffman coding[EB/OL]. (2016-02-15)[2025-05-10]. https://arxiv.org/abs/1510.00149.
[14]	Yang H C, Zhu Y H, Liu J. Energy-constrained compression for deep neural networks via weighted sparse projection and layer input masking[EB/OL]. (2019-06-02)[2025-05-10]. https://arxiv.org/abs/1806.04321.
[15]	Gong R H, Yong Y, Wang Z N, et al. Fast and controllable post-training sparsity: learning optimal sparsity allocation with global constraint in minutes[C]//Proceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence and Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence and Fourteenth Symposium on Educational Advances in Artificial Intelligence. New York: ACM, 2024: 12190-12198.
[16]	Li H, Kadav A, Durdanovic I, et al. Pruning filters for efficient ConvNets[EB/OL]. (2017-03-10)[2025-05-10]. https://arxiv.org/abs/1608.08710.
[17]	刘宇, 雷雪梅. 融合MobileNetV3特征的结构化剪枝方法[J]. 上海交通大学学报, 2023, 57(9): 1203-1213. Liu Y, Lei X M. A structured pruning method integrating characteristics of MobileNetV3[J]. Journal of Shanghai Jiao Tong University, 2023, 57(9): 1203-1213(in Chinese).
[18]	Hu H Y, Peng R, Tai Y W, et al. Network trimming: a data-driven neuron pruning approach towards efficient deep architectures[EB/OL]. (2016-07-12)[2025-05-10]. https://arxiv.org/abs/1607.03250.
[19]	Liu Z, Li J G, Shen Z Q, et al. Learning efficient convolutional networks through network slimming[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway: IEEE Press, 2017: 2755-2763.
[20]	Lin M B, Cao L J, Zhang Y X, et al. Pruning networks with cross-layer ranking & k-reciprocal nearest filters[J]. IEEE Transactions on Neural Networks and Learning Systems, 2023, 34(11): 9139-9148.
[21]	Akimoto Y, Shirakawa S. Analysis of search space design for neural architecture search with weight sharing[C]//Proceedings of the 2024 International Joint Conference on Neural Networks. Piscataway: IEEE Press, 2024: 1-8.
[22]	Wang H, Qin C, Bai Y, et al. Why is the state of neural network pruning so confusing on the fairness, comparison setup, and trainability in network pruning[EB/OL]. (2023-02-21)[2025-05-10]. https://arxiv.org/abs/2301.05219.
[23]	Feng Q T, Xu K, Li Y H, et al. Edge-wise one-level global pruning on NAS generated networks[C]//Proceedings of the Pattern Recognition and Computer Vision. Berlin: Springer, 2021: 3-15.
[24]	Jiang M N, Wang J C, Eldebiky A, et al. Class-aware pruning for efficient neural networks[C]//Proceedings of the 2024 Design, Automation & Test in Europe Conference & Exhibition. Piscataway: IEEE Press, 2024: 1-6.
[25]	Chollet F. Xception: deep learning with depthwise separable convolutions[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2017: 1800-1807.
[26]	Chatzikonstantinou C, Konstantinidis D, Dimitropoulos K, et al. Recurrent neural network pruning using dynamical systems and iterative fine-tuning[J]. Neural Networks, 2021, 143: 475-488.
[27]	Wei S Y, Ye T Z, Zhang S, et al. Joint token pruning and squeezing towards more aggressive compression of vision Transformers[C]//Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2023: 2092-2101.
[28]	Iandola F N, Han S, Moskewicz M W, et al. SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5 MB model size[EB/OL]. (2016-11-04)[2025-05-10]. https://arxiv.org/abs/1602.07360.