-
摘要:
针对状态受限条件下高超声速变外形飞行器的高精度控制问题，提出一种基于强化学习的参数自适应模型预测控制(MPC)方法。考虑传统组合体外形难以在高超声速条件下表现良好的气动性能，设计一种基于乘波体为基准外形的变外形飞行器气动布局，并建立该类飞行器的面向控制的姿态动力学模型。针对飞行过程中控制约束与飞行器状态约束问题，基于MPC滚动生成控制指令。进一步考虑飞行器连续变形过程中参数时变引起的控制器性能变化问题，设计一种基于近端策略优化(PPO)的在线参数自适应方案。理论分析和仿真结果证明，所提方法能够在满足飞行器约束条件下，提高控制精度，并降低计算耗时。
-
关键词:
- 高超声速变外形飞行器 /
- 姿态控制 /
- 模型预测控制 /
- 强化学习 /
- 气动特性分析
Abstract:The attitude control system is affected by hypersonic morphing vehicles due to complex disturbances, unknown dynamics, and state constraints. To address these problems, an adaptive model predictive control (MPC) system is proposed in this paper. Focusing on the low aerodynamic character of traditionally combined aircraft, a hypersonic morphing waverider is designed to achieve higher flight performance, and a control-oriented model is established for attitude control. Furthermore, in order to solve the problems of state constraints, including rudder and angular velocity limits, a MPC method is used to optimize the control command online. However, a model predictive controller based on a fixed control parameter usually faces performance degradation under time-varying parameters perturbation. A proximal policy optimization (PPO) agent is designed to adaptively output control parameter online. In addition to improving control accuracy and saving computational resources, the developed controller can accomplish high performance control under aircraft state limitations. Simulation results prove the effectiveness and superiority of the method.
-
图 4 基于PPO-MPC的高超声速变外形乘波体姿态控制系统设计框图
Figure 4. Attitude controller designed structure diagram for HMW based on PPO-MPC
图 11 PPO智能体控制参数输出曲线
Figure 11. Control parameters output curves of PPO intelligent agent
图 12 蒙特卡罗仿真姿态角跟踪误差曲线
Figure 12. Attitude angle tracking error curves of Monte Carlo simulation
图 13 蒙特卡罗仿真姿态角速度响应曲线
Figure 13. Attitude angular velocity response curves of Monte Carlo simulation
图 15 蒙特卡罗仿真控制量参数输出曲线
Figure 15. Control parameters output curves of Monte Carlo simulation
表 1 高超声速变外形乘波体飞行器总体设计参数
Table 1. Overall designed parameters of HMW
固定机身
质量$ {m}_{\text{b}} $/kg可变机翼
质量$ {m}_{\text{w}} $/kg参考长度
$ {L}_{\text{ref}} $/m参考面积
$ {S}_{\text{ref}} $/$ {\text{m}}^{2} $最小翼展
$ {b}_{\min } $/m最大翼展
$ {b}_{\max } $/m绕机体x轴转动
惯量$ {I}_{xx}/(\text{kg}\cdot {\text{m}}^{2}) $绕机体y轴转动
惯量$ {I}_{yy}/(\text{kg}\cdot {\text{m}}^{2}) $绕机体z轴转动
惯量$ {I}_{\textit{zz}}/(\text{kg}\cdot {\text{m}}^{2}) $惯性积
$ {I}_{xy}/(\text{kg}\cdot {\text{m}}^{2}) $1 000 50 2.4 1.2 0.1 0.5 63 508 467 106 
下载: 导出CSV
表 2 气动仿真工况条件
Table 2. Aerodynamic simulation conditions
高度$ H $/km 马赫数$ M a $ 迎角$ \alpha $/(°) 侧滑角$ \beta $/(°) 左升降舵偏角$ {\delta }_{\text{D1}} $/(°) 右升降舵偏角$ {\delta }_{\text{D2}} $/(°) 方向舵偏角$ {\delta }_{\text{D3}} $/(°) 25 3～15(2) 0～20(4) −3～3(3) −30～30(10) −30～30(10) −30～30(10) 注：括号中数字表示取值间隔。
下载: 导出CSV
表 3 PPO智能体离线训练奖励参数
Table 3. Reward parameters of PPO intelligent agents offline training
$ {a}_{\alpha } $ $ {a}_{\beta } $ $ {a}_{\sigma } $ $ {k}_{\alpha } $ $ {k}_{\sigma } $ $ {k}_{{{N}_{\text{p}}}} $ $ {b}_{\alpha } $ $ {b}_{\beta } $ $ {b}_{\sigma } $ $ {k}_{\beta } $ $ {k}_{{{\dot{\delta }}_{\text{D}}}} $ $ {r}_{s} $ 3 3 3 10 5 3 2 2 2 2 2×10−3 0.05
下载: 导出CSV
表 4 PPO智能体离线训练超参数
Table 4. Offline training hyperparameters of PPO intelligent agents
学习率 折扣因子 采样时间 批学习数 经验回放池样本数量 截断因子 3×10−4 0.99 0.1 128 25 600 0.2
下载: 导出CSV
表 5 高超声速变外形乘波体初始状态参数
Table 5. Initial state parameters of HMW
$ H\text{/km} $ $ V_{{0}} $/(m·s−1) $ {\alpha }_{0}{\text{/}}({\text{°}}) $ $ {\beta }_{0}{\text{/}}({\text{°}}) $ $ {\sigma }_{0}{\text{/}}({\text{°}}) $ $ {\omega }_{x}{}_{0} $/((°)·s−1) $ {\omega }_{y0} $/((°)·s−1) $ {\omega }_{z0} $/((°)·s−1) 30 2 500 0 0 0 0 0 0
下载: 导出CSV
表 6 不同控制器平均单步计算耗时
Table 6. Averaged single step calculation time in different controllers
控制方法 计算耗时/ms MPC 1.0277 PPO-MPC 0.5616
下载: 导出CSV
表 7 偏差条件设置
Table 7. Deviation condition settings
偏差/% 变形时间
偏差/s偏差/(°) 气动力矩系数 大气密度 $ \Delta {\alpha }_{0} $ $ \Delta {\beta }_{0} $ $ \Delta {\sigma }_{0} $ $ \pm 20 $ $ \pm 20 $ ±2 ±2 ±1 ±5
下载: 导出CSV
-
[1] 董泽洪, 李颖晖, 吕茂隆, 等. 迎角受限的高超声速飞行器固定时间鲁棒控制[J]. 宇航学报, 2022, 43(12): 1708-1721.DONG Z H, LI Y H, LYU M L, et al. Fixed-time robust control for hypersonic flight vehicle with constrained angle of attack[J]. Journal of Astronautics, 2022, 43(12): 1708-1721(in Chinese). [2] ZHAO D, REN W J. Resilient control of flexible hypersonic vehicles against unmatched distributed faults[J]. Chinese Journal of Aeronautics, 2024, 37(5): 60-70. [3] AJAJ R M, BEAVERSTOCK C S, FRISWELL M I. Morphing aircraft: the need for a new design philosophy[J]. Aerospace Science and Technology, 2016, 49: 154-166. [4] 岳彩红, 唐胜景, 王肖, 等. 高超声速伸缩式变形飞行器再入制导方法[J]. 北京航空航天大学学报, 2021, 47(6): 1288-1298.YUE C H, TANG S J, WANG X, et al. Reentry guidance method of hypersonic telescopic deformable vehicle[J]. Journal of Beijing University of Aeronautics and Astronautics, 2021, 47(6): 1288-1298(in Chinese). [5] BAO C Y, WANG P, TANG G J. Integrated method of guidance, control and morphing for hypersonic morphing vehicle in glide phase[J]. Chinese Journal of Aeronautics, 2021, 34(5): 535-553. [6] 王忠森, 廖宇新, 魏才盛, 等. 高超声速飞行器快速终端滑模保性能容错控制[J]. 航空学报, 2023, 44(24): 157-170.WANG Z S, LIAO Y X, WEI C S, et al. Fast terminal sliding mode fault-tolerant control of hypersonic vehicle with guaranteed performance[J]. Acta Aeronautica et Astronautica Sinica, 2023, 44(24): 157-170(in Chinese). [7] ZHANG B L, ZHOU D. Optimal predictive sliding-mode guidance law for intercepting near-space hypersonic maneuvering target[J]. Chinese Journal of Aeronautics, 2022, 35(4): 320-331. [8] 李小兵, 赵思源, 卜祥伟, 等. 高超声速飞行器预设性能反演控制方法设计[J]. 北京航空航天大学学报, 2019, 45(4): 650-661.LI X B, ZHAO S Y, BU X W, et al. Design of prescribed performance backstepping control method for hypersonic flight vehicles[J]. Journal of Beijing University of Aeronautics and Astronautics, 2019, 45(4): 650-661(in Chinese). [9] WANG L, QI R Y, JIANG B. Adaptive fault-tolerant control for non-minimum phase hypersonic vehicles based on adaptive dynamic programming[J]. Chinese Journal of Aeronautics, 2024, 37(3): 290-311. [10] 胡军. 高超声速飞行器非线性自适应姿态控制[J]. 宇航学报, 2017, 38(12): 1281-1288.HU J. The nonlinear adaptive attitude control for hypersonic vehicle[J]. Journal of Astronautics, 2017, 38(12): 1281-1288(in Chinese). [11] 刘继承, 江驹, 阴浩博, 等. 高超声速飞行器自适应固定时间抗饱和控制[J]. 哈尔滨工程大学学报, 2022, 43(7): 1013-1022.LIU J C, JIANG J, YIN H B, et al. Fixed-time antisaturation adaptive control of a hypersonic vehicle[J]. Journal of Harbin Engineering University, 2022, 43(7): 1013-1022(in Chinese). [12] ZHANG H, WANG P, TANG G J, et al. Fuzzy disturbance observer-based dynamic sliding mode control for hypersonic morphing vehicles[J]. Aerospace Science and Technology, 2023, 142: 108633. [13] 张远, 黄万伟, 路坤锋, 等. 高超声速变外形飞行器建模与有限时间控制[J]. 北京航空航天大学学报, 2022, 48(10): 1979-1993.ZHANG Y, HUANG W W, LU K F, et al. Modeling and finite-time control for hypersonic morphing flight vehicle[J]. Journal of Beijing University of Aeronautics and Astronautics, 2022, 48(10): 1979-1993(in Chinese). [14] 张豪, 王鹏, 汤国建, 等. 高超声速变外形飞行器事件触发有限时间控制[J]. 航空学报, 2023, 44(15): 325-338.ZHANG H, WANG P, TANG G J, et al. Event-triggered fast finite time control for hypersonic morphing vehicles[J]. Acta Aeronautica et Astronautica Sinica, 2023, 44(15): 325-338(in Chinese). [15] DAI P, YAN B B, HAN T, et al. Barrier Lyapunov function based model predictive control of a morphing waverider with input saturation and full-state constraints[J]. IEEE Transactions on Aerospace and Electronic Systems, 2023, 59(3): 3071-3081. [16] SONG Z Y, YAN G F, ZHAO D J. ESO-based robust predictive control of lunar module with fuel sloshing dynamics[J]. Journal of Central South University, 2017, 24(3): 589-598. [17] 康国华, 金晨迪, 郭玉洁, 等. 基于深度学习的组合体航天器模型预测控制[J]. 宇航学报, 2019, 40(11): 1322-1331.KANG G H, JIN C D, GUO Y J, et al. Model predictive control of combined spacecraft based on deep learning[J]. Journal of Astronautics, 2019, 40(11): 1322-1331(in Chinese). [18] NIU X J, GUO J G, ZHOU J, et al. Robust attitude predictive control design for hypersonic vehicles with state constraints[J]. Proceedings of the Institution of Mechanical Engineers, Part G: Journal of Aerospace Engineering, 2019, 233(7): 2592-2609. [19] CHENG L, WANG Z B, GONG S P. Adaptive control of hypersonic vehicles with unknown dynamics based on dual network architecture[J]. Acta Astronautica, 2022, 193: 197-208. [20] 王冠, 夏红伟. 一种基于学习的高超声速飞行器智能控制方法[J]. 宇航学报, 2023, 44(2): 233-242.WANG G, XIA H W. A learning-based intelligent control method for hypersonic flight vehicle[J]. Journal of Astronautics, 2023, 44(2): 233-242(in Chinese). [21] 刘深深, 罗磊, 韩青华, 等. 动量增升高升阻比飞行器横航向稳定性研究[J]. 北京航空航天大学学报, 2023, 49(11): 3010-3021.LIU S S, LUO L, HAN Q H, et al. Study on lateral-directional stability of a practical high lift-to-drag ratio hypersonic vehicle with momentum lift augmentation[J]. Journal of Beijing University of Aeronautics and Astronautics, 2023, 49(11): 3010-3021(in Chinese). [22] VAN SOEST W R, CHU Q P, MULDER J A. Combined feedback linearization and constrained model predictive control for entry flight[J]. Journal of Guidance, Control, and Dynamics, 2006, 29(2): 427-434. [23] NOCEDAL J, WRIGHT S J. Numerical optimization second edition[M]. Singapore: World Scientific, 1999. -
下载:
点击查看大图
计量
- 文章访问数: 646
- HTML全文浏览量: 305
- PDF下载量: 14
- 被引次数: 0
