| Citation: | BAI F C，YANG X X，DENG X L，et al. Station keeping control for aerostat in wind fields based on deep reinforcement learning[J]. Journal of Beijing University of Aeronautics and Astronautics，2024，50（7）：2354-2366 （in Chinese） doi: 10.13700/j.bh.1001-5965.2022.0629
|
In this paper, a stratospheric aerostat station keeping model is established. Based on Markov decision process, Double Deep Q-learning with prioritized experience replay is applied to stratospheric aerostat station keeping control under dynamic and non-dynamic conditions. Ultimately, metrics like the average station keeping radius and the station keeping effective time ratio are used to assess the effectiveness of the station keeping control approach. The simulation analysis results show that: under the mission the station keeping radius is 50 km and the station keeping time is three days, in the case of no power propulsion, the average station keeping radius of the stratospheric aerostat is 28.16 km, the station keeping effective time ratio is 83%. In the case of powered propulsion, the average station keeping radius of the stratospheric aerostat is significantly increased. The powered stratospheric aerostat can achieve flight control with a station keeping radius of 20 km, an average station keeping radius of 8.84 km, and a station keeping effective time ratio of 100%.
| [1] |
侯中喜, 杨希祥, 乔凯, 等. 平流层飞艇技术[M]. 北京: 科学出版社, 2019.
HOU Z X, YANG X X, QIAO K, et al. Stratospheric airship technology [M]. Beijing: Science Press, 2019.
|
| [2] |
李智斌, 黄宛宁, 张钊, 等. 2020年临近空间科技热点回眸[J]. 科技导报, 2021, 39(1): 54-68.
LI Z B, HUANG W N, ZHANG Z, et al. Summary of the hot spots of near space science and technology in 2020[J]. Science & Technology Review, 2021, 39(1): 54-68(in Chinese).
|
| [3] |
NOCK K T, HEUN M K, AARON K M. Global stratospheric balloon constellations[J]. Advances in Space Research, 2002, 30(5): 1233-1238. doi: 10.1016/S0273-1177(02)00528-8
|
| [4] |
赵达, 刘东旭, 孙康文, 等. 平流层飞艇研制现状、技术难点及发展趋势[J]. 航空学报, 2016, 37(1): 45-56.
ZHAO D, LIU D X, SUN K W, et al. Research status, technical difficulties and development trend of stratospheric airship[J]. Acta Aeronautica et Astronautica Sinica, 2016, 37(1): 45-56(in Chinese).
|
| [5] |
CATHEY H M, TUTTLE J W, FAIRBROTHER D A, et al. Qualification of the NASA super pressure balloon[C]// AIAA Balloon Systems Conference. Reston: AIAA, 2015: 2909.
|
| [6] |
刘东旭, 樊彦斌, 马云鹏, 等. 氦气渗透对高空长航时浮空器驻空能力影响[J]. 宇航学报, 2010, 31(11): 2477-2482.
LIU D X, FAN Y B, MA Y P, et al. Effect of helium permeability on working endurance high altitude long duration LTA vehicle[J]. Journal of Astronautics, 2010, 31(11): 2477-2482(in Chinese).
|
| [7] |
杨燕初, 张航悦, 赵荣. 零压式高空气球球形设计与参数敏感性分析[J]. 国防科技大学学报, 2019, 41(1): 58-64. doi: 10.11887/j.cn.201901009
YANG Y C, ZHANG H Y, ZHAO R. Shape design of zero pressure high altitude balloon and sensitivity analysis of key parameters[J]. Journal of National University of Defense Technology, 2019, 41(1): 58-64(in Chinese). doi: 10.11887/j.cn.201901009
|
| [8] |
杨跃能. 平流层飞艇动力学建模与控制方法研究[D]. 长沙: 国防科学技术大学, 2013.
YANG Y N. Dynamics modeling and flight control for a stratospheric airship[D]. Changsha: National University of Defense Technology, 2013(in Chinese).
|
| [9] |
杨希祥, 朱炳杰, 邓小龙, 等. Stratobus平流层飞艇项目研究进展与仿真分析[J]. 航空学报, 2021, 42(9): 224579.
YANG X X, ZHU B J, DENG X L, et al. Development status and simulation analysis of stratospheric airship Stratobus[J]. Acta Aeronautica et Astronautica Sinica, 2021, 42(9): 224579(in Chinese).
|
| [10] |
RONEY J A. Statistical wind analysis for near-space applications[J]. Journal of Atmospheric and Solar-Terrestrial Physics, 2007, 69(13): 1485-1501. doi: 10.1016/j.jastp.2007.05.005
|
| [11] |
邓小龙, 丛伟轩, 李魁, 等. 风场综合利用的新型平流层浮空器轨迹设计[J]. 宇航学报, 2019, 40(7): 748-757.
DENG X L, CONG W X, LI K, et al. Trajectory design of a novel stratospheric aerostat based on comprehensive utilization of wind fields[J]. Journal of Astronautics, 2019, 40(7): 748-757(in Chinese).
|
| [12] |
翟嘉琪, 杨希祥, 邓小龙, 等. 不确定风场下平流层浮空器全局路径规划[J]. 北京航空航天大学学报, 2023, 49(5): 1116-1126.
ZHAI J Q, YANG X X, DENG X L, et al. Global path planning of stratospheric aerostat in uncertain wind field[J]. Journal of Beijing University of Aeronautics and Astronautics, 2023, 49(5): 1116-1126(in Chinese).
|
| [13] |
SMITH M S. Demonstration of fine altitude control on stratospheric balloons to achieve a desired ground track[C]// AIAA Balloon Systems Conference. Reston: AIAA, 2017: 3287.
|
| [14] |
李魁, 邓小龙, 杨希祥, 等. 基于平流层风场预测的浮空器轨迹控制[J]. 北京航空航天大学学报, 2019, 45(5): 1008-1018.
LI K, DENG X L, YANG X X, et al. Trajectory control of aerostat based on prediction of stratospheric wind field[J]. Journal of Beijing University of Aeronautics and Astronautics, 2019, 45(5): 1008-1018(in Chinese).
|
| [15] |
TRAN N K, HE X, ZLOTNIK D E, et al. Attitude sensing and control of a stratospheric ballon platform[C]// AIAA Balloon Systems (BAL) Conference. Reston: AIAA, 2013: 1373.
|
| [16] |
DU H F, SUN T F, LV M Y, et al. Dynamic coverage performance of wind-assisted balloons mesh based on Voronoi partition and energy constraint[J]. Advances in Space Research, 2022, 70(2): 470-484. doi: 10.1016/j.asr.2022.04.051
|
| [17] |
YODER C D, GEMMER T R, MAZZOLENI A P. Modelling and performance analysis of a tether and sail-based trajectory control system for extra-terrestrial scientific balloon missions[J]. Acta Astronautica, 2019, 160: 527-537. doi: 10.1016/j.actaastro.2018.12.030
|
| [18] |
RAMESH S S, MA J L, LIM K M, et al. Numerical evaluation of station-keeping strategies for stratospheric balloons[J]. Aerospace Science and Technology, 2018, 80: 288-300. doi: 10.1016/j.ast.2018.07.010
|
| [19] |
BELLEMARE M G, CANDIDO S, CASTRO P S, et al. Autonomous navigation of stratospheric balloons using reinforcement learning[J]. Nature, 2020, 588(7836): 77-82. doi: 10.1038/s41586-020-2939-8
|
| [20] |
DU H F, LV M Y, LI J, et al. Station-keeping performance analysis for high altitude balloon with altitude control system[J]. Aerospace Science and Technology, 2019, 92: 644-652. doi: 10.1016/j.ast.2019.06.035
|
| [21] |
DU H F, LV M Y, ZHANG L C, et al. Energy management strategy design and station-keeping strategy optimization for high altitude balloon with altitude control system[J]. Aerospace Science and Technology, 2019, 93: 105342. doi: 10.1016/j.ast.2019.105342
|
| [22] |
王益平, 周飞, 徐明. 临近空间浮空器区域驻留控制策略研究[J]. 中国空间科学技术, 2018, 38(1): 63-69.
WANG Y P, ZHOU F, XU M. Research on control strategy of territory-hovering aerostat in near space[J]. Chinese Space Science and Technology, 2018, 38(1): 63-69(in Chinese).
|
| [23] |
邓小龙, 杨希祥, 麻震宇, 等. 基于风场环境利用的平流层浮空器区域驻留关键问题研究进展[J]. 航空学报, 2019, 40(8): 022941.
DENG X L, YANG X X, MA Z Y, et al. Review of key technologies for station-keeping of stratospheric aerostats based on wind field utilization[J]. Acta Aeronautica et Astronautica Sinica, 2019, 40(8): 022941(in Chinese).
|
| [24] |
JIANG Y, LV M Y, ZHU W Y, et al. A method of 3-D region controlling for scientific balloon long-endurance flight in the real wind[J]. Aerospace Science and Technology, 2020, 97: 105618. doi: 10.1016/j.ast.2019.105618
|
| [25] |
JIANG Y, LV M Y, LI J. Station-keeping control design of double balloon system based on horizontal region constraints[J]. Aerospace Science and Technology, 2020, 100: 105792. doi: 10.1016/j.ast.2020.105792
|
| [26] |
杨思明, 单征, 丁煜, 等. 深度强化学习研究综述[J]. 计算机工程, 2021, 47(12): 19-29.
YANG S M, SHAN Z, DING Y, et al. Survey of research on deep reinforcement learning[J]. Computer Engineering, 2021, 47(12): 19-29(in Chinese).
|
| [27] |
张悦. 多智能体深度强化学习方法及应用研究[D]. 西安: 西安电子科技大学, 2018.
ZHANG Y. Research on multi-agent deep reinforcement learning methods and applications[D]. Xi’an: Xidian University, 2018(in Chinese).
|
| [28] |
XU Z Y, LIU Y, DU H F, et al. Station-keeping for high-altitude balloon with reinforcement learning[J]. Advances in Space Research, 2022, 70(3): 733-751. doi: 10.1016/j.asr.2022.05.006
|
| [29] |
张小达, 张鹏, 李小龙. 《标准大气与参考大气模型应用指南》介绍[J]. 航天标准化, 2010(3): 8-11.
ZHANG X D, ZHANG P, LI X L. Introduction of “application guide of standard atmosphere and reference atmosphere model”[J]. Aerospace Standardization, 2010(3): 8-11(in Chinese).
|
| [30] |
张顶立. 基于深度强化学习的城市场景无人机避撞决策研究[D]. 广汉: 中国民用航空飞行学院, 2022.
ZHANG D L. Research on autonomous collision avoidance decisionmaking of UAV in urban airspace based on deep reinforcement learning[D]. Guanghan: Civil Aviation Flight University of China, 2022(in Chinese).
|
| [31] |
LI J X, CHEN Y T, ZHAO X N, et al. An improved DQN path planning algorithm[J]. The Journal of Supercomputing, 2022, 78(1): 616-639. doi: 10.1007/s11227-021-03878-2
|
| [32] |
SUTTON R S, BARTO A G. Reinforcement learning: an introduction[M]. 2nd ed. London: MIT, 2018.
|
| [33] |
SCHAUL T, QUAN J, ANTONOGLOU I, et al. Prioritized experience replay[EB/OL]. (2016-02-25)[2022-06-19]. https://doi.org/10.48550/arXiv.1511.05952.
|
Figures(13) / Tables(8)
Copyright © Journal of Beijing University of Aeronautics and Astronautics
Address: Editorial Department of Journal of Beijing University of Aeronautics and Astronautics, 37 Xueyuan Road, Haidian District, Beijing Post Code: 100191 Email: jbuaa@buaa.edu.cn
Supported by:
Beijing Renhe Information Technology Co., Ltd.
DownLoad:
