Geomagnetic sensing navigation method based on deep reinforcement learning and simulated annealing
-
摘要:
在无先验条件的未知环境中，水下自主潜航器（AUV）的路径规划与导航是一大挑战。基于此，提出一种无需先验地磁图的感知导航方法，该方法融合了深度强化学习（DRL）和模拟退火（SA）算法，以实现有效的路径规划和地磁图构建。通过搭建深度Q网络（DQN）对载体进行环境探索，收集局部地磁数据，进而利用所收集的数据训练一个回归模型以预测全局地磁图。与此同时，采用模拟退火算法对水下自主无人潜航器的路径进行优化，以避免载体空间搜索陷入局部极小问题。通过一系列仿真实验，验证了所提方法在路径长度、探索效率及地磁图准确性方面的有效性。研究结果表明：所提方法能够显著提高水下自主无人潜航器在未知环境中的导航性能，并为地磁图的构建提供了一种新的途径。
Abstract:In the unknown environment without prior conditions, the path planning and navigation of an underwater autonomous underwater vehicle (AUV) is a big challenge. This study presents a perceptive navigation approach without a prior geomagnetic map. It achieves efficient path planning and geomagnetic map creation by combining deep reinforcement learning (DRL) with a simulated annealing (SA) algorithm. A deep Q network (DQN) is built to explore the environment of the carrier, collect local geomagnetic data, and then use the collected data to train a regression model to predict the global geomagnetic map. At the same time, the simulated annealing algorithm is used to optimize the path of an underwater AUV to avoid the local minimum problem of carrier space search. The success of the suggested approach is confirmed through a number of simulated tests in terms of path length, exploration efficiency, and geomagnetic map correctness. The results show that this method can significantly improve the navigation performance of underwater AUV in an unknown environment, and provide a new way for the construction of geomagnetic maps.
-
图 5 不同探索率下的奖励曲线对比
Figure 5. Comparison of reward curves under different exploration rates
图 6 不同探索率下成功率对比
Figure 6. Comparison of success rates under different exploration rates
表 1 参数设置
Table 1. Parameter settings
DQN模型
学习率$ \alpha $DQN中
折扣因子$ \gamma $探索率
衰减率$ \eta $转向角度
数量$ {A}_{\text{N}} $模拟退火
初始温度$ {T}_{0} $模拟退火
冷却率$ \lambda $年份 0.001 0.95 0.998 360 5 000 0.98 2023 
下载: 导出CSV
-
[1] 张伟, 王乃新, 魏世琳, 等. 水下无人潜航器集群发展现状及关键技术综述[J]. 哈尔滨工程大学学报, 2020, 41(2): 289-297.Zhang W, Wang N X, Wei S L, et al. Overview of unmanned underwater vehicle swarm development status and key technologies[J]. Journal of Harbin Engineering University, 2020, 41(2): 289-297(in Chinese). [2] 赵曦, 赵建虎. 水下地形匹配导航现状及发展趋势[J]. 哈尔滨工程大学学报, 2023, 44(11): 1927-1936.Zhao X, Zhao J H. Current progress and development trend of underwater terrain-matching navigation[J]. Journal of Harbin Engineering University, 2023, 44(11): 1927-1936(in Chinese). [3] 张涛, 夏茂栋, 张佳宇, 等. 水下导航定位技术综述[J]. 全球定位系统, 2022, 47(4): 1-16.Zhang T, Xia M D, Zhang J Y, et al. Review of underwater navigation and positioning technology[J]. GNSS World of China, 2022, 47(4): 1-16(in Chinese). [4] 张小跃, 杨功流, 张春熹. 捷联惯导/里程计组合导航方法[J]. 北京航空航天大学学报, 2013, 39(7): 922-926.Zhang X Y, Yang G L, Zhang C X. Integrated navigation method for SINS and odometer[J]. Journal of Beijing University of Aeronautics and Astronautics, 2013, 39(7): 922-926(in Chinese). [5] 马腾, 李晔, 赵玉新, 等. AUV的图优化海底地形同步定位与建图方法[J]. 导航定位与授时, 2020, 7(2): 42-49.Ma T, Li Y, Zhao Y X, et al. AUV bathymetric simultaneous localization and mapping based on graph optimization method[J]. Navigation Positioning and Timing, 2020, 7(2): 42-49(in Chinese). [6] 李明, 何继玮, 王一蕾, 等. 基于重力场数据的水下无源导航 [C]//第三届无人系统高峰论坛论文集. 南京: 南京理工大学, 2023: 1-7.Li M, He J W, Wang Y L, et al. Underwater passive navigation based on gravity field data[C]//Proceedings of the 3rd Unmanned Systems Summit Forum. Nanjing: Nanjing University of Science and Technology, 2023: 1-7(in Chinese). [7] Bobrov D S. Development of methods and means for creating navigational gravity maps[D]. Mendeleevo: All-Russian Scientific Research Institute of Physical-Technical and Radio Engineering Measurements, 2020. [8] 林沂, 孙晶京, 闫旭. 地磁导航定位技术原理与方法综述[J]. 全球定位系统, 2023, 48(6): 32-41.Lin Y, Sun J J, Yan X. A review of the principles and methods of geomagnetic navigation and positioning technology[J]. GNSS World of China, 2023, 48(6): 32-41(in Chinese). [9] 王巍, 邢朝洋, 冯文帅. 自主导航技术发展现状与趋势[J]. 航空学报, 2021, 42(11): 11-29.Wang W, Xing C Y, Feng W S. State of the art and perspectives of autonomous navigation technology[J]. Acta Aeronautica et Astronautica Sinica, 2021, 42(11): 11-29(in Chinese). [10] Lohmann K J, Lohmann C M F, Ehrhart L M, et al. Geomagnetic map used in sea-turtle navigation[J]. Nature, 2004, 428(6986): 909-910. [11] Chen Z, Liu K J, Zhang Q, et al. Geomagnetic vector pattern recognition navigation method based on probabilistic neural network[J]. IEEE Transactions on Geoscience and Remote Sensing, 2023, 61: 5909608. [12] 董豪, 杨静, 李少波, 等. 基于深度强化学习的机器人运动控制研究进展[J]. 控制与决策, 2022, 37(2): 278-292.Dong H, Yang J, Li S B, et al. Research progress of robot motion control based on deep reinforcement learning[J]. Control and Decision, 2022, 37(2): 278-292(in Chinese). [13] Panda J P, Mitra A, Warrior H V. A review on the hydrodynamic characteristics of autonomous underwater vehicles[J]. Proceedings of the Institution of Mechanical Engineers, Part M: Journal of Engineering for the Maritime Environment, 2021, 235(1): 15-29. [14] Grando R B, De Jesus J C, Kich V A, et al. Deep reinforcement learning for mapless navigation of a hybrid aerial underwater vehicle with medium transition[C]//Proceedings of the IEEE International Conference on Robotics and Automation. Piscataway: IEEE Press, 2021: 1088-1094. [15] 陈科胜, 鲜思东, 郭鹏. 求解旅行商问题的自适应升温模拟退火算法[J]. 控制理论与应用, 2021, 38(2): 245-254.Chen K S, Xian S D, Guo P. Adaptive temperature rising simulated annealing algorithm for traveling salesman problem[J]. Control Theory & Applications, 2021, 38(2): 245-254(in Chinese). [16] 王保宪, 唐林波, 陈聪葱, 等. 基于圆形采样和稀疏表示模型的鲁棒目标跟踪[J]. 北京理工大学学报, 2016, 36(9): 983-990.Wang B X, Tang L B, Chen C C, et al. Robust object tracking based on circle sampling and sparse representation[J]. Transactions of Beijing Institute of Technology, 2016, 36(9): 983-990(in Chinese) . [17] Wang C, Niu Y, Liu M Y, et al. Geomagnetic navigation for AUV based on deep reinforcement learning algorithm[C]//Proceeding of the IEEE International Conference on Robotics and Biomimetics. Piscataway: IEEE Press, 2020: 2571-2575. [18] 郭娇娇, 刘明雍, 刘坤, 等. 基于进化梯度搜索的 AUV 地磁仿生导航研究[J]. 西北工业大学学报, 2019, 5.Guo J J, Liu M Y, Liu K, et al. Research on AUV geomagnetic bionic navigation based on evolutionary gradient search[J]. Journal of Northwestern Polytechnical University, 2019, 37(5): 987-994(in Chinese) . [19] 张晓明, 赵剡. 基于克里金插值的局部地磁图的构建[J]. 电子测量技术, 2009, 32(4): 122-125.Zhang X M, Zhao S. Construction of local geomagnetic maps based on Kriging interpolation[J]. Electronic Measurement Technology, 2009, 32(4): 122-125(in Chinese) . [20] Keys R. Cubic convolution interpolation for digital image processing[J]. IEEE Transactions on Acoustics, Speech, and Signal Processing, 1981, 29(6): 1153-1160. [21] Friedman J H. Greedy function approximation: a gradient boosting machine[J]. Annals of Statistics, 2001, 29(5): 1189-1232. -
下载:
点击查看大图
计量
- 文章访问数: 400
- HTML全文浏览量: 134
- PDF下载量: 40
- 被引次数: 0
