Hierarchical Markov decision processes based path planning for UAV in three-dimensional environment

The ability of path planning is an important ensure for unmanned aerial vehicle (UAV) in autonomous flight. A path planning model was based on Markov decision processes (MDP), in which the problem of path planning was regarded as looking for the best tactic through the model of environment and the principle of rewards and punishment. To solve the problem such as huge space-time spending and changing course at high frequency, the hierarchical Markov decision processes (HMDP) were introduced based on the method of clustering states. The arithmetic was also used for path planning in three-dimensional environment. The results of simulation show the HMDP model can be used to path planning for UAV in three-dimensional environment. It lays the foundation for local path planning in real flight.