Reinforcement Learning for UAV Path Planning Under Complicated Constraints with GNSS Quality Awareness

Requirements for Unmanned Aerial Vehicle (UAV) applications in low-altitude operations are escalating demands on the heavy reliance on Global Navigation Satellite System (GNSS) services for Position, Navigation and Timing (PNT) solutions. This paper presents the integration of forecasted GNSS services into UAV path planning to meet such criteria.
UAVs often operate under stringent environments with rigid resilience requirements, and face challenges caused by dense, dynamic, complicated, and uncertain obstructions. When flying in complex environments, it is important to consider signal degradation caused by reflections (multipath), and obscuration (Non-Line of Sight (NLOS)), which can lead to positioning errors that must be minimized to ensure flight safety.
Recent works integrate GNSS reliability maps derived from pseudorange error estimations into path planning to reduce loss-of-GNSS risks with PNT degradations. To accommodate multiple constraint conditions attempting to improve flight resilience against GNSS-degraded environments, this paper proposes a Reinforcement Learning (RL) approach to feature GNSS quality awareness during path planning. The non-linearity relations between GNSS quality in the form of Dilution of Precision (DOP), geographic locations, and the policy of searching sub-minima points are learned by clipped Proximal Policy Optimization (PPO) method. Other constraints considered include static obstacle occurrence, altitude boundary, forbidden flying regions, and operational volumes. The reward and punishment functions and the training method are designed for maximising the success criteria of approaching destinations. The proposed RL approach is demonstrated using a real 3D map of Indianapolis, USA in the Godot engine, incorporating forecasted DOP data generated by a Geospatial Augmentation system named GNSS Foresight from Spirent. Results indicate a 36\% enhancement in mission success rates when GNSS performance is included in the path planning training. Additionally, the varying tensor size, representing the UAV's DOP perception range, exhibits a positive proportion relation to a higher mission rate, despite an increment in computational complexity.