Exploring Multi-Task Reinforcement Learning for Spatial Reasoning in Remote Sensing Vision-Language Models

Remote sensing visual question answering requires vision-language models to understand complex geospatial scenes and reason over object locations, spatial relations, and region-level semantics. Although reinforcement learning has shown potential for enhancing the reasoning ability of vision-language models, its application to multi-task remote sensing scenarios remains underexplored, especially under diverse imaging conditions and complex object distributions. In our preliminary experiments, we observe that directly mixing different remote sensing tasks during reinforcement learning may lead to imbalanced optimization, where some tasks dominate the training process while others improve slowly.

To investigate this problem, we explore a multi-task reinforcement learning framework for remote sensing vision-language models. The framework jointly considers several spatial reasoning tasks, including referring expression comprehension, referring expression segmentation, open-vocabulary detection, and RSVQA-style question answering, which together cover localization, segmentation, recognition, and question-driven reasoning. To improve training balance, we introduce a performance-aware task sampling strategy that adjusts task proportions according to task-level learning dynamics. In addition, we incorporate supervised anchor responses into the rollout process to provide more stable guidance during the early exploration stage, where purely sampled responses may produce noisy or unreliable reward signals.

This work provides an initial study of multi-task reinforcement learning for remote sensing vision-language models, aiming to enhance spatial reasoning, localization robustness, and generalization in RSVQA scenarios.