Agriculture performs a vital role in ensuring the growing world population's feeding demands. Therefore, the water resource is essential for achieving food demands since it supports the crop's production conditions, improves productivity, reduces extreme weather impacts, and promotes environmental conservation. Nevertheless, proper agricultural water management is complex due to temporary alterations in water resources availability, climate change impacts, and multiple water decision-makers without mutual consensus adding uncertainty and increasing the risk in the decision-making process. Therefore, it is necessary to apply methods that support decision-makers allowing optimal conditions for crop productivity at minimum risk. In this sense, Mathematical Programming (MP) represents a strategy that offers various modeling techniques for supporting decision-makers on water optimization under uncertain conditions. Multistage Stochastic Programming (MSP) provides stage-structured decision-making schemes for supporting water decision-making based on scenario analysis. Consequently, the study develops a literature review on MSP for optimum agricultural water resources management and allocation under uncertainty, answering the following guiding questions: 1. What are the implications of proper water resources allocation in improving farmers' benefits? 2. What are the main difficulties faced in water allocation on agricultural irrigation practices? 3. What are the main uncertain modeling strategies related to MSP? The early findings indicate the effectiveness of MSP, Interval, and Fuzzy Programming combinations to tackle different uncertain sources providing better allocation schemes, flexible decision-making, and low-cost solutions. Besides, this work concludes that water availability based on climate change represents the primary source of uncertainty, and farmers face the most significant risk as water end-users.