A Robust, Adaptive Hybrid Nonlinear Conjugate Gradient Method Ensuring Global Convergence

Nonlinear conjugate gradient (NCG) methods are among the most effective techniques for solving large-scale unconstrained optimization problems. Their efficiency arises from low memory requirements, straightforward implementation, and strong performance in high-dimensional settings. A key component of NCG methods is the parameter βk\beta_kβk​, which governs how the new search direction is generated from the current gradient and the previous direction. The selection of βk\beta_kβk​ plays a crucial role in determining both the convergence behavior and computational efficiency of the algorithm.

Several well-known formulas have been proposed for βk\beta_kβk​, including the Fletcher–Reeves (FR), Polak–Ribiere (PRP), and Hestenes–Stiefel (HS) methods. The FR method is recognized for its solid theoretical convergence properties under standard line search conditions, yet it may exhibit slow progress in practice. On the other hand, PRP and HS often achieve faster numerical performance and better practical behavior, particularly for nonconvex problems. However, these methods may fail to guarantee global convergence unless additional safeguards are introduced to preserve the descent property.

To address these challenges, this work proposes a new adaptive hybrid formulation for βk\beta_kβk​ that combines the complementary strengths of FR, PRP, and HS. The proposed strategy dynamically balances robustness and efficiency by adjusting the influence of each formula based on the local characteristics of the objective function. A safeguarding mechanism is incorporated to ensure that the generated search direction maintains the descent condition at every iteration. Under standard assumptions and appropriate line search strategies, global convergence of the proposed method is established. Numerical experiments on benchmark problems demonstrate improved stability and competitive performance compared to classical NCG methods, without sacrificing theoretical guarantees.