Conjugate gradient line search implementation from: W. W. Hager and H. Zhang (2006) Algorithm 851: CG_DESCENT, a conjugate gradient method with guaranteed descent. ACM Transactions on Mathematical Software 32: 113–137.


Solve an iteration using TRS to produce stepsizes, alpha: extrapolation gamma: gradient step


Solving TrustRegionSubproblem` via a given regularizer or a radius or radius-free

the strict regularization mode

strictly solve a quadratic regularization problems given a regularizer :λ

the radius free mode

compute the Lagrangian multiplier according to the adaptive :γ instead. see the paper for details.

the strict radius mode

if you choose a radius, this function is a bisection procedure following the complexity of O(loglog(1/ε)) by Ye you may find other popular methods like Steihaug-CG, Steihaug-Toint Lanczos method, et cetera, which are the most practical TRS solvers, as far as I know, you can find their implementations in GLTR and GLRT in GALAHAD library.