Reference

​

Contents

​

​

Index

​

​

AdaptiveRegularization.HessOp — Type
HessOp(::AbstractNLPModel{T,S}, n)

Return a structure used for the evaluation of the Hessian matrix as an operator.

AdaptiveRegularization.PDataNLSST — Type
PDataNLSST(::Type{S}, ::Type{T}, n)

Return a structure used for the preprocessing of Steihaug-Toint methods for Gauss-Newton approximation of nonlinear least squares.

AdaptiveRegularization.TRARCWorkspace — Type
TRARCWorkspace(nlp, ::Type{Hess}, n)

Pre-allocate the memory used during the TRARC call for the problem nlp of size n. The possible values for Hess are: HessDense, HessSparse, HessSparseCOO, HessOp. Return a TRARCWorkspace structure.

AdaptiveRegularization.TrustRegion — Type
TrustRegion(α₀::T;kwargs...)

Select the main parameters used in the TRARC algorithm with α₀ as initial TR/ARC parameter. The keyword arguments are:

  • max_α::T: Maximum value for α. Default T(1) / sqrt(eps(T)).
  • acceptance_threshold::T: Ratio over which the step is successful. Default T(0.1).
  • increase_threshold::T: Ratio over which we increase α. Default T(0.75).
  • reduce_threshold::T: Ratio under which we decrease α. Default T(0.1).
  • increase_factor::T: Factor of increase of α. Default T(5.0).
  • decrease_factor::T: Factor of decrease of α. Default T(0.1).
  • max_unsuccinarow::Int: Limit on the number of successive unsucessful iterations. Default 30.

Returns a TrustRegion structure.

This can be compared to https://github.com/JuliaSmoothOptimizers/SolverTools.jl/blob/main/src/trust-region/basic-trust-region.jl

AdaptiveRegularization.TRARC — Function
TRARC(nlp; kwargs...)

Compute a local minimum of an unconstrained optimization problem using trust-region (TR)/adaptive regularization with cubics (ARC) methods.

Arguments

  • nlp::AbstractNLPModel: the model solved, see NLPModels.jl.

The keyword arguments include

  • TR::TrustRegion: structure with trust-region/ARC parameters, see TrustRegion. Default: TrustRegion(T(10.0)).
  • hess_type::Type{Hess}: Structure used to handle the hessian. The possible values are: HessDense, HessSparse, HessSparseCOO, HessOp. Default: HessOp.
  • pdata_type::Type{ParamData} Structure used for the preprocessing step. Default: PDataKARC.
  • solve_model::Function Function used to solve the subproblem. Default: solve_modelKARC.
  • robust::Bool: true implements a robust evaluation of the model. Default: true.
  • verbose::Bool: true prints iteration information. Default: false.

Additional kwargs are used for stopping criterion, see Stopping.jl.

Output

The returned value is a GenericExecutionStats, see SolverCore.jl.

This implementation uses Stopping.jl. Therefore, it is also possible to used

TRARC(stp; kwargs...)

which returns the stp::NLPStopping updated.

For advanced usage, the principal call to the solver uses a TRARCWorkspace.

TRARC(stp, pdata, workspace, trust_region_parameters; kwargs...)

Some variants of TRARC are already implemented and listed in AdaptiveRegularization.ALL_solvers.

References

This method unifies the implementation of trust-region and adaptive regularization with cubics as described in

Dussault, J.-P. (2020).
A unified efficient implementation of trust-region type algorithms for unconstrained optimization.
INFOR: Information Systems and Operational Research, 58(2), 290-309.
10.1080/03155986.2019.1624490

Examples

using AdaptiveRegularization, ADNLPModels
nlp = ADNLPModel(x -> 100 * (x[2] - x[1]^2)^2 + (x[1] - 1)^2, [-1.2; 1.0]);
stats = TRARC(nlp)

# output

"Execution stats: first-order stationary"
AdaptiveRegularization.compute_r — Method
compute_r(nlp, f, Δf, Δq, slope, d, xnext, gnext, robust)

Compute the actual vs predicted reduction ratio ∆f/Δq.

Arguments:

  • nlp: Current model we are trying to solve
  • f: current objective value
  • Δf: = f - f_trial is the actual reduction is an objective/merit/penalty function,
  • Δq: q - q_trial is the reduction predicted by the model q of f.
  • slope: current slope
  • d: potential next direction
  • xnext: potential next iterate
  • gnext: current gradient value, if good_grad is true, then this value has been udpated.
  • robust: if true, try to trap potential cancellation errors

Output:

  • r: reduction ratio ∆f/Δq
  • good_grad: true if gnext has been recomputed
  • gnext: gradient.

We assume that qis being minimized, and therefore thatΔq > 0`.

AdaptiveRegularization.hessian! — Function
hessian!(workspace::HessDense, nlp, x)

Return the Hessian matrix of nlp at x in-place with memory update of workspace.

AdaptiveRegularization.init — Method
init(::Type{Hess}, nlp::AbstractNLPModel{T,S}, n)

Return the hessian structure Hess and its composite type.

AdaptiveRegularization.preprocess — Method
preprocess(PData::TPData, H, g, gNorm2, n1, n2, α)

Function called in the TRARC algorithm every time a new iterate has been accepted.

Arguments

  • PData::TPData: data structure used for preprocessing.
  • H: current Hessian matrix.
  • g: current gradient.
  • gNorm2: 2-norm of the gradient.
  • n1: Current count on the number of Hessian-vector products.
  • n2: Maximum number of Hessian-vector products accepted.
  • α: current value of the TR/ARC parameter.

It returns PData.

AdaptiveRegularization.solve_model — Method
solve_model(PData::TPData, H, g, gNorm2, n1, n2, α)

Function called in the TRARC algorithm to solve the subproblem.

Arguments

  • PData::TPData: data structure used for preprocessing.
  • H: current Hessian matrix.
  • g: current gradient.
  • gNorm2: 2-norm of the gradient.
  • n1: Current count on the number of Hessian-vector products.
  • n2: Maximum number of Hessian-vector products accepted.
  • α: current value of the TR/ARC parameter.

It returns a couple (PData.d, PData.λ). Current implementations include: solve_modelKARC, solve_modelTRK, solve_modelST_TR.