SimpleSolvers

simple bisection line search

An OptimizationAlgorithm is a datastructe that is used to dispatch on different algorithms.

It needs to implement three important methods,

initialize!(alg::OptimizationAlgorithm, ::AbstractVector)
update!(alg::OptimizationAlgorithm, ::AbstractVector)
solver_step!(::AbstractVector, alg::OptimizationAlgorithm)

that initialize and update the state of the algorithm and perform an actual optimization step.

Further the following convenience methods should be implemented,

objective(alg::OptimizationAlgorithm)
gradient(alg::OptimizationAlgorithm)
hessian(alg::OptimizationAlgorithm)
linesearch(alg::OptimizationAlgorithm)

which return the objective to optimize, its gradient and (approximate) Hessian as well as the linesearch algorithm used in conjunction with the optimization algorithm if any.

Configurable options with defaults (values 0 and NaN indicate unlimited):

x_abstol::Real = 1e-50,
x_reltol::Real = 2eps(),
f_abstol::Real = 1e-50,
f_reltol::Real = 2eps(),
f_mindec::Real = 1e-4,
g_restol::Real = sqrt(eps()),
x_abstol_break::Real = Inf,
x_reltol_break::Real = Inf,
f_abstol_break::Real = Inf,
f_reltol_break::Real = Inf,
g_restol_break::Real = Inf,
f_calls_limit::Int = 0,
g_calls_limit::Int = 0,
h_calls_limit::Int = 0,
allow_f_increases::Bool = true,
min_iterations::Int = 0,
max_iterations::Int = 1_000,
warn_iterations::Int = max_iterations,
show_trace::Bool = false,
store_trace::Bool = false,
extended_trace::Bool = false,
show_every::Int = 1,
verbosity::Int = 1

Quadratic Polynomial line search

Evaluates the objective value at x. Returns f(x), but does not store the value in obj.f

Get the most recently evaluated objective value of obj.

Evaluates the objective value at x. Returns f(x), but does not store the value in obj.F

Get the most recently evaluated objective value of obj.

Force (re-)evaluation of the derivative of the objective at x. Returns f'(x) and stores the derivative in obj.D

Evaluates the derivative of the objective at x. Returns f'(x) and stores the derivative in obj.D

Evaluates the derivative of the objective at x. Returns f'(x), but does not store the derivative in obj.D

Get the most recently evaluated derivative of the objective of obj.

Force (re-)evaluation of the gradient at x. Returns ∇f(x) and stores the value in obj.g.

Evaluates the gradient at x. Returns ∇f(x) and stores the value in obj.g.

Evaluates the gradient at x. This does not update obj.g or obj.x_g.

Get the most recently evaluated gradient of obj.

Verifies if an object implements the OptimizationAlgorithm interface.

create univariate objective for linesearch algorithm

create univariate objective for linesearch algorithm

compute objective and gradient at new solution and update result

Force (re-)evaluation of the objective at x. Returns f(x) and stores the value in obj.f

Force (re-)evaluation of the objective value at x. Returns f(x) and stores the value in obj.F

Evaluates the objective at x. Returns f(x) and stores the value in obj.f

Evaluates the objective value at x. Returns f(x) and stores the value in obj.F