Descent Subroutines

NewtonDescent(; linsolve = nothing, precs = DEFAULT_PRECS)

Compute the descent direction as $J δu = -fu$. For non-square Jacobian problems, this is commonly referred to as the Gauss-Newton Descent.

See also Dogleg, SteepestDescent, DampedNewtonDescent.

SteepestDescent(; linsolve = nothing, precs = DEFAULT_PRECS)

Compute the descent direction as $δu = -Jᵀfu$. The linear solver and preconditioner are only used if J is provided in the inverted form.

See also Dogleg, NewtonDescent, DampedNewtonDescent.

DampedNewtonDescent(; linsolve = nothing, precs = DEFAULT_PRECS, initial_damping,
    damping_fn)

A Newton descent algorithm with damping. The damping factor is computed using the damping_fn function. The descent direction is computed as $(JᵀJ + λDᵀD) δu = -fu$. For non-square Jacobians, we default to solving for Jδx = -fu and √λ⋅D δx = 0 simultaneously. If the linear solver can't handle non-square matrices, we use the normal form equations $(JᵀJ + λDᵀD) δu = Jᵀ fu$. Note that this factorization is often the faster choice, but it is not as numerically stable as the least squares solver.

The damping factor returned must be a non-negative number.

Keyword Arguments

• initial_damping: The initial damping factor to use
• damping_fn: The function to use to compute the damping factor. This must satisfy the NonlinearSolve.AbstractDampingFunction interface.

Dogleg(; linsolve = nothing, precs = DEFAULT_PRECS)

Switch between Newton's method and the steepest descent method depending on the size of the trust region. The trust region is specified via keyword argument trust_region to solve!.

See also SteepestDescent, NewtonDescent, DampedNewtonDescent.

GeodesicAcceleration(; descent, finite_diff_step_geodesic, α)

Uses the descent algorithm to compute the velocity and acceleration terms for the geodesic acceleration method. The velocity and acceleration terms are then combined to compute the descent direction.

This method in its current form was developed for LevenbergMarquardt. Performance for other methods are not theorectically or experimentally verified.

Keyword Arguments

• descent: the descent algorithm to use for computing the velocity and acceleration.
• finite_diff_step_geodesic: the step size used for finite differencing used to calculate the geodesic acceleration. Defaults to 0.1 which means that the step size is approximately 10% of the first-order step. See Section 3 of [1].
• α: a factor that determines if a step is accepted or rejected. To incorporate geodesic acceleration as an addition to the Levenberg-Marquardt algorithm, it is necessary that acceptable steps meet the condition $\frac{2||a||}{||v||} \le \alpha_{\text{geodesic}}$, where $a$ is the geodesic acceleration, $v$ is the Levenberg-Marquardt algorithm's step (velocity along a geodesic path) and α_geodesic is some number of order 1. For most problems α_geodesic = 0.75 is a good value but for problems where convergence is difficult α_geodesic = 0.1 is an effective choice. Defaults to 0.75. See Section 3 of Transtrum and Sethna [1].