This sections provides details on how to instantiate and solve a constrained least squares problem with Enlsip.jl As a reminder from Home, problems to solve are of the following form:

\[\begin{aligned} \underset{x}{\min} \quad & \dfrac{1}{2} \|r(x)\|^2 \\ \text{s.t.} \quad & c_i(x) = 0, \quad i =1,\ldots,q \\ & c_j(x) \geq 0, \quad j=q+1,\ldots,\ell, \\ \end{aligned}\]

Note that with this formulation, bounds constraints are not distinguished from general inequality constraints. Though, as shown later in this section, they can be provided as vectors of lower and/or upper bounds, which is more convenient for this type of constraints.

Also, the Enlsip solver works with double precision float numbers (i.e. type Float64).

Instantiate a model

Solving a problem with Enlsip is organized in two steps.

First, a model of type CnlsModel must be instantiated.

The CnlsModel constructor requires the evaluation functions of residuals, constraints, their associated jacobian matrices and dimensions of the problem.

Although the package enables one to create linear unconstrained least squares, it is recommended to use it to solve nonlinear least squares with general constraints.

The three following positional arguments are mandatory to create a model:

  • residuals : function that computes the vector of residuals
  • nb_parameters : number of variables
  • nb_residuals : number of residuals

The following keywords arguments are optionnal and deal with constraints and jacobian matrices computations. If the jacobian matrices functions are not provided, they are computed numerically by forward differences within Enlsip.

starting_pointinitial solution (can be an infeasbile point)
jacobian_residualsfunction computing the jacobian matrix of the residuals
eq_constraintsfunction computing the equality constraints
jacobian_eqconsfunction computing the jacobian matrix of the equality constraints
nb_eqconsnumber of equality constraints
ineq_constraintsfunction computing the inequality constraints
jacobian_ineqconsfunction computing the jacobian matrix of the inequality constraints
nb_ineqconsnumber of inequality constraints
x_lowvector of lower bounds
x_uppvector of upper bounds

Solving a model

Then, the Enlsip solver can be used by calling the solve! function on a instantiated model.

Missing docstring.

Missing docstring for Enlsip.solve!. Check Documenter's build log for details.

By default, the algorithm will print some details about the iterations performed through during execution.

Here are some details on how to read and understand the different columns of the output:

iteriteration number
objectivevalue of the sum of squared residuals (i.e. objective function) at current point
$\vert\vert$ active_constraints $\vert\vert^2$value of the sum of squared active constraints at current point
$\vert\vert$ p $\vert\vert$norm of the search direction computed at current iteration
$\alpha$value of the steplength computed at current iteration
reductionreduction in the objective function performed at curret iteration

Once the Enlsip solver has been called, one can get additional info about the success, or failure, of the algorithm by calling one of the following functions:

Missing docstring.

Missing docstring for Enlsip.solution. Check Documenter's build log for details.

Missing docstring.

Missing docstring for Enlsip.status. Check Documenter's build log for details.

Missing docstring.

Missing docstring for Enlsip.objective_value. Check Documenter's build log for details.


Problem 65 from Hock and Schittkowski collection[1]

We show how to implement and solve the following problem:

\[\begin{aligned} \min \quad & (x_1-x_2)^2 + \dfrac{(x_1+x_2-10)^2}{9}+(x_3-5)^2 \\ \text{s.t.} \quad & 48-x_1^2-x_2^2-x_3^2 \geq 0\\ & -4.5\leq x_i \leq 4.5, \quad i=1,2\\ & -5 \leq x_3 \leq 5. \end{aligned}.\]

First, we provide the dimensions of the problems.

# Dimensions of the problem

n = 3 # number of parameters
m = 3 # number of residuals
nb_eq = 0 # number of equality constraints
nb_constraints = 7 # number of inequality constraints

Then, we define the functions required to compute the residuals, constraints, their respective jacobian matrices and a starting point.

# Residuals and jacobian matrix associated
r(x::Vector) = [x[1] - x[2]; (x[1]+x[2]-10.0) / 3.0; x[3]-5.0]

jac_r(x::Vector) = [1. -1. 0;
    1/3 1/3 0.;
    0. 0. 1.]

# Constraints (one equality and box constraints)

c(x::Vector) = [48.0 - x[1]^2-x[2]^2-x[3]^2] # evaluation function for the equality constraint
jac_c(x::Vector) = [ -2x[1] -2x[2] -2x[3]] # jacobian matrix of the equality constraint

x_l = [-4.5, -4.5, -5.0] # lower bounds
x_u = [4.5, 4.5, 5.0] # upper bounds

# Starting point
x0 = [-5.0, 5.0, 0.0]

A CnlsModel can now be instantiated.

# Instantiate a model associated with the problem
hs65_model = Enlsip.CnlsModel(r, n, m ;starting_point=x0, ineq_constraints = c,
nb_ineqcons = 1, x_low=x_l, x_upp=x_u, jacobian_residuals=jac_r, jacobian_ineqcons=jac_c)

Finally, the solve! function can be called on our model. In this example, we set the silent optionnal argument to true so no output will be printed. See previous Solving a model section for details about what is printed after solving a problem. Other optionnal arguments remain to default values.

Enlsip.solve!(hs65_model, silent=true)

Once Enlsip solver has been executed on a problem, one can check if the problem has been successfully solved or not.


One can also get the optimal solution obtained and value of objective function at that point.

hs65_solution = Enlsip.solution(hs65_model)
3-element Vector{Float64}:
hs65_objective = Enlsip.objective_value(hs65_model)
  • 1W. Hock and K. Schittkowski. Test Examples for Nonlinear Programming Codes, volume 187 of Lecture Notes in Economics and Mathematical Systems. Springer, second edition, 1980.