## Optimization

To show how the **Evolutionary** package can be used, we minimize the Rosenbrock function, a classical test problem for numerical optimization. We'll assume that you've already installed the **Evolutionary** package using Julia's package manager.

### Objective Function Definition

First, we load **Evolutionary** and define the Rosenbrock function:

```
using Evolutionary
f(x) = (1.0 - x[1])^2 + 100.0 * (x[2] - x[1]^2)^2 # Rosenbrock
```

There are various ways to define your objective function:

- For single-objective optimization, the objective function has to have one parameter and return a scalar value, e.g.

`soofun(x) = x[1] + x[2]`

- For multi-objective optimization, the objective function has to return an vector of values, e.g.

`moofun(x) = [ x[1], x[2]+1 ]`

- If multi-objective function has one parameter, the resulting value array will be copied. To reduce, additional data copy, the function can be defined with two parameters to perform in-place change of the result, e.g.

```
function moofun(F, x)
F[1] = x
F[2] = x +1
F
end
```

### Perform Optimization

Once we've defined this function, we can find the minimizer (the input that minimizes the objective) and the minimum (the value of the objective at the minimizer) using any of our favorite optimization algorithms. With a function defined, we just specify a form of an individual `x`

of the population for an evolutionary algorithm, and call `optimize`

with a starting individual `x0`

and a particular optimization algorithm, e.g. `CMAES()`

:

```
x0 = [0.0, 0.0];
Evolutionary.optimize(f, x0, CMAES())
```

`Evolutionary.optimize`

— Function```
optimize(f[, F], indiv, algo[, opts])
optimize(f[, F], constr, algo[, opts])
optimize(f[, F], constr, indiv, algo[, opts])
optimize(f[, F], constr, algo, poplt[, opts])
```

Perform optimization of the function `f`

using the algorithm `algo`

with the population, composed of the initial population `poplt`

, or individuals similar to the original individual `indiv`

, or generated from the constraints `constr`

, with the options `opts`

.

- For multi-objective optimization, the objective value
`F`

*must*be provided.

## Configurable options

There are several options that simply take on some default values if the user doesn't provide any.

### Algorithm options

There are different algorithms available in `Evolutionary`

, and they are all listed below. Notice that the constructors are written without input here, but they generally take keywords to tweak the way they work. See the pages describing each solver for more detail.

### General options

In addition to the algorithm, you can alter the behavior of the optimization procedure by using the following `Options`

keyword arguments:

`Evolutionary.Options`

— TypeThere are following options available:

`abstol::Float64`

: the absolute tolerance used in the convergence test`reltol::Float64`

: the relative tolerance used in the convergence test`successive_f_tol::Integer`

: the additional number of the iterations of the optimization algorithm after the convergence test is satisfied (*default: 10*)`iterations::Integer`

: the total number of the iterations of the optimization algorithm (*default: 1000*)`show_trace::Bool`

: enable the trace information display during the optimization (*default: false*).`store_trace::Bool`

: enable the trace information capturing during the optimization (*default: false*). The trace can be accessed by using`trace`

function after optimization is finished.`show_every::Integer`

: show every`n`

s successive trace message (*default: 1*)`time_limit::Float64`

: the time limit for the optimization run in seconds. If the value set to`NaN`

then the limit is not set. (*default: NaN*)`callback`

: the callback function that is called after each iteration of the optimization algorithm. The function accepts as parameter a trace dictionary, and**must**return a`Bool`

value which if`true`

terminates the optimization. (*default: nothing*)`parallelization::Symbol`

: allows parallelization of the population fitness evaluation if set to`:thread`

using multiple threads (*default:*)`:serial`

`rng::AbstractRNG`

: a random number generator object that is used to control generation of random data during the evolutionary optimization (*default:*)`Random.default_rng()`

We currently recommend the statically dispatched interface by using the `Evolutionary.Options`

constructor:

```
res = Evolutionary.optimize(x->-sum(x),
BitVector(zeros(30)),
GA(selection=uniformranking(5),
mutation=flip, crossover=SPX),
Evolutionary.Options(iterations=10))
```

## Obtaining results

After we have our results in `res`

object, we can use the API for getting optimization results. This consists of a collection of functions. They are not exported, so they have to be prefixed by `Evolutionary.`

. Say we do the following optimization:

```
julia> res = Evolutionary.optimize(x->-sum(x), BitVector(zeros(3)), GA())
* Status: success
* Candidate solution
Minimizer: [false, false, false]
Minimum: 0
Iterations: 11
* Found with
Algorithm: GA[P=50,x=0.8,μ=0.1,ɛ=0]
* Convergence measures
|f(x) - f(x')| = 0.0 ≤ 1.0e-12
* Work counters
Seconds run: 0.0354 (vs limit Inf)
Iterations: 11
f(x) calls: 600
```

You can inspect the result by using a collection of the auxiliary functions, e.g. the `minimizer`

and `minimum`

of the objective functions, which can be found using

```
julia> Evolutionary.minimizer(res)
3-element BitVector:
0
0
0
julia> Evolutionary.minimum(res)
0
```

#### Complete list of functions

An `OptimizationResults`

interface for representing an optimization result.

`Evolutionary.OptimizationResults`

— TypeAbstract evolutionary optimization result type

`Base.summary`

— Method`summary(result)`

Shows the optimization algorithm that produced this `result`

.

`Evolutionary.minimizer`

— Method`minimizer(result)`

A minimizer object from the optimization `result`

.

`Base.minimum`

— Method`minimum(result)`

A minimum value from the optimization `result`

.

`Evolutionary.iterations`

— Method`iterations(result)`

A number of iterations to reach the minimum.

`Evolutionary.iteration_limit_reached`

— Method`iteration_limit_reached(result)`

Returns `true`

if the iteration limit was reached.

`Evolutionary.trace`

— Method`trace(result)`

Returns a trace of optimization states from the optimization `result`

.

`NLSolversBase.f_calls`

— Method`f_calls(result)`

Returns a number of an objective function calls.

`Evolutionary.abstol`

— Method`abstol(result)`

Returns an absolute tolerance value of the optimization `result`

.

`Evolutionary.reltol`

— Method`reltol(result)`

Returns a relative tolerance value of the optimization `result`

.

An implementation of the result object for evolutionary optimizations.

`Evolutionary.EvolutionaryOptimizationResults`

— TypeEvolutionary optimization result type

`Evolutionary.converged`

— Method`converged(result)`

Returns `true`

if the optimization successfully converged to a minimum value.

## Trace

When `store_trace`

and/or `show_trace`

options are set to `true`

in the `Options`

object, an optimization trace is either captured and/or shown on the screen. By default, only the current state minimum value is displayed in the trace. In order to extend trace record, you need to override `trace!`

function providing specialize function behavior on one of specific parameters.

`Evolutionary.trace!`

— Method`trace!(record::Dict{String,Any}, objfun, state, population, method, options)`

Update the trace `record`

. This function allows to supplement an additional information into the optimization algorithm trace by modifying a trace `record`

. It can be overridden by specifying particular parameter types.

Commonly, you would define a specializations of a `state`

, `population`

, or `method`

parameters of `trace!`

function, e.g.

```
function trace!(record::Dict{String,Any}, objfun, state, population, method::CMAES, options)
record["σ"] = state.σ
end
```

## Parallelization

If the objective function is heavily CPU-bound, it's possible to utilize multiple processors/threads to speed up computations. To enable multi-threading evaluation of the objective function, set `parallelization`

option to `:thread`

in the `Options`

object.