The collection of CEC09 unconstrained multi-objective problems.



Default parameters for all convenience methods that are exported to the end user.

See OptRunController for the description.


JADE collection of optimization problems.

We skip (for now) f12 and f13 in the JADE paper since they are penalized functions which are quite nonstandard. We also skip f8 since we are unsure about its proper implementation.


Multi-objective optimization methods accepted by bboptimize().

The values are the method initialization routines or types derived from Optimizer.


Single objective optimization methods accepted by bboptimize().

The values are the method initialization routines or types derived from Optimizer.


The point of the SearchSpace.

The abstract type. It allows different actual implementations to be used, e.g Vector or SubArray.


Specific data and functions for adaptation An Adaptive DE typically changes parameters of the search dynamically. This is typically done in the tell!() function when we know if the trial vector was better than the target vector.


Archive saves information about promising solutions during an optimization run.


A mixture of 2 Cauchy distributions. Random values are further constrained to [0.0, 1.0] range either by truncating the initial unconstrained value or by generating new random value until it fits the range.


Borg MOEA algorithm.

Based on Hadka & Reed, "Borg: An Auto-Adaptive Many-Objective Evolutionary Computing Framework", Evol. Comp. 2013


Candidate solution to the problem.

Candidate can be either a member of the population (index > 0) or a standalone solution (index == -1). Can carry additional information, like the tag or the genetic operator applied (extra).


Modifies NC "children" by transferring some information from NP "parents".

The concrete implementations must provide apply!() method.


An ordered set of dicts that are examined one after another to find the parameter value. Returns nothing if no param setting is found in any of the dicts.


Set of points in fitness space that Pareto-dominate (MIN is true) or are dominated by (MIN is false) the given point pt.

The point pt itself is not part of its DominanceCone.

If the goal is to maximize each individual fitness component (i.e. is_minimizing(fs::FitnessScheme) == false), the cone of Pareto-dominating points should use MIN = false and Pareto-dominated points should use MIN = true.


ϵ-box archive saves only the solutions that are not ϵ-box dominated by any other solutions in the archive.

It also counts the number of candidate solutions that have been added and how many ϵ-box progresses have been made.


EpsBoxDominanceFitnessScheme defines ϵ-box dominance for N-tuple (N≧1) fitnesses. It operates with fitnesses of type IndexedTupleFitness.

aggregator::AGG is a function mapping tuple fitness to a single numerical value. Might be used for comparisons (or not, depending on the setup). Always used when printing fitness vectors though.


ϵ-dominance for N-tuple (N≧1) fitnesses.

aggregator::AGG is a function mapping tuple fitness to a single numerical value. Might be used for comparisons (or not, depending on the setup). Always used when printing fitness vectors though.


The abstract base for types that manage the objective function evaluation. P is the optimization problem it is used for.


A point in the problem's search space with the corresponding fitness value.

F is the original problem's fitness type


FitnessScheme defines how fitness vectors/values are compared, presented and aggregated. Fitness represents the score of one and the same individual on one or a set of evaluations. A scheme is a specific way to consider the scores in a coherent way. Type parameter F specifies the type of fitness values.

FitnessScheme could also be used as a function that defines the fitness ordering, i.e. fs(x, y) == true iff fitness x is better than y.


A FrequencyAdapter adapts the frequencies with which a set of values/strategies/methods should be applied/tried in an optimization problem. It is based on the Adaptive Coordinate Frequencies scheme described in:

T. Glasmachers and U. Dogan, "Accelerated Coordinate Descent with Adaptive Coordinate Frequencies", 2013.

but generalized so that it can support more than the adaptation of only the coordinates in a Coordinate Descent scheme. The things that are being adapted are identified by integers in 1:n, with n being the main parameter.


OptimizationProblem with the objective function defined by some Julia Function and search space.

Optionally, a known optimal value could be provided to terminate the optimization once it is reached.


Generating Set Search as described in Kolda2003: Kolda, Tamara G., Robert Michael Lewis, and Virginia Torczon. "Optimization by direct search: New perspectives on some classical and modern methods." SIAM review 45.3 (2003): 385-482.


RectSearchSpace that allows both continuous and discrete dimensions. Discrete dimensions are defined by the number of digits (dimdigits(ss)) after decimal point (i.e. precision)

If dimdigits(ss, i) is negative, i-th dimension is considered continuous.


Mutation clock operator is a more efficient way to mutate vectors than to generate a random value per variable in the vectors. It instead generates the number of variables to skip until the next mutation. Then it uses a sub-mutation operator to do the actual mutation. This is based on the paper: Deb and Deb (2012), "Analyzing Mutation Schemes for Real-Parameter Genetic Algorithms" but we use a Poisson distribution.


Optimization Controller.

Applies specific optimization method to a given problem. Supports restarts and modifying parameter of the method between runs. runcontrollers field maintains the list of OptRunController instances applied so far.

See OptRunController.


Create OptController for a given optimizer and a problem.

params are any of OptRunController parameters plus * :RngSeed and :RandomizeRngSeed params for controlling the random seed * :RecoverResults if intermediate results are returned upon InterruptException() (on by default)

OptRunController(optimizer::O, evaluator::E, params)

Create OptRunController for a given problem using specified optimizer.


  • optimizer initialized optimization method
  • evaluator the evaluator of the problem fitness
  • params controller settings, see DefaultParameters for the default values:
    • :MaxTime max time in seconds (takes precedence over the other budget-related params if specified), 0.0 disables the check
    • :MaxFuncEvals max fitness evals (takes precedence over max iterations, but not max time), 0 disables the check
    • :MaxSteps max iterations gives the least control since different optimizers have different "size" of their "iterations"
    • :MaxStepsWithoutProgress max iterations without fitness improvement
    • :MinDeltaFitnessTolerance minimum delta fitness (difference between the two consecutive best fitness improvements) we can accept before terminating
    • :FitnessTolerance stop the optimization when the best fitness found is within this distance of the actual optimum (if known)
    • :MaxNumStepsWithoutFuncEvals stop optimization if no new fitness evals in this many steps (indicates a converged/degenerate search)
    • :NumRepetitions number of repetitions to run for each optimizer for each problem
    • :TraceMode how the optimizer state is traced to the STDOUT during the optimization (one of :silent, :verbose)
    • :TraceInterval the trace interval (in seconds)
    • :SaveTrace whether to save it to a file (defaults to false)
    • :SaveFitnessTraceToCsv whether the history of fitness changes during optimization should be save to a csv file
    • :SaveParameters save method/controller parameters to a JSON file

The results of running optimization method.

Returned by run!(oc::OptRunController). Should be compatible (on the API level) with the Optim package. See make_opt_results().


Parent Centric Crossover (PCX).

See Deb, K., Anand, A., and Joshi, D., "A Computationally Efficient Evolutionary Algorithm for Real-Parameter Optimization," Evolutionary Computation, vol. 10, no. 4, pp. 371-395, 2002.


Pareto dominance for N-tuple (N≧1) fitnesses.

aggregator::AGG is a function mapping tuple fitness to a single numerical value. Might be used for comparisons (or not, depending on the setup). Always used when printing fitness vectors though.


Polynomial mutation as presented in the paper: Deb and Deb (2012), "Analyzing Mutation Schemes for Real-Parameter Genetic Algorithms"


The base abstract type for the collection of candidate solutions in the population-based optimization methods.


Default implementation of the Evaluator.

FP is the original problem's fitness type FA is the fitness type actually stored by the archive.


Base class for problem families.

It is an abstraction for problem parameterization (e.g by the number of the search space dimensions) that allows to instantiate OptimizationProblem for the concrete parameters.


IndividualsSelector that implements a "trivial geography" similar to Spector and Kline (2006) by first sampling an individual randomly and then selecting additional individuals for the same tournament within a certain deme of limited size (radius) for the sub-sequent individuals in the population.

The version we implement here is from: I. Harvey, "The Microbial Genetic Algorithm", in Advances in Artificial Life Darwin Meets von Neumann, Springer, 2011. The original paper is: Spector, L., and J. Klein. 2005. Trivial Geography in Genetic Programming. In Genetic Programming Theory and Practice III, edited by T. Yu, R.L. Riolo, and B. Worzel, pp. 109-124. Boston, MA: Kluwer Academic Publishers.


Embedding operator that randomly samples between parent's value and the nearest parameter boundary to get the new valid value if target's parameter is out-of-bounds.


In RatioFitnessScheme the fitness values can be ranked on a ratio scale so pairwise comparisons are not required. The default scale used is the aggregate of the fitness components.


Specification of a plot for the real-time tracking of the fitness progress.

To use the VegaLite front-end via BlackBoxOptimRealtimePlotServerExt extension, HTTP.jl and Sockets.jl are required.


A SearchSpace with N-dimensional rectangle as a set of valid points. I.e. dimmin(ss, i)x[i]dimmax(ss, i) for each dimension i.

RectSearchSpace(dimranges::AbstractVector; [dimdigits = nothing])

Create RectSearchSpace with given range of valid values for each dimension and, if specified, dimdigits precision for each dimension. Returns MixedPrecisionRectSearchSpace if there's at least one dimension with specified precision (dimdigits[i] ≥ 0), otherwise ContinuousRectSearchSpace.


The variants of the memetic search algorithms RS and RIS. However, we have modified them since they did not give very good performance when implemented as described in the papers below. Possibly, the papers are not unambigous and I have misinterpreted something from them...

The "Resampling Search" (RS) memetic algorithm is described in:

F. Caraffini, F. Neri, M. Gongora and B. N. Passow, "Re-sampling Search: A
Seriously Simple Memetic Approach with a High Performance", 2013.

and its close sibling "Resampling Inheritance Search" (RIS) is described in:

F. Caraffini, F. Neri, B. N. Passow and G. Iacca, "Re-sampled Inheritance
Search: High Performance Despite the Simplicity", 2013.

Float64-valued scalar fitness scheme. The boolean type parameter MIN specifies if smaller fitness values are better (true) or worse (false).


A base abstract type for OptimizationProblem search space. A concrete SearchSpace subtype specifies the valid candidate points that could be considered in a search/optimization.


A TransformedProblem subclass that shifts the objective argument and offsets the value: $f(x - xshift) + fitshift$ → min/max, $x$$X$ ($X$ stays intact).


Simplex Crossover (SPX).

ϵ>0 controls how the original simplex is inflated, ϵ=1 means no inflation.

See Tsutsui, Yamamura & Higuchi "Multi-parent recombination with simplex crossover in real coded genetic algorithms", 1999, Proc. of the Genetic and Evolutionary Computation Conference


BitSet-based container that could be used to e.g. store the ids of completed jobs by MultithreadEvaluator. Assumes there is max_seq_el element, so that all 1:maxseqel elements are in the set.


Optimizers derived from SteppingOptimizer implement classical iterative optimization scheme step!()step!() → …


Archive that maintains a top list of the best performing (best fitness) candidates seen so far.

The two last best fitness values could be used to estimate a confidence interval for how much improvement potential there is.


TransformedProblem{FS, P} is an abstract class for optimization problems derived from some original problem of type P by introducing a few changes.

The concrete derived types must implement:

  • objfunc() method
  • fitness_scheme() and search_space()

Base class for tuple-based fitness schemes.

Type parameters:

  • N is the number of the objectives
  • F is the type of each objective
  • FA is the actual type of the multi-objective fitness
  • MIN if objectives should be minimized or maximized
  • AGG the type of aggregator

Unimodal Normal Distribution Crossover (UNDX).

See Kita, H., Ono, I., and Kobayashi, S., "Multi-parental Extension of the Unimodal Normal Distribution Crossover for Real-coded Genetic Algorithms," Proceedings of the 1999 Congress on Evolutionary Computation, pp. 1581-1588, 1999. Deb, K., Anand, A., and Joshi, D., "A Computationally Efficient Evolutionary Algorithm for Real-Parameter Optimization," Evolutionary Computation, vol. 10, no. 4, pp. 371-395, 2002.

in(ind::AbstractIndividual, ss::SearchSpace)

Check if given individual lies in the given search space.

IGD(A::Vector{NTuple{N,F}}, B::Vector{NTuple{N,F}}, [two_sided=true])

The average Euclidean distance from the points of A to the points of B.

IGD(ref::Hypersurface, sol::Vector{FitIndividual}, [two_sided=true])

Average Euclidean distance from the exact Pareto frontier of the problem (ref) to the solution (sol) produced by the optimization method. If two_sided is on, returns the maximum of IGD(sol, ref) and IGD(nondominated(ref), sol).


Put the candidate back into the pool and copy the values into the corresponding individual of the population (candi.index should be set).

acquire_candi(pop::FitPopulation[, {ix::Int, candi::Candidate}])

Get individual from a pool, or create one if the pool is empty. By default the individual is not initialized, but if ix or candi is specified, the corresponding fields of the new candidate are set to the given values.

add_candidate!(a::TopListArchive, fitness::F, candidate[, tag=0][, num_fevals=-1])

Add a candidate with a fitness to the archive (if it is good enough).


Adjust the internal parameters of the genetic operator op taking into account the fitness change.

The default implementation does nothing.


Ask for a new candidate solution to be generated, and a list of individuals it should be ranked with.

The individuals are supplied as an array of tuples with the individual and its index.

See also tell!()

assign_weights!(weights, rankedCandidates, sortedWeights)

Assigns the candidate weights according to the candidate index. rankedCandidates are ranked by their fitness, sortedWeights are the corresponding weights.

Returns candidate weights sorted by the individual's index in the population.

async_update_fitness!(eval::AbstractAsyncEvaluator, candidates::Any;
                      force::Bool=false) ->
                      Union{AbstractFitnessEvaluationJob, Nothing}

Asynchronously calculate the fitnesses. candidates should support the iterate() interface and return Candidate elements.

Returns AbstractFitnessEvaluationJob or nothing if candidates does not contain any candidates.


  • force::Bool: force fitness calculation even if the candidate already has non-NA fitness

See also: sync_update_fitness

bboptimize(problem[, x0, parameters::Associative; kwargs...])

Solve given optimization problem. Optionally a starting point x0 can be specified.

See setup_problem() for the types of problem supported. In addition, the problem could be OptController containing the results of the previous optimization runs.

The optimization method parameters could be specified via kwargs or parameters arguments.

Returns OptimizationResults instance.

See also bbsetup() and BlackBoxOptim.OptRunController for a full list of supported parameters.

bbsetup(problem[; parameters::Associative, kwargs...])

Set up optimization method for a given problem.

See setup_problem() for the types of problem supported. The optimization method parameters could be specified via kwargs or parameters arguments.

Returns the initialized OptController instance. To actually run the method call bboptimize() or run!().

See also BlackBoxOptim.OptRunController for a full list of supported parameters.


Report the number of times each key was encountered in a count dict.

Returns a percentage dict calculated while iterating over the counted items.


Generator for the family of deceptive functions from the Cuccu2011 paper on novelty-based restarts. We have vectorized it to allow more than 1D versions. The Cuccu2011 paper uses the following values for

(l, w) = [(5, 0),  (15, 0),  (30, 0),
          (5, 2),  (15, 2),  (30, 2),
          (5, 10), (15, 10), (30, 10)]

and notes that (15, 2) and (30, 2) are the most difficult instances.


The difference between the current best fitness and the former best fitness.

dimdelta(ss::SearchSpace, [i::Integer])

A delta of maximal and minimal valid values for i-th dimension of ss (diameter of i-th dimension), or a vector of deltas for each dimension if no i given.

dimmax(ss::SearchSpace, [i::Integer])

A maximal valid value for i-th dimension of ss, or a vector of maximal valid values for each dimension of ss if no i was given.

dimmin(ss::SearchSpace, [i::Integer])

A minimal valid value for i-th dimension of ss, or a vector of minimal valid values for each dimension of ss if no i was given.

dimrange(ss::SearchSpace, [i::Integer])

Gets a ParamsRange tuple of minimal and maximal valid values for i-th dimension of ss, or a vector of ParamsRange tuples for each dimension if no i given.


From section 3, page 7, of Tsallis1996 paper: Tsallis and Stariolo, "Generalized simulated annealing", Physica A, 1996. available from the original paper used this as a 4-dimensional problem but here it is generalized.

fitness(params::Individual, e::ProblemEvaluator, tag::Int=0)

Evaluate the fitness and implicitly update the archive with the provided parameters and calculated fitness.

Returns the fitness in the archived format.

fitness_improvement_potential(a::Archive[, p = 0.01])

Calculate the solution improvement potential.

The percentage improvement that can be expected from the current fitness value at a given p-value. In theory, an optimization run should be terminated when this value is very small, i.e. there is little improvement potential left in the run.


Get the type of fitness values for fitness scheme fs.

format_fitness(fit, [problem::OptimizationProblem])

Format fitness into a string. Calls show_fitness() under the hood.

generate(surf::Hypersurface, fs::EpsBoxDominanceFitnessScheme, param_step = 0.1*fs.ϵ)

Generate the points of the hypersurface using the discretization defined by ϵ-box fitness schema.


Hartman 3D is a multi-minima, non-separable test problem. Our implementation is based on: However, we get a different global minima than the one stated on that page. The minima should be -3.86278 but we get a different one so use that in the problem spec:


Hartman 6D is a multi-minima, non-separable test problem. Our implementation is based on:


Check whether f1 or f2 fitness is better.


  • -1 if f1 is better than f2
  • 1 if f2 is better than f1
  • 0 if f1 and f2 are equal.

Returns a tuple of u and v comparison:

  • -1: u≺v
  • 0: u and v non-dominating
  • 1: u≻v

and whether u index fully matches v index.


Calculates the initial $log B$ matrix for xNES based on the deltas of each dimension.

instantiate(family::FunctionBasedProblemFamily, ndim::Int)

Construct FunctionBasedProblem with the given number of dimensions.

instantiate(family::FunctionBasedProblemFamily, ndim::Int)

Construct search space for FunctionBasedProblem with the given number of dimensions.

is_stopping(eval::AbstractAsyncEvaluator) -> Bool

Check if the evaluator is in the shutdown sequence.

When the evaluator is being shut down, it waits for the workers to finish their jobs and stop, no new jobs could be submitted.


Get the fitness of the last evaluated candidate.

Leads to nicer code if we can first test if it is better or worse than existing candidates and only want the fitness itself if the criteria fulfilled.


Get a tuple of the sign and the magnitude of the value rounded to the first digit. Used for archiving candidates separately for each magnitude class.


Merge the collection of multiple fitness histories and calculate the min, max, avg and median values for fitness and FIR (fitness improvement ratio) at each point where the fitness is changing.


Gets the random genetic operator from the mixture.

The probability to select an operator is proportional to its weight.

Returns a tuple of the genetic operator and its index in the mix.


Give the index of the method that should be used at the next iteration.

FrequencyAdapter maintains a block of randomly shuffled methods. The function takes the next available method in the block. If the block is empty, it is repopulated.

The methods are randomly shuffled each time the block is regenerated, since we need to know their effectiveness at every period of the optimization process.

noprogress_streak(a::EpsBoxArchive, [since_restart])

Get the number of add_candidate!() calls since the last ϵ-progress. If since_restart is specified, the number is relative to the last restart.


Get the problem parameters (a point in the search space) of the individual.


Generate a population for a given optimization problem.

method specifies a method for sampling random individuals, defaults to :latin_hypercube.

problem_set(ps::Dict{Any, FunctionBasedProblemFamily}, dims::Union{Int,Vector{Int}})

Construct a fixed-dimensional version of each problem from ps for each dimension given in dims.

Returns a dictionary of problems.


Get random Pareto frontier element.

Returns nothing if frontier is empty.

rand_individuals(ss, n; [method=:latin_hypercube])

Generate n individuals by randomly sampling in the ss search space using specified sampling method. The supported methods are:

  • uniform: uniform independent sampling for each dimension
  • latin_hypercube (the default): use latin hypercube sampling method

Print a report based on a result dict from one set of repeated runs of an optimization method. Returns the success rate, i.e. number of times the termination reason was "Within fitness tolerance...".


Reset the current ParallelEvaluationState and the vector of candidates that need fitness evaluation.


Reset the candidate fitness.

Need it when the candidate parameters have changed, but the stored fitness is still for the old parameter set.!Method

Start a new optimization run, possibly with new parameters and report on results.!Method

Run optimization until one of the stopping conditions are satisfied.

select(selector::IndividualsSelector, population, numSamples::Int)

Select numSamples random candidates from the population.

set_candidate!(o::Optimizer, x0)

Set a candidate as a starting points for optimization. For population-based optimizers this will randomly overwrite one of the candidate solutions of the current population with x0. For optimizers that maintain a single candidate that will be set to x0.

set_candidates!(o::Optimizer, x0)

Set a vector of candidates as starting points for optimization. For population-based optimizers this will randomly overwrite positions in the current population. Optimizers that maintain a single candidate doesn't implement this method since it would not be clear which of the supplied starting points should be chosen.

setup_problem(problem, parameters::Parameters)

Set up a fixed-dimensional optimization problem.

There are several setup_problem() method that accept different type of problem argument: * OptimizationProblem * function (:NumDimensions has to be specified in parameters) * FunctionBasedProblemFamily (:NumDimensions has to be specified in parameters)


Shekel10 is a 4D, multi-minima, non-separable test problem. Our implementation is based on the C code in:


Shekel5 is a 4D, multi-minima, non-separable test problem. Our implementation is based on the C code in:


Shekel7 is a 4D, multi-minima, non-separable test problem. Our implementation is based on the C code in:

show_fitness(io, fit, [problem::OptimizationProblem])

Output fitness to the given IO stream. show_fitness() method could be overridden for a specific problem, e.g. to print the names of each objective.


Count the tags of individuals on the ϵ-box frontier. Each restart the individual remains in the frontier discounts it by θ.

Returns the tagcount dictionary.

tell!(ato::AskTellOptimizer, rankedCandidates)

Tell the optimizer about the ranking of candidates. Returns the number of rankedCandidates that were inserted into the population, because of the improved fitness.

See also ask().

trace_state(io, op::GeneticOperator, mode::Symbol)

Trace the state of the operator. Called by trace_progress() during OptRunController run by some of the genetic optimizers.

Override the method to trace the state of your genetic operator.

trace_state(io::IO, optimizer::Optimizer, mode::Symbol)

Trace the current optimization state to a given IO stream. Called by OptRunController trace_progress().

Override it for your optimizer to generate method-specific diagnostic traces.


Update the internal model of progress and success rate of each method based on the latest progress value of one method. Progress values should be larger the larger the progress/improvement was.

update_fitness!([f], eval::Evaluator, candidates; force::Bool=false)

Calculate fitness of candidates and optionally apply f to each processed one. force specifies if already existing non-NA fitnesses should be re-evaluated.

update_fitness!([f], eval::Evaluator, candidate::Candidate; force::Bool=false) -> Candidate

Calculate fitness of candidate and optionally apply f. force specifies whether to re-evaluate fitness, if the candidate already has non-NA one.

viewer(population, individual_index)

Get vector-slice of the population matrix for the specified individual. Does not allocate any additional space, while still providing the same lookup performance.


Calculate the width of the confidence interval at a certain p-value. This is based on the paper: Carvalho (2011), "Confidence intervals for the minimum of a function using extreme value statistics"

This means that the current estimate of the confidence interval for the minimum of the optimized function lies within the interval

] l1 - w, l1 [

with probability $(1-p)$ as the number of sampled function points goes to infinity, where

w = width_of_confidence_interval(a, p)
l1 = best_fitness(a)
haltonnumber(base, index)

Generate the n-th Halton number in the sequence with base b.


Implementation is based on the psudo code in:
latin_hypercube_sampling(mins, maxs, n)

Randomly sample n vectors from the parallelogram defined by mins and maxs using the Latin hypercube algorithm.

Returns dims×n matrix.