This package exports

  • BOpt, boptimize!
  • acquisition types: ExpectedImprovement, ProbabilityOfImprovement, UpperConfidenceBound, ThompsonSamplingSimple, MutualInformation
  • scaling of standard deviation in UpperConfidenceBound: BrochuBetaScaling, NoBetaScaling
  • GP hyperparameter optimizer: MAPGPOptimizer, NoModelOptimizer
  • Initializer: ScaledSobolIterator, ScaledLHSIterator
  • optimization sense: Min, Max
  • verbosity levels: Silent, Timings, Progress
  • helper: maxduration!, maxiterations!

Use the REPL help, e.g. ?Bopt, to get more information.

BOpt(func, model, acquisition, modeloptimizer, lowerbounds, upperbounds;
          sense = Max, maxiterations = 10^4, maxduration = Inf,
          acquisitionoptions = NamedTuple(), repetitions = 1,
          verbosity = Progress,
          initializer_iterations = 5*length(lowerbounds),
          initializer = ScaledSobolIterator(lowerbounds, upperbounds,

Scales βt of UpperConfidenceBound as

βt = √(2 * log(t^(D/2 + 2) * π^2/(3δ)))

where t is the number of observations, D is the dimensionality of the input data points and δ is a small constant (default δ = 0.1).

See Brochu E., Cora V. M., de Freitas N. (2010), "A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning", page 16.


The expected improvement measures the expected improvement x - τ of a point x upon an incumbent target τ. For Gaussian distributions it is given by

(μ(x) - τ) * ϕ[(μ(x) - τ)/σ(x)] + σ(x) * Φ[(μ(x) - τ)/σ(x)]

where ϕ is the standard normal distribution function and Φ is the standard normal cumulative function, and μ(x), σ(x) are mean and standard deviation of the distribution at point x.

MAPGPOptimizer(; every = 10, kwargs...)

Set the GP hyperparameters to the maximum a posteriori (MAP) estimate every number of steps. Run BayesianOptimization.defaultoptions(MAPGPOptimizer) to see the default options. By default, all priors are flat. Uniform priors in an interval can be specified by setting the bounds of the form [lowerbound, upperbound], e.g. for a kernel with 3 parameters one would set kernbounds = [[-3, -3, -4], [2, 3, 1]]. Non-flat priors can be specified directly on the GP parameters, e.g. using Distributions; set_priors!(mean, [Normal(0., 1.)])


The mutual information measures the amount of information gained by querying at x. The parameter γ̂ gives a lower bound for the information on f from the queries {x}. For a Gaussian this is γ̂ = ∑σ²(x) and the mutual information at x is μ(x) + √(α)*(√(σ²(x)+γ̂) - √(γ̂))

where μ(x), σ(x) are mean and standard deviation of the distribution at point x.

See Contal E., Perchet V., Vayatis N. (2014), "Gaussian Process Optimization with Mutual Information"


The probability of improvement measures the probability that a point x leads to an improvement upon an incumbent target τ. For Gaussian distributions it is given by

Φ[(μ(x) - τ)/σ(x)]

where Φ is the standard normal cumulative distribution function and μ(x), σ(x) are mean and standard deviation of the distribution at point x.

ScaledSobolIterator(lowerbounds, upperbounds, N;
                    seq = SobolSeq(length(lowerbounds)))

Returns an iterator over N elements of a Sobol sequence between lowerbounds and upperbounds. The first N elements of the Sobol sequence are skipped for better uniformity (see


The acquisition function associated with ThompsonSamplingSimple draws independent samples for each input x a function value from the model. Together with a gradient-free optimization method this leads to proposal points that might be similarly distributed as the maxima of true Thompson samples from GPs. True Thompson samples from a GP are simply functions from a GP. Maximizing these samples can be tricky, see e.g. chapter 6.


For Gaussian distributions the upper confidence bound at x is given by μ(x) + βt * σ(x)

where βt is a fixed parameter in the case of NoBetaScaling or an observation size dependent parameter in the case of e.g. BrochuBetaScaling.

 ScaledLHSIterator(lowerbounds, upperbounds, N)

Returns an iterator over N elements of a latin hyper cube sample between lowerbounds and upperbounds. See also ScaledSobolIterator for an iterator that has arguably better uniformity.

 maxduration!(s::IterationCounter, duration)
 maxduration!(o::BOpt, duration)

Sets the maximal duration per call of boptimize! to duration.

 maxiterations!(s::IterationCounter, N)
 maxiterations!(o::BOpt, N)

Sets the maximal number of iterations per call of boptimize! to N.