`BayesianOptimization.BayesianOptimization`

— ModuleThis package exports

`BOpt`

,`boptimize!`

- acquisition types:
`ExpectedImprovement`

,`ProbabilityOfImprovement`

,`UpperConfidenceBound`

,`ThompsonSamplingSimple`

,`MutualInformation`

- scaling of standard deviation in
`UpperConfidenceBound`

:`BrochuBetaScaling`

,`NoBetaScaling`

- GP hyperparameter optimizer:
`MAPGPOptimizer`

,`NoModelOptimizer`

- Initializer:
`ScaledSobolIterator`

,`ScaledLHSIterator`

- optimization sense:
`Min`

,`Max`

- verbosity levels:
`Silent`

,`Timings`

,`Progress`

- helper: maxduration!, maxiterations!

Use the REPL help, e.g. `?Bopt`

, to get more information.

`BayesianOptimization.BOpt`

— Method```
BOpt(func, model, acquisition, modeloptimizer, lowerbounds, upperbounds;
sense = Max, maxiterations = 10^4, maxduration = Inf,
acquisitionoptions = NamedTuple(), repetitions = 1,
verbosity = Progress,
initializer_iterations = 5*length(lowerbounds),
initializer = ScaledSobolIterator(lowerbounds, upperbounds,
initializer_iterations))
```

`BayesianOptimization.BrochuBetaScaling`

— TypeScales `βt`

of `UpperConfidenceBound`

as

`βt = √(2 * log(t^(D/2 + 2) * π^2/(3δ)))`

where `t`

is the number of observations, `D`

is the dimensionality of the input data points and δ is a small constant (default δ = 0.1).

See Brochu E., Cora V. M., de Freitas N. (2010), "A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning", https://arxiv.org/abs/1012.2599v1 page 16.

`BayesianOptimization.ExpectedImprovement`

— TypeThe expected improvement measures the expected improvement `x - τ`

of a point `x`

upon an incumbent target `τ`

. For Gaussian distributions it is given by

`(μ(x) - τ) * ϕ[(μ(x) - τ)/σ(x)] + σ(x) * Φ[(μ(x) - τ)/σ(x)]`

where `ϕ`

is the standard normal distribution function and `Φ`

is the standard normal cumulative function, and `μ(x)`

, `σ(x)`

are mean and standard deviation of the distribution at point `x`

.

`BayesianOptimization.MAPGPOptimizer`

— Method`MAPGPOptimizer(; every = 10, kwargs...)`

Set the GP hyperparameters to the maximum a posteriori (MAP) estimate `every`

number of steps. Run `BayesianOptimization.defaultoptions(MAPGPOptimizer)`

to see the default options. By default, all priors are flat. Uniform priors in an interval can be specified by setting the bounds of the form `[lowerbound, upperbound]`

, e.g. for a kernel with 3 parameters one would set `kernbounds = [[-3, -3, -4], [2, 3, 1]]`

. Non-flat priors can be specified directly on the GP parameters, e.g. `using Distributions; set_priors!(mean, [Normal(0., 1.)])`

`BayesianOptimization.MutualInformation`

— TypeThe mutual information measures the amount of information gained by querying at x. The parameter γ̂ gives a lower bound for the information on f from the queries {x}. For a Gaussian this is γ̂ = ∑σ²(x) and the mutual information at x is μ(x) + √(α)*(√(σ²(x)+γ̂) - √(γ̂))

where `μ(x)`

, `σ(x)`

are mean and standard deviation of the distribution at point `x`

.

See Contal E., Perchet V., Vayatis N. (2014), "Gaussian Process Optimization with Mutual Information" http://proceedings.mlr.press/v32/contal14.pdf

`BayesianOptimization.NoBetaScaling`

— TypeApplies no scaling to `βt`

of `UpperConfidenceBound`

.

`BayesianOptimization.NoModelOptimizer`

— TypeDon't optimize the model ever.

`BayesianOptimization.ProbabilityOfImprovement`

— TypeThe probability of improvement measures the probability that a point `x`

leads to an improvement upon an incumbent target `τ`

. For Gaussian distributions it is given by

`Φ[(μ(x) - τ)/σ(x)]`

where `Φ`

is the standard normal cumulative distribution function and `μ(x)`

, `σ(x)`

are mean and standard deviation of the distribution at point `x`

.

`BayesianOptimization.ScaledSobolIterator`

— Method```
ScaledSobolIterator(lowerbounds, upperbounds, N;
seq = SobolSeq(length(lowerbounds)))
```

Returns an iterator over `N`

elements of a Sobol sequence between `lowerbounds`

and `upperbounds`

. The first `N`

elements of the Sobol sequence are skipped for better uniformity (see https://github.com/stevengj/Sobol.jl)

`BayesianOptimization.ThompsonSamplingSimple`

— TypeThe acquisition function associated with `ThompsonSamplingSimple`

draws independent samples for each input `x`

a function value from the model. Together with a gradient-free optimization method this leads to proposal points that might be similarly distributed as the maxima of true Thompson samples from GPs. True Thompson samples from a GP are simply functions from a GP. Maximizing these samples can be tricky, see e.g. http://hildobijl.com/Downloads/GPRT.pdf chapter 6.

`BayesianOptimization.UpperConfidenceBound`

— TypeFor Gaussian distributions the upper confidence bound at `x`

is given by μ(x) + βt * σ(x)

where `βt`

is a fixed parameter in the case of `NoBetaScaling`

or an observation size dependent parameter in the case of e.g. `BrochuBetaScaling`

.

`BayesianOptimization.UpperConfidenceBound`

— Method`UpperConfidenceBound(; scaling = BrochuBetaScaling(.1), βt = 1)`

`BayesianOptimization.ScaledLHSIterator`

— Method` ScaledLHSIterator(lowerbounds, upperbounds, N)`

Returns an iterator over `N`

elements of a latin hyper cube sample between `lowerbounds`

and `upperbounds`

. See also `ScaledSobolIterator`

for an iterator that has arguably better uniformity.

`BayesianOptimization.boptimize!`

— Method`boptimize!(o::BOpt)`

`BayesianOptimization.maxduration!`

— Method```
maxduration!(s::IterationCounter, duration)
maxduration!(o::BOpt, duration)
```

Sets the maximal duration per call of `boptimize!`

to `duration`

.

`BayesianOptimization.maxiterations!`

— Method```
maxiterations!(s::IterationCounter, N)
maxiterations!(o::BOpt, N)
```

Sets the maximal number of iterations per call of `boptimize!`

to `N`

.