function DE(;
    n_groups = 4, 
    priors = nothing, 
    Np, 
    burnin = 1000, 
    discard_burnin = true, 
    α = .1,
    β = .1, 
    ϵ = .001,
    σ = .05, 
    κ = 1.0, 
    θsnooker = 0.0, 
    bounds, 
    n_initial = 0, 
    generate_proposal = random_gamma, 
    update_particle! = mh_update!,
    evaluate_fitness! = compute_posterior!, 
    sample = sample,
    blocking_on = x -> false,
    blocks = [false])

Differential Evolution MCMC object.

Keywords

• n_groups=4: number of groups of particles.
• Np: number of particles per group.
• burnin=1000: number of burnin iterations
• discard_burnin: indicates whether burnin samples are discarded. Default is true.
• α=.1: migration probability.
• β=.1: mutation probability.
• ϵ=.001: noise in crossover step.
• σ=.05: standard deviation of noise added to parameters for mutation.
• κ=1.0: recombination with probability (1-κ) during crossover.
• θsnooker=0: sample along line x_i - z. 0.1 is recommended if > 0.
• n_initial: initial number of samples from the prior distribution when sample=resample. 10 times the number of parameters is a typical value
• bounds: a vector of tuples for lower and upper bounds of each parameter
• iter: current iteration
• generate_proposal: a function that generates proposals. Default is the two mode proposal described in Turner et al. 2012. You can also choose fixed_gamma, variable_gamma (see help) or pass a custom function
• update_particle!: a function for updating the particle with a proposal value. Default: mh_update!, which uses the Metropolis-Hastings rule.
• evaluate_fitness!: a function for evaluating the fitness of a posterior. The default is to compute the posterior loglikelihood with compute_posterior!. Select evaluate_fun! for optimization rather than MCMC sampling.
• sample: a function for sampling particles during the crossover step. The default sample uses current particle parameter values whereas resample samples from the history of accepted values for each particle. Np must 3 or greater when using resample.
• blocking_on = x -> false: a function that indicates whether block updating is used on each iteration. The function requires optimization_tests arguement for the DE sampler object and must return a true or false value.
• blocks: a vector of boolean vectors indicating which parameters to update. Each sub-vector represents a block and each element in the sub-vector indicates which parameters are updated within the block. For example, [[true,false],[false,true]] indicates that the parameter in the first position is updated on the first block and the parameter in the second position is updated on the second block. If a parameter is a vector or matrix, they are nested within the block sub-vector.

References

• Ter Braak, C. J. A Markov Chain Monte Carlo version of the genetic algorithm Differential Evolution: easy Bayesian computing for real parameter spaces.

• Ter Braak, Cajo JF, and Jasper A. Vrugt. Differential evolution Markov chain with snooker updater and fewer chains. Statistics and Computing 18.4 (2008): 435-446

• Turner, B. M., Sederberg, P. B., Brown, S. D., & Steyvers, M. (2013). A method for efficiently sampling from distributions with correlated dimensions. Psychological methods, 18(3), 368.

• Turner, B. M., & Sederberg, P. B. (2012). Approximate Bayesian computation with differential evolution. Journal of Mathematical Psychology, 56(5), 375-385.

DEModel{F,L,T,S} <: AbstractModel where {F <: Function,L,T,S}

A model object containing the log likelihood function and prior distributions.

Fields

• prior_loglike::L: log likelihood of posterior sample. A function must be defined for sample, but not for optimize.
• loglike::F: a log likelihood function for Bayesian parameter estimation or an objective function for optimization.
• sample_prior::S: a function for initial values. Typically, a prior distribution is ideal.
• names::T: parameter names

Constructor

function DEModel(
    args...; 
    prior_loglike = nothing, 
    loglike, 
    names, 
    sample_prior, 
    data, 
    kwargs...)

Arguments

• args...: optional positional arguments for loglike

Keywords

• prior_loglike=nothing: log likelihood of posterior sample. A function must be defined for sample, but not for optimize.
• loglike: a log likelihood function for Bayesian parameter estimation or an objective function for optimization.
• sample_prior: a function for initial values. Typically, a prior distribution is ideal.
• names: parameter names
• kwargs...: optional keyword arguments for loglike

Particle{T}

Fields

• Θ::Vector{T}: a vector of parameters
• accept::Vector{Bool}: proposal acceptance. 1: accept, 0: reject
• weight::Float64: particle weight based on model fit (currently posterior log likelihood)
• lp::Vector{Float64}: a vector of log posterior probabilities associated with each accepted proposal
• id::Int: particle id

bundle_samples(model::DEModel, de::DE, groups, n_iter)

Converts group particles to a chain object capable of generating convergence diagnostics and posterior summaries.

Arguments

• model: model containing a likelihood function with data and priors
• de: differential evolution object
• groups: a vector of groups of particles
• n_iter: number of iterations

accept(proposal, current, log_adj=0.0)

Metropolis-Hastings proposal selection Note: assumes weights are posterior log likelihoods

Arguments

• proposal: weight of proposal e.g. posterior log likelihood
• current: weight of current value e.g. posterior log likelihood
• adj: an adjustment term for the snooker update

adjust_loglike(Pt, proposal, Pz)

The adjusted log likelihood component for a snooker update.

• Pt: the target particle from iteration n - 1
• proposal: the proposal particle on iteration n
• Pz: the particle formed by the projection of particles m and n

compute_posterior!(de, model, proposal)

Computes posterior log likelihood of proposal particle. The value -Inf is returned if the proposal is out of bounds.

Arguments

• de: differential evolution object
• model: model containing a likelihood function with data and priors
• proposal: the proposed particle

crossover!(model, de, group)

Performs crossover step for each particle pt in the chain

Arguments

• model: model containing a likelihood function with data and priors
• de: differential evolution object
• group: a group of particles
• block: a vector of boolean values indicating which parameters to update

crossover!(model, de, group, pt::Particle, block)

Performs crossover step for a given particle pt.

Arguments

• model: model containing a likelihood function with data and priors
• de: differential evolution object
• group: a group of particles
• block: a vector of boolean values indicating which parameters to update

crossover!(model, de, group, pt::Particle)

Performs crossover step for for a given particle pt

Arguments

• model: model containing a likelihood function with data and priors
• de: differential evolution object
• pt: a target particle to be updated

crossover!(model, de, group)

Performs crossover step for each particle pt in the chain

Arguments

• model: model containing a likelihood function with data and priors
• de: differential evolution object
• group: a group of particles

evaluate_fun!(de, model, proposal))

Evaluates the fitness of an arbitrary function called loglike. This is used for point estimation as it does not use a prior distribution.

Arguments

• de: differential evolution object
• model: model containing a likelihood function with data and priors
• proposal: the proposed particle

fixed_gamma(de, Pt, group)

Generate proposal according to θ' = θt + γ(θm − θn) + b where γ = 2.38.

Arguments

• de: differential evolution object
• Pt: current particle
• group: a group of particles

get_names(model, p)

Returns parameters names.

Arguments

• p: a particle
• model: model containing a likelihood function with data and priors

in_bounds(b, θ::Real)

Checks whether parameter is within lower and upper bounds.

Arguments

• b: boundary (lowerbound,upperbound)
• θ: a parameter value

init_particle!(model, de, p, n_iter)

Initializes values for a particle.

Arguments

• model: model containing a likelihood function with data and priors
• de: differential evolution object
• p: a particle
• n_iter: the number of iterations

mh_update!(de, current, proposal, log_adj=0.0)

Update particle based on Metropolis-Hastings rule.

Arguments

• de: differential evolution object
• current: current particle
• proposal: proposal particle
• log_adj: an adjustment term for snooker update

migration!(de, groups)

Select a subset of groups and swap one poorly particle from each selected groups.

Arguments

• de: differential evolution object
• groups: groups of particles

mutate_or_crossover!(model, de, group, seed::Number)

Selects between mutation and crossover step with probability β.

Arguments

• model: model containing a likelihood function with data and priors
• de: differential evolution object
• group: a vector of interacting particles (e.g. chains)
• seed::Number: RNG seed

mutation!(model, de, group)

Mutates each particle by adding random noise y ~ normal(0, σ), where σ is a small number.

Arguments

• model: model containing a likelihood function with data and priors
• de: differential evolution object
• group: a group of particles

optimize(model::DEModel, de::DE, n_iter::Int; progress=false, kwargs...)

Finds optimal set of parameters.

Arguments

• model: a model containing likelihood function with data and priors
• de: differential evolution object
• n_iter: number of iterations or samples

Keywords

• progress=false: show progress of algorithm
• kwargs...: optional keyword arguments

optimize(model::DEModel, de::DE, ::MCMCThreads, n_iter::Int; progress=false, kwargs...)

Finds optimal set of parameters.

Arguments

• model: a model containing likelihood function with data and priors
• de: differential evolution object
• MCMCThreads: pass MCMCThreads() object to run on multiple threads
• n_iter: number of iterations or samples

Keywords

• progress=false: show progress of algorithm
• kwargs...: optional keyword arguments

project(p1::Particle, p2::Particle)

Returns the projection of particle p1 onto particle p2. This method works for vectors and nested arrays.

Arguments

• p1::Particle: base particle
• p2::Particle: particle projected onto p1

pstep!(model::DEModel, de::DE, groups)

Perform a single step for DE-MCMC with each particle group on a separate thread.

Arguments

• model: model containing a likelihood function with data and priors
• de: DE-MCMC sampler object
• groups: Array of vectors of particles

random_gamma(de, Pt, group)

Generate proposal according to θ' = θt + γ1(θm − θn) + γ2(θb − θt) + b. γ2=0 after burnin

Arguments

• de: differential evolution object
• Pt: current particle
• group: a group of particles

recombination!(de, pt::Particle, pp::Particle)

Resets parameters of proposal to previous value with probability (1-κ).

• de: differential evolution object
• pt: current partical
• pp: proposal particle

resample(de, group, n, replace)

Sample a random particle from previously accepted values for snooker update.

Arguments

• de: differential evolution object
• group: a group of particles
• n: number of particles to sample
• replace: sample with replacement if true

reset!(p1::Particle, p2::Particle, idx)

During block updates, all parameters are updated. This function resets parameters in the proposal p1 with values from the previous particle.

Arguments

p1: proposal particle which will have values reset p2: previous particle idx: boolean vector indicating which values are updated in block. False values are reset.

sample_init(model::DEModel, de::DE, n_iter)

Creates vectors of particles and samples initial parameter values from priors.

Arguments

• model: model containing a likelihood function with data and priors
• de: differential evolution object
• n_iter: number of iterations

select_base(group)

Selects base particle θb with probability proportional to weight.

• group: a group of particles

select_groups(de, groups)

Select a subset of groups for migration and return their indices.

Arguments

• de: differential evolution object
• groups: groups of particles

select_particle(group)

Select particle from a single chain inversely proportional to its weight.

Arguments

• group: a group of particles

select_particles(sub_group)

Select particles from groups for migration. Returns particle index and particles.

Arguments

• sub_group: vector of particles

shift_particles!(sub_group, p_idx, particles)

Swap the particles so that p1->pn, p2 -> p1,..., pn -> pn-1 where pi is the particle belonging to the ith group.

Arguments

• sub_group: group of particles
• p_idx: particle index
• particles: particle objects representing position in parameter space

snooker_update!(de, Pt, group)

Performs snooker update during crossover

Arguments

• de: differential evolution object
• Pt: current particle
• group: a group of particles

step!(model::DEModel, de::DE, groups)

Perform a single step for DE-MCMC.

Arguments

• model: model containing a likelihood function with data and priors
• de: DE-MCMC sampler object
• groups: Array of vectors of particles

store_samples!(de, groups)

Store samples after burnin period.

Arguments

• de: differential evolution object
• groups: groups of particles

update!(model, de, groups)(model, de, groups)

Particles are updated by block or simultaneously on a single thread.

Arguments

• model: model containing a likelihood function with data and priors
• de: differential evolution object
• group: a vector of interacting particles (e.g. chains)

variable_gamma(de, Pt, group)

Generate proposal according to θ' = θt + γ(θm − θn) + b where γ = 2.38/√(2d) where d is the number of parameters

Arguments

• de: differential evolution object
• Pt: current particle
• group: a group of particles

sample(de::DE, group_diff, n, replace)

Sample a random particle.

Arguments

• de: differential evolution object
• group: a group of particles
• n: number of particles to sample
• replace: sample with replacement if true

sample(model::DEModel, de::DE, n_iter::Int; progress=false, kwargs...)

Samples from the posterior distribution.

Arguments

• model: a model containing likelihood function with data and priors
• de: differential evolution object
• n_iter: number of iterations or samples

Keywords

• progress=false: show progress of sampler
• kwargs...: optional keyword arguments

sample(model::DEModel, de::DE, ::MCMCThreads, n_iter::Int; progress=false, kwargs...)

Samples from the posterior distribution with each group of particles on a seperarate thread for the mutation and crossover steps.

Arguments

• model: a model containing likelihood function with data and priors
• de: differential evolution object
• MCMCThreads: pass MCMCThreads() object to run on multiple threads
• n_iter: number of iterations or samples

Keywords

• progress=false: show progress of sampler
• kwargs...: optional keyword arguments