API

Likelihoods

GPLikelihoods.BernoulliLikelihoodType
BernoulliLikelihood(l=logistic)

Bernoulli likelihood is to be used if we assume that the uncertainity associated with the data follows a Bernoulli distribution. The link l needs to transform the input f to the domain [0, 1]

\[ p(y|f) = \operatorname{Bernoulli}(y | l(f))\]

On calling, this would return a Bernoulli distribution with l(f) probability of true.

GPLikelihoods.CategoricalLikelihoodType
CategoricalLikelihood(l=BijectiveSimplexLink(softmax))

Categorical likelihood is to be used if we assume that the uncertainty associated with the data follows a Categorical distribution.

Assuming a distribution with n categories:

n-1 inputs (bijective link)

One can work with a bijective transformation by wrapping a link (like softmax) into a BijectiveSimplexLink and only needs n-1 inputs:

\[ p(y|f_1, f_2, \dots, f_{n-1}) = \operatorname{Categorical}(y | l(f_1, f_2, \dots, f_{n-1}, 0))\]

The default constructor is a bijective link around softmax.

n inputs (non-bijective link)

One can also pass directly the inputs without concatenating a 0:

\[ p(y|f_1, f_2, \dots, f_n) = \operatorname{Categorical}(y | l(f_1, f_2, \dots, f_n))\]

This variant is over-parametrized, as there are n-1 independent parameters embedded in a n dimensional parameter space. For more details, see the end of the section of this Wikipedia link where it corresponds to Variant 1 and 2.

GPLikelihoods.GammaLikelihoodType
GammaLikelihood(α::Real=1.0, l=exp)

Gamma likelihood with fixed shape α.

\[ p(y|f) = \operatorname{Gamma}(y | α, l(f))\]

On calling, this returns a Gamma distribution with shape α and scale invlink(f).

GPLikelihoods.GaussianLikelihoodType
GaussianLikelihood(σ²)

Gaussian likelihood with σ² variance. This is to be used if we assume that the uncertainity associated with the data follows a Gaussian distribution.

\[ p(y|f) = \operatorname{Normal}(y | f, σ²)\]

On calling, this would return a normal distribution with mean f and variance σ².

GPLikelihoods.HeteroscedasticGaussianLikelihoodType
HeteroscedasticGaussianLikelihood(l=exp)

Heteroscedastic Gaussian likelihood. This is a Gaussian likelihood whose mean and variance are functions of latent processes.

\[ p(y|[f, g]) = \operatorname{Normal}(y | f, sqrt(l(g)))\]

On calling, this would return a normal distribution with mean f and variance l(g). Where l is link going from R to R^+

GPLikelihoods.PoissonLikelihoodType
PoissonLikelihood(l=exp)

Poisson likelihood with rate defined as l(f).

\[ p(y|f) = \operatorname{Poisson}(y | θ=l(f))\]

This is to be used if we assume that the uncertainity associated with the data follows a Poisson distribution.

Negative Binomial

GPLikelihoods.NegativeBinomialLikelihoodType
NegativeBinomialLikelihood(param::NBParam, invlink::Union{Function,Link})

There are many possible parametrizations for the Negative Binomial likelihood. We follow the convention laid out in p.137 of [^Hilbe'11] and provide some common parametrizations. The NegativeBinomialLikelihood has a special structure; the first type parameter NBParam defines in what parametrization the latent function is used, and contains the other (scalar) parameters. NBParam itself has two subtypes:

  • NBParamProb for parametrizations where f -> p, the probability of success of a Bernoulli event
  • NBParamMean for parametrizations where f -> μ, the expected number of events

NBParam predefined types

NBParamProb types with p = invlink(f) the probability of success

NBParamMean types with μ = invlink(f) the mean/expected number of events

  • NBParamI: Mean is linked to f and variance is given by μ(1 + α)
  • NBParamII: Mean is linked to f and variance is given by μ(1 + α * μ)
  • NBParamPower: Mean is linked to f and variance is given by μ(1 + α * μ^ρ)

To create a new parametrization, you need to:

  • create a new type struct MyNBParam{T} <: NBParam; myparams::T; end;
  • dispatch (l::NegativeBinomialLikelihood{<:MyNBParam})(f::Real), which must return a NegativeBinomial from Distributions.jl.

NegativeBinomial follows the parametrization of NBParamSuccess, i.e. the first argument is the number of successes and the second argument is the probability of success.

Examples

julia> NegativeBinomialLikelihood(NBParamSuccess(10), logistic)(2.0)
NegativeBinomial{Float64}(r=10.0, p=0.8807970779778824)
julia> NegativeBinomialLikelihood(NBParamFailure(10), logistic)(2.0)
NegativeBinomial{Float64}(r=10.0, p=0.11920292202211757)

julia> d = NegativeBinomialLikelihood(NBParamI(3.0), exp)(2.0)
NegativeBinomial{Float64}(r=2.4630186996435506, p=0.25)
julia> mean(d) ≈ exp(2.0)
true
julia> var(d) ≈ exp(2.0) * (1 + 3.0)
true

[^Hilbe'11]: Hilbe, Joseph M. Negative binomial regression. Cambridge University Press, 2011.

GPLikelihoods.NBParamSuccessType
NBParamSuccess(successes)

Negative Binomial parametrization with successes the number of successes and invlink(f) the probability of success. This corresponds to the definition used by Distributions.jl.

\[ p(y|\text{successes}, p=\text{invlink}(f)) = \frac{\Gamma(y+\text{successes})}{y! \Gamma(\text{successes})} p^\text{successes} (1 - p)^y\]

GPLikelihoods.NBParamFailureType
NBParamFailure(failures)

Negative Binomial parametrization with failures the number of failures and invlink(f) the probability of success. This corresponds to the definition used by Wikipedia.

\[ p(y|\text{failures}, p=\text{invlink}(f)) = \frac{\Gamma(y+\text{failures})}{y! \Gamma(\text{failures})} p^y (1 - p)^{\text{failures}}\]

GPLikelihoods.NBParamIType
NBParamI(α)

Negative Binomial parametrization with mean μ = invlink(f) and variance v = μ(1 + α) where α > 0.

GPLikelihoods.NBParamIIType
NBParamII(α)

Negative Binomial parametrization with mean μ = invlink(f) and variance v = μ(1 + α * μ) where α > 0.

GPLikelihoods.NBParamPowerType
NBParamPower(α, ρ)

Negative Binomial parametrization with mean μ = invlink(f) and variance v = μ(1 + α * μ^ρ) where α > 0 and ρ > 0.

GPLikelihoods.BijectiveSimplexLinkType
BijectiveSimplexLink(link)

Wrapper to preprocess the inputs by adding a 0 at the end before passing it to the link link. This is a necessary step to work with simplices. For example with the SoftMaxLink, to obtain a n-simplex leading to n+1 categories for the CategoricalLikelihood, one needs to pass n+1 latent GP. However, by wrapping the link into a BijectiveSimplexLink, only n latent are needed.

The rest of the links ExpLink, LogisticLink, etc., are aliases for the corresponding wrapped functions in a Link. For example ExpLink == Link{typeof(exp)}.

When passing a Link to a likelihood object, this link corresponds to the transformation p=link(f) while, as mentioned in the Constrained parameters section, the statistics literature usually uses the denomination inverse link or mean function for it.

Expectations

GPLikelihoods.expected_loglikelihoodFunction
expected_loglikelihood(
    quadrature,
    lik,
    q_f::AbstractVector{<:Normal},
    y::AbstractVector,
)

This function computes the expected log likelihood:

\[ ∫ q(f) log p(y | f) df\]

where p(y | f) is the process likelihood. This is described by lik, which should be a callable that takes f as input and returns a Distribution over y that supports loglikelihood(lik(f), y).

q(f) is an approximation to the latent function values f given by:

\[ q(f) = ∫ p(f | u) q(u) du\]

where q(u) is the variational distribution over inducing points. The marginal distributions of q(f) are given by q_f.

quadrature determines which method is used to calculate the expected log likelihood.

Extended help

q(f) is assumed to be an MvNormal distribution and p(y | f) is assumed to have independent marginals such that only the marginals of q(f) are required.

expected_loglikelihood(::DefaultExpectationMethod, lik, q_f::AbstractVector{<:Normal}, y::AbstractVector)

The expected log likelihood, using the default quadrature method for the given likelihood. (The default quadrature method is defined by default_expectation_method(lik), and should be the closed form solution if it exists, but otherwise defaults to Gauss-Hermite quadrature.)