# API

`GPLikelihoods.BernoulliLikelihood`

`GPLikelihoods.BijectiveSimplexLink`

`GPLikelihoods.CategoricalLikelihood`

`GPLikelihoods.ChainLink`

`GPLikelihoods.ExpLink`

`GPLikelihoods.ExponentialLikelihood`

`GPLikelihoods.GammaLikelihood`

`GPLikelihoods.GaussianLikelihood`

`GPLikelihoods.HeteroscedasticGaussianLikelihood`

`GPLikelihoods.InvLink`

`GPLikelihoods.Link`

`GPLikelihoods.LogLink`

`GPLikelihoods.LogisticLink`

`GPLikelihoods.LogitLink`

`GPLikelihoods.NBParamFailure`

`GPLikelihoods.NBParamI`

`GPLikelihoods.NBParamII`

`GPLikelihoods.NBParamPower`

`GPLikelihoods.NBParamSuccess`

`GPLikelihoods.NegativeBinomialLikelihood`

`GPLikelihoods.NormalCDFLink`

`GPLikelihoods.PoissonLikelihood`

`GPLikelihoods.ProbitLink`

`GPLikelihoods.SoftMaxLink`

`GPLikelihoods.SqrtLink`

`GPLikelihoods.SquareLink`

`GPLikelihoods.expected_loglikelihood`

## Likelihoods

`GPLikelihoods.BernoulliLikelihood`

— Type`BernoulliLikelihood(l=logistic)`

Bernoulli likelihood is to be used if we assume that the uncertainity associated with the data follows a Bernoulli distribution. The link `l`

needs to transform the input `f`

to the domain [0, 1]

\[ p(y|f) = \operatorname{Bernoulli}(y | l(f))\]

On calling, this would return a Bernoulli distribution with `l(f)`

probability of `true`

.

`GPLikelihoods.CategoricalLikelihood`

— Type`CategoricalLikelihood(l=BijectiveSimplexLink(softmax))`

Categorical likelihood is to be used if we assume that the uncertainty associated with the data follows a Categorical distribution.

Assuming a distribution with `n`

categories:

`n-1`

inputs (bijective link)

One can work with a bijective transformation by wrapping a link (like `softmax`

) into a `BijectiveSimplexLink`

and only needs `n-1`

inputs:

\[ p(y|f_1, f_2, \dots, f_{n-1}) = \operatorname{Categorical}(y | l(f_1, f_2, \dots, f_{n-1}, 0))\]

The default constructor is a bijective link around `softmax`

.

`n`

inputs (non-bijective link)

One can also pass directly the inputs without concatenating a `0`

:

\[ p(y|f_1, f_2, \dots, f_n) = \operatorname{Categorical}(y | l(f_1, f_2, \dots, f_n))\]

This variant is over-parametrized, as there are `n-1`

independent parameters embedded in a `n`

dimensional parameter space. For more details, see the end of the section of this Wikipedia link where it corresponds to Variant 1 and 2.

`GPLikelihoods.ExponentialLikelihood`

— Type`ExponentialLikelihood(l=exp)`

Exponential likelihood with scale given by `l(f)`

.

\[ p(y|f) = \operatorname{Exponential}(y | l(f))\]

`GPLikelihoods.GammaLikelihood`

— Type`GammaLikelihood(α::Real=1.0, l=exp)`

Gamma likelihood with fixed shape `α`

.

\[ p(y|f) = \operatorname{Gamma}(y | α, l(f))\]

On calling, this returns a Gamma distribution with shape `α`

and scale `invlink(f)`

.

`GPLikelihoods.GaussianLikelihood`

— Type`GaussianLikelihood(σ²)`

Gaussian likelihood with `σ²`

variance. This is to be used if we assume that the uncertainity associated with the data follows a Gaussian distribution.

\[ p(y|f) = \operatorname{Normal}(y | f, σ²)\]

On calling, this would return a normal distribution with mean `f`

and variance σ².

`GPLikelihoods.HeteroscedasticGaussianLikelihood`

— Type`HeteroscedasticGaussianLikelihood(l=exp)`

Heteroscedastic Gaussian likelihood. This is a Gaussian likelihood whose mean and variance are functions of latent processes.

\[ p(y|[f, g]) = \operatorname{Normal}(y | f, sqrt(l(g)))\]

On calling, this would return a normal distribution with mean `f`

and variance `l(g)`

. Where `l`

is link going from R to R^+

`GPLikelihoods.PoissonLikelihood`

— Type`PoissonLikelihood(l=exp)`

Poisson likelihood with rate defined as `l(f)`

.

\[ p(y|f) = \operatorname{Poisson}(y | θ=l(f))\]

This is to be used if we assume that the uncertainity associated with the data follows a Poisson distribution.

### Negative Binomial

`GPLikelihoods.NegativeBinomialLikelihood`

— Type`NegativeBinomialLikelihood(param::NBParam, invlink::Union{Function,Link})`

There are many possible parametrizations for the Negative Binomial likelihood. We follow the convention laid out in p.137 of [^Hilbe'11] and provide some common parametrizations. The `NegativeBinomialLikelihood`

has a special structure; the first type parameter `NBParam`

defines in what parametrization the latent function is used, and contains the other (scalar) parameters. `NBParam`

itself has two subtypes:

`NBParamProb`

for parametrizations where`f -> p`

, the probability of success of a Bernoulli event`NBParamMean`

for parametrizations where`f -> μ`

, the expected number of events

`NBParam`

predefined types

`NBParamProb`

types with `p = invlink(f)`

the probability of success

`NBParamSuccess`

: This is the definition used in`Distributions.jl`

.`NBParamFailure`

: This is the definition used in Wikipedia.

`NBParamMean`

types with `μ = invlink(f)`

the mean/expected number of events

`NBParamI`

: Mean is linked to`f`

and variance is given by`μ(1 + α)`

`NBParamII`

: Mean is linked to`f`

and variance is given by`μ(1 + α * μ)`

`NBParamPower`

: Mean is linked to`f`

and variance is given by`μ(1 + α * μ^ρ)`

To create a new parametrization, you need to:

- create a new type
`struct MyNBParam{T} <: NBParam; myparams::T; end`

; - dispatch
`(l::NegativeBinomialLikelihood{<:MyNBParam})(f::Real)`

, which must return a`NegativeBinomial`

from`Distributions.jl`

.

`NegativeBinomial`

follows the parametrization of `NBParamSuccess`

, i.e. the first argument is the number of successes and the second argument is the probability of success.

**Examples**

```
julia> NegativeBinomialLikelihood(NBParamSuccess(10), logistic)(2.0)
NegativeBinomial{Float64}(r=10.0, p=0.8807970779778824)
julia> NegativeBinomialLikelihood(NBParamFailure(10), logistic)(2.0)
NegativeBinomial{Float64}(r=10.0, p=0.11920292202211757)
julia> d = NegativeBinomialLikelihood(NBParamI(3.0), exp)(2.0)
NegativeBinomial{Float64}(r=2.4630186996435506, p=0.25)
julia> mean(d) ≈ exp(2.0)
true
julia> var(d) ≈ exp(2.0) * (1 + 3.0)
true
```

[^Hilbe'11]: Hilbe, Joseph M. Negative binomial regression. Cambridge University Press, 2011.

`GPLikelihoods.NBParamSuccess`

— Type`NBParamSuccess(successes)`

Negative Binomial parametrization with `successes`

the number of successes and `invlink(f)`

the probability of success. This corresponds to the definition used by Distributions.jl.

\[ p(y|\text{successes}, p=\text{invlink}(f)) = \frac{\Gamma(y+\text{successes})}{y! \Gamma(\text{successes})} p^\text{successes} (1 - p)^y\]

`GPLikelihoods.NBParamFailure`

— Type`NBParamFailure(failures)`

Negative Binomial parametrization with `failures`

the number of failures and `invlink(f)`

the probability of success. This corresponds to the definition used by Wikipedia.

\[ p(y|\text{failures}, p=\text{invlink}(f)) = \frac{\Gamma(y+\text{failures})}{y! \Gamma(\text{failures})} p^y (1 - p)^{\text{failures}}\]

`GPLikelihoods.NBParamI`

— Type`NBParamI(α)`

Negative Binomial parametrization with mean `μ = invlink(f)`

and variance `v = μ(1 + α)`

where `α > 0`

.

`GPLikelihoods.NBParamII`

— Type`NBParamII(α)`

Negative Binomial parametrization with mean `μ = invlink(f)`

and variance `v = μ(1 + α * μ)`

where `α > 0`

.

`GPLikelihoods.NBParamPower`

— Type`NBParamPower(α, ρ)`

Negative Binomial parametrization with mean `μ = invlink(f)`

and variance `v = μ(1 + α * μ^ρ)`

where `α > 0`

and `ρ > 0`

.

## Links

`GPLikelihoods.Link`

— Type`Link(f)`

General construction for a link with a function `f`

.

`GPLikelihoods.ChainLink`

— Type`ChainLink(links)`

Create a composed chain of different links.

`GPLikelihoods.BijectiveSimplexLink`

— Type`BijectiveSimplexLink(link)`

Wrapper to preprocess the inputs by adding a `0`

at the end before passing it to the link `link`

. This is a necessary step to work with simplices. For example with the `SoftMaxLink`

, to obtain a `n`

-simplex leading to `n+1`

categories for the `CategoricalLikelihood`

, one needs to pass `n+1`

latent GP. However, by wrapping the link into a `BijectiveSimplexLink`

, only `n`

latent are needed.

The rest of the links `ExpLink`

, `LogisticLink`

, etc., are aliases for the corresponding wrapped functions in a `Link`

. For example `ExpLink == Link{typeof(exp)}`

.

When passing a `Link`

to a likelihood object, this link corresponds to the transformation `p=link(f)`

while, as mentioned in the Constrained parameters section, the statistics literature usually uses the denomination **inverse link or mean function** for it.

`GPLikelihoods.LogLink`

— Type`LogLink()`

`log`

link, f:ℝ⁺->ℝ . Its inverse is the `ExpLink`

.

`GPLikelihoods.ExpLink`

— Type`ExpLink()`

`exp`

link, f:ℝ->ℝ⁺. Its inverse is the `LogLink`

.

`GPLikelihoods.InvLink`

— Type`InvLink()`

`inv`

link, f:ℝ/{0}->ℝ/{0}. It is its own inverse.

`GPLikelihoods.SqrtLink`

— Type`SqrtLink()`

`sqrt`

link, f:ℝ⁺∪{0}->ℝ⁺∪{0}. Its inverse is the `SquareLink`

.

`GPLikelihoods.SquareLink`

— Type`SquareLink()`

`^2`

link, f:ℝ->ℝ⁺∪{0}. Its inverse is the `SqrtLink`

.

`GPLikelihoods.LogitLink`

— Type`LogitLink()`

`log(x/(1-x))`

link, f:[0,1]->ℝ. Its inverse is the `LogisticLink`

.

`GPLikelihoods.LogisticLink`

— Type`LogisticLink()`

`1/(1+exp(-x))`

link. f:ℝ->[0,1]. Its inverse is the `LogitLink`

.

`GPLikelihoods.ProbitLink`

— Type`ProbitLink()`

`ϕ⁻¹(y)`

link, where `ϕ⁻¹`

is the `invcdf`

of a `Normal`

distribution, f:[0,1]->ℝ. Its inverse is the `NormalCDFLink`

.

`GPLikelihoods.NormalCDFLink`

— Type`NormalCDFLink()`

`ϕ(y)`

link, where `ϕ`

is the `cdf`

of a `Normal`

distribution, f:ℝ->[0,1]. Its inverse is the `ProbitLink`

.

`GPLikelihoods.SoftMaxLink`

— Type`SoftMaxLink()`

`softmax`

link, i.e `f(x)ᵢ = exp(xᵢ)/∑ⱼexp(xⱼ)`

. f:ℝⁿ->Sⁿ⁻¹, where Sⁿ⁻¹ is an (n-1)-simplex It has no defined inverse

## Expectations

`GPLikelihoods.expected_loglikelihood`

— Function```
expected_loglikelihood(
quadrature,
lik,
q_f::AbstractVector{<:Normal},
y::AbstractVector,
)
```

This function computes the expected log likelihood:

\[ ∫ q(f) log p(y | f) df\]

where `p(y | f)`

is the process likelihood. This is described by `lik`

, which should be a callable that takes `f`

as input and returns a Distribution over `y`

that supports `loglikelihood(lik(f), y)`

.

`q(f)`

is an approximation to the latent function values `f`

given by:

\[ q(f) = ∫ p(f | u) q(u) du\]

where `q(u)`

is the variational distribution over inducing points. The marginal distributions of `q(f)`

are given by `q_f`

.

`quadrature`

determines which method is used to calculate the expected log likelihood.

**Extended help**

`q(f)`

is assumed to be an `MvNormal`

distribution and `p(y | f)`

is assumed to have independent marginals such that only the marginals of `q(f)`

are required.

`expected_loglikelihood(::DefaultExpectationMethod, lik, q_f::AbstractVector{<:Normal}, y::AbstractVector)`

The expected log likelihood, using the default quadrature method for the given likelihood. (The default quadrature method is defined by `default_expectation_method(lik)`

, and should be the closed form solution if it exists, but otherwise defaults to Gauss-Hermite quadrature.)