# Exponential Family Distributions

## Exponential Family

All the distributions provided by this package are members of the exponential family of distribution, i.e. they have the follotwing canonical form:

where:

- $\eta$ is the natural parameter (scalar or vector)
- $T(x)$ is the sufficient statistic (scalar or vector)
- $A(\eta)$ is the log-normalizer (scalar)
- $B(x)$ is the base measure (scalar)

Practically, the package provide the following abstract type

`ExpFamilyDistributions.Distribution`

— Type`abstract type Distribution end`

Supertype for distributions member of the exponential family.

Which represents the supter-type of members of the exponential family.

## Parameterization

All subtypes of `Distribution`

have the following form:

```
struct MyDistribution{P<:AbstractParam} <: Distribution
param::P
end
```

This particular form allows each distribution to be agnostic to their concrete parameterization. The parameter type inherits from:

`ExpFamilyDistributions.AbstractParameter`

— Type`abstract type AbstractParameter{T} end`

Abstract type for parameters of a member of the exponential family.

and supports the following methods

`ExpFamilyDistributions.naturalform`

— Function`ExpFamilyDistributions.jacobian`

— Function`jacobian(param)`

Returns the Jacobian of ξ (the real form) w.r.t. the natural form η.

`ExpFamilyDistributions.realform`

— Function`realform(param)`

Returns the vector of parameters as stored in `param`

. Note that this function is just an accessor of the internal storage of the parameter, modifying the returned value should modify the parameter accordingly.

See also: `naturalform`

.

`ExpFamilyDistributions.reallocate`

— Function`reallocate(param, bufferType)`

Reallocate (i.e. copy) `param`

with all interal buffers stored as `bufferType`

.

`ExpFamilyDistributions.todict`

— Function`ExpFamilyDistributions.fromdict`

— Function## Distribution interface

Each subtype of [`Distribution`

] implements the following interface:

`ExpFamilyDistributions.basemeasure`

— Function`basemeasure(p, x)`

Returns the base measure of `x`

for the distribution `p`

.

`ExpFamilyDistributions.gradlognorm`

— Function`gradlognorm(p)`

Returns the gradient of the log-normalizer of `p`

w.r.t. its natural parameters.

`ExpFamilyDistributions.kldiv`

— Function`kldiv(q::T, p::T[, μ = gradlognorm(q)]) where T<:Distribution`

Compute the KL-divergence between two distributions of the same type (i.e. `kldiv(Normal, Normal)`

, `kldiv(Dirichlet, Dirichlet)`

, ...). You can specify directly the expectation of the sufficient statistics `μ`

.

`ExpFamilyDistributions.lognorm`

— Function`lognorm(p)`

Returns the log-normalization constant of `p`

.

`ExpFamilyDistributions.loglikelihood`

— Function`loglikelihood(p, x)`

Returns the log-likelihood of `x`

for the distribution `p`

.

`ExpFamilyDistributions.sample`

— Function`sample(p, n=1)`

Draw `n`

samples from the distribution `p`

.

`ExpFamilyDistributions.splitgrad`

— Function`splitgrad(p, μ)`

Split the gradient of the log-normalizer into its "standard" components. For instance, for the Normal distribution, the output will be the expected value of $x$ and $xxᵀ$.

`ExpFamilyDistributions.stats`

— Function`stats(p, x)`

Returns the sufficient statistics of `x`

for the distribution `p`

.

`ExpFamilyDistributions.stdparam`

— Function`stdparam(p, η)`

Returns the standard parameters corresponding to the natural parameters `η`

for distributions with the same type of `p`

.

## Utilities

The package also provides the following utility functions:

`ExpFamilyDistributions.vec_tril`

— Function`vec_tril(M)`

Returns the vectorized low-triangular part (diagonal not included) of the matrix.

See also `inv_vec_tril`

, `matrix`

.

`ExpFamilyDistributions.inv_vec_tril`

— Function`ExpFamilyDistributions.matrix`

— Function`matrix(diagM, vec_trilM)`

Returns a symmetrix matrix from the diagonal and the "tril" form of a matrix.

See also `vec_tril`

, `inv_vec_tril`

`ExpFamilyDistributions.pdmat_logdet`

— Function`pdmat_logdet(M)`

Log-determinant of a positive definite matrix.

`ExpFamilyDistributions.pdmat_inverse`

— Function`pdmat_inverse(M)`

Inverse of a positive definite matrix.