Exponential Family Distributions

Exponential Family

All the distributions provided by this package are members of the exponential family of distribution, i.e. they have the follotwing canonical form:

\[p(x) = \exp \{ \eta^\top T(x) - A(\eta) + B(x) \}\]


  • $\eta$ is the natural parameter (scalar or vector)
  • $T(x)$ is the sufficient statistic (scalar or vector)
  • $A(\eta)$ is the log-normalizer (scalar)
  • $B(x)$ is the base measure (scalar)

Practically, the package provide the following abstract type

Which represents the supter-type of members of the exponential family.


All subtypes of Distribution have the following form:

struct MyDistribution{P<:AbstractParam} <: Distribution

This particular form allows each distribution to be agnostic to their concrete parameterization. The parameter type inherits from:

and supports the following methods


Returns the vector of parameters as stored in param. Note that this function is just an accessor of the internal storage of the parameter, modifying the returned value should modify the parameter accordingly.

See also: naturalform.

Distribution interface

Each subtype of [Distribution] implements the following interface:

kldiv(q::T, p::T[, μ = gradlognorm(q)]) where T<:Distribution

Compute the KL-divergence between two distributions of the same type (i.e. kldiv(Normal, Normal), kldiv(Dirichlet, Dirichlet), ...). You can specify directly the expectation of the sufficient statistics μ.

splitgrad(p, μ)

Split the gradient of the log-normalizer into its "standard" components. For instance, for the Normal distribution, the output will be the expected value of $x$ and $xxᵀ$.

stdparam(p, η)

Returns the standard parameters corresponding to the natural parameters η for distributions with the same type of p.


The package also provides the following utility functions: