Introduction

Some problems, especially in numerical integration and Markov Chain Monte Carlo, benefit from transformation of variables: for example, if $σ > 0$ is a standard deviation parameter, it is usually better to work with log(σ) which can take any value on the real line. However, in general such transformations require correcting density functions by the determinant of their Jacobian matrix, usually referred to as "the Jacobian".

Also, is usually easier to code MCMC algorithms to work with vectors of real numbers, which may represent a "flattened" version of parameters, and would need to be decomposed into individual parameters, which themselves may be arrays, tuples, or special objects like lower triangular matrices.

This package is designed to help with both of these use cases. For example, consider the "8 schools" problem from Chapter 5.5 of Gelman et al (2013), in which SAT scores $y_{ij}$ in $J=8$ schools are modeled using a conditional normal

\[y_{ij} ∼ N(θⱼ, σ²)\]

and the $θⱼ$ are assume to have a hierarchical prior distribution

\[θⱼ ∼ N(μ, τ²)\]

For this problem, one could define a transformation

using TransformVariables
t = as((μ = asℝ, σ = asℝ₊, τ = asℝ₊, θs = as(Array, 8)))
dimension(t)
11

which would then yield a NamedTuple with the given names, with one of them being a Vector:

julia> x = randn(dimension(t))
11-element Array{Float64,1}:
  0.2972879845354616
  0.3823959677906078
 -0.5976344767282311
 -0.01044524463737564
 -0.839026854388764
  0.31111133849833383
  2.2950878238373105
 -2.2670863488005306
  0.5299655761667461
  0.43142152642291204
  0.5837082875687786

julia> y = transform(t, x)
(μ = 0.2972879845354616, σ = 1.4657923768059056, τ = 0.5501113994952206, θs = [-0.01044524463737564, -0.839026854388764, 0.31111133849833383, 2.2950878238373105, -2.2670863488005306, 0.5299655761667461, 0.43142152642291204, 0.5837082875687786])

julia> keys(y)
(:μ, :σ, :τ, :θs)

julia> y.θs
8-element Array{Float64,1}:
 -0.01044524463737564
 -0.839026854388764
  0.31111133849833383
  2.2950878238373105
 -2.2670863488005306
  0.5299655761667461
  0.43142152642291204
  0.5837082875687786

Further worked examples of using this package can be found in the DynamicHMCExamples.jl repository. It is recommended that the user reads those first, and treats this documentation as a reference.

General interface

TransformVariables.dimension — Function
dimension(t::AbstractTransform)

The dimension (number of elements) that t transforms.

Types should implement this method.

TransformVariables.inverse — Function
inverse(t::AbstractTransform, y)

Return x so that transform(t, x) ≈ y.

inverse(t)

Return a callable equivalen to y -> inverse(t, y).

TransformVariables.inverse! — Function
inverse!(x, transformation, y)

Put inverse(t, y) into a preallocated vector x, returning x.

Generalized indexing should be assumed on x.

See inverse_eltype for determining the type of x.

TransformVariables.transform_logdensity — Function
transform_logdensity(t, f, x)

Let $y = t(x)$, and $f(y)$ a log density at y. This function evaluates f ∘ t as a log density, taking care of the log Jacobian correction.

TransformVariables.random_arg — Function
random_arg(x; kwargs...)

A random argument for a transformation.

Keyword arguments

A standard multivaritate normal or Cauchy is used, depending on cauchy, then scaled with scale. rng is the random number generator used.

TransformVariables.random_value — Function
random_value(t; kwargs...)

Random value from a transformation.

Keyword arguments

A standard multivaritate normal or Cauchy is used, depending on cauchy, then scaled with scale. rng is the random number generator used.

Defining transformations

The as constructor and aggregations

Some transformations, particularly aggregations use the function as as the constructor. Aggregating transformations are built from other transformations to transform consecutive (blocks of) real numbers into the desired domain.

It is recommended that you use as(Array, ...) and friends (as(Vector, ...), as(Matrix, ...)) for repeating the same transformation, and named tuples such as as((μ = ..., σ = ...)) for transforming into named parameters. For extracting parameters in log likelihoods, consider Parameters.jl.

See methods(as) for all the constructors, ?as for their documentation.

TransformVariables.as — Function
as(T, args...)

Shorthand for constructing transformations with image in T. args determines or modifies behavior, details depend on T.

Not all transformations have an as method, some just have direct constructors. See methods(as) for a list.

Examples

as(Real, -∞, 1)          # transform a real number to (-∞, 1)
as(Array, 10, 2)         # reshape 20 real numbers to a 10x2 matrix
as(Array, as𝕀, 10)       # transform 10 real numbers to (0, 1)
as((a = asℝ₊, b = as𝕀)) # transform 2 real numbers a NamedTuple, with a > 0, 0 < b < 1

Scalar transforms

The symbol ∞ is a placeholder for infinity. It does not correspond to Inf, but acts as a placeholder for the correct dispatch. -∞ is valid.

TransformVariables.∞ — Constant

Placeholder representing of infinity for specifing interval boundaries. Supports the - operator, ie -∞.

as(Real, a, b) defines transformations to finite and (semi-)infinite subsets of the real line, where a and b can be -∞ and ∞, respectively. The following constants are defined for common cases.

TransformVariables.asℝ — Constant

Transform to the real line (identity).

asℝ and as_real are equivalent alternatives.

TransformVariables.as𝕀 — Constant

Transform to the unit interval (0, 1).

as𝕀 and as_unit_interval are equivalent alternatives.

Special arrays

TransformVariables.CorrCholeskyFactor — Type
CorrCholeskyFactor(n)

Cholesky factor of a correlation matrix of size n.

Transforms $n×(n-1)/2$ real numbers to an $n×n$ upper-triangular matrix U, such that U'*U is a correlation matrix (positive definite, with unit diagonal).

Notes

If

  • z is a vector of n IID standard normal variates,

  • σ is an n-element vector of standard deviations,

  • U is obtained from CorrCholeskyFactor(n),

then Diagonal(σ) * U' * z will be a multivariate normal with the given variances and correlation matrix U' * U.

source

Defining custom transformations

TransformVariables.logjac_forwarddiff — Function
logjac_forwarddiff(f, x; handleNaN, chunk, cfg)

Calculate the log Jacobian determinant of f at x using `ForwardDiff.

Note

f should be a bijection, mapping from vectors of real numbers to vectors of equal length.

When handleNaN = true (the default), NaN log Jacobians are converted to -Inf.

TransformVariables.value_and_logjac_forwarddiff — Function
value_and_logjac_forwarddiff(f, x; flatten, handleNaN, chunk, cfg)

Calculate the value and the log Jacobian determinant of f at x. flatten is used to get a vector out of the result that makes f a bijection.

TransformVariables.CustomTransform — Type
CustomTransform(g, f, flatten; chunk, cfg)

Wrap a custom transform y = f(transform(g, x))in a type that calculates the log Jacobian of∂y/∂xusingForwardDiff` when necessary.

Usually, g::TransformReals, but when an integer is used, it amounts to the identity transformation with that dimension.

flatten should take the result from f, and return a flat vector with no redundant elements, so that $x ↦ y$ is a bijection. For example, for a covariance matrix the elements below the diagonal should be removed.

chunk and cfg can be used to configure ForwardDiff.JacobianConfig. cfg is used directly, while chunk = ForwardDiff.Chunk{N}() can be used to obtain a type-stable configuration.