ADTypes.jl

Documentation for ADTypes.jl.

ADTypes.ADTypes — Module
ADTypes.jl

ADTypes.jl is a multi-valued logic system to choose an automatic differentiation (AD) package and specify its parameters.

Dense AD

Forward mode

Algorithmic differentiation:

ADTypes.AutoForwardDiff — Type
AutoForwardDiff{chunksize,T}

Chooses ForwardDiff.jl.

Type parameters

  • chunksize: the preferred chunk size to evaluate several derivatives at once

Fields

  • tag::T: a custom tag to handle nested differentiation calls (usually not necessary)

Constructors

AutoForwardDiff(; chunksize=nothing, tag=nothing)

Finite differences:

ADTypes.AutoFiniteDiff — Type
AutoFiniteDiff{T1,T2,T3}

Chooses FiniteDiff.jl.

Fields

  • fdtype::T1: finite difference type
  • fdjtype::T2: finite difference type for the Jacobian
  • fdhtype::T3: finite difference type for the Hessian

Constructor

AutoFiniteDiff(; fdtype=Val(:forward), fdjtype=fdtype, fdhtype=Val(:hcentral))

Reverse mode

Forward or reverse mode

ADTypes.AutoEnzyme — Type
AutoEnzyme{M}

Chooses Enzyme.jl.

Fields

  • mode::M: can be either
    • an object subtyping EnzymeCore.Mode (like EnzymeCore.Forward or EnzymeCore.Reverse) if a specific mode is required
    • nothing to choose the best mode automatically

Constructors

AutoEnzyme(; mode=nothing)

Symbolic mode

Sparse AD

ADTypes.AutoSparse — Type
AutoSparse{D,S,C}

Wraps an ADTypes.jl object to deal with sparse Jacobians and Hessians.

Fields

Constructors

AutoSparse(
    dense_ad;
    sparsity_detector=ADTypes.NoSparsityDetector(),
    coloring_algorithm=ADTypes.NoColoringAlgorithm()
)
ADTypes.dense_ad — Function
dense_ad(ad::AutoSparse)::AbstractADType

Return the underlying AD package for a sparse AD choice.

See also

Sparsity detector

ADTypes.jacobian_sparsity — Function
jacobian_sparsity(f, x, sd::AbstractSparsityDetector)::AbstractMatrix{Bool}
jacobian_sparsity(f!, y, x, sd::AbstractSparsityDetector)::AbstractMatrix{Bool}

Use detector sd to construct a (typically sparse) matrix S describing the pattern of nonzeroes in the Jacobian of f (resp. f!) applied at x (resp. (y, x)).

ADTypes.hessian_sparsity — Function
hessian_sparsity(f, x, sd::AbstractSparsityDetector)::AbstractMatrix{Bool}

Use detector sd to construct a (typically sparse) matrix S describing the pattern of nonzeroes in the Hessian of f applied at x.

Coloring algorithm

ADTypes.AbstractColoringAlgorithm — Type
AbstractColoringAlgorithm

Abstract supertype for Jacobian/Hessian coloring algorithms, defined for example in SparseDiffTools.jl.

Required methods

Note

The terminology and definitions are taken from the following paper:

"What Color Is Your Jacobian? Graph Coloring for Computing Derivatives"

Assefaw Hadish Gebremedhin, Fredrik Manne, and Alex Pothen (2005)

https://epubs.siam.org/doi/10.1137/S0036144504444711

ADTypes.column_coloring — Function
column_coloring(M::AbstractMatrix, ca::ColoringAlgorithm)::AbstractVector{<:Integer}

Use algorithm ca to construct a structurally orthogonal partition of the columns of M.

The result is a coloring vector c of length size(M, 2) such that for every non-zero coefficient M[i, j], column j is the only column of its color c[j] with a non-zero coefficient in row i.

ADTypes.row_coloring — Function
row_coloring(M::AbstractMatrix, ca::ColoringAlgorithm)::AbstractVector{<:Integer}

Use algorithm ca to construct a structurally orthogonal partition of the rows of M.

The result is a coloring vector c of length size(M, 1) such that for every non-zero coefficient M[i, j], row i is the only row of its color c[i] with a non-zero coefficient in column j.

ADTypes.symmetric_coloring — Function
symmetric_coloring(M::AbstractMatrix, ca::ColoringAlgorithm)::AbstractVector{<:Integer}

Use algorithm ca to construct a symetrically structurally orthogonal partition of the columns (or rows) of the symmetric matrix M.

The result is a coloring vector c of length size(M, 1) == size(M, 2) such that for every non-zero coefficient M[i, j], at least one of the following conditions holds:

  • column j is the only column of its color c[j] with a non-zero coefficient in row i;
  • column i is the only column of its color c[i] with a non-zero coefficient in row j.

Modes

ADTypes.ForwardMode — Type
ForwardMode

Trait for AD choices that rely on forward mode algorithmic differentiation or finite differences.

These two paradigms are classified together because they can both efficiently compute Jacobian-vector products.