API

Modules

Types and constants

Functions and macros

Documentation

AutoDiffOperators.ADModule — Type
struct ADModule{m}

Speficies and automatic differentiation backend via it's module name.

We recommend to use the AD-backend types defined in AbstractDifferentiation.jl and [ADTypes.jl}(https://github.com/SciML/ADTypes.jl) instead of ADModule. The function convert_ad provides a conversion mechanism between all of these different AD-backend-speficiers.

Examples:

ADModule(:FiniteDifferences)
ADModule(:ForwardDiff)
ADModule(:Zygote)
ADModule(:Enzyme)

Can be converted to a Val for compatibily with approaches like LogDensityProblemsAD.ADgradient (see LogDensityProblemsAD):

Val(ADModule(:ForwardDiff)) == Val(:ForwardDiff)

Constructing an instance of ADModule will fail if the package hosting the corresponding module is not loaded (either directly, or indirectly via dependencies of other loaded packages).

Module(ad::ADModule) returns the Module object that corresponds to ad.

AutoDiffOperators.AdjointMatrixLikeOperator — Type
struct AutoDiffOperators.AdjointMatrixLikeOperator{T<:Number,sym,herm,posdef} <: MatrixLikeOperator{T,sym,herm,posdef}

Represents the adjoint of a MatrixLikeOperator.

User code should not instantiate this type directly, use adjoint(::MatrixLikeOperator) instead.

AutoDiffOperators.FwdRevADSelector — Type
AutoDiffOperators.FwdRevADSelector{Fwd<:ADSelector,Rev<:ADSelector} <: ADSelector

Represent an automatic differentiation backend that forwards forward-mode and reverse-mode AD to two separate selectors fwd::ADSelector and rev::ADSelector.

User code should not instantiate AutoDiffOperators.FwdRevADSelector directly, but use ADSelector(fwd, rev) or ADSelector(fwd = fwd, rev = rev) instead.

AutoDiffOperators.MatrixLikeOperator — Type
abstract type MatrixLikeOperator{T<:Number} <: AbstractMatrix{T} end

Abstract type for matrix-like operators that support multiplication with vectors.

An op::MatrixLikeOperator can be constructed from a multiplication function op_mul(x) that implements op * x, and ajoint multiplication function adjoint_op_mul(x) that implements op' * x, and a size sz via

MatrixLikeOperator(
    ovp, op_mul_function, adjoint_op_mul_function, sz::Dims
    issymmetric::Bool, ishermitian::Bool, isposdef::Bool
)

Typically specialized subtypes of MatrixLikeOperator should be used for specific kinds of operators, though.

MatrixLikeOperator supports conversion to LinearMaps.jl operator types via constructor and convert methods:

LinearMaps.FunctionMap(op::MatrixLikeOperator) isa LinearMaps.FunctionMap
LinearMaps.LinearMap(op::MatrixLikeOperator) isa LinearMaps.FunctionMap

Subtypes must implement:

  • Base.size(op::MatrixLikeOperator)
  • AutoDiffOperators.op_mul(op::MatrixLikeOperator, x::AbstractVector{<:Number})
  • AutoDiffOperators.adjoint_op_mul(op::MatrixLikeOperator, x::AbstractVector{<:Number})

And may implement, resp. specialize (if possible):

  • Base.adjoint(op::MatrixLikeOperator)
  • Base.transpose(op::MatrixLikeOperator)
  • AutoDiffOperators.transpose_op_mul(op::MatrixLikeOperator, x::AbstractVector{<:Number})
AutoDiffOperators.ADSelector — Type
const ADSelector = Union{
    AbstractDifferentiation.AbstractBackend,
    ADTypes.AbstractADType,
    ADModule
}

Instances speficy an automatic differentiation backend.

Unifies the AD-backend selector types defined in AbstractDifferentiation.jl and [ADTypes.jl}(https://github.com/SciML/ADTypes.jl), as well as AutoDiffOperators.ADModule.

AutoDiffOperators currently supports the following AD-backends, and its functions will, in general, accept any subtype of ADSelector as AD-selectors that match them:

  • ForwardDiff
  • Zygote

Some operations that specifically require forward-mode or reverse-mode AD will only accept a subset of these backends though.

The following functions must be specialized for subtypes of ADSelector: convert_ad, with_jvp, with_vjp_func and AutoDiffOperators.supports_structargs

A default implementation is provided for with_gradient, but specialized implementations may often be more performant.

Selector types that forward forward and reverse-mode ad to other selector types should specialize forward_ad_selector and reverse_ad_selector.

AutoDiffOperators.mulfunc_operator — Function
AutoDiffOperators.mulfunc_operator(
    ::Type{OP},
    ::Type{T}, sz::Dims{2}, ovp, vop,
    ::Val{sym}, ::Val{herm}, ::Val{posdef}
) where {OP, T<:Real, sym, herm, posdef}

Generates a linear operator object of type OP that supports multiplication and with (adjoint) vectors based on a multiplication function ovp and an adjoint multiplication function vop.

An operator op = mulfunc_operator(OP, T, sz, ovp, vop, Val(sym), ::Val{herm}, ::Val{posdef}) must show show following behavior:

op isa OP
eltype(op) == T
size(op) == sz
op * x_r == ovp(x_r)
x_l' * op == vop(x_l)
issymmetric(op) == sym
ishermitian(op) == herm
isposdef(op) = posdef

where x_l and x_r are vectors of size sz[1] and sz[2] respectively.

AutoDiffOperators.supports_structargs — Function
AutoDiffOperators.supports_structargs(ad::ADSelector)::Boolean

Returns true if ad supports structured function arguments or false if ad only supports vectors of real numbers.

Since ad may use different backends for forward- and reverse-mode AD, use supports_structargs(forward_ad_selector(ad)) and supports_structargs(reverse_ad_selector(ad)) to check if ad supports structured arguments for the desired operation.

AutoDiffOperators.with_gradient!! — Function
with_gradient!!(f, δx, x, ad::ADSelector)

Returns a tuple (f(x), ∇f(x)) with the gradient ∇f(x)offatx`.

δx may or may not be reused/overwritten and returned as ∇f(x).

The default implementation falls back to with_gradient(f, x, ad), subtypes of ADSelector may specialized with_gradient!! to provide more efficient implementations.

AutoDiffOperators.with_jacobian — Function
with_jacobian(f, x, OP, ad::ADSelector)

Returns a tuple (f(x), J) with a multiplicative Jabobian operator J of type OP.

Example:

using AutoDiffOperators, LinearMaps
y, J = with_jacobian(f, x, LinearMap, ad)
y == f(x)
_, J_explicit = with_jacobian(f, x, Matrix, ad)
J * z_r ≈ J_explicit * z_r
z_l' * J ≈ z_l' * J_explicit

OP may be LinearMaps.LinearMap (resp. LinearMaps.FunctionMap) or Matrix. Other operator types can be supported by specializing AutoDiffOperators.mulfunc_operator for the operator type.

The default implementation of with_jacobian uses jvp_func and with_vjp_func to implement (adjoint) multiplication of J with (adjoint) vectors.