API reference

Entry points

DifferentiationInterfaceTest.test_differentiationFunction
test_differentiation(
    backends::Vector{<:ADTypes.AbstractADType};
    ...
)
test_differentiation(
    backends::Vector{<:ADTypes.AbstractADType},
    scenarios::Vector{<:AbstractScenario};
    correctness,
    type_stability,
    call_count,
    sparsity,
    ref_backend,
    detailed,
    input_type,
    output_type,
    first_order,
    second_order,
    onearg,
    twoarg,
    inplace,
    outofplace,
    excluded,
    logging,
    isapprox,
    atol,
    rtol
)

Cross-test a list of backends on a list of scenarios, running a variety of different tests.

Default arguments

Keyword arguments

Testing:

  • correctness=true: whether to compare the differentiation results with the theoretical values specified in each scenario
  • type_stability=false: whether to check type stability with JET.jl (thanks to JET.@test_opt)
  • sparsity: whether to check sparsity of the jacobian / hessian
  • ref_backend: if not nothing, an ADTypes.AbstractADType object to use instead of the scenario-specific reference to provide true values
  • detailed=false: whether to print a detailed or condensed test log

Filtering:

  • input_type=Any, output_type=Any: restrict scenario inputs / outputs to subtypes of this
  • first_order=true, second_order=true: include first order / second order operators
  • onearg=true, twoarg=true: include one-argument / two-argument functions
  • inplace=true, outofplace=true: include in-place / out-of-place operators

Options:

  • logging=false: whether to log progress
  • isapprox=isapprox: function used to compare objects, with the standard signature isapprox(x, y; atol, rtol)
  • atol=0: absolute precision for correctness testing (when comparing to the reference outputs)
  • rtol=1e-3: relative precision for correctness testing (when comparing to the reference outputs)
test_differentiation(
    backend::ADTypes.AbstractADType,
    args...;
    kwargs...
)

Shortcut for a single backend.

DifferentiationInterfaceTest.benchmark_differentiationFunction
benchmark_differentiation(
    backends::Vector{<:ADTypes.AbstractADType},
    scenarios::Vector{<:AbstractScenario};
    input_type,
    output_type,
    first_order,
    second_order,
    onearg,
    twoarg,
    inplace,
    outofplace,
    excluded,
    logging
) -> Vector{DifferentiationBenchmarkDataRow}

Benchmark a list of backends for a list of operators on a list of scenarios.

The object returned is a Vector of DifferentiationBenchmarkDataRow.

The keyword arguments available here have the same meaning as those in test_differentiation.

DifferentiationInterfaceTest.DifferentiationBenchmarkDataRowType
DifferentiationBenchmarkDataRow

Ad-hoc storage type for differentiation benchmarking results.

If you have a vector rows::Vector{DifferentiationBenchmarkDataRow}, you can turn it into a DataFrame as follows:

using DataFrames

df = DataFrame(rows)

The resulting DataFrame will have one column for each of the following fields.

Fields

  • backend::ADTypes.AbstractADType: backend used for benchmarking

  • scenario::AbstractScenario: scenario used for benchmarking

  • operator::Symbol: differentiation operator used for benchmarking, e.g. :gradient or :hessian

  • calls::Int64: number of calls to the differentiated function for one call to the operator

  • samples::Int64: number of benchmarking samples taken

  • evals::Int64: number of evaluations used for averaging in each sample

  • time::Float64: minimum runtime over all samples, in seconds

  • allocs::Float64: minimum number of allocations over all samples

  • bytes::Float64: minimum memory allocated over all samples, in bytes

  • gc_fraction::Float64: minimum fraction of time spent in garbage collection over all samples, between 0.0 and 1.0

  • compile_fraction::Float64: minimum fraction of time spent compiling over all samples, between 0.0 and 1.0

See the documentation of Chairmarks.jl for more details on the measurement fields.

Pre-made scenario lists

Scenario types

DifferentiationInterfaceTest.AbstractScenarioType
AbstractScenario

Store a testing scenario composed of a function and its input + output.

This abstract type should never be used directly: construct one of the subtypes corresponding to the operator you want to test.

Subtypes

Fields

All subtypes have the following fields:

  • f: function to apply
  • x: primal input
  • y: primal output
  • ref: reference to compare against

In addition, some subtypes contain an additional seed (dx or dy).

Constructor

If y is provided, f is interpreted as a 2-argument function f!(y, x) = nothing. Otherwise, f is interpreted as an 1-argument function f(x) = y.

The reference keyword ref should be a function that takes x (and a potential seed dx or dy) to return the correct object.

The keyword place should be either :inplace or :outofplace depending on what kind of operator must be tested.

Internals

This is not part of the public API.