ExaModels

ExaModels.ExaModelsModule
ExaModels

An algebraic modeling and automatic differentiation tool in Julia Language, specialized for SIMD abstraction of nonlinear programs.

For more information, please visit https://github.com/sshin23/ExaModels.jl

ExaModels.AdjointNode1Type
AdjointNode1{F, T, I}

A node with one child for first-order forward pass tree

Fields:

  • x::T: function value
  • y::T: first-order sensitivity
  • inner::I: children
ExaModels.AdjointNode2Type
AdjointNode2{F, T, I1, I2}

A node with two children for first-order forward pass tree

Fields:

  • x::T: function value
  • y1::T: first-order sensitivity w.r.t. first argument
  • y2::T: first-order sensitivity w.r.t. second argument
  • inner1::I1: children #1
  • inner2::I2: children #2
ExaModels.AdjointNodeSourceType
AdjointNodeSource{VT}

A source of AdjointNode. adjoint_node_source[i] returns an AdjointNodeVar at index i.

Fields:

  • inner::VT: variable vector
ExaModels.AdjointNodeVarType
AdjointNodeVar{I, T}

A variable node for first-order forward pass tree

Fields:

  • i::I: index
  • x::T: value
ExaModels.CompressorType
Compressor{I}

Data structure for the sparse index

Fields:

  • inner::I: stores the sparse index as a tuple form
ExaModels.ExaCoreType

ExaCore([array_type::Type, backend])

Returns an intermediate data object ExaCore, which later can be used for creating ExaModel

Example

julia> using ExaModels

julia> c = ExaCore()
An ExaCore

  Float type: ...................... Float64
  Array type: ...................... Vector{Float64}
  Backend: ......................... Nothing

  number of objective patterns: .... 0
  number of constraint patterns: ... 0

julia> c = ExaCore(Float32)
An ExaCore

  Float type: ...................... Float32
  Array type: ...................... Vector{Float32}
  Backend: ......................... Nothing

  number of objective patterns: .... 0
  number of constraint patterns: ... 0

julia> using CUDA

julia> c = ExaCore(Float32, CUDABackend())
An ExaCore

  Float type: ...................... Float32
  Array type: ...................... CUDA.CuArray{Float32, 1, CUDA.Mem.DeviceBuffer}
  Backend: ......................... CUDA.CUDAKernels.CUDABackend

  number of objective patterns: .... 0
  number of constraint patterns: ... 0
ExaModels.ExaModelMethod
ExaModel(core)

Returns an ExaModel object, which can be solved by nonlinear optimization solvers within JuliaSmoothOptimizer ecosystem, such as NLPModelsIpopt or MadNLP.

Example

julia> using ExaModels

julia> c = ExaCore();                      # create an ExaCore object

julia> x = variable(c, 1:10);              # create variables

julia> objective(c, x[i]^2 for i in 1:10); # set objective function

julia> m = ExaModel(c)                     # creat an ExaModel object
An ExaModel{Float64, Vector{Float64}, ...}

  Problem name: Generic
   All variables: ████████████████████ 10     All constraints: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0
            free: ████████████████████ 10                free: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0
           lower: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0                lower: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0
           upper: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0                upper: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0
         low/upp: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0              low/upp: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0
           fixed: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0                fixed: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0
          infeas: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0               infeas: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0
            nnzh: ( 81.82% sparsity)   10              linear: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0
                                                    nonlinear: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0
                                                         nnzj: (------% sparsity)

julia> using NLPModelsIpopt

julia> result = ipopt(m; print_level=0)    # solve the problem
"Execution stats: first-order stationary"
ExaModels.Node1Type
Node1{F, I}

A node with one child for symbolic expression tree

Fields:

  • inner::I: children
ExaModels.Node2Type
Node2{F, I1, I2}

A node with two children for symbolic expression tree

Fields:

  • inner1::I1: children #1
  • inner2::I2: children #2
ExaModels.ParIndexedType
ParIndexed{I, J}

A parameterized data node

Fields:

  • inner::I: parameter for the data
ExaModels.SIMDFunctionType
SIMDFunction(gen::Base.Generator, o0 = 0, o1 = 0, o2 = 0)

Returns a SIMDFunction using the gen.

Arguments:

  • gen: an iterable function specified in Base.Generator format
  • o0: offset for the function evaluation
  • o1: offset for the derivative evalution
  • o2: offset for the second-order derivative evalution
ExaModels.SecondAdjointNode1Type
SecondAdjointNode1{F, T, I}

A node with one child for second-order forward pass tree

Fields:

  • x::T: function value
  • y::T: first-order sensitivity
  • h::T: second-order sensitivity
  • inner::I: DESCRIPTION
ExaModels.SecondAdjointNode2Type
SecondAdjointNode2{F, T, I1, I2}

A node with one child for second-order forward pass tree

Fields:

  • x::T: function value
  • y1::T: first-order sensitivity w.r.t. first argument
  • y2::T: first-order sensitivity w.r.t. first argument
  • h11::T: second-order sensitivity w.r.t. first argument
  • h12::T: second-order sensitivity w.r.t. first and second argument
  • h22::T: second-order sensitivity w.r.t. second argument
  • inner1::I1: children #1
  • inner2::I2: children #2
ExaModels.SecondAdjointNodeSourceType
SecondAdjointNodeSource{VT}

A source of AdjointNode. adjoint_node_source[i] returns an AdjointNodeVar at index i.

Fields:

  • inner::VT: variable vector
ExaModels.VarType
Var{I}

A variable node for symbolic expression tree

Fields:

  • i::I: (parameterized) index
ExaModels.constraintMethod
constraint(core, n; start = 0, lcon = 0,  ucon = 0)

Adds empty constraints of dimension n, so that later the terms can be added with constraint!.

ExaModels.constraintMethod
constraint(core, generator; start = 0, lcon = 0,  ucon = 0)

Adds constraints specified by a generator to core, and returns an Constraint object.

Keyword Arguments

  • start: The initial guess of the solution. Can either be Number, AbstractArray, or Generator.
  • lcon : The constraint lower bound. Can either be Number, AbstractArray, or Generator.
  • ucon : The constraint upper bound. Can either be Number, AbstractArray, or Generator.

Example

julia> using ExaModels

julia> c = ExaCore();

julia> x = variable(c, 10);

julia> constraint(c, x[i] + x[i+1] for i=1:9; lcon = -1, ucon = (1+i for i=1:9))
Constraint

  s.t. (...)
       g♭ ≤ [g(x,p)]_{p ∈ P} ≤ g♯

  where |P| = 9
ExaModels.constraintMethod
constraint(core, expr, pars; start = 0, lcon = 0,  ucon = 0)

Adds constraints specified by a expr and pars to core, and returns an Constraint object.

ExaModels.drpassMethod
drpass(d::D, y, adj)

Performs dense gradient evaluation via the reverse pass on the computation (sub)graph formed by forward pass

Arguments:

  • d: first-order computation (sub)graph
  • y: result vector
  • adj: adjoint propagated up to the current node
ExaModels.gradient!Method
gradient!(y, f, x, adj)

Performs dense gradient evalution

Arguments:

  • y: result vector
  • f: the function to be differentiated in SIMDFunction format
  • x: variable vector
  • adj: initial adjoint
ExaModels.grpassMethod
grpass(d::D, comp, y, o1, cnt, adj)

Performs dsparse gradient evaluation via the reverse pass on the computation (sub)graph formed by forward pass

Arguments:

  • d: first-order computation (sub)graph
  • comp: a Compressor, which helps map counter to sparse vector index
  • y: result vector
  • o1: index offset
  • cnt: counter
  • adj: adjoint propagated up to the current node
ExaModels.hdrpassMethod
hdrpass(t1::T1, t2::T2, comp, y1, y2, o2, cnt, adj)

Performs sparse hessian evaluation ((df1/dx)(df2/dx)' portion) via the reverse pass on the computation (sub)graph formed by second-order forward pass

Arguments:

  • t1: second-order computation (sub)graph regarding f1
  • t2: second-order computation (sub)graph regarding f2
  • comp: a Compressor, which helps map counter to sparse vector index
  • y1: result vector #1
  • y2: result vector #2 (only used when evaluating sparsity)
  • o2: index offset
  • cnt: counter
  • adj: second adjoint propagated up to the current node
ExaModels.jrpassMethod
jrpass(d::D, comp, i, y1, y2, o1, cnt, adj)

Performs sparse jacobian evaluation via the reverse pass on the computation (sub)graph formed by forward pass

Arguments:

  • d: first-order computation (sub)graph
  • comp: a Compressor, which helps map counter to sparse vector index
  • i: constraint index (this is i-th constraint)
  • y1: result vector #1
  • y2: result vector #2 (only used when evaluating sparsity)
  • o1: index offset
  • cnt: counter
  • adj: adjoint propagated up to the current node
ExaModels.multipliersMethod
multipliers(result, y)

Returns the multipliers for constraints y associated with result, obtained by solving the model.

Example

julia> using ExaModels, NLPModelsIpopt

julia> c = ExaCore();                     

julia> x = variable(c, 1:10, lvar = -1, uvar = 1);

julia> objective(c, (x[i]-2)^2 for i in 1:10);

julia> y = constraint(c, x[i] + x[i+1] for i=1:9; lcon = -1, ucon = (1+i for i=1:9));

julia> m = ExaModel(c);                   

julia> result = ipopt(m; print_level=0);

julia> val = multipliers(result, y);


julia> val[1] ≈ 0.81933930
true
ExaModels.multipliers_LMethod
multipliers_L(result, x)

Returns the multipliers_L for variable x associated with result, obtained by solving the model.

Example

julia> using ExaModels, NLPModelsIpopt

julia> c = ExaCore();                     

julia> x = variable(c, 1:10, lvar = -1, uvar = 1);

julia> objective(c, (x[i]-2)^2 for i in 1:10);

julia> m = ExaModel(c);                   

julia> result = ipopt(m; print_level=0);

julia> val = multipliers_L(result, x);

julia> isapprox(val, fill(0, 10), atol=sqrt(eps(Float64)), rtol=Inf)
true
ExaModels.multipliers_UMethod
multipliers_U(result, x)

Returns the multipliers_U for variable x associated with result, obtained by solving the model.

Example

julia> using ExaModels, NLPModelsIpopt

julia> c = ExaCore();                     

julia> x = variable(c, 1:10, lvar = -1, uvar = 1);

julia> objective(c, (x[i]-2)^2 for i in 1:10);

julia> m = ExaModel(c);                   

julia> result = ipopt(m; print_level=0);

julia> val = multipliers_U(result, x);

julia> isapprox(val, fill(2, 10), atol=sqrt(eps(Float64)), rtol=Inf)
true
ExaModels.objectiveMethod
objective(core::ExaCore, generator)

Adds objective terms specified by a generator to core, and returns an Objective object.

Example

julia> using ExaModels

julia> c = ExaCore();

julia> x = variable(c, 10);

julia> objective(c, x[i]^2 for i=1:10)
Objective

  min (...) + ∑_{p ∈ P} f(x,p)

  where |P| = 10
ExaModels.objectiveMethod
objective(core::ExaCore, expr, pars)

Adds objective terms specified by a expr and pars to core, and returns an Objective object.

ExaModels.sgradient!Method

sgradient!(y, f, x, adj)

Performs sparse gradient evalution

Arguments:

  • y: result vector
  • f: the function to be differentiated in SIMDFunction format
  • x: variable vector
  • adj: initial adjoint
ExaModels.shessian!Method
shessian!(y1, y2, f, x, adj1, adj2)

Performs sparse jacobian evalution

Arguments:

  • y1: result vector #1
  • y2: result vector #2 (only used when evaluating sparsity)
  • f: the function to be differentiated in SIMDFunction format
  • x: variable vector
  • adj1: initial first adjoint
  • adj2: initial second adjoint
ExaModels.sjacobian!Method
sjacobian!(y1, y2, f, x, adj)

Performs sparse jacobian evalution

Arguments:

  • y1: result vector #1
  • y2: result vector #2 (only used when evaluating sparsity)
  • f: the function to be differentiated in SIMDFunction format
  • x: variable vector
  • adj: initial adjoint
ExaModels.solutionMethod
solution(result, x)

Returns the solution for variable x associated with result, obtained by solving the model.

Example

julia> using ExaModels, NLPModelsIpopt

julia> c = ExaCore();                     

julia> x = variable(c, 1:10, lvar = -1, uvar = 1);

julia> objective(c, (x[i]-2)^2 for i in 1:10);

julia> m = ExaModel(c);                   

julia> result = ipopt(m; print_level=0);

julia> val = solution(result, x);

julia> isapprox(val, fill(1, 10), atol=sqrt(eps(Float64)), rtol=Inf)
true
ExaModels.variableMethod
variable(core, dims...; start = 0, lvar = -Inf, uvar = Inf)

Adds variables with dimensions specified by dims to core, and returns Variable object. dims can be either Integer or UnitRange.

Keyword Arguments

  • start: The initial guess of the solution. Can either be Number, AbstractArray, or Generator.
  • lvar : The variable lower bound. Can either be Number, AbstractArray, or Generator.
  • uvar : The variable upper bound. Can either be Number, AbstractArray, or Generator.

Example

julia> using ExaModels

julia> c = ExaCore();

julia> x = variable(c, 10; start = (sin(i) for i=1:10))
Variable

  x ∈ R^{10}

julia> y = variable(c, 2:10, 3:5; lvar = zeros(9,3), uvar = ones(9,3))
Variable

  x ∈ R^{9 × 3}
ExaModels.@register_bivariateMacro
register_bivariate(f, df1, df2, ddf11, ddf12, ddf22)

Register a bivariate function f to ExaModels, so that it can be used within objective and constraint expressions

Arguments:

  • f: function
  • df1: derivative function (w.r.t. first argument)
  • df2: derivative function (w.r.t. second argument)
  • ddf11: second-order derivative funciton (w.r.t. first argument)
  • ddf12: second-order derivative funciton (w.r.t. first and second argument)
  • ddf22: second-order derivative funciton (w.r.t. second argument)

Example

julia> using ExaModels

julia> relu23(x) = (x > 0 || y > 0) ? (x + y)^3 : zero(x)
relu23 (generic function with 1 method)

julia> drelu231(x) = (x > 0 || y > 0) ? 3 * (x + y)^2 : zero(x)
drelu231 (generic function with 1 method)

julia> drelu232(x) = (x > 0 || y > 0) ? 3 * (x + y)^2  : zero(x)
drelu232 (generic function with 1 method)

julia> ddrelu2311(x) = (x > 0 || y > 0) ? 6 * (x + y) : zero(x)
ddrelu2311 (generic function with 1 method)

julia> ddrelu2312(x) = (x > 0 || y > 0) ? 6 * (x + y) : zero(x)
ddrelu2312 (generic function with 1 method)

julia> ddrelu2322(x) = (x > 0 || y > 0) ? 6 * (x + y) : zero(x)
ddrelu2322 (generic function with 1 method)

julia> @register_bivariate(relu23, drelu231, drelu232, ddrelu2311, ddrelu2312, ddrelu2322)
ExaModels.@register_univariateMacro
@register_univariate(f, df, ddf)

Register a univariate function f to ExaModels, so that it can be used within objective and constraint expressions

Arguments:

  • f: function
  • df: derivative function
  • ddf: second-order derivative funciton

Example

julia> using ExaModels

julia> relu3(x) = x > 0 ? x^3 : zero(x)
relu3 (generic function with 1 method)

julia> drelu3(x) = x > 0 ? 3*x^2 : zero(x)
drelu3 (generic function with 1 method)

julia> ddrelu3(x) = x > 0 ? 6*x : zero(x)
ddrelu3 (generic function with 1 method)

julia> @register_univariate(relu3, drelu3, ddrelu3)