Limitations

FD does not support expressions with conditionals on FD variables. For example, you can do this:

julia> f(a,b,c) = a< 1.0 ? cos(b) : sin(c)
f (generic function with 2 methods)

julia> f(0.0,x,y)
cos(x)

julia> f(1.0,x,y)
sin(y)

but you can't do this:

julia> f(a,b) = a < b ? cos(a) : sin(b)
f (generic function with 2 methods)

julia> f(x,y)
ERROR: MethodError: no method matching isless(::FastDifferentiation.Node{Symbol, 0}, ::FastDifferentiation.Node{Symbol, 0})

Closest candidates are:
  isless(::Any, ::DataValues.DataValue{Union{}})
   @ DataValues ~/.julia/packages/DataValues/N7oeL/src/scalar/core.jl:291
  isless(::S, ::DataValues.DataValue{T}) where {S, T}
   @ DataValues ~/.julia/packages/DataValues/N7oeL/src/scalar/core.jl:285
  isless(::DataValues.DataValue{Union{}}, ::Any)
   @ DataValues ~/.julia/packages/DataValues/N7oeL/src/scalar/core.jl:293
  ...

This is because the call f(x,y) creates an expression graph. At graph creation time the FD variables x,y are unevaluated variables with no specific value so they cannot be compared with any other value.

The algorithm can be extended to work with conditionals applied to FD variables but the processing time and graph size may grow exponentially with conditional nesting depth. A future version may allow for limited conditional nesting. See Future Work for a potential long term solution to this problem.

FD does not support looping internally. All operations with loops, such as matrix vector multiplication, are unrolled into scalar operations. The corresponding executable functions generated by make_function have size proportional to the number of operations.

Expressions with ≈10⁵ scalar operations have reasonable symbolic preprocessing and compilation times. Beyond this size LLVM compilation time can become extremely long and eventually the executables become so large that their caching behavior is not good and performance declines.

A possible solution to this problem is to do what is called rerolling: detecting repreating indexing patterns in the FD expressions and automatically generating loops to replace inlined code. This rerolling step would be performed on the FD expressions graphs before function compliation.

It is not necessary to completely undo the unrolling back to the original expresssion, just to reduce code size enough to get reasonable compilation times and better caching behavior.

For example, in this matrix vector multiplication

julia> a = make_variables(:a,3,3)
3×3 Matrix{FastDifferentiation.Node}:
 a1_1  a1_2  a1_3
 a2_1  a2_2  a2_3
 a3_1  a3_2  a3_3

julia> b = make_variables(:b,3)
3-element Vector{FastDifferentiation.Node}:
 b1
 b2
 b3

julia> a*b
3-element Vector{Any}:
 (((a1_1 * b1) + (a1_2 * b2)) + (a1_3 * b3))
 (((a2_1 * b1) + (a2_2 * b2)) + (a2_3 * b3))
 (((a3_1 * b1) + (a3_2 * b2)) + (a3_3 * b3))

the goal is to replace concrete index numbers with symbolic index variables that represent offsets rather than absolute indices

[
  a[i,j]*b[j] + a[i, j+1]*b[j+1] + a[i,j+2]*b[j+2]
  a[i+1,j]*b[j] + a[i+1, j+1]*b[j+1] + a[i+1,j+2]*b[j+2]
  a[i+2,j]*b[j] + a[i+2, j+1]*b[j+1] + a[i+2,j+2]*b[j+2]
]

and then to extract looping structure from these indices.