The default alignment for operators when not specified.

const EMPTY_NODE_STATE

A singleton instance of TimeDag.EmptyNodeEvaluationState.

INTERSECT

For inputs (A, B, ...), tick whenever all inputs tick simultaneously.

LEFT

For inputs (A, B, ...), tick whenever A ticks so long as all inputs are active.

UNION

For inputs (A, B, ...), tick whenever any input ticks so long as all inputs are active.

Represent a technique for aligning two timeseries.

BCast(f::Function)

Represent a function which should be broadcasted when called.

That is, BCast{f}(x, y...) is identical to f.(x, y...).

Binary operator accumulated from inception.

BinaryNodeOp{T,A<:Alignment} <: NodeOp{T}

An abstract type representing a node op with two parents, and using alignment A.

Time-windowed associative binary operator, potentially emitting early before the window is full.

Windowed associative binary operator, potentially emitting early before the window is full.

Block{T}()
Block(times::AbstractVector{DateTime}, values::AbstractVector{T})
Block(unchecked, times, values)

Represent some data in timeseries.

Conceptually this is a list of (time, value) pairs, or "knots". Times must be strictly increasing — i.e. no repeated timestamps are allowed.

The constructor Block(times, values) will verify that the input data satisfies this constraint, however Block(unchecked, times, values) will skip the checks. This is primarily intended for internal use, where the caller assumes responsibility for the validity of times & values.

A node which just wraps a block.

A node which fundamentally represents a constant available over all time.

State to keep track of the number of knots that we have seen on the input.

State for an n-dimensional covariance matrix.

State for an n-dimensional covariance matrix of statically known dimension.

A node which will never tick, but has a definite value type.

EmptyNodeEvaluationState()

A NodeEvaluationState which has no content, to be used by non-stateful nodes.

In practice, the common instance TimeDag.EMPTY_NODE_STATE can be used.

EvaluationState

All state necessary for the evaluation of some nodes.

This should be created with start_at.

Fields

• ordered_node_to_children::OrderedDict{Node,Set{Node}}: a map from every node which we need to run, to its children. The ordering of the elements is such that, if evaluated in this order, all dependencies will be evaluated before they are required.
• node_to_state::IdDict{Node,NodeEvaluationState}: maintain the state for every node being evaluated.
• current_time::DateTime: the time to which this state corresponds.
• evaluated_node_to_blocks::IdDict{Node,Vector{Block}}: the output blocks that we care about.

A node which emits the last window knots as an array.

IdentityMap

An abstract identity map.

Any implementation of this type needs to implement obtain_node!.

A node op which ticks once a day at the specified local time of day.

A node which lags its input by a fixed number of knots.

Maybe{T}()
Maybe(value::T)

A structure which can hold a value of type T, or represent the absence of a value.

The API is optimised for speed over memory usage, by allowing a function that may otherwise return Union{T, Nothing} to instead always return Maybe{T}, and hence be type-stable.

NaryNodeOp{N,T,A<:Alignment} <: NodeOp{T}

An abstract type representing a node op with N parents, and using alignment A.

This type should be avoided for N < 3, since in these cases it would be more appropriate to use either TimeDag.UnaryNodeOp or TimeDag.BinaryNodeOp.

Node(parents, op)

A node in the computational graph that combines zero or more parents with op to produce a timeseries.

Due to subgraph elimination, nodes that are equivalent should always be identical objects. We therefore leave hash & == defined in terms of the objectid.

abstract type NodeEvaluationState end

Represents any and all state that a node must retain between evaluating batches.

Instances of subtypes of NodeEvaluationState are given to run_node!.

abstract type NodeOp{T} end

Represent a time-series operation whose output will be a time-series with value type T.

A node op which ticks every delta, such that a knot would appear on epoch.

SimpleBinary{f,TimeAgnostic,T,A}

Represents a stateless binary operator that will always emit a value.

The value of the TimeAgnostic type parmater is coupled to time_agnostic.

SimpleBinaryLeftInitial{f,T,R}

Represents a stateless binary operator that will always emit a value.

The value of the TimeAgnostic type parmater is coupled to time_agnostic.

Unlike SimpleBinary, this also contains initial values for its right parent. See Initial values for more details.

SimpleBinaryUnionInitial{f,TimeAgnostic,T,L,R}

Represents a stateless binary operator that will always emit a value.

The value of the TimeAgnostic type parmater is coupled to time_agnostic.

Unlike SimpleBinary, this also contains initial values for its parent nodes. See Initial values for more details.

SimpleNary{f,TimeAgnostic,N,T,A}

Represents a stateless Nary operator that will always emit a value.

The value of the TimeAgnostic type parmater is coupled to time_agnostic.

SimpleNaryInitial{f,TimeAgnostic,N,T,A,Types}

Represents a stateless binary operator that will always emit a value.

The value of the TimeAgnostic type parmater is coupled to time_agnostic.

Unlike SimpleNary, this also contains initial values. See Initial values for more details.

SimpleUnary{f,TimeAgnostic,T}

Represents a stateless unary operator that will always emit a value.

The value of the TimeAgnostic type parmater is coupled to time_agnostic.

A node which lags its input by a fixed time duration.

A node which will read from an appropriately formatted TeaFile.

State to keep track of the number of knots that we have seen on the input since the last output.

Unary operator accumulated from inception.

UnaryNodeOp{T} <: NodeOp{T}

An abstract type representing a node op with a single parent.

Time-windowed associative unary operator, potentially emitting early before the window is full.

Windowed associative unary operator, potentially emitting early before the window is full.

Sentinel type for use in the constructor for Block below.

WeakIdentityMap

Represent a collection of nodes which doesn't hold strong references to any nodes.

This is useful, as it allows the existence of this cache to be somewhat transparent to the user, and they only have to care about holding on to references for nodes that they care about.

This structure contains nodes, but also node weak nodes – these allow us to determine whether we ought to create a given node.

WeakNode(parents, op)

Represent a node-like object that doesn't hold strong references to its parents.

This exists purely such that hash and == do allow multiple instances of WeakNode to compare equal if they have the same parents and op.

Wrapped{f}

Represent a function that, when called, will expect its arguments to be nodes and try to convert them as such.

!(x::Node)

Obtain a node with values constructed by applying ! to each input value.

*(x, y[, alignment=DEFAULT_ALIGNMENT; kwargs...])

Obtain a node with values constructed by applying * to the input values.

An alignment can optionally be specified. x and y should be nodes, or constants that can be converted to nodes.

Other keyword arguments are passed to apply.

+(x, y[, alignment=DEFAULT_ALIGNMENT; kwargs...])

Obtain a node with values constructed by applying + to the input values.

An alignment can optionally be specified. x and y should be nodes, or constants that can be converted to nodes.

Other keyword arguments are passed to apply.

-(x, y[, alignment=DEFAULT_ALIGNMENT; kwargs...])

Obtain a node with values constructed by applying - to the input values.

An alignment can optionally be specified. x and y should be nodes, or constants that can be converted to nodes.

Other keyword arguments are passed to apply.

-(x::Node)

Obtain a node with values constructed by applying - to each input value.

/(x, y[, alignment=DEFAULT_ALIGNMENT; kwargs...])

Obtain a node with values constructed by applying / to the input values.

An alignment can optionally be specified. x and y should be nodes, or constants that can be converted to nodes.

Other keyword arguments are passed to apply.

<(x, y[, alignment=DEFAULT_ALIGNMENT; kwargs...])

Obtain a node with values constructed by applying < to the input values.

An alignment can optionally be specified. x and y should be nodes, or constants that can be converted to nodes.

Other keyword arguments are passed to apply.

<=(x, y[, alignment=DEFAULT_ALIGNMENT; kwargs...])

Obtain a node with values constructed by applying <= to the input values.

An alignment can optionally be specified. x and y should be nodes, or constants that can be converted to nodes.

Other keyword arguments are passed to apply.

>(x, y[, alignment=DEFAULT_ALIGNMENT; kwargs...])

Obtain a node with values constructed by applying > to the input values.

An alignment can optionally be specified. x and y should be nodes, or constants that can be converted to nodes.

Other keyword arguments are passed to apply.

>=(x, y[, alignment=DEFAULT_ALIGNMENT; kwargs...])

Obtain a node with values constructed by applying >= to the input values.

An alignment can optionally be specified. x and y should be nodes, or constants that can be converted to nodes.

Other keyword arguments are passed to apply.

^(x, y[, alignment=DEFAULT_ALIGNMENT; kwargs...])

Obtain a node with values constructed by applying ^ to the input values.

An alignment can optionally be specified. x and y should be nodes, or constants that can be converted to nodes.

Other keyword arguments are passed to apply.

cbrt(x::Node)

Obtain a node with values constructed by applying cbrt to each input value.

abs(x::Node)

Obtain a node with values constructed by applying abs to each input value.

acos(x::Node)

Obtain a node with values constructed by applying acos to each input value.

acosh(x::Node)

Obtain a node with values constructed by applying acosh to each input value.

asin(x::Node)

Obtain a node with values constructed by applying asin to each input value.

asinh(x::Node)

Obtain a node with values constructed by applying asinh to each input value.

atan(x::Node)

Obtain a node with values constructed by applying atan to each input value.

atanh(x::Node)

Obtain a node with values constructed by applying atanh to each input value.

cos(x::Node)

Obtain a node with values constructed by applying cos to each input value.

cosh(x::Node)

Obtain a node with values constructed by applying cosh to each input value.

diff(x::Node[, n=1])

Compute the n-knot difference of x, i.e. x - lag(x, n).

exp(x::Node)

Obtain a node with values constructed by applying exp to each input value.

filter(f::Function, x::Node) -> Node

Obtain a node that removes knots for which f(value) is false.

The value_type of the returned node is the same as that for the input x.

getindex(x::Node, args...)

A node whose values are generated by calling getindex(value, args...) on each value obtained from the node x.

inv(x::Node)

Obtain a node with values constructed by applying inv to each input value.

log(x::Node)

Obtain a node with values constructed by applying log to each input value.

log10(x::Node)

Obtain a node with values constructed by applying log10 to each input value.

log2(x::Node)

Obtain a node with values constructed by applying log2 to each input value.

max(x, y[, alignment=DEFAULT_ALIGNMENT; kwargs...])

Obtain a node with values constructed by applying max to the input values.

An alignment can optionally be specified. x and y should be nodes, or constants that can be converted to nodes.

Other keyword arguments are passed to apply.

merge(x::Node...) -> Node

Given at least one node x, create a node that emits the union of knots from all x.

If one or more of the inputs contain knots at the same time, then only one will be emitted. The last input in which a knot occurs at a particular time will take precedence.

If the inputs x have different value types, then the resultant value type will be promoted as necessary to accommodate all inputs.

min(x, y[, alignment=DEFAULT_ALIGNMENT; kwargs...])

Obtain a node with values constructed by applying min to the input values.

An alignment can optionally be specified. x and y should be nodes, or constants that can be converted to nodes.

Other keyword arguments are passed to apply.

prod(x::Node, window::Int; emit_early::Bool=false) -> Node
prod(x::Node, window::TimePeriod; emit_early::Bool=false) -> Node

Create a node of the rolling product of x over the last window knots, or time interval.

If emit_early is false, then the node returned will only start ticking once the window is full. Otherwise, it will tick immediately with a partially-filled window.

prod(x::Node) -> Node

Create a node which ticks when x ticks, with values of the cumulative product of x.

rand([rng=...,] alignment::Node[, S, dims...])

Generate random numbers aligned to alignment, with the given rng if provided.

Semantics are otherwise very similar to the usual Base.rand:

• If specified, S will be one of

  • the element type
  • a set of values from which to select

  S will default to Float64 otherwise.

• If specified, dims should be a tuple or vararg of integers representing the dimensions of an array.

sign(x::Node)

Obtain a node with values constructed by applying sign to each input value.

sin(x::Node)

Obtain a node with values constructed by applying sin to each input value.

sinh(x::Node)

Obtain a node with values constructed by applying sinh to each input value.

skip(node::Node, n::Int)

Produces a TimeDag.Node which is equal to x less the first n knots.

skipmissing(x::Node{T}) -> Node{nonmissingtype(T)}

Obtain a node which ticks with the values of x, so long as that value is not missing.

The value_type of the node that is returned will always be the nonmissingtype of the value_type of x.

In the case that x cannot tick with missing (based on its value_type), we just return x.

sqrt(x::Node)

Obtain a node with values constructed by applying sqrt to each input value.

sum(x::Node, window::Int; emit_early::Bool=false) -> Node
sum(x::Node, window::TimePeriod; emit_early::Bool=false) -> Node

Create a node of the rolling sum of x over the last window knots, or time interval.

If emit_early is false, then the node returned will only start ticking once the window is full. Otherwise, it will tick immediately with a partially-filled window.

sum(x::Node) -> Node

Create a node which ticks when x ticks, with values of the cumulative sum of x.

tan(x::Node)

Obtain a node with values constructed by applying tan to each input value.

tanh(x::Node)

Obtain a node with values constructed by applying tanh to each input value.

vec(x::Node{<:AbstractArray}) -> Node{<:AbstractVector}

Return a vector whose values are those of x, but flattened into a single vector.

If x has values which are already an AbstractVector, this will be a no-op.

dot(x, y[, alignment=DEFAULT_ALIGNMENT; kwargs...])

Obtain a node with values constructed by applying dot to the input values.

An alignment can optionally be specified. x and y should be nodes, or constants that can be converted to nodes.

Other keyword arguments are passed to apply.

cor(x, y[, alignment]; corrected::Bool=true) -> Node

Create a node which ticks with values of the running correlation of x and y.

The specified alignment controls the behaviour when x and y tick at different times, as per the documentation in Alignment. When not specified, it defaults to UNION.

This is equivalent to a sample correlation over the n values of (x, y) pairs observed at and before a given time.

cor(x, y, window::Int[, alignment]; emit_early=false) -> Node
cor(x, y, window::TimePeriod[, alignment]; emit_early=false) -> Node

Create a node of the rolling covariance of x and y over the last window knots, or time interval.

The specified alignment controls the behaviour when x and y tick at different times, as per the documentation in Alignment. When not specified, it defaults to UNION.

If emit_early is false, then the node returned will only start ticking once the window is full. Otherwise, it will tick immediately with a partially-filled window.

This is equivalent to a sample correlation over the n values of (x, y) pairs observed at and before a given time, with n capped at window.

cov(x, y[, alignment]; corrected::Bool=true) -> Node

Create a node which ticks with values of the running covariance of x and y.

The specified alignment controls the behaviour when x and y tick at different times, as per the documentation in Alignment. When not specified, it defaults to UNION.

This is equivalent to a sample covariance over the n values of (x, y) pairs observed at and before a given time. If corrected is true (the default), we normalise by n-1, otherwise we normalise by n.

cov(x, y, window::Int[, alignment]; emit_early=false, corrected=true) -> Node
cov(x, y, window::TimePeriod[, alignment]; emit_early=false, corrected=true) -> Node

Create a node of the rolling covariance of x and y over the last window knots, or time interval.

The specified alignment controls the behaviour when x and y tick at different times, as per the documentation in Alignment. When not specified, it defaults to UNION.

If emit_early is false, then the node returned will only start ticking once the window is full. Otherwise, it will tick immediately with a partially-filled window.

This is equivalent to a sample covariance over the n values of (x, y) pairs observed at and before a given time, with n capped at window. If corrected is true (the default), we normalise by n-1, otherwise we normalise by n.

cov(x::Node{<:AbstractVector}; corrected::Bool=true) -> Node

Create a node which ticks with values of the running covariance of x.

cov(x::Node{<:AbstractVector}, window::Int; corrected::Bool=true) -> Node

Create a node of the rolling covariance of x over the last window knots.

If emit_early is false, then the node returned will only start ticking once the window is full. Otherwise, it will tick immediately with a partially-filled window.

mean(x::Node, window::Int; emit_early::Bool=false) -> Node
mean(x::Node, window::TimePeriod; emit_early::Bool=false) -> Node

Create a node of the rolling mean of x over the last window knots, or time interval.

If emit_early is false, then the node returned will only start ticking once the window is full. Otherwise, it will tick immediately with a partially-filled window.

mean(x::Node) -> Node

Create a node which ticks when x ticks, with values of the running mean of x.

std(x::Node, window::Int; emit_early::Bool=false, corrected::Bool=true) -> Node
std(x::Node, window::TimePeriod; emit_early::Bool=false, corrected::Bool=true) -> Node

Create a node of the rolling standard deviation of x over the last window knots, or time interval.

If emit_early is false, then the node returned will only start ticking once the window is full. Otherwise, it will tick immediately with a partially-filled window.

This is equivalent to sqrt(var(x, window; emit_early, corrected)).

std(x::Node; corrected::Bool=true) -> Node

Create a node which ticks when x ticks, with values of the running standard deviation of x.

This is equivalent to sqrt(var(x; corrected)).

var(x::Node, window::Int; emit_early::Bool=false, corrected::Bool=true) -> Node
var(x::Node, window::TimePeriod; emit_early::Bool=false, corrected::Bool=true) -> Node

Create a node of the rolling variance of x over the last window knots, or time interval.

If emit_early is false, then the node returned will only start ticking once the window is full. Otherwise, it will tick immediately with a partially-filled window.

This is equivalent to a sample variance over the n values of x (capped at window), observed at and before a given time. If corrected is true (the default), we normalise by n-1, otherwise we normalise by n.

var(x::Node; corrected::Bool=true) -> Node

Create a node which ticks when x ticks, with values of the running variance of x.

This is equivalent to a sample variance over the n values of x observed at and before a given time. If corrected is true (the default), we normalise by n-1, otherwise we normalise by n.

_allocate_values(T, n::Int) -> AbstractVector{T}

Allocate some uninitialized memory that looks like a vector{T} of length n.

Apply, assuming input_l and input_r have identical alignment.

Apply, assuming all inputs have identical alignment.

_can_propagate_constant(::NodeOp) -> Bool

Return true for ops which can propagate constant values if all their parents are constant.

_combine(op, data_1, data_2) -> Data

This should be defined for all inception and windowed ops. Given two data objects, combine them into a new data object.

Wrapper that avoids the need to define create_evaluation_state for stateless nodes.

_create_node!(id_map::WeakIdentityMap, parents, op, weak_node) -> Node

Create a node and insert it into id_map.

_create_operator_evaluation_state(parents, op) -> NodeEvaluationState

Internal function that will look at stateless_operator, and iff it is false call create_operator_evaluation_state. Otherwise return an empty node state.

_data_type(op)

Return the type of data used for the given op.

Whether or not this window op is set to emit with a non-full window.

Whether or not this window op is set to emit with a non-full window.

Identity if the argument is a node, otherwise wrap it in a constant node.

_equal_times(a::Block, b::Block) -> Bool

true => the times in blocks a and b are the same. false => the times in blocks a and b may be different, or the same.

This function will try to return true for as many cases as possible, with the guarantee that it will always run in constant time; i.e. it will never explicitly compare time values.

_extract(op, data) -> value

This should be defined for all inception and windowed ops. Given some data object, it should compute the appropriate output value for the node.

_get_constant_inputs(parents, op)

For use only in the specialised obtain_node.

It assumes that all parents are either constant or empty, and that at least one of parents is empty.

Convenience method to dispatch to reduced-argument operator! calls.

_propagate_constant_value(op::NodeOp{T}, parents::NTuple{N, Node}) -> T

Given that all parents are constants, get the value of the constant node we should output. This assumes that _can_propagate_constant(op) is true.

_rand(alignment::Node, args...)
_rand(rng::AbstractRNG, alignment::Node, args...)

Internal generation of a Rand node.

_should_tick(op, data) -> Bool

This should be defined for any op that does not have _unfiltered(op) returning true. The return value determines whether a knot should be emitted for this value.

Slice the block so we contain values in the interval [timestart, timeend).

_to_utc(dt::DateTime, tz::TimeZone) -> DateTime

Interpret naive dt to be in timezone tz. Return a naive DateTime, but in UTC.

Resize the vector to length n and attempt to reclaim unused space.

_unfiltered(op) -> Bool

Returns true iff _should_tick will always return true.

_update(op, data_1, x...) -> Data

Given a data object, and a new observation (of potentially multiple arguments), generate a new data field.

If only creating an InceptionOp, it is sufficient to define this instead of _combine. By default this will use _wrap and _combine.

Internal implementation of vcat.

Expects blocks to be non-empty, and all blocks therein to also be non-empty.

_window(::TWindowOp) -> Millisecond

Return the time duration of the window for the specified op. The default implementation expects a field called window on the op structure.

_window(::WindowOp) -> Int64

Return the number of knots in the window for the specified op. The default implementation expects a field called window on the op structure.

_wrap(::Type{T}, x...)

Wrap value(s) into a data object of the given type, for use with associative combinations.

The default implementation handles the identity case, and also common conversions where possible (calling single-argument constructor). A custom method should be added when needed for user-defined types.

active_count(nodes...) -> Node{Int64}

Get a node of the number of the given nodes (at least one) which are active.

align(x, y) -> Node

Form a node that ticks with the values of x whenever y ticks.

align_once(x, y) -> Node

Similar to align(x, y), except knots from x will be emitted at most once.

This means that the resulting node will tick at a subset of the times that y ticks.

always_ticks(node) -> Bool
always_ticks(op) -> Bool

Returns true iff the return value from operator! can be assumed to always be valid.

If true, operator!(::Node{T}, ...) should return a T. If false, operator!(::Node{T}, ...) should return a Maybe{T}.

Note, that for sensible performance characteristics, this should be knowable from typeof(op)

ancestors(nodes)

Get a list of all nodes in the graph defined by nodes, including all parents. * Every node in the graph will be visited exactly once. * The parents of any vertex will always come before the vertex itself.

ancestors(graph::AbstractGraph{T}, sources::AbstractVector{T}) -> Vector{T}

Given a (directed) graph, with edges from child to parent, find all ancestor vertices of the given sources.

The result will be ordered such that the parents of any given vertex will always come before the vertex itself.

apply(f::Function, x; out_type=nothing, time_agnostic=true)
apply(
    f::Function, x, y[, z, ..., alignment=DEFAULT_ALIGNMENT];
    out_type=nothing, time_agnostic=true, initial_values=nothing
)

Obtain a node with values constructed by applying the pure function f to the input values.

Iff time_agnostic is false, f will be passed the time of the current knot as the first argument, in addition to any values.

With more than one nodal argument, alignment will be performed. In this case, the alignment argument can be specified as one of INTERSECT, LEFT or UNION. If unspecified, DEFAULT_ALIGNMENT will be used.

Internally this will infer the output type of f applied to the arguments, and will also ensure that subgraph elimination occurs when possible.

If out_type is not specified, we attempt to infer the value type of the resulting node automatically, using output_type. Alternatively, if out_type is given as anything other than nothing, it will be used instead.

If initial_values is specified, it should be a tuple of the same length as the number of node-like arguments passed in. See Initial values for more details.

block_node(block::Block)

Construct a node whose values are read directly from the given block.

coalign(x, [...; alignment::Alignment]) -> Node...

Given at least one node(s) x, or values that are convertible to nodes, align all of them.

We guarantee that all nodes that are returned will have the same alignment. The values of each node will correspond to the values of the input nodes.

The choice of alignment is controlled by alignment, which defaults to UNION.

constant(value) -> Node
constant(T, value) -> Node{T}

Explicitly wrap value into a TimeDag constant node, regardless of its type.

If T is provided, this allows creation of a node with a value_type that is a supertype of the type of the value — otherwise the constant node will always just use the concrete type of value.

In many cases this isn't required, since many TimeDag functions which expect nodes will automatically wrap non-node arguments into a constant node.

convert_value(T, x::Node[; upcast=false]) -> Node

Convert the values of node x to type T.

The value type of the resulting node is guaranteed to be T if and only if upcast=true. See further discussion in the note.

julia> x = convert_value(Any, constant("hello"));

julia> value_type(x)
String

julia> x = convert_value(Any, constant("hello"); upcast=true);

julia> value_type(x)
Any

count_knots(x) -> Node{Int64}

Return a node that ticks with the number of knots seen in x since evaluation began.

create_evaluation_state(node::Node) -> NodeEvaluationState
create_evaluation_state(parents, node::NodeOp) -> NodeEvaluationState

Create an empty evaluation state for the given node, when starting evaluation at the specified time.

create_operator_evaluation_state(parents, op::NodeOp) -> NodeEvaluationState

Create an empty evaluation state for the given node, when starting evaluation at the specified time.

Note that this is state that will be passed to operator. The overall node may additionally wrap this state with further state, if this is necessary for e.g. alignment.

duplicate(x)

Return an object that is equal to x, but fully independent of it.

Note that for any parts of x that TimeDag considers to be immutable (e.g. Blocks), this can return the identical object.

Conceptually this is otherwise very similar to deepcopy(x).

Internal implementation of duplicate.

By default delegates to deepcopy, but this can be avoided where it is known to be unnecessary.

ema(x::Node, α::AbstractFloat) -> Node
ema(x::Node, w_eff::Integer) -> Node

Create a node which computes the exponential moving average of x.

The decay is specified either by α, which should satisfy 0 < α < 1, or by w_eff, which should be an integer greater than 1. If the latter is specified, then we compute α = 2 / (w_eff + 1).

For internal state $s_t$, with $s_0 = 0$, and resulting EMA series $m_t$, this has the form:

\[\begin{aligned} s_t &= s_{t-1} + (1 - \alpha) x_t \\ m_t &= \frac{\alpha s_t}{1 - (1 - \alpha)^t}. \end{aligned}\]

For further information, see the notational conventions and discussion on Wikipedia. Note that this function implements the variant including the correction for the initial convergence problem.

empty_node(T)

Construct a node with value type T which, if evaluated, will never tick.

equivalence_classes(f, collection) -> Vector{Vector{eltype(collection)}}

Generate the set of equivalence classes for collection generated by the equivalence relation f : T × T → Bool.

Note that behaviour is undefined if f is non-transitive.

evaluate(nodes::AbstractVector{Node}, t0, t1[; batch_interval]) -> Vector{Block}
evaluate(node::Node, t0, t1[; batch_interval]) -> Block

Evaluate the specified node(s) over the specified time range [t0, t1), and return the corresponding Block(s).

If nodes have common dependencies, work will not be repeated when performing this evaluation.

evaluate_until!(state::EvaluationState, time_end::DateTime)

Update the evaluation state by performing the evalution for each node.

first_knot(x::Node{T}) -> Node{T}

Get a node which ticks with only the first knot of x, and then never ticks again.

global_identity_map() -> IdentityMap

Get the global IdentityMap instance used in TimeDag.

has_initial_values(op) -> Bool

If this returns true, it indicates that initial values for the op's parents are specified.

See the documentation on Initial values for further information.

history(x::Node{T}, window::Int) -> Node{Vector{T}}

Create a node whose values represent the last window values seen in x.

Each value will be vector of length window, and the result will only start ticking once window values have been seen. The vector value contains time-ordered observations, with the most recent observation last.

initial_left(op::BinaryNodeOp) -> L

Specify the initial value to use for the first parent of the given op.

Needs to be defined if has_initial_values returns true, and alignment is UNION. For other alignments it is not required.

initial_right(op::BinaryNodeOp) -> R

Specify the initial value to use for the second parent of the given op.

Needs to be defined if has_initial_values(op) returns true, and alignment is UNION or LEFT. For INTERSECT alignment it is not required.

initial_values(op::NaryNodeOp) -> Tuple

Specify the initial values to use for all parents of the given op.

Needs to be defined for nary ops for which has_initial_values returns true.

iterdates(time_of_day::Time=Time(0), tz::TimeZone=tz"UTC", occurrence=1)

Create a node which ticks exactly once a day at time_of_day in timezone tz.

This defaults to midnight in UTC. If tz is set otherwise, then each knot will appear at time_of_day in that timezone.

Note that: * All knot times in TimeDag are considered to be in UTC. * It is possible to select a time_of_day that does not exist for every day. This will lead to an exception being raised during evaluation.

In a given knot, each value will be of type DateTime, and equal the time of the knot.

lag(x::Node, w::TimePeriod)

Construct a node which takes values from x, but lags them by period w.

lag(x::Node, n::Integer)

Construct a node which takes values from x, but lags them by n knots.

This means that we do not introduce any new timestamps that do not appear in x, however we will not emit knots for the first n values that appear when evaluating x.

left(x, y[, alignment::Alignment; initial_values=nothing]) -> Node

Construct a node that ticks according to alignment with the latest value of x.

It is "left", in the sense of picking the left-hand of the two arguments x and y.

obtain_node!(id_map::IdentityMap, parents::NTuple{N,Node}, op::NodeOp) -> Node

If a node with parents and op doesn't exist inside id_map, create and insert it.

Return either the new or existing node.

obtain_node(parents::NTuple{N,Node}, op::NodeOp) -> Node

Get a node for the given op and parents. If an equivalent node already exists in the global identity map, use that one, otherwise create a new node, add to the identity map, and return it.

Constant propagation

If all parents are constant nodes, and op has a well-defined operation on constant inputs, we will immediately perform the computation and return a constant node wrapping the computed value.

operator!(op::UnaryNodeOp{T}, (state,), (time,) x) -> T / Maybe{T}
operator!(op::BinaryNodeOp{T}, (state,), (time,) x, y) -> T / Maybe{T}
operator!(op::NaryNodeOp{N,T}, (state,), (time,) x, y, z...) -> T / Maybe{T}

Perform the operation for this node.

When defining a method of this for a new op, follow these rules:

• state should be omitted iff TimeDag.stateless_operator.
• time should be omitted iff TimeDag.time_agnostic.
• All values x, y, z... should be omittted iff TimeDag.value_agnostic.

For stateful operations, this operator should mutate state as required.

The return value out should be of type T iff TimeDag.always_ticks is true, otherwise it should be of type TimeDag.Maybe{T}.

If out <: Maybe{T}, and has !valid(out), this indicates that we do not wish to emit a knot at this time, and it will be skipped. Otherwise, value(out) will be used as the output value.

output_type(f, arg_types...)

Return the output type of the specified function. Tries to be fast where possible.

parents(node::Node) -> NTuple{N, Node} where {N}

Get immediate parents of the given node.

prepend(x, y) -> Node

Create a node that ticks with knots from x until y is active, and thereafter from y.

Note that the value_type of the returned node will be that of the promoted value types of x and y.

pulse(delta::TimePeriod[; epoch::DateTime])

Obtain a node which ticks every delta. Each value will equal the time of the knot.

Knots will be placed such that the difference between its time and epoch will always be an integer multiple of delta. By default epoch is set to the Julia DateTime epoch, which is DateTime(0, 12, 31).

right(x, y[, alignment::Alignment; initial_values=nothing]) -> Node

Construct a node that ticks according to alignment with the latest value of y.

It is "right", in the sense of picking the right-hand of the two arguments x and y.

run_node!(
    op::NodeOp{T},
    state::NodeEvaluationState,
    time_start::DateTime,
    time_end::DateTime,
    input_blocks::Block...
) -> Block{T}

Evaluate the given node from time_start until time_end, with the initial state. Zero or more blocks will be passed as an input; these correspond to the parents of a node, and are passed in the same order as that returned by parents(node).

We return a new Block of output knots from this node.

start_at(nodes, time_start::DateTime) -> EvaluationState

Create a sufficient EvaluationState for the evaluation of nodes.

Internally, this will determine the subgraph that needs evaluating, i.e. all the ancestors of nodes, and create a NodeEvaluationState for each of these.

stateless(node) -> Bool
stateless(op) -> Bool

Returns true iff op can be assumed to be stateless; that is, if the node evaluation state is TimeDag.EMPTY_NODE_STATE.

stateless_operator(node) -> Bool
stateless_operator(op) -> Bool

Returns true iff operator(op, ...) would never look at or modify the evaluation state.

If this returns true, create_operator_evaluation_state will not be used.

Note that if an op has stateless(op) returning true, then it necessarily should also return true here. The default implementation is to return stateless(op), meaning that if one is creating a node that is fully stateless, one need only define stateless.

For optimal performance, this should be knowable from the type of op alone.

tea_file(path::AbstractString, value_field_name)

Get a node that will read data from the tea file at path.

Such a tea file must observe the following properties, which will be verified at runtime:

• Have a primary time field which is compatible with a Julia DateTime.
• Have exactly one column with name value_field_name.
• Have strictly increasing times.

Upon node creation, the metadata section of the file will be parsed to infer the value type of the resulting node. However, the bulk of the data will only be read at evaluation time.

See also

• The tea file spec
• TeaFiles.jl

throttle(x::Node, n::Integer) -> Node

Return a node that only ticks every n knots.

The first knot encountered on the input will always be emitted.

time_agnostic(node) -> Bool
time_agnostic(op) -> Bool

Returns true iff op does not care about the time of the input knot(s).

For optimal performance, this should be knowable from the type of op alone.

unsafe_value(x::Maybe{T}) -> T

Returns the value stored in x.

It is "unsafe" when !valid(x), in that the return value of this function is undefined. If T is a reference type, calling this function will result in an UndefRefError being thrown.

valid(x::Maybe) -> Bool

Return true iff x holds a value.

value(x::Maybe{T}) -> T

Returns the value stored in x, or throws an ArgumentError if !valid(x).

Note that, in a tight loop, it is preferable to use a combination of calls to valid and unsafe_value, as it will generate more optimal code.

value_agnostic(node) -> Bool
value_agnostic(op) -> Bool

Returns true iff op does not care about the value(s) of the input knot(s).

For optimal performance, this should be knowable from the type of op alone.

value_type(node::Node{T}) -> T

The type of each value emitted for this node.

wrap(f::Function; time_agnostic=true)

Return a callable object that acts on nodes, and returns a node.

It is assumed that f is stateless (this will therefore not work with closures). We also assume that we will always emit a knot when the alignment semantics say we should — thus f must always return a valid output value.

Iff time_agnostic is false, f will be passed the time of the current knot as the first argument, followed by the value of every input.

If the object is called with more than one node, alignment will be performed. If an alignment other than the default should be used, provide it as the final argument.

Internally this will call TimeDag.apply(f, args...; kwargs...); see there for further details.

wrapb(f::Function; time_agnostic=true)

wrapb is like wrap, however f will be broadcasted over all input values.

@simple_binary(f)

Define methods f(x, y) that will obtain the correct instance of SimpleBinary{f}.

These will internally infer the output value, and perform subgraph elimination.

@simple_unary(f)

Define a method f(::Node) that will obtain the correct instance of SimpleUnary{f}.

This will internally infer the output value, and perform subgraph elimination.

@simple_unary_self_inverse(f)

Define a method f(::Node) that will obtain the correct instance of SimpleUnary{f}.

This must ONLY be used if f(f(x)) == x for all nodes x.

This will internally infer the output value, and perform subgraph elimination.