Layer Types

Layer Types

Currently, a basic Merger layer and a Splitter layer are implemented. In addition, there are several convenience layers for the preimplemented models.



An element in a FeedbackChain in which the forward stream and feedback stream are combined according to an operation op.


  • splitname::String: name of the Splitter node from which the feedback is taken
  • fb::F: feedback branch
  • op::O: operation to combine forward and feedback branches


fb typically takes the form of a Flux operation or chain. When a FeedbackChain encounters a Merger, it will look up the state s of the Splitter given by forkname from the previous timestep, apply fb to it and combine it with the forward input x according to op(x, fb(s))


Return the name of the Splitter from which m gets its input.



An element in a FeedbackChain that marks locations where a feedback branch forks off from the forward branch.


  • name::String: unique name used to identify the fork for the backward pass.


In the forward stream, a is essentially an identity operation. It only alerts the FeedbackChain to add the current Array to the chain's state and mark it with the Splitters name so that Mergers can access it for feedback to the next timestep.


Return name of s.

Other layers

The preimplemented models use a flattening layer and local response normalization.


Turns a high-dimensional array (e.g., a batch of feature maps) into a 2-d array, linearizing all except the last (batch) dimension.

Implementation of local response normalization.


Local response normalization layer. Input i is processed according to out(x,y,f,b) = x * [b + α * sum( i(x, y, f-k÷2:f+k÷2, b)^2 )]^(-β)

Todo: β is currently ignored (always set to 0.5)


Applies a local response normalization layer according to: out(x,y,f,b) = x * [c + α * sum( i(x, y, f-k÷2:f+k÷2, b)^2 )]^(-β)

Todo: β is currently ignored (always set to 0.5)