FluxArchitectures

Documentation for FluxArchitectures.

Installation

Download Julia 1.6 or later, if you haven't already. You can add FluxArchitectures from Julia's package manager, by typing 

] add FluxArchitectures

in the Julia prompt.

Models

  • LSTnet: This "Long- and Short-term Time-series network" follows the paper by Lai et. al..

  • DARNN: The "Dual-Stage Attention-Based Recurrent Neural Network for Time Series Prediction" is based on the paper by Qin et. al..

  • TPA-LSTM: The Temporal Pattern Attention LSTM network is based on the paper "Temporal Pattern Attention for Multivariate Time Series Forecasting" by Shih et. al..

  • DSANet: The "Dual Self-Attention Network for Multivariate Time Series Forecasting" is based on the paper by Siteng Huang et. al.

Quickstart

Activate the package and load some sample-data:

using FluxArchitectures
poollength = 10; horizon = 6; datalength = 1000;
input, target = get_data(:exchange_rate, poollength, datalength, horizon)

Define a model and a loss function:

model = LSTnet(size(input, 1), 2, 3, poollength, 120)
loss(x, y) = Flux.mse(model(x), y')

Train the model:

Flux.train!(loss, Flux.params(model),Iterators.repeated((input, target), 20), ADAM(0.01))

GPU Calculations

Finally, GPU support for all models has been added! However, it cannot be guaranteed that this is done in an optimal way. Currently, only DSANet sees an improvement in training speed on my hardware, whereas all other models train faster on a CPU. The deeper reason is the current limitation of Flux's implementation of RNNs, see this issue that affects all models except DSANet, which doesn't use recurrence.