cpu(x)

Transfer x to CPU

gpu(x)

Transfer x to GPU

glorot_normal(rng::AbstractRNG, size...; gain = 1)

Return an Array{Float32} of the given size containing random numbers drawn from a normal distribution with standard deviation gain * sqrt(2 / (fan_in + fan_out)). This method is described in [1] and also known as Xavier initialization.

References

[1] Glorot, Xavier, and Yoshua Bengio. "Understanding the difficulty of training deep feedforward neural networks." Proceedings of the thirteenth international conference on artificial intelligence and statistics. 2010.

glorot_uniform(rng::AbstractRNG, size...; gain = 1)

Return an Array{Float32} of the given size containing random numbers drawn from a uniform distribution on the interval $[-x, x]$, where x = gain * sqrt(6 / (fan_in + fan_out)). This method is described in [1] and also known as Xavier initialization.

References

[1] Glorot, Xavier, and Yoshua Bengio. "Understanding the difficulty of training deep feedforward neural networks." Proceedings of the thirteenth international conference on artificial intelligence and statistics. 2010.

ones32(rng::AbstractRNG, size...) = ones(Float32, size...)

Return an Array{Float32} of ones of the given size. (rng is ignored)

zeros32(rng::AbstractRNG, size...) = zeros(Float32, size...)

Return an Array{Float32} of zeros of the given size. (rng is ignored)

applyactivation(f::Function, x::AbstractArray)

Apply the function f on x elementwise, i.e. f.(x). Dispatches to CUDNN if possible.

elementwise_add(x, y)

Computes x .+ y. Dispatches to CUDNN if possible

elementwise_mul(x, y)

Computes x .* y. Dispatches to CUDNN if possible

istraining(::Val{training})
istraining(st::NamedTuple)

Returns true if training is true or if st contains a training field with value true. Else returns false.

Method undefined if st.training is not of type Val.

multigate(x::AbstractArray, ::Val{N})

Split up x into N equally sized chunks (along dimension 1).

replicate(rng::AbstractRNG)
replicate(rng::CUDA.RNG)

Creates a copy of the rng state depending on its type.