construct_diff_version(f, g) -> pass_trough_function

Constructs a pass trough function for the given function f and gradient function g. The pass trough function is a function that returns the same value as f but the gradient is taken from g. Optional parameter k controls the steepnes of the gradient, default is 1. Supports both scalars and arrays.

heaviside(Number, [steepnes]) -> Number

Implements the Heaviside function in the forward pass. It is 1 if x > 0 and 0 otherwise. The derivative is taken to be the sigmoid. An extra parameter k controls the steepnes of the sigmoid, default is 1.

ignore_gradient(x) -> x

Drops the gradient in any computation. In reverse mode it uses ChainRulesCore API. In forward mode it simply returns a new number with a zero dual.

sign_diff(Number, [steepnes]) -> Number

Implements the sign function in the forward pass. It is 1 if x > 0, -1 if x < 0 and 0 if x = 0. The derivative is taken to be the tanh. An extra parameter k controls the steepnes of the sigmoid, default is 1.