AddGlobalParameters(out_dim, loc; init_theta, activation)

Adds learnable global parameters θ to the input vector at indexes loc to create a vector of size out_dim.

Arguments:

• out_dim::Int: Length of the resulting output vector.
• loc::Int: Indexes of θ in the output vector.
• init_theta: Initialization function for θ from WeightInitializers.jl. Default = glorot_uniform.
• activation: Activation function to use on θ. Default = softplus.

Examples

julia> layer = AddGlobalParameters(4, [2, 4])
AddGlobalParameters()  # 2 parameters, plus 16 non-trainable
 
julia> layer([1; 2;;], ps, st)[1] # returns y, st
4×1 Matrix{Float32}:
 1.0
 θ₁
 2.0
 θ₂

Additive

Additive error model following y = f(x) + ϵ

Arguments

• init_f: Function to initialize parameters. Default = init_sigma.

BasicIndividual(...)

Struct holding the data for a single subject for standard PK or PD analyses.

BasicIndividual(x, t, y, callback; id, initial, eta)

Arguments

• x: Subject specific covariates. If Matrix is passed, predictions can change over time.
• t: Time points of observations. If the subject is time-variable, a NamedTuple (x = [...], y = [...]) should be passed.
• y::AbstractVector: Observations. Must be a vector. Only supports a single DV.
• callback: Differential equation callback containing treatment interventions.
• id: Patient id to store in the Individual instance.
• initial: Initial value of the dependent value at t = 0. Default = [].
• eta: Random effect estimate for the individual. Default = [].

Combine([out_dim], pairs...)

Connects specific branches to specific outputs in a Lux model containing a BranchLayer. Output from branches pointing to the same output are combined using a product.

Arguments:

• pairs...: Pairs denoting what branch to connected to what output(s). Pairs take the following format: i => [j] or [j, k, ...]

Examples

julia> layer = Combine(1 => [1], 2 => [1], 3 => [2]) # Connects branch 1 to output 1, branch 2 to output 1, etc.
Combine()

Proportional

Proportional error model following y = f(x) + ϵ₁ + f(x) ⋅ ϵ₂

Arguments

• init_f: Function to initialize parameters. Default = init_sigma.

Custom

Custom error model. Requires to definition of a variance(::Custom, p, y) function describing the variance of the observations and initialization function for its parameters.

Arguments

• init_f: Function to initialize parameters. Default = init_sigma.

DeepCompartmentModel{O,D,M,P,R}

Model architecture originally described in [janssen2022]. Uses a Neural Network to learn the relationship between the covariates and the parameters of a system of differential equations, for example describing a compartment model.
[janssen2022] Janssen, Alexander, et al. "Deep compartment models: a deep learning approach for the reliable prediction of time‐series data in pharmacokinetic modeling." CPT: Pharmacometrics & Systems Pharmacology 11.7 (2022): 934-945.

DeepCompartmentModel(ode_f, num_compartments, args...; kwargs...)

Convenience constructor creating an ODE model based on the user supplied ode_f function. The number of compartments needs to be supplied in order to correctly initialize the ODEProblem.

Arguments

• ode_f::AbstractDEProblem: DE problem describing the dynamical system.
• num_compartments::Int: Number of partial differential equations in the ODE.

DeepCompartmentModel(problem, ann; kwargs...)

Convenience constructor also initializing the model parameters.

Arguments

• prob::AbstractDEProblem: DE problem describing the dynamical system.
• ann::AbstractExplicitLayer: Lux model representing the ann.

DeepCompartmentModel(prob, ann, p; rng, objective, dv_compartment)

Arguments

• prob::AbstractDEProblem: DE problem describing the dynamical system.
• ann::AbstractExplicitLayer: Lux model representing the ann.
• p: Model parameters, containing all model parameters.
• rng: Randomizer used for initialization of the parameters.
• objective::AbstractObjective: Objective function to optimize. Currently supports SSE, LogLikelihood, and VariationalELBO (for mixed effects estimation). Default = SSE.
• dv_compartment::Int: The index of the compartment for the prediction of the dependent variable. Default = 1.

LogLikelihood{D, E}()

LogLikelihood based objective function:

L(p) = p(y | p, σ)

Default uses a Normal / MultivariateNormal loglikelihood function (as represented by D). Different distributions can be passed, and custom parameters can be controlled using the Custom ErrorModel.

Normalize(lb, ub)

Performs min-max scaling of the input according to (x - lb) / (ub - lb). The length of lb and ub should match the input vector.

Arguments:

• lb: lower bound, default = zero(ub).
• ub: upper bound.

Population(AbstractIndividual[...])

Combines a vector of individuals into a Population. Makes sure all the Individuals are of the same type. If any subject has time-dependent effects, all Individuals are transformed to the time-variable format.

Proportional

Proportional error model following y = f(x) + f(x) ⋅ ϵ

Arguments

• init_f: Function to initialize parameters. Default = init_sigma.

SSE()

Sum of squared errors objective function:

L(p) = Σᵢ (yᵢ - f(xᵢ; p))²

StandardNeuralNetwork(...)

Standard neural network architecture that directly predicts the observations y based on covariates x, time point t, and dose d.

StandardNeuralNetwork(ann; objective, rng)

Arguments

• ann: Lux model representing the neural network architecture.
• objective: Objective fuction to optimize. Default = SSE.
• rng: Randomizer. Default = default_rng().

StandardNeuralNetwork(ann, ps, st; objective, rng)

Convenience constructor initializing the remaining model parameters with user initialized neural network weights ps and state st.

Arguments

• ann: Lux model representing the neural network architecture.
• ps: Initial parameters for the neural network.
• st: Initial state for the neural network.
• objective: Objective fuction to optimize. Default = SSE.
• rng: Randomizer. Default = default_rng().

DCM(args...; kwargs...)

Alias for DeepCompartmentModel(args...; kwargs...)

Individual(...)

Alias for constructing a BasicIndividual.

LowDimensionalNODE(dudt, ann, node; kwargs...)

Constructor for describing low-dimensional Neural-ODE based models. See [bram2023] for examples of ways of defining dudt.

Arguments:

• dudt: Function describing the system of partial differential equations.

Should take the form of f(u, p, t; model) = .... p.weights contains the parameters for the node, p.I the drug infusions at time t, and p.z the parameters produced by the ann.

• ann: Neural network for predicting V according to f(x) = V.
• node: Neural-ODE model.
• kwargs: keyword arguments given to the HybridDCM constructor.

[bram2023] Bräm, Dominic Stefan, et al. "Low-dimensional neural ODEs and their application in pharmacokinetics." Journal of Pharmacokinetics and Pharmacodynamics (2023): 1-18.

LowDimensionalNODE(ann, node; kwargs...)

Convenience function for constructing low-dimensional Neural-ODE based models. This simple implementation learns a model according to:

du/dt = node([u; t]) + I / V

Where the model learns the system dynamics based on the previous state u and current time point t. Rate of drug infusions I is divided by a subject-specific volume of distribution estimate V, which is learned by another neural network. See [bram2023] for more details.

Arguments:

• ann: Neural network for predicting V according to f(x) = V.
• node: Neural-ODE model.
• kwargs: keyword arguments given to the HybridDCM constructor.

[bram2023] Bräm, Dominic Stefan, et al. "Low-dimensional neural ODEs and their application in pharmacokinetics." Journal of Pharmacokinetics and Pharmacodynamics (2023): 1-18.

MultiHeadedBranch(covariate_idx, neurons, heads; activation, init_bias).

Convenience function for creating branches in Lux models. Constructs a neural network containing a single hidden layer connecting to multiple independent layers in parallel specified by heads. This way, the covariate effect shares a similar base, but can still learn independent effects for each head. All hidden layers contain the same number of neurons. It is advisable to use at most 2 covariates as input to the model to facilitate interpretation.

Arguments:

• covariate_idx::Union{AbstractVector{<:Int}, Int}: Index of the covariate(s) to use in the branch.
• neurons::Int: The amount of neurons in the hidden layer.
• heads::Int: The number of independent heads.
• activation: Activation function to use in the hidden layer. Default = swish.
• init_bias: Initialization function of the bias parameters. Default = ones32. Can help with improving the initial estimates.

SNN(...)

Alias for StandardNeuralNetwork.

SingleHeadedBranch(covariate_idx, neurons; activation, init_bias).

Convenience function for creating branches in Lux models. Constructs a single hidden layer neural network with the specified number of neurons. It is advisable to use at most 2 covariates as input to the model to facilitate interpretation.

Arguments:

• covariate_idx::Union{AbstractVector{<:Int}, Int}: Index of the covariate(s) to use in the branch.
• neurons::Int: The amount of neurons in the hidden layer.
• activation: Activation function to use in the hidden layer. Default = swish.
• init_bias: Initialization function of the bias parameters. Default = ones32. Can help with improving the initial estimates.

constrain(p::NamedTuple)

Transforms the unconstrained parameter vector to constrained space.

Examples

σ* ∈ ℝ → softplus(σ*) ∈ ℝ⁺
ω, C → ω ⋅ C ⋅ ω'

constrain_phi(::FullRank, 𝜙::NamedTuple)

Transforms unconstrained 𝜙 to constrained space. For a FullRank approximation this function returns μ and the lower cholesky factor L.

constrain_phi(::MeanField, 𝜙::NamedTuple)

Transforms unconstrained 𝜙 to constrained space. For a MeanField approximation this function returns μ and standard deviations σ.

fit!(model::AbstractModel, population::Population, opt, epochs; callback)

Fits the model in place to the data from the population. Updated model parameters are stored in the model. Can be passed a callback function that can be used to monitor training. The callback is called with the current epoch and loss after gradient calculation and before updating model parameters.

forward(model::DeepCompartmentModel, container, p; full, interpolate, get_dv, saveat, sensealg)

Predicts the differential equation parameters and returns the solution.

Arguments

• model::DeepCompartmentModel: The model to use to perform the prediction.
• container::Union{AbstractIndividual, Population}: A population or Individual to perform the predictions for.
• p: Model parameters. Default = model.p.
• full: Return the output for all model compartments.
• interpolate: Saves additional time points to return a continuous solution of the DE.
• get_dv: Directly returns the predictions for the dv compartment.
• saveat: Custom time points to save the solution.
• sensealg: Sensitivity algorithm to use for gradient calculations.

generate_dosing_callback(I; S1)

Returns a DiscreteCallback implementing the dosing events in intervention matrix I.

Arguments

• I: Matrix with rows containing events with time, dose, rate, and duration columns.
• S1: Scaling factor for the doses. Used to get the dose in the same unit as model parameters. Default = 1.

init_omega(rng, n)

Initialization function for the MultivariateNormal prior on random effect parameters.

init_sigma(rng, error; init_dist)

Initialization function for σ parameters based on the error model.

Arguments

• rng::AbstractRNG: Randomizer to use.
• error: Error model. One of Additive, Proportional, Combined, or Custom.
• init_dist::Sampleable: Distribution from which to sample the initial σ. Default = Uniform(0, 1)

interpret_branch(model, covariate_idx, anchor; x)

Convenience function for running the interpretation function on models following the AbstractModel interface.

Arguments:

• model::AbstractModel: AbstractModel (e.g. DeepCompartmentModel(...)) using an Multi-branch based neural network.
• covariate_idx: Index of the covariate in the input vector.
• anchor: Unnormalized covariate value to which the output is "anchored", i.e. f(anchor) = 1.
• x: Normalized dummy input to the branch.

interpret_branch(model, covariate_idx, anchor, ps, st; x)

Returns the output of a specific branch based on dummy input. Can be used to interpret covariate effects. Returns the unnormalized x and the branch output y divided by the prediction at the location of anchor:

y = f(x) ./ f(anchor)

Arguments:

• ann: Lux model.
• ps: Model parameters.
• st: Model state.
• covariate_idx: Index of the covariate in the input vector.
• anchor: Unnormalized covariate value to which the output is "anchored", i.e. f(anchor) = 1.
• x: Normalized dummy input to the branch.

is_timevariable(individual)

Returns whether the individual has time variable effects.

make_branch(covariate_idx, layers...)

Internal function used to create branches. Allows one to create more specific branch layers. Appends the layers to the following Lux.Chain:

Chain(
    SelectDim(1, covariate_idx),
    ReshapeLayer((1,)),
    layers...
)

Arguments:

• covariate_idx::Union{AbstractVector{<:Int}, Int}: Index of the covariate(s) to use in the branch.
• layers: Layers used to build the neural network.

Expects that forward returns a one-dimensional array

Expects that forward returns a one-dimensional array

one_comp!(dA, A, p, t)

In-place function for the one-compartment model: dA/dt = (I / Vd) - A₁ ⋅ k₁₀

Model parameters

• CL: Drug clearance estimate.
• Vd: Drug volume of distribution estimate.
• I: Rate of drug infusion; handled by callback function.

Inferred parameters

• k₁₀: CL / Vd

one_comp_abs!(dA, A, p, t)

In-place function for the one-compartment model with an absorption compartment:

dA₁/dt = I - kₐ ⋅ A₁
dA₂/dt = kₐ ⋅ A₁ - A₂ ⋅ k₁₀

Model parameters

• kₐ: Rate of drug absorption.
• k₁₀: Rate of drug elimination.
• I: Rate of drug infusion; handled by callback function.

predict(...)

Alias for the forward function.

softplus_inv(x::Real)

Returns the inverse of the softplus function such that:
y = softplus(x)
x = softplus_inv(y)

two_comp!(dA, A, p, t)

In-place function for the two-compartment model:

dA₁/dt = (I / V₁) + A₂ ⋅ k₂₁ - A₁ ⋅ (k₁₀ + k₁₂)
dA₂/dt = A₁ ⋅ k₁₂ - A₂ ⋅ k₂₁

Model Parameters

• CL: Drug clearance from the first compartment.
• V₁: Central volume of distribution (in A₁).
• Q: Inter-compartmental clearance.
• V₂: Peripheral volume of distribution (in A₂).
• I: Rate of drug infusion; handled by callback function.

Inferred parameters

• k₁₀: CL / V₁
• k₁₂: Q / V₁
• k₂₁: Q / V₂

two_comp_abs!(dA, A, p, t)

In-place function for the two-compartment model with an absorption compartment:

dA₁/dt = I - kₐ ⋅ A₁
dA₂/dt = kₐ ⋅ A₁ + A₃ ⋅ k₃₂ - A₂ ⋅ (k₂₀ + k₂₃)
dA₃/dt = A₂ ⋅ k₂₃ - A₃ ⋅ k₃₂

Model parameters

• kₐ: Rate of drug absorption.
• k₂₀: Rate of drug elimination.
• k₂₃: Rate of drug moving from compartment 2 to 3.
• k₃₂: Rate of drug moving from compartment 3 to 2.
• I: Rate of drug infusion; handled by callback function.

unpack(p::AbstractMatrix, t::Real)

Enables the use of parameters that change over time. Collects the column of the parameter matrix that matches the current intergrator time point.

unpack(p::AbstractVector, t::Real)

For Vector-based p this function just returns p.

update!(model::AbstractModel, p::Tuple)

Performs an in-place update of model parameters when P <: NamedTuple.

variance(model, p, y)

Returns the variance of y based on the model.

Arguments:

• model::AbstractModel: The model.
• p: Constrained model parameters.
• y: Observations or predictions for which to calculate the variance.

std(model::AbstractModel, prediction::AbstractVector)

Returns the standard deviation of predictions based on the model.

Arguments:

• model::AbstractModel: The model.
• y: Observations or predictions for which to calculate standard deviations.