premade_agents::Dict{String,Function}

Global dictionary constant that contains premade agents. This is updated by other packages that add new premade agents.

Type for shared parameters containing both the parameter value and a vector of parameter names that will share that value

Type to use for specifying a paramter that sets a state's initial value

Type for specifying a group of parameters

Custom error type which will result in rejection of a sample

binary_rescorla_wagner_softmax(agent::Agent, input::Bool)

Action model that learns from binary inputs with a classic Rescorla-Wagner model. Passes learnt probabilities through a softmax to get the action prpbability distribution.

Parameters: "learningrate" and "actionprecision". States: "value", "valueprobability", "actionprobability".

Function for checking the structure of the agent

create_agent_model(agent,param_priors,inputs,actions,impute_missing_actions)

Create a Turing model object used for fitting an ActionModels agent.

"" fitmodel(agent::Agent, inputs::Array, actions::Vector, parampriors::Dict, kwargs...) Use Turing to fit the parameters of an agent to a set of inputs and corresponding actions.

Arguments

• 'agent::Agent': an ActionModels agent object created with either premadeagent or initagent.
• 'param_priors::Dict': dictionary containing priors (as Distribution objects) for fitted parameters. Keys are parameter names, values are priors.
• 'inputs:Array': array of inputs. Each row is a timestep, and each column is a single input value.
• 'actions::Array': array of actions. Each row is a timestep, and each column is a single action.
• 'fixed_parameters::Dict = Dict()': dictionary containing parameter values for parameters that are not fitted. Keys are parameter names, values are priors. For parameters not specified here and without priors, the parameter values of the agent are used instead.
• 'sampler::Union{DynamicPPL.AbstractSampler, Turing.Inference.InferenceAlgorithm}: specify the type of Turing sampler, defaults to NUTS(-1, 0.65; adtype=AutoReverseDiff(true))
• 'n_cores = 1': set number of cores to use for parallelization. If set to 1, no parallelization is used.
• 'n_iterations = 1000': set number of iterations per chain.
• 'n_chains = 2': set number of amount of chains.
• 'verbose = true': set to false to hide warnings.
• 'showsamplerejections = false': set whether to show warnings whenever samples are rejected.
• 'imputemissingactions = false': set whether the values of missing actions should also be estimated by Turing.
• 'sampler_kwargs...': additional keyword arguments to be passed to the sampler` call.

Examples

#Create a premade agent: binary Rescorla-Wagner
agent = premade_agent("premade_binary_rescorla_wagner_softmax")
#Set priors for the learning rate
param_priors = Dict("learning_rate" => Uniform(0, 1))
#Set inputs and actions
inputs = [1, 0, 1]
actions = [1, 1, 0]
#Fit the model
fit_model(agent, param_priors, inputs, actions, n_chains = 1, n_iterations = 10)

"" fit_model(agent::Agent, inputs::Array, actions::Vector, priors::Dict, kwargs...)

Use Turing to fit the parameters of an agent to a set of inputs and corresponding actions.

Arguments

• 'agent::Agent': an ActionModels agent object created with either premadeagent or initagent.
• 'priors::Dict': dictionary containing priors (as Distribution objects) for fitted parameters. Keys are parameter names, values are priors.
• 'inputs:Array': array of inputs. Each row is a timestep, and each column is a single input value.
• 'actions::Array': array of actions. Each row is a timestep, and each column is a single action.
• 'fixed_parameters::Dict = Dict()': dictionary containing parameter values for parameters that are not fitted. Keys are parameter names, values are priors. For parameters not specified here and without priors, the parameter values of the agent are used instead.
• 'sampler::Union{DynamicPPL.AbstractSampler, Turing.Inference.InferenceAlgorithm}': specify the type of Turing sampler. Defaults to NUTS(-1, 0.65; adtype=AutoReverseDiff(true))
• 'n_cores = 1': set number of cores to use for parallelization. If set to 1, no parallelization is used.
• 'n_iterations = 1000': set number of iterations per chain.
• 'n_chains = 2': set number of amount of chains.
• 'verbose = true': set to false to hide warnings.
• 'showsamplerejections = false': set whether to show warnings whenever samples are rejected.
• 'imputemissingactions = false': set whether the values of missing actions should also be estimated by Turing.
• 'sampler_kwargs...': additional keyword arguments to be passed to the sampler` call.

Examples

#Create a premade agent: binary Rescorla-Wagner
agent = premade_agent("premade_binary_rescorla_wagner_softmax")

#Set priors for the learning rate
priors = Dict("learning_rate" => Uniform(0, 1))

#Set inputs and actions
inputs = [1, 0, 1]
actions = [1, 1, 0]

#Fit the model
fit_model(agent, priors, inputs, actions, n_chains = 1, n_iterations = 10)

get_history(agent::Agent, target_state::Union{String,Tuple})

Get the history of a single state from an agent. Returns a vector.

get_history(agent::Agent, target_states::Vector)

Get set of a vector of states from an agent. Returns a dictionary of states and their histories.

get_history(agent::Agent)

Get histories for states from an agent. Returns a dictionary of states and their histories.

get_parameters(agent::Agent, target_param::Union{String,Tuple})

Get a single parameter from an agent. Returns a single value.

get_parameters(agent::Agent, target_param::Vector)

Get a set of parameter values from an agent. Returns a dictionary of parameters and their values.

get_parameters(agent::Agent)

Get all parameters from an agent. Returns a dictionary of parameters and their values.

get_posteriors(chain::Chains; type::String = "median")

Extract parameters from a Turing Chain object. Returns a dictionary of parameters and their posteriors. 'type' can be set to either 'median', in which case median values are extracted, or 'distribution', in which case full posterior distributions are extracted.

get_states(agent::Agent, target_state::Union{String,Tuple})

Get a single state from an agent. Returns a single value.

get_states(agent::Agent, target_state::Vector)

Get a set of state values from an agent. Returns a dictionary of state names and their values.

get_states(agent::Agent)

Get all states from an agent. Returns a dictionary of state names and their values.

give_inputs!(agent::Agent, inputs)

Give inputs to an agent. Input can be a single value, a vector of values, or an array of values. Returns the agent's action trajectory, without the initial state.

init_agent(action_model::Function; substruct::Any = nothing, parameters::Dict = Dict(), states::Union{Dict, Vector} = Dict(),
settings::Dict = Dict(), parameter_groups::Dict = Dict())

Initialize an agent.

Note that actionmodel can also be specified as a vector of action models: actionmodel::Vector{Function}. In this case the action models will be stored in the agent's settings. In that case use the function 'multiple_actions'

Arguments

• 'action_model::Function': a function specifying the agent's action model. Can be any function that takes an agent and a single input as arguments, and returns a probability distribution from which actions are sampled.
• 'substruct::Any = nothing': struct containing additional parameters and states. This structure also get passed to utility functions. Check advanced usage guide.
• 'parameters::Dict = Dict()': dictionary containing parameters of the agent. Keys are parameter names (strings, or tuples of strings), values are parameter values.
• 'states::Union{Dict, Vector} = Dict()': dictionary containing states of the agent. Keys are state names (strings, or tuples of strings), values are initial state values. Can also be a vector of state name strings.
• 'settings::Dict = Dict()': dictionary containing additional settings for the agent. Keys are setting names, values are setting values.
• 'parameter_groups::Dict = Dict()': dictionary containing shared parameters. Keys are the the name of the shared parameter, values are the value of the shared parameter followed by a vector of the parameters sharing that value.

Examples

```julia

Create agent with a binary Rescorla-Wagner action model

Create action model function

function binaryrescorlawagner_softmax(agent::Agent, input::Union{Bool,Integer})

#Read in parameters
learning_rate = agent.parameters["learning_rate"]
action_precision = agent.parameters["action_precision"]

#Read in states
old_value = agent.states["value"]

#Sigmoid transform the value
old_value_probability = 1 / (1 + exp(-old_value))

#Get new value state
new_value = old_value + learning_rate * (input - old_value_probability)

#Pass through softmax to get action probability
action_probability = 1 / (1 + exp(-action_precision * new_value))

#Create Bernoulli normal distribution with mean of the target value and a standard deviation from parameters
action_distribution = Distributions.Bernoulli(action_probability)

#Update states
agent.states["value"] = new_value
agent.states["value_probability"], 1 / (1 + exp(-new_value))
agent.states["action_probability"], action_probability
#Add to history
push!(agent.history["value"], new_value)
push!(agent.history["value_probability"], 1 / (1 + exp(-new_value)))
push!(agent.history["action_probability"], action_probability)

return action_distribution

end

#Define requried parameters parameters = Dict( "learningrate" => 1, "actionprecision" => 1, ("initial", "value") => 0, )

#Define required states states = Dict( "value" => missing, "valueprobability" => missing, "actionprobability" => missing, )

#Create agent agent = initagent( binaryrescorlawagnersoftmax, parameters = parameters, states = states, settings = settings, )

multiple_actions(agent::Agent, input::Any)

Action model that combines multiple action models stored in the agent.

plot_parameter_distribution(fitted_model, param_priors;
    subplot_titles = Dict(),
    show_distributions = true,
    show_intervals = true,
    prior_color = :green,
    posterior_color = :orange,
    prior_interval_offset = 0,
    posterior_interval_offset = 0.01,
    inner_interval = 0.5,
    outer_interval = 0.8,
    plot_width = 900,
    plot_height = 300)

Plot the prior and posterior distributions of the parameters of a fitted model.

Arguments

• 'subplot_titles': A dictionary of parameter names and their corresponding plot titles.
• 'show_distributions': Whether to show full distributions.
• 'show_intervals': Whether to show uncertainty intervals.
• 'prior_color': Color of the prior distribution.
• 'posterior_color': Color of the posterior distribution.
• 'priorintervaloffset': Offset of the prior interval bars.
• 'posteriorintervaloffset': Offset of the posterior interval bars.
• 'inner_interval': Size of the inner uncertainty interval.
• 'outer_interval': Size of the outer uncertainty interval.
• 'plot_width': Width of the plot.
• 'plot_height': Height of the plot.

plotpredictivesimulation(paramdistributions::Union{Chains,Dict}, agent::Agent, inputs::Array, targetstate::Union{String,Tuple}; fixedparameters::Dict = Dict(), nsimulations::Int = 100, verbose::Bool = true, mediancolor::Union{String,Symbol} = :red, title::String = "Sampled trajectories", label::Union{String,Tuple} = targetstate, alpha::Real = 0.1, linewidth::Real = 2, )

Simulate distributions of states and actions for an agent, with parameters sampled from a specified distirbutions, and given a series of inputs.

Arguments

• 'param_distributions::Union{Chains,Dict}': Distributions to sample parameters from. Can be a dictionary containing keys and distributions, or a Turing Chain object containing the posterior distributions after fitting.
• 'agent::Agent': an ActionModels agent object created with either premadeagent or initagent.
• 'inputs:Array': array of inputs. Each row is a timestep, and each column is a single input value.
• 'target_state::Union{String,Tuple}': the state for which to plot the simulated distribution. If set to 'action', plot the action distribution. Note that the target state must be in the agent's history.
• 'fixed_parameters::Dict = Dict()': dictionary containing parameter values for parameters that are not fitted. Keys are parameter names, values are priors. For parameters not specified here and without priors, the parameter values of the agent are used instead.
• 'n_simulations::Int = 100': set number of simulations you want to run.
• 'verbose = true': set to false to hide warnings.
• 'median_color::Union{String,Symbol} = :red': specify color of median value in the plot.
• 'title::String = "Sampled trajectories"': title on graph.
• 'label::Union{String,Tuple} = target_state': label on graph.
• 'alpha::Real = 0.1': the transparency of each simulated trajectory line.
• 'linewidth::Real = 2': specify linewidth on your plot.

plot_trajector(agent::Agent, target_state::Union{String,Tuple}; kwargs...)

Plot trajectory of a state from an agent. Keyword arguments are passed to Plots.

premade_agent(model_name::String, config::Dict = Dict(); verbose::Bool = true)

Create an agent from the list of premade agents.

Arguments

• 'model_name::String': Name of the premade model. Returns a list of possible model names if set to 'help'.
• 'config::Dict = Dict()': A dictionary with configurations for the agent, like parameters and settings.
• 'verbose::Bool = true': If set to false, warnings are hidden.

reset!(agent::Agent)

Reset an agent to its initial state. Use initial state parameters when specified, and otherwise just use the first value in the state history.

set_parameters!(agent::Agent, target_param::Union{String,Tuple}, param_value::Any)

Setting a single parameter value for an agent.

set_parameters!(agent::Agent, parameter_values::Dict)

Set mutliple parameters values for an agent. Takes a dictionary of parameter names and values.

single_input!(agent::Agent, input::Any)

Give a single input to an Agent and let it evolve. Returns the agent's action.

warn_premade_defaults(defaults::Dict, config::Dict, prefix::String = "")

Check if any config values have not been set by the user, and warns them that it is using defaults.