ReinforcementLearning.jl, as the name says, is a package for reinforcement learning research in Julia.
Our design principles are:
- Reusability and extensibility: Provide elaborately designed components and interfaces to help users implement new algorithms.
- Easy experimentation: Make it easy for new users to run benchmark experiments, compare different algorithms, evaluate and diagnose agents.
- Reproducibility: Facilitate reproducibility from traditional tabular methods to modern deep reinforcement learning algorithms.
Project Structure
ReinforcementLearning.jl itself is just a wrapper around several other packages inside the JuliaReinforcementLearning org. The relationship between different packages is described below:
+-------------------------------------------------------------------------------------------+ | | | ReinforcementLearning.jl | | | | +------------------------------+ | | | ReinforcementLearningBase.jl | | | +--------|---------------------+ | | | | | | +--------------------------------------+ | | | | ReinforcementLearningEnvironments.jl | | | | | | | | | | (Conditionally depends on) | | | | | | | | | | ArcadeLearningEnvironment.jl | | | +-------->+ OpenSpiel.jl | | | | | POMDPs.jl | | | | | PyCall.jl | | | | | ViZDoom.jl | | | | | Maze.jl(WIP) | | | | +--------------------------------------+ | | | | | | +------------------------------+ | | +-------->+ ReinforcementLearningCore.jl | | | +--------|---------------------+ | | | | | | +-----------------------------+ | | |--------->+ ReinforcementLearningZoo.jl | | | | +-----------------------------+ | | | | | | +----------------------------------------+ | | +--------->+ ReinforcementLearningAnIntroduction.jl | | | +----------------------------------------+ | +-------------------------------------------------------------------------------------------+
Installation
This package can be installed from the package manager in Julia's REPL:
] add ReinforcementLearning
A Quick Example
using ReinforcementLearning
using StatsBase:mean
env = CartPoleEnv(;T=Float32, seed=123)
agent = Agent(
policy = RandomPolicy(env;seed=456),
trajectory = CircularCompactSARTSATrajectory(; capacity=3, state_type=Float32, state_size = (4,)),
)
hook = ComposedHook(TotalRewardPerEpisode(), TimePerStep())
run(agent, env, StopAfterEpisode(10_000), hook)
@info "stats for random policy" avg_reward = mean(hook[1].rewards) avg_fps = 1 / mean(hook[2].times)
# ┌ Info: stats for random policy
# │ avg_reward = 21.0591
# └ avg_fps = 1.6062450808744398e6
See also here for detailed explanation.