AWSBatch

run_batch(;
    name::AbstractString="",
    queue::AbstractString="",
    region::AbstractString="",
    definition::Union{AbstractString, JobDefinition, Nothing}=nothing,
    image::AbstractString="",
    vcpus::Integer=1,
    memory::Integer=-1,
    role::AbstractString="",
    cmd::Cmd=``,
    num_jobs::Integer=1,
    parameters::Dict{String, String}=Dict{String, String}(),
) -> BatchJob

Handles submitting a BatchJob based on various potential defaults. For example, default job fields can be inferred from an existing job definition or an existing job (if currently running in a batch job).

Order of priority from highest to lowest:

1. Explict arguments passed in via kwargs.
2. Inferred environment (e.g., AWS_BATCH_JOB_ID environment variable set)
3. Job definition parameters

If no valid job definition exists (see AWSBatch.job_definition_arn then a new job definition will be created and registered based on the job parameters.

BatchJob

Stores a batch job id in order to:

• describe a job and its parameters
• check on the status of a job
• wait for a job to complete
• fetch log_events

Fields

• id::AbstractString: jobId

submit(
    name::AbstractString,
    definition::JobDefinition,
    queue::JobQueue;
    container::AbstractDict=Dict(),
    parameters::Dict{String,String}=Dict{String, String}(),
    num_jobs::Integer=1,
) -> BatchJob

Handles submitting the batch job. Returns a BatchJob wrapper for the id.

describe(job::BatchJob) -> Dict

Provides details about the AWS batch job.

JobDefinition

Returns the job definition corresponding to a batch job.

status(job::BatchJob) -> JobState

Returns the current status of a job.

wait(
    cond::Function,
    job::BatchJob;
    timeout=600,
    delay=5
)

Polls the batch job state until it hits one of the conditions in cond. The loop will exit if it hits a failure condition and will not catch any excpetions. The polling interval can be controlled with delay and timeout provides a maximum polling time.

Examples

julia> wait(state -> state < SUCCEEDED, job)
true

wait(
    job::BatchJob,
    cond::Vector{JobState}=[RUNNING, SUCCEEDED],
    failure::Vector{JobState}=[FAILED];
    kwargs...,
)

Polls the batch job state until it hits one of the conditions in cond. The loop will exit if it hits a failure condition and will not catch any excpetions. The polling interval can be controlled with delay and timeout provides a maximum polling time.

log_events(job::BatchJob) -> Union{Vector{LogEvent}, Nothing}

Fetches the logStreamName, fetches the CloudWatch logs, and returns a vector of log events. If the log stream does not currently exist then nothing is returned.

NOTES:

• The logStreamName isn't available until the job is RUNNING, so you may want to use wait(job) or wait(job, [AWSBatch.SUCCEEDED]) prior to calling this function.

JobDefinition

Stores the job definition arn including the revision.

ComputeEnvironment

An object representing an AWS batch compute environment.

See AWSBatch.create_compute_environment.

create_compute_environment(name;
                           managed=true,
                           role="",
                           resources=Dict(),
                           enabled=true,
                           tags=Dict(),
                           aws_config=global_aws_config())

Create a compute environment of type type with name name.

See the AWS docs here.

list_job_definitions(;aws_config=global_aws_config())

Get a list of JobDefinition objects via Batch.decsribe_job_definitions().

job_definition_arn(
    definition_name::AbstractString;
    image::AbstractString="",
    role::AbstractString="",
    aws_config::AbstractAWSConfig=global_aws_config(),
) -> Union{AbstractString, Nothing}

Looks up the ARN (Amazon Resource Name) for the latest job definition that can be reused. Returns a JobDefinition with the ARN that can be reused or nothing.

A job definition can only be reused if:

1. status = ACTIVE
2. type = container
3. image = the current job's image
4. jobRoleArn = the current job's role

register(
    definition_name::AbstractString;
    role::AbstractString="",
    image::AbstractString="",
    vcpus::Integer=1,
    memory::Integer=1024,
    cmd::Cmd=``,
    region::AbstractString="",
    parameters::Dict{String,String}=Dict{String, String}(),
) -> JobDefinition

Registers a new job definition.

deregister(job::JobDefinition)

Deregisters an AWS Batch job.

isregistered(definition::JobDefinition; aws_config=global_aws_config()) -> Bool

Checks if a JobDefinition is registered.

describe(definition::JobDefinition; aws_config=global_aws_config()) -> Dict

Describes a job definition as a dictionary. Requires the IAM permissions "batch:DescribeJobDefinitions".

JobQueue

An object representing and AWS batch job queue.

See AWSBatch.create_job_queue.

create_job_queue(name, envs, priority=1; aws_config=global_aws_config())

Create a job queue with name name and priority priority returning the associated JobQueue object. envs must be an iterator of compute environments given by ARN.

See the AWS docs here.

list_job_queues(;aws_config=global_aws_config())

Get a list of JobQueue objects as returned by Batch.describe_job_queues().

JobState

An enum for representing different possible AWS Batch job states.

See docs for details.

LogEvent

A struct for representing an event in an AWS Batch job log.