CloudTest.Config

Convenience struct passed to the user-provided functions for Minio.with and Azurite.with that holds credentials and the store object automatically created when the services were started. Can be iterated to get the credentials and store, like credentials, bucket = conf.

_cmd(`cmd`)

This utility function ensures that we can run binaries in commands on Linux in Nix. Use it as a wrapper around an existing cmd.

Example:

julia> run(_cmd(`ls`))

CloudBase.Azure

Submodule that contains a custom HTTP.jl client for performing Azure requests. For authenticated requests, an Azure.Credentials object should be passed as the credentials keyword argument. Otherwise, the request methods operate just like the HTTP equivalents and supports all the same keyword arguments.

CloudBase.Azure.Container(name, account)

Object representation of an Azure storage bucket with the given name and account. Aliased in the CloudStore.jl package as Blobs.Container.

CloudBase.Azure.Credentials(; expireThreshold=Dates.Minute(5))
CloudBase.Azure.Credentials(account, shared_key)
CloudBase.Azure.Credentials(access_token)

Credentials object used for authenticating Azure requests. By default, calling Azure.Credentials() will search the normal Azure credential locations (files, environment variables, etc.) to find the account, shared key, or access_token. Otherwise, the 2nd constructor allows providing the account & shared key or access token directly, ignoring any existing configurations. Azure VM credentials are also automatically detected and retrieved. Temporary credentials via Azure VM that include expirations will automatically be refreshed expireThreshold before expiration when a request is made.

Azure.get(url, headers, body; credentials, kw...)
Azure.put(url, headers, body; kw...)
Azure.post(url, headers, body; kw...)
Azure.delete(url, headers, body; kw...)
Azure.head(url, headers; kw...)
Azure.patch(url, headers, body; kw...)
Azure.request(method, url, headers, body; kw...)
Azure.open(method, url, headers[, body]; kw...)

HTTP.jl client methods that additionally each take a credentials keyword argument, which should be an Azure.Credentials object. If the credentials object was inferred from the environment and is set to expire soon, it will be refreshed automatically.

Otherwise, these methods operate exactly like their HTTP.method counterparts, accepting all the same positional and keyword arguments.

Azure.get(url, headers, body; credentials, kw...)
Azure.put(url, headers, body; kw...)
Azure.post(url, headers, body; kw...)
Azure.delete(url, headers, body; kw...)
Azure.head(url, headers; kw...)
Azure.patch(url, headers, body; kw...)
Azure.request(method, url, headers, body; kw...)
Azure.open(method, url, headers[, body]; kw...)

HTTP.jl client methods that additionally each take a credentials keyword argument, which should be an Azure.Credentials object. If the credentials object was inferred from the environment and is set to expire soon, it will be refreshed automatically.

Otherwise, these methods operate exactly like their HTTP.method counterparts, accepting all the same positional and keyword arguments.

Azure.get(url, headers, body; credentials, kw...)
Azure.put(url, headers, body; kw...)
Azure.post(url, headers, body; kw...)
Azure.delete(url, headers, body; kw...)
Azure.head(url, headers; kw...)
Azure.patch(url, headers, body; kw...)
Azure.request(method, url, headers, body; kw...)
Azure.open(method, url, headers[, body]; kw...)

HTTP.jl client methods that additionally each take a credentials keyword argument, which should be an Azure.Credentials object. If the credentials object was inferred from the environment and is set to expire soon, it will be refreshed automatically.

Otherwise, these methods operate exactly like their HTTP.method counterparts, accepting all the same positional and keyword arguments.

Azure.get(url, headers, body; credentials, kw...)
Azure.put(url, headers, body; kw...)
Azure.post(url, headers, body; kw...)
Azure.delete(url, headers, body; kw...)
Azure.head(url, headers; kw...)
Azure.patch(url, headers, body; kw...)
Azure.request(method, url, headers, body; kw...)
Azure.open(method, url, headers[, body]; kw...)

HTTP.jl client methods that additionally each take a credentials keyword argument, which should be an Azure.Credentials object. If the credentials object was inferred from the environment and is set to expire soon, it will be refreshed automatically.

Otherwise, these methods operate exactly like their HTTP.method counterparts, accepting all the same positional and keyword arguments.

Azure.get(url, headers, body; credentials, kw...)
Azure.put(url, headers, body; kw...)
Azure.post(url, headers, body; kw...)
Azure.delete(url, headers, body; kw...)
Azure.head(url, headers; kw...)
Azure.patch(url, headers, body; kw...)
Azure.request(method, url, headers, body; kw...)
Azure.open(method, url, headers[, body]; kw...)

HTTP.jl client methods that additionally each take a credentials keyword argument, which should be an Azure.Credentials object. If the credentials object was inferred from the environment and is set to expire soon, it will be refreshed automatically.

Otherwise, these methods operate exactly like their HTTP.method counterparts, accepting all the same positional and keyword arguments.

Azure.get(url, headers, body; credentials, kw...)
Azure.put(url, headers, body; kw...)
Azure.post(url, headers, body; kw...)
Azure.delete(url, headers, body; kw...)
Azure.head(url, headers; kw...)
Azure.patch(url, headers, body; kw...)
Azure.request(method, url, headers, body; kw...)
Azure.open(method, url, headers[, body]; kw...)

HTTP.jl client methods that additionally each take a credentials keyword argument, which should be an Azure.Credentials object. If the credentials object was inferred from the environment and is set to expire soon, it will be refreshed automatically.

Otherwise, these methods operate exactly like their HTTP.method counterparts, accepting all the same positional and keyword arguments.

Azure.get(url, headers, body; credentials, kw...)
Azure.put(url, headers, body; kw...)
Azure.post(url, headers, body; kw...)
Azure.delete(url, headers, body; kw...)
Azure.head(url, headers; kw...)
Azure.patch(url, headers, body; kw...)
Azure.request(method, url, headers, body; kw...)
Azure.open(method, url, headers[, body]; kw...)

HTTP.jl client methods that additionally each take a credentials keyword argument, which should be an Azure.Credentials object. If the credentials object was inferred from the environment and is set to expire soon, it will be refreshed automatically.

Otherwise, these methods operate exactly like their HTTP.method counterparts, accepting all the same positional and keyword arguments.

Azure.get(url, headers, body; credentials, kw...)
Azure.put(url, headers, body; kw...)
Azure.post(url, headers, body; kw...)
Azure.delete(url, headers, body; kw...)
Azure.head(url, headers; kw...)
Azure.patch(url, headers, body; kw...)
Azure.request(method, url, headers, body; kw...)
Azure.open(method, url, headers[, body]; kw...)

HTTP.jl client methods that additionally each take a credentials keyword argument, which should be an Azure.Credentials object. If the credentials object was inferred from the environment and is set to expire soon, it will be refreshed automatically.

Otherwise, these methods operate exactly like their HTTP.method counterparts, accepting all the same positional and keyword arguments.

Minio.with(f; dir, bucket, public, startupDelay, debug)

Starts a minio server on a random open port, and passes a CloudTest.Config to f, which contains the credentials that should be used for requests made, as well as an AWS.Bucket that is created to help bootstrap the testing process. Supported keyword arguments include:

• dir: directory to use for the minio server, defaults to a temporary directory (which is deleted when the server is stopped)
• bucket: name of the bucket to create, defaults to "test"
• public: whether the bucket should be public, defaults to false
• startupDelay: number of seconds to wait after starting the server before creating the bucket, defaults to 0.25; this can be useful on slower systems to allow time for the server to fully startup
• debug: whether to turn on minio debug logging, defaults to false

Azurite.with(f; dir, bucket, public, startupDelay, debug)

Starts an azurite server on a random open port, and passes a CloudTest.Config to f, which contains the credentials that should be used for requests made, as well as an Azure.Container that is created to help bootstrap the testing process. Supported keyword arguments include:

• dir: directory to use for the minio server, defaults to a temporary directory (which is deleted when the server is stopped)
• bucket: name of the bucket to create, defaults to "test"
• public: whether the bucket should be public, defaults to false
• startupDelay: number of seconds to wait after starting the server before creating the bucket, defaults to 0.25; this can be useful on slower systems to allow time for the server to fully startup
• debug: whether to turn on minio debug logging, defaults to false

AbstractStore

Abstract type that specific cloud providers subtype to represent a "store". This is used to construct cloud urls and perform operations on cloud objects. See AWS.Bucket and Azure.Container for examples.

CloudCredentials

Abstract type that specific cloud providers subtype to represent a "credentials" object. This is passed to cloud request methods as the credentials keyword argument and is used to authenticate cloud requests. See AWS.Credentials and Azure.Credentials for examples.

SignedPermission(; kw...)
SignedPermission(::String)

Specify signed permissions when generating an Azure SAS URL.

For account-level SAS, the following permissions are allowed (char, keyword arg: desc):

• r, read: Read
• w, write: Write
• d, delete: Delete
• y, permanentDelete: Permanent delete
• l, list: List
• a, add: Add
• c, create: Create
• u, update: Update
• p, process: Process
• t, tag: Tag
• f, filter: Filter
• i, setImmutabilityPolicy: Set Immutability Policy

For service-level SAS, the following permissions are allowed (char, keyword arg: desc):

• r, read: Read
• a, add: Add
• c, create: Create
• w, write: Write
• d, delete: Delete
• x, deleteVersion: Delete Version
• y, permanentDelete: Permanent delete
• l, list: List
• u, update: Update
• t, tag: Tag
• f, find: Find
• m, move: Move
• e, execute: Execute
• o, ownership: Ownership
• p, permissions: Permissions
• i, setImmutabilityPolicy: Set Immutability Policy

CloudBase.AWS

Submodule that contains a custom HTTP.jl client for performing AWS requests. For authenticated requests, an AWS.Credentials object should be passed as the credentials keyword argument. Otherwise, the request methods operate just like the HTTP equivalents and supports all the same keyword arguments.

CloudBase.AWS.Bucket(name, [region="us-east-1"]; accelerate::Bool=false)

Object representation of an AWS storage bucket with the given name. If not provided, the region is assumed to be "us-east-1". Aliased in the CloudStore.jl package as S3.Bucket. If accelerate=true is passed, requests with the bucket will use the bucket.s3-accelerate.amazonaws.com style url instead of the traditional bucket.s3.amazonaws.com.

CloudBase.AWS.Credentials([profile]; expireThreshold=Dates.Minute(5))
CloudBase.AWS.Credentials(access_key_id, secret_access_key[, session_token])

Credentials object used for authenticating AWS requests. By default, calling AWS.Credentials() or AWS.Credentials(profile), will search the normal AWS credential locations (files, environment variables, etc.) to find the access key and secret. Otherwise, the 2nd constructor allows providing the access key & secret directly, ignoring any existing configurations. If a profile includes a role_arn, an STS request will be made with source credentials to get temporary credentials. AWS EC2 and ECS credentials are also automatically detected and retrieved. Temporary credentials via EC2, EC2, or role_arn that include expirations will automatically be refreshed expireThreshold before expiration when a request is made.

AWS.get(url, headers, body; credentials, awsv2=false, kw...)
AWS.put(url, headers, body; kw...)
AWS.post(url, headers, body; kw...)
AWS.delete(url, headers, body; kw...)
AWS.head(url, headers; kw...)
AWS.patch(url, headers, body; kw...)
AWS.request(method, url, headers, body; kw...)
AWS.open(method, url, headers[, body]; kw...)

HTTP.jl client methods that additionally each take a credentials keyword argument, which should be an AWS.Credentials object. To have AWSV2 request signing instead of AWSV4, pass awsv2=true. If the credentials object was inferred from the environment and is set to expire soon, it will be refreshed automatically.

Otherwise, these methods operate exactly like their HTTP.method counterparts, accepting all the same positional and keyword arguments.

Note that due to the nature of AWS signing requirements, streaming request bodies are not supported.

AWS.get(url, headers, body; credentials, awsv2=false, kw...)
AWS.put(url, headers, body; kw...)
AWS.post(url, headers, body; kw...)
AWS.delete(url, headers, body; kw...)
AWS.head(url, headers; kw...)
AWS.patch(url, headers, body; kw...)
AWS.request(method, url, headers, body; kw...)
AWS.open(method, url, headers[, body]; kw...)

HTTP.jl client methods that additionally each take a credentials keyword argument, which should be an AWS.Credentials object. To have AWSV2 request signing instead of AWSV4, pass awsv2=true. If the credentials object was inferred from the environment and is set to expire soon, it will be refreshed automatically.

Otherwise, these methods operate exactly like their HTTP.method counterparts, accepting all the same positional and keyword arguments.

Note that due to the nature of AWS signing requirements, streaming request bodies are not supported.

AWS.get(url, headers, body; credentials, awsv2=false, kw...)
AWS.put(url, headers, body; kw...)
AWS.post(url, headers, body; kw...)
AWS.delete(url, headers, body; kw...)
AWS.head(url, headers; kw...)
AWS.patch(url, headers, body; kw...)
AWS.request(method, url, headers, body; kw...)
AWS.open(method, url, headers[, body]; kw...)

HTTP.jl client methods that additionally each take a credentials keyword argument, which should be an AWS.Credentials object. To have AWSV2 request signing instead of AWSV4, pass awsv2=true. If the credentials object was inferred from the environment and is set to expire soon, it will be refreshed automatically.

Otherwise, these methods operate exactly like their HTTP.method counterparts, accepting all the same positional and keyword arguments.

Note that due to the nature of AWS signing requirements, streaming request bodies are not supported.

AWS.get(url, headers, body; credentials, awsv2=false, kw...)
AWS.put(url, headers, body; kw...)
AWS.post(url, headers, body; kw...)
AWS.delete(url, headers, body; kw...)
AWS.head(url, headers; kw...)
AWS.patch(url, headers, body; kw...)
AWS.request(method, url, headers, body; kw...)
AWS.open(method, url, headers[, body]; kw...)

HTTP.jl client methods that additionally each take a credentials keyword argument, which should be an AWS.Credentials object. To have AWSV2 request signing instead of AWSV4, pass awsv2=true. If the credentials object was inferred from the environment and is set to expire soon, it will be refreshed automatically.

Otherwise, these methods operate exactly like their HTTP.method counterparts, accepting all the same positional and keyword arguments.

Note that due to the nature of AWS signing requirements, streaming request bodies are not supported.

AWS.get(url, headers, body; credentials, awsv2=false, kw...)
AWS.put(url, headers, body; kw...)
AWS.post(url, headers, body; kw...)
AWS.delete(url, headers, body; kw...)
AWS.head(url, headers; kw...)
AWS.patch(url, headers, body; kw...)
AWS.request(method, url, headers, body; kw...)
AWS.open(method, url, headers[, body]; kw...)

HTTP.jl client methods that additionally each take a credentials keyword argument, which should be an AWS.Credentials object. To have AWSV2 request signing instead of AWSV4, pass awsv2=true. If the credentials object was inferred from the environment and is set to expire soon, it will be refreshed automatically.

Otherwise, these methods operate exactly like their HTTP.method counterparts, accepting all the same positional and keyword arguments.

Note that due to the nature of AWS signing requirements, streaming request bodies are not supported.

AWS.get(url, headers, body; credentials, awsv2=false, kw...)
AWS.put(url, headers, body; kw...)
AWS.post(url, headers, body; kw...)
AWS.delete(url, headers, body; kw...)
AWS.head(url, headers; kw...)
AWS.patch(url, headers, body; kw...)
AWS.request(method, url, headers, body; kw...)
AWS.open(method, url, headers[, body]; kw...)

HTTP.jl client methods that additionally each take a credentials keyword argument, which should be an AWS.Credentials object. To have AWSV2 request signing instead of AWSV4, pass awsv2=true. If the credentials object was inferred from the environment and is set to expire soon, it will be refreshed automatically.

Otherwise, these methods operate exactly like their HTTP.method counterparts, accepting all the same positional and keyword arguments.

Note that due to the nature of AWS signing requirements, streaming request bodies are not supported.

AWS.get(url, headers, body; credentials, awsv2=false, kw...)
AWS.put(url, headers, body; kw...)
AWS.post(url, headers, body; kw...)
AWS.delete(url, headers, body; kw...)
AWS.head(url, headers; kw...)
AWS.patch(url, headers, body; kw...)
AWS.request(method, url, headers, body; kw...)
AWS.open(method, url, headers[, body]; kw...)

HTTP.jl client methods that additionally each take a credentials keyword argument, which should be an AWS.Credentials object. To have AWSV2 request signing instead of AWSV4, pass awsv2=true. If the credentials object was inferred from the environment and is set to expire soon, it will be refreshed automatically.

Otherwise, these methods operate exactly like their HTTP.method counterparts, accepting all the same positional and keyword arguments.

Note that due to the nature of AWS signing requirements, streaming request bodies are not supported.

AWS.get(url, headers, body; credentials, awsv2=false, kw...)
AWS.put(url, headers, body; kw...)
AWS.post(url, headers, body; kw...)
AWS.delete(url, headers, body; kw...)
AWS.head(url, headers; kw...)
AWS.patch(url, headers, body; kw...)
AWS.request(method, url, headers, body; kw...)
AWS.open(method, url, headers[, body]; kw...)

HTTP.jl client methods that additionally each take a credentials keyword argument, which should be an AWS.Credentials object. To have AWSV2 request signing instead of AWSV4, pass awsv2=true. If the credentials object was inferred from the environment and is set to expire soon, it will be refreshed automatically.

Otherwise, these methods operate exactly like their HTTP.method counterparts, accepting all the same positional and keyword arguments.

Note that due to the nature of AWS signing requirements, streaming request bodies are not supported.