EncodedArrays.AbstractArrayCodecType
abstract type AbstractArrayCodec <: Codecs.Codec end

Abstract type for arrays codecs.

Subtypes must implement the AbstractEncodedArray API. Most coded should use EncodedArray as the concrete subtype of AbstractArrayCodec. Codecs that use a custom subtype of AbstractEncodedArray must implement

EncodedArrays.encarraytype(::Type{<:AbstractArrayCodec},::Type{<:AbstractArray{T,N}})::Type{<:AbstractEncodedArray{T,N}}
EncodedArrays.AbstractEncodedArrayType
AbstractEncodedArray{T,N} <: AbstractArray{T,N}

Abstract type for arrays that store their elements in encoded/compressed form.

In addition to the standard AbstractArray API, an AbstractEncodedArray must support the functions

  • EncodedArrays.getcodec(A::EncodedArray): Returns the codec.
  • Base.codeunits(A::EncodedArray): Returns the internal encoded data representation.

Encoded arrays will typically be created via

A_enc = (codec::AbstractArrayCodec)(A::AbstractArray)

or

A_enc = AbstractEncodedArray(undef, codec::AbstractArrayCodec)
append!(A_enc, B::AbstractArray)

Decoding happens via standard array conversion or assignment:

A_dec = Array(A)
A_dec = convert(Array,A)
A_dec = A[:]

A_dec = Array{T,N}(undef, size(A_enc)...)
A_dec[:] = A_enc
EncodedArrays.EncodedArrayType
EncodedArray{T,N,C,DV} <: AbstractEncodedArray{T,N}

Concrete type for AbstractEncodedArrays.

Constructor:

EncodedArray{T}(
    codec::AbstractArrayCodec,
    size::NTuple{N,Integer},
    encoded::AbstractVector{UInt8}
)

Codecs using EncodedArray only need to implement EncodedArrays.encode_data! and EncodedArrays.decode_data!.

If length of the decoded data can be inferred from the encoded data, a constructor

EncodedArray{T,N}(codec::MyCodec,encoded::AbstractVector{UInt8})

should also be defined. By default, two EncodedArrays that have the same codec and size are assumed to be equal if and only if their code units are equal.

Generic methods for the rest of the AbstractEncodedArray API are already provided for EncodedArray.

EncodedArrays.VectorOfEncodedArraysType
const VectorOfEncodedArrays{T,N,...} = StructArray{EncodedArray{...},...}

Alias for vectors of encoded arrays that store the code units of all elements in contiguous fashion using StructArrays.StructArray and ArraysOfArray.VectorOfArrays.

Base.:|>Method
¦>(A::AbstractArray{T}, codec::AbstractArrayCodec)::AbstractEncodedArray

Encode A using codec and return an AbstractEncodedArray. The default implementation returns an EncodedArray.

EncodedArrays.decode_data!Function
decode_data!(data::AbstractArray, codec::AbstractArrayCodec, encoded::AbstractVector{UInt8})

Depending on codec, may or may not resize decoded to fit the size of the decoded data. Codecs may require decoded to be of correct size (e.g. to improved performance or when the size/shape of the decoded data cannot be easily inferred from the encoded data.

Returns data.

EncodedArrays.encode_data!Function
encode_data!(encoded::AbstractVector{UInt8}, codec::AbstractArrayCodec, data::AbstractArray)

Will resize encoded as necessary to fit the encoded data.

Returns encoded.

EncodedArrays.getcodecFunction
EncodedArrays.getcodec(A::AbstractEncodedArray)::AbstractArrayCodec

Returns the codec used to encode/compress A.

EncodedArrays.read_autozz_varlenFunction
read_autozz_varlen(io::IO, ::Type{<:Integer})

Read an integer of type T from io, using zig-zag decoding depending on whether T is signed or unsigned.

EncodedArrays.read_varlenMethod
read_varlen(io::IO, T::Type{<:Unsigned})

Read an unsigned variable-length integer value of type T from io. If the next value encoded in x is too large to be represented by T, an exception is thrown.

See EncodedArrays.write_varlen.

EncodedArrays.write_varlenMethod
write_varlen(io::IO, x::Unsigned)

Write unsigned integer value x to IO using variable-length coding. Data is written in LSB fashion in units of one byte. The highest bit of each byte indicates if more bytes will need to be read, the 7 lower bits contain the next 7 bits of x.