abstract type AbstractArrayCodec <: Codecs.Codec end

Abstract type for arrays codecs.

Subtypes must implement the AbstractEncodedArray API. Most coded should use EncodedArray as the concrete subtype of AbstractArrayCodec. Codecs that use a custom subtype of AbstractEncodedArray must implement

AbstractEncodedArray{T,N} <: AbstractArray{T,N}

Abstract type for arrays that store their elements in encoded/compressed form.

In addition to the standard AbstractArray API, an AbstractEncodedArray must support the functions

  • EncodedArrays.getcodec(A::EncodedArray): Returns the codec.
  • Base.codeunits(A::EncodedArray): Returns the internal encoded data representation.

Encoded arrays will typically be created via

A_enc = (codec::AbstractArrayCodec)(A::AbstractArray)


A_enc = AbstractEncodedArray(undef, codec::AbstractArrayCodec)
append!(A_enc, B::AbstractArray)

Decoding happens via standard array conversion or assignment:

A_dec = Array(A)
A_dec = convert(Array,A)
A_dec = A[:]

A_dec = Array{T,N}(undef, size(A_enc)...)
A_dec[:] = A_enc
EncodedArray{T,N,C,DV} <: AbstractEncodedArray{T,N}

Concrete type for AbstractEncodedArrays.



Codecs using EncodedArray only need to implement EncodedArrays.encode_data! and EncodedArrays.decode_data!.

If length of the decoded data can be inferred from the encoded data, a constructor


should also be defined. By default, two EncodedArrays that have the same codec and size are assumed to be equal if and only if their code units are equal.

Generic methods for the rest of the AbstractEncodedArray API are already provided for EncodedArray.

const VectorOfEncodedArrays{T,N,...} = StructArray{EncodedArray{...},...}

Alias for vectors of encoded arrays that store the code units of all elements in contiguous fashion using StructArrays.StructArray and ArraysOfArray.VectorOfArrays.

¦>(A::AbstractArray{T}, codec::AbstractArrayCodec)::AbstractEncodedArray

Encode A using codec and return an AbstractEncodedArray. The default implementation returns an EncodedArray.

decode_data!(data::AbstractArray, codec::AbstractArrayCodec, encoded::AbstractVector{UInt8})

Depending on codec, may or may not resize decoded to fit the size of the decoded data. Codecs may require decoded to be of correct size (e.g. to improved performance or when the size/shape of the decoded data cannot be easily inferred from the encoded data.

Returns data.

encode_data!(encoded::AbstractVector{UInt8}, codec::AbstractArrayCodec, data::AbstractArray)

Will resize encoded as necessary to fit the encoded data.

Returns encoded.


Returns the codec used to encode/compress A.

read_autozz_varlen(io::IO, ::Type{<:Integer})

Read an integer of type T from io, using zig-zag decoding depending on whether T is signed or unsigned.

read_varlen(io::IO, T::Type{<:Unsigned})

Read an unsigned variable-length integer value of type T from io. If the next value encoded in x is too large to be represented by T, an exception is thrown.

See EncodedArrays.write_varlen.

write_varlen(io::IO, x::Unsigned)

Write unsigned integer value x to IO using variable-length coding. Data is written in LSB fashion in units of one byte. The highest bit of each byte indicates if more bytes will need to be read, the 7 lower bits contain the next 7 bits of x.