Druid

A Druid SQL querying library.

See also: Client, execute

Aggregator

Supertype of all aggregators.

AndF(filters::Vector{<:Filter})

Match all filters.

AndH(havingSpecs::Vector{<:HavingSpec})

Match all havingSpecs.

Arithmetic(name::String, fn::String, fields::Vector{<:PostAggregator}; ordering=nothing)

Apply the fn aggregation to the fields.

ordering should be a String if provided.

Bound(dimension::String;
    lower=nothing, upper=nothing, lowerStrict=nothing, upperStrict=nothing,
    ordering=nothing, extractionFn=nothing)

Filter values bounded in a range.

lower, upper and ordering should be Strings if provided. lowerStrict and upperStrict should be Bools if provided. extractionFn should be an ExtractionFunction if provided.

BucketEF(; size=nothing, offset=nothing)

Bucket numeric values into size buckets starting at offset.

size and offset should be non negative integers if provided.

CascadeEF(extractionFns::Vector{<:ExtractionFunction})

Apply multiple extraction functions in sequence.

Client

Client(url::URIs.URI)
Client(url::AbstractString)
Client(host, port, scheme)

Druid queryable host connection information store.

Do not provide the endpoint, it will be set based on the type of query. For example, "druid/v2/sql" for SQL queries and "druid/v2" for all other queries.

Examples

julia> client = Client("http://localhost:8888/")
Client(URI("http://localhost:8888/"))

julia> client = Client("localhost", 8082, "http")
Client(URI("http://localhost:8082"))

ColumnComparison(dimensions::Tuple{DimensionSpec, DimensionSpec})

ColumnComparison(dim1::DimensionSpec, dim2::DimensionSpec)

dimension1 = dimension2 filter.

ConstantPA(name::String, value::Real)

Return a constant value as name.

Contains(value::String; case_sensitive=nothing)

Search query spec for strings containing value.

case_sensitive should be a Bool if provided.

Count(output_name)

Count number of rows.

DataSource

Supertype of all data sources.

DatasourceMetadata(dataSource::DataSource; context=nothing)

Returns a Dict{String, String} with "timestamp" and "maxIngestedEventTime" as keys.

context should be a Dict if provided.

DefaultDS(dimension::String;
    outputName=nothing, outputType=nothing)

The "default" type dimension spec.

outputName and outputType should be Strings if provided.

DefaultLS(; limit=nothing, offset=nothing, columns=nothing)

Default limitSpec.

limit and offset should be non negative integers if provided. columns should be Vector{OrderByColumn} if provided.

DimSelector(dimension::String, value)

dimension = value filter.

Dimension()
Dimension(; ordering=nothing, previousStop=nothing)

Dimension topN metric spec.

ordering and previousStop should be Strings if provided.

DimensionSpec

Supertype of all dimension specs.

DurationGranularity(duration::UInt64; origin=nothing)

Duration type granularity where the duration is specified as milliseconds since origin.

Specifying origin is not required (defaults to Druid's default), but should be a ISO8601 datetime string if you do specify it.

EqualTo(aggregation::String, value::Real)

aggregation = value filter.

Expression(name::String, expression::String; outputType=nothing)

Create a virtual column using expression.

outputType should be String if provided.

ExtractionDS(dimension::String, extractionFn::ExtractionFunction;
    outputName=nothing, outputType=nothing)

Dimension spec to use with an extraction function.

outputName and outputType should be Strings if provided.

ExtractionFilter(dimension::String, value, extractionFn::ExtractionFunction)

Filter to value extracted using extractionFn.

ExtractionFunction

Supertype of all extraction functions.

FieldAccess(name::String, aggregator::String)
FieldAccess(aggregator::String)

Access the aggregator and return as name.

Filter

Supertype of all filters.

Filtered(filter, aggregator)

An aggregator that combines a filter with the aggregator.

FinalizingFieldAccess(name::String, aggregator::String)
FinalizingFieldAccess(aggregator::String)

Access the aggregator and return as name.

Fragment(value::String; case_sensitive=nothing)

Fragment search query spec.

case_sensitive should be a Bool if provided.

Granularity

Supertype of all granularities.

GreaterThan(aggregation::String, value::Real)

aggregation >= value filter.

Greatest(name::String, fields::Vector{<:PostAggregator}, dtype::String)

Find the largest value per row across the fields of type dtype (double, long) and return as name.

GroupBy(
    dataSource::DataSource,
    dimensions::Vector{<:DimensionSpec},
    intervals::Vector{<:Interval},
    granularity::Granularity;
    <keyword arguments>
)

Returns a Druid.GroupByResult which is compatible with the Tables.jl interface.

A method with all arguments as keyword arguments is also provided.

Arguments

• limitSpec::LimitSpec = nothing
• having::HavingSpec = nothing
• filter::Filter = nothing
• aggregations::Vector{<:Aggregator} = nothing
• postAggregations::Vector{<:PostAggregator} = nothing
• virtualColumns::Vector{<:VirtualColumn} = nothing
• subtotalsSpec::Vector{Vector{String}} = nothing
• context::Dict = nothing

Grouping(groupings, name)

Grouping aggregator

HavingSpec

Supertype of all having filters.

HyperUniqueCardinality(name::String, field::String)
HyperUniqueCardinality(field::String)

Post aggregator of a hyper unique aggregation.

InF(dimension::String, values::Vector)

Filter if value is in values.

Inline(column_names::Vector{String}, rows::Vector{Vector{T}})

Inline datasource.

InlineLookupEF(lookup::Map;
    retainMissingValue=nothing, replaceMissingValueWith=nothing, injective=nothing, optimize=nothing)

Extract using an inline lookup.

retainMissingValue, injective, optimize should be Bools and replaceMissingValueWith should be a String if provided.

InsensitiveContains(value::String)

Case insensitive string contains searhc query spec.

Interval(first::String, last::String)

A type to represent ISO8601 intervals.

IntervalF(dimension::String, intervals::Vector{<:Interval}; extractionFn=nothing)

Filter by time falling in one of the intervals.

extractionFn should be an ExtractionFunction if provided.

Inverted(metric::TopNMetricSpec)

Inverting topN metric spec.

JavaScriptEF(jsfunction::String; injective=nothing)

Extract using a custom JavaScript function.

injective should be a Bool if provided.

JavaScriptF(dimension::String, jsfunction::String)

Filter using a custom JavaScript function.

JavaScriptPA(name::String, fieldNames::Vector{String}, jsfunction::String)

Post aggregate the fieldNames aggregators using a custom jsfunction JavaScript function.

Join(left::Union{Table, Lookup, Inline, QuerySource, Join},
    right::Union{Lookup, Inline, QuerySource}, rightPrefix::String,
    condition::String, joinType::String)

Join datasource.

Least(name::String, fields::Vector{<:PostAggregator}, dtype::String)

Find the smallest value per row across the fields of type dtype (double, long) and return as name.

LessThan(aggregation::String, value::Real)

aggregation <= value filter.

Like(dimension::String, pattern::String; escape=nothing, extractionFn=nothing)

Filter strings matching pattern.

escape should be a String and extractionFn an ExtractionFunction if provided.

LimitSpec

Supertype of all limit specs.

ListFiltered(delegate::DimensionSpec, values::Vector{String}; isWhitelist=nothing)

Filter the delegate dimension spec with the values.

isWhitelist should be a Bool if provided.

Lookup(name::String)

Lookup datasource.

LookupDS(dimension::String, name::String; outputName=nothing)

A dimension spec to use with the name lookup.

outputName should be a String if provided.

LowerEF(; locale=nothing)

Extract strings as lower case.

locale should be a String if provided.

Map(isOneToOne::Bool, dict::Dict)

The lookup map used in the map lookup dimension spec.

MapLookupDS(dimension::String, lookup::Map;
    outputName=nothing, retainMissingValue=nothing, replaceMissingValueWith=nothing, optimize=nothing)

Map lookup dimension spec.

outputName and replaceMissingValueWith should be Strings, retainMissingValue and optimize should be Bools if provided.

NotF(filter::Filter)

Do not match filter.

NotH(havingSpec::HavingSpec)

Do not match the havingSpec.

Numeric(metric::String)

Numeric topN metric spec.

OrF(filters::Vector{<:Filter})

Match at least one of the filters.

OrH(havingSpecs::Vector{<:HavingSpec})

Match at leats one of the havingSpecs.

OrderByColumn(dimension::String, direction::String; dimensionOrder=nothing)

dimensionOrder should be a String if provided.

Parameter(type::String, value)

SQL query parameter, where type is one of the Druid SQL data types and value is the parameter value.

PartialEF(expr::String)

Regex based extraction function that return nulls for non-matches.

PeriodGranularity(period::String; origin=nothing, timezone=nothing)

Period type granularity where the period is specified as an ISO8601 period string. Period starts on origin in the timezone.

Specifying origin and timezone is not required (defaults to Druid's default). But if you do specify them, origin should be a ISO8601 datetime string and timezone should be one of those supported by Druid.

PostAggregator

Supertype of all post aggregators.

PrefixFiltered(delegate::DimensionSpec, prefix::String)

Filter the delegate dimension spec with the prefix.

Query

Supertype of all queries.

QueryResult

Supertype of all query results.

QuerySource(query::Query)

Query datasource.

RegexEF(expr::String; index=nothing, replaceMissingValue=nothing, replaceMissingValueWith=nothing)

Regex based extraction function.

index should be a non negative integer, replaceMissingValue a Bool and replaceMissingValueWith a String if provided.

RegexF(dimension::String, pattern::String)

Regex pattern matching filter.

RegexFiltered(delegate::DimensionSpec, pattern::String)

Filter the delegate dimension spec with the regex pattern.

RegisteredLookupEF(lookup::String;
    retainMissingValue=nothing, replaceMissingValueWith=nothing, injective=nothing, optimize=nothing)

Extract using a registered lookup.

retainMissingValue, injective, optimize should be Bools and replaceMissingValueWith should be a String if provided.

Scan(dataSource::DataSource, intervals::Vector{<:Interval}; <keyword arguments>)

A method with all arguments as keyword arguments is also provided.

Arguments

• columns::Vector{String} = nothing
• filter::Filter = nothing
• virtualColumns::Vector{<:VirtualColumn} = nothing
• order::String = nothing
• limit::Integer = nothing
• offset::Integer = nothing
• resultFormat::String = nothing
• batchSize::Integer = nothing
• context::Dict = nothing
• legacy::Bool = nothing

Search(dataSource::DataSource, intervals::Vector{<:Interval}, query::SearchQuerySpec; <keyword arguments>)

Returns a Druid.SearchResult which is compatible with the Tables.jl interface.

A method with all arguments as keyword arguments is also provided.

Arguments

• granularity::Granularity = nothing
• filter::Filter = nothing
• virtualColumns::Vector{<:VirtualColumn} = nothing
• sort::String = nothing
• limit::Integer = nothing
• context::Dict = nothing

SearchF(dimension::String, query::SearchQuerySpec; extractionFn=nothing)

Filter using a query and optionally extraction function.

extractionFn should be an ExtractionFunction if provided.

SearchQueryEF(query::SearchQuerySpec)

Extraction using a search query spec.

SearchQuerySpec

Supertype of all search query specs.

SegmentMetadata(dataSource::DataSource; <keyword arguments>)

Returns a Dict with the query output.

A method with all arguments as keyword arguments is also provided.

Arguments

• intervals::Vector{<:Interval} = nothing
• toInclude::Union{String, Vector{String}} = nothing
• merge::Bool = nothing
• analysisTypes::Vector{String} = nothing
• lenientAggregatorMerge::Bool = nothing
• virtualColumns::Vector{<:VirtualColumn} = nothing
• context::Dict = nothing

Selector(dimension::String, value::String)

dimension = value filter.

SimpleGranularity(name::String)

One of the simple predefined granularities of Druid.

SingleField(type, field, output_name)

Apply type aggregator to the field and get result as output_name.

Available types: "longSum", "doubleSum", "floatSum", "longMin", "doubleMin", "floatMin", "longMax", "doubleMax", "floatMax", "longFirst", "doubleFirst", "floatFirst", "longLast", "doubleLast", "floatLast", "longAny", "doubleAny", "floatAny", "doubleMean"

Sql(query::String; <keyword arguments>)

A method with all arguments as keyword arguments is also provided.

Arguments

• parameters::Vector{Parameter} = nothing
• resultFormat::String = nothing
• header::Bool = true
• context::Dict = nothing

StringAgg(type, field, output_name; maxStringBytes=nothing)

Apply type string aggregator to field and get result as output_name.

Available types: "first", "last", "any"

See Druid documentation for maxStringBytes.

StringFormatEF(format::String; nullHandling=nothing)

Extract strings using a format.

nullHandling should be a String if provided.

StrlenEF()

Extract string length.

SubstringEF(index::UInt64; length=nothing)

Substring extraction based on index and length.

length should be a non negative integer if provided.

Table(name::String)

Table datasource.

TimeBoundary(dataSource::DataSource; <keyword arguments>)

Returns a Dict{String, String} with "timestamp", "minTime" and/or "maxTime" (depending on bound) as keys.

A method with all arguments as keyword arguments is also provided.

Arguments

• bound::String = nothing
• filter::Filter = nothing
• virtualColumns::Vector{<:VirtualColumn} = nothing
• context::Dict = nothing

TimeFormatEF(; format=nothing, timeZone=nothing, locale=nothing, granularity=nothing, asMillis=nothing)

Extract timestamps.

format, timeZone, locale should be Strings, granularity should be a Granularity and asMillis should be a Bool if provided.

TimeParseEF(timeFormat::String, resultFormat::String; joda=nothing)

Parse and extract time.

joda should be a Bool if provided.

Timeseries(dataSource::DataSource, intervals::Vector{<:Interval}, granularity::Granularity; <keyword arguments>)

Returns a Druid.TimeseriesResult which is compatible with the Tables.jl interface.

A method with all arguments as keyword arguments is also provided.

Arguments

• filter::Filter = nothing
• aggregations::Vector{<:Aggregator} = nothing
• postAggregations::Vector{<:PostAggregator} = nothing
• virtualColumns::Vector{<:VirtualColumn} = nothing
• descending::Bool = nothing
• limit::Integer = nothing
• context::Dict = nothing

TopN(dataSource::DataSource, intervals::Vector{Interval}, granularity::Granularity,
    dimension::DimensionSpec, threshold::Uint64, metric::TopNMetricSpec; <keyword arguments>)

Returns a Druid.TopNResult which is compatible with the Tables.jl interface.

A method with all arguments as keyword arguments is also provided.

Arguments

• aggregations::Vector{<:Aggregator} = nothing
• postAggregations::Vector{<:PostAggregator} = nothing
• filter::Filter = nothing
• virtualColumns::Vector{<:VirtualColumn} = nothing
• context::Dict = nothing

TopNMetricSpec

Supertype of all topN metric specs.

TrueF()

Filter everything.

Unioned(names::Vector{String})

Union of table datasources.

UpperEF(; locale=nothing)

Extract strings as upper case.

locale should be a String if provided.

VirtualColumn

Supertype of all virtual columns.

execute(client, query::Sql; names=:auto)

Executes the Druid SQL query on the client::Client. Returns the query results as a String. Throws exception if query execution fails.

A Vector{Symbol} can be provided for the column names through names.

Examples

julia> query = Sql("SELECT * FROM some_datasource LIMIT 10")
julia> execute(client, query)
"[...\n]"

julia> query = Sql("SELECT * FROM non_existent_datasource")
julia> execute(client, query)
ERROR: Druid error during query execution
Dict{String,Any}("errorClass" => "org.apache.calcite.tools.ValidationException"...

execute(client, query; pretty=false)

Executes the native Druid query on the client::Client. Returns the query results as a String. Throws exception if query execution fails.

Convert a structure into a Dict but only with the properties with non-nothing values.