# AcuteBenchmark

AcuteBenchmark allows you to benchmark functions that get Arrays as their input.

It is used inside IntelVectorMath for benchmarking its functions. A fully working example available here: https://github.com/JuliaMath/VML.jl/blob/AcuteBenchmark/benchmark/benchmark.jl

Creates random inputs for a function based on limits, types, and dims specified.

config = Funb(
fun = sin,
limits = [(-1,1)],
types = [Float32, Float64],
dims = [100],
)

or just in a compact form:

config = Funb( sin, [(-1,1)], [Float32, Float64], [10])

use benchmark! to run the benchmark:

using AcuteBenchmark

configs = FunbArray([
Funb( sin, [(-1,1)],[Float32, Float64], [10] );
Funb( atan, [(-1,1), (-1,1)],[Float32, Float64],[10, 10] );
Funb( *, [(-1, 1), (-1, 1), (-1, 1)], [Float32, Float64], [(10,10), (10,10)] );
])

benchmark!(configs)

Plot the benchmark result using:

bar(configs)

To have a same color for the same types use:

bar(configs, uniqueType = true, dimAnnotation = true)

To plot the relative speed, pass a pair of configs:

bar(configsRealBase => configsRealIVM, uniqueType = true, dimAnnotation = false, uniqueDim = true, "Base" => "IntelVectorMath")

To plot how the function acts over different dimension sets:

configs2 = Funb( sin, [(-1,1)],[Float32, Float64], [10 20 30 40] );
benchmark!(configs2)
dimplot(configs2)

To compare different sets pass an array of configs:

dimplot([configsRealBase,configsRealIVM],["Base", "IntelVectorMath"])

AcuteBenchmark.FunbType
Funb(;fun, limits, types, dims)

Creates random inputs for a function based on limits, types, and dims specified.

Arguments

• fun: the function :fun or :(Module.fun)
• limits: min and max of possible values
• types : type of elements
• dims: Array of dimensions of the input vectors for each argument. Each column is for a new set of sizes, and each row is for different input arguments. So:
• each element gives the size of the input, and it is a:
• Number (for 1D)
• Tuple (for N-D)
• each row for each function argument
• each column for each dimension set

Examples

config = Funb(
fun = sin,
limits = [(-1,1)],
types = [Float32, Float64],
dims = [100],
)

or just in a compact form:

config = Funb( sin, [(-1,1)], [Float32, Float64], [100])
AcuteBenchmark.FunbArrayMethod
FunbArray

Array of Funb configs for different functions.

Examples

using AcuteBenchmark

configs = FunbArray([
Funb( sin, [(-1,1)],[Float32, Float64], [100] );
Funb( atan, [(-1,1), (-1,1)],[Float32, Float64],[100, 100] );
Funb( *, [(-1, 1), (-1, 1), (-1, 1)], [Float32, Float64], [(100,100), (100,100)] );
])


You can also directly give the configs in vectors:

configs = FunbArray(
fun =   [sin,
atan,
*],
limits = [[(-1,1)],
[(-1,1), (-1,1)],
[(-1, 1), (-1, 1), (-1, 1)]],
types = fill([Float32, Float64], (3)),
dims = [ [100],
[100, 100],
[(100,100), (100,100)] ],
)
AcuteBenchmark.barMethod
bar(config::StructArray{Funb}; uniqueType::Bool = false, dimAnnotation::Bool = true, uniqueDim::Bool = false)

Plots bars for each dimension set.

It is assumed that number of dimension sets are the same.

• To have a same color for the same types, set true as the 2nd argument.
• To turn off dimension annotations pass false as the 3rd argument.
• In case of unique dimensions pass true as 4th argument to print dimension in title instead.

Examples

bar(configs)
bar(configs, uniqueType = true, dimAnnotation = false, uniqueDim = true)
AcuteBenchmark.barMethod
bar(config::Pair{StructArray{Funb}}; uniqueType::Bool = false, dimAnnotation::Bool = false, uniqueDim::Bool =false, configName::Pair{String,String} = "1" => "2")

Gets a pair of StructArrays and calculates relative speed of coresponding elements and plots them.

Uses the first element of the pair for the configurations and only uses runtimes from the 2nd element of the pair. The relative speed is calculated like this:

1st element runtimes / 2nd element runtimes

Give the configsets names as a pair for the ylabel.

bar(configs => configs, uniqueType = true, dimAnnotation = false, uniqueDim =true, "group 1" => "group 2")
AcuteBenchmark.benchmark!Method
benchmark!(config::StructArray{Funb}) # FunbArray{Funb}
benchmark!(config::Array{Funb})

Performs the benchmarking on a given Funb.

Examples

using AcuteBenchmark

configs = FunbArray([
Funb( sin, [(-1,1)],[Float32, Float64], [100] );
Funb( atan, [(-1,1), (-1,1)],[Float32, Float64],[100, 100] );
Funb( *, [(-1, 1), (-1, 1), (-1, 1)], [Float32, Float64], [(100,100), (100,100)] );
])

benchmark!(configs)
AcuteBenchmark.dimplotMethod
dimplot(config::StructArray{Funb})

Shows runtime of a function for different dimension sets. The axes are logarithmic.

Examples

configs2 = Funb( sin, [(-1,1)],[Float32, Float64], [10 20 30 40] );

benchmark!(configs2)

dimplot(configs2)
AcuteBenchmark.dimplotMethod
dimplot(config::Vector{StructArray{Funb}}, labels::Vector{String})

By passing a vector different benchmark sets can be grouped together to be shown in a single plot. The axes are logarithmic.

Examples

dimplot([config1,config2, config3], ["group 1", "group 2", "group3"])
AcuteBenchmark.loadMethod
load(file_name)

Examples

configs = AcuteBenchmark.load("benchmarkdata.jld2")
AcuteBenchmark.numArgsDimsMethod
numArgsDims(in)

Finding number of arguments and number of dimension sets

Examples

numArgs, numDimsSets = numArgsDims(in)
AcuteBenchmark.saveMethod
save(file_name, config)

Save benchmark data

Examples

AcuteBenchmark.save("benchmarkdata.jld2", config)