EffectSizes.jl

EffectSizes.jl is a Julia package for effect size measures. Confidence intervals are assigned to effect sizes using either the normal distribution or by bootstrap resampling. The package implements types for the following measures:

MeasureType
Cohen's dCohenD
Hedge's gHedgeG
Glass's ΔGlassΔ

Installation

] add https://github.com/harryscholes/EffectSizes.jl

Examples

julia> using Random, EffectSizes; Random.seed!(1);

julia> xs = randn(10^3);

julia> ys = randn(10^3) .+ 0.5;

julia> es = CohenD(xs, ys, quantile=0.95); # normal CI (idealised distribution)

julia> typeof(es)
CohenD{Float64,ConfidenceInterval{Float64}}

julia> effectsize(es)
-0.503…

julia> quantile(es)
0.95

julia> ci = confint(es);

julia> typeof(ci)
ConfidenceInterval{Float64}

julia> confint(ci)
(-0.592…, -0.414…)

julia> es = CohenD(xs, ys, 10^4, quantile=0.95); # bootstrap CI (empirical distribution)

julia> effectsize(es) # effect size is the same
-0.503…

julia> typeof(es)
CohenD{Float64,BootstrapConfidenceInterval{Float64}}

julia> ci = confint(es); # confidence interval is different

julia> lower(ci)
-0.591…

julia> upper(ci)
-0.415…

Index

API

EffectSizes.AbstractEffectSize — Type
AbstractEffectSize

An abstract type to represent an effect size.

EffectEffect size
Small0.2
Medium0.5
Large0.8

Subtypes implement:

MethodDescription
effectsizereturns the effect size index
confintreturns the confidence interval
EffectSizes.CohenD — Type
CohenD(xs, ys[, bootstrap]; [quantile=0.95])

Calculate Cohen's $d$ effect size index between two vectors xs and ys.

A confidence interval for the effect size is calculated at the quantile quantile. If bootstrap is provided, the confidence interval is calculated by resampling from xs and ysbootstrap times.

\[ d = \frac{m_A - m_B}{s}\]

where $m$ is the mean and $s$ is the pooled standard deviation:

\[ s = \sqrt{\frac{(n_A - 1) s_A^2 + (n_B - 1) s_B^2}{n_A + n_B - 2}}\]

If $m_A$ > $m_B$, $d$ will be positive and if $m_A$ < $m_B$, $d$ will be negative.

Note

HedgeG outperforms CohenD when sample sizes are < 20.

Examples

xs = randn(100000)
ys = randn(100000) .+ 0.01

using EffectSizes
CohenD(xs, ys)

using HypothesisTests
EqualVarianceTTest(xs, ys)
EffectSizes.HedgeG — Type
HedgeG(xs, ys[, bootstrap]; [quantile=0.95])

Calculate Hedge's $g$ effect size index between two vectors xs and ys.

A confidence interval for the effect size is calculated at the quantile quantile. If bootstrap is provided, the confidence interval is calculated by resampling from xs and ysbootstrap times.

\[ g = \frac{m_A - m_B}{s}\]

where $m$ is the mean and $s$ is the pooled standard deviation:

\[ s = \sqrt{\frac{(n_A - 1) s_A^2 + (n_B - 1) s_B^2}{n_A + n_B}}\]

If $m_A$ > $m_B$, $g$ will be positive and if $m_A$ < $m_B$, $g$ will be negative.

Note

HedgeG outperforms CohenD when sample sizes are < 20.

EffectSizes.GlassΔ — Type
GlassΔ(treatment, control[, bootstrap]; [quantile=0.95])

Calculate Glass's $Δ$ effect size index between two vectors treatment and control.

A confidence interval for the effect size is calculated at the quantile quantile. If bootstrap is provided, the confidence interval is calculated by resampling from xs and ysbootstrap times.

\[ Δ = \frac{m_T - m_C}{s_C}\]

where $m$ is the mean, $s$ is the standard deviation, $T$ is the treatment group and $C$ is the control group.

If $m_T$ > $m_C$, $Δ$ will be positive and if $m_T$ < $m_C$, $Δ$ will be negative.

Note

GlassΔ should be used when the standard deviations between the two groups are very different.

StatsAPI.confint — Method
confint(es::AbstractEffectSize) -> ConfidenceInterval

Return the confidence interval of an effect size as a ConfidenceInterval object.

Statistics.quantile — Method
quantile(es::AbstractEffectSize) -> Float64

Returns the quantile of a confidence interval.

EffectSizes.AbstractConfidenceInterval — Type
AbstractConfidenceInterval{T<:Real}

A type representing a confidence interval.

Subtypes implement:

MethodDescription
confintreturns the lower and upper bounds
quantilereturns the quantile
EffectSizes.ConfidenceInterval — Type
ConfidenceInterval(lower, upper, quantile)

A type representing the lower and upper bounds of an effect size confidence interval at a specified quantile.

ConfidenceInterval(xs, ys, es; quantile)

Calculate a confidence interval for the effect size es between two vectors xs and ys at a specified quantile.

EffectSizes.BootstrapConfidenceInterval — Type
BootstrapConfidenceInterval(lower, upper, quantile, bootstrap)

A type representing the lower and upper bounds of an effect size confidence interval at a specified quantile with bootstrap resamples.

BootstrapConfidenceInterval(f, xs, ys, bootstrap; quantile)

Calculate a bootstrap confidence interval between two vectors xs and ys at a specified quantile by applying f to bootstrap resamples of xs and ys.

StatsAPI.confint — Method
confint(ci::AbstractConfidenceInterval{T}) -> Tuple{T,T}

Return the lower and upper bounds of a confidence interval.

EffectSizes.lower — Function
lower(ci::AbstractConfidenceInterval{T}) -> T

Return the lower bound of a confidence interval.

EffectSizes.upper — Function
upper(ci::AbstractConfidenceInterval{T}) -> T

Return the upper bound of a confidence interval.

Statistics.quantile — Method
quantile(ci::AbstractConfidenceInterval) -> Float64

Returns the quantile of a confidence interval.