DiscreteEntropy

Summary

DiscreteEntropy is a Julia package to estimate the Shannon entropy of discrete data.

DiscreteEntropy implements a large collection of entropy estimators.

Installing DiscreteEntropy

1. If you have not done so already, install Julia. Julia 1.8 and

higher are supported. Nightly is not (yet) supported.

1. Install DiscreteEntropy using

using Pkg; Pkg.add("DiscreteEntropy")

or

] add DiscreteEntropy

Basic Usage

using DiscreteEntropy

data = [1,2,3,4,3,2,1];

7-element Vector{Int64}:
 1
 2
 3
 4
 3
 2
 1

Most of the estimators take a CountData object. This is a compact representation of the histogram of the random variable. The easiest way to create it is via from_data

# if `data` is a histogram already
cd = from_data(data, Histogram)

# or if `data` is actually a vector of samples

cds = from_data(data, Samples)

CountData([2.0 1.0; 3.0 1.0], 7.0, 4)

# or if `data` is actually a vector of samples

cds = from_data(data, Samples)

CountData([2.0 1.0; 3.0 1.0], 7.0, 4)

# now we can estimate
h = estimate_h(from_data(data, Histogram), ChaoShen)

# treating data as a vector of samples
h = estimate_h(from_data(data, Samples), ChaoShen)

1.6310218225019266

DiscreteEntropy.jl outputs Shannon measures in nats.

h = to_bits(estimate_h(cd, ChaoShen))

2.997302182277761