abstract type AbstractCorrespondence

Method for determining the correspondence matching between blobs and measurements.

Supports functions assign(c::AbstractCorrespondence, blobs, coordinates), too_far(c::AbstractCorrespondence, blob, coordinate)


  • HungarianCorrespondence
  • NearestNeighborCorrespondence

Represents a blob. Internally stores a Kalman filter, a counter that is incremented when the blob is not assigned a measurement and a trace of all locations and all seen measurements. If no measurement was seen for a particular time step, OOB = CartesianIndex(0,0) is recorded.

This type supports location, trace, tracem, lifetime, draw!

Blob(params::KalmanParams, coord::CartesianIndex)

Spawn a blob using settings from params at given coordinates.

BlobTracker{T <: AbstractCorrespondence}

Example bt = BlobTracker(sizes=3:5, σw=2.0, σe = 10.0)

Optional Keyword arguments:

  • params::KalmanParams: Holds the parameters for the kalman filter σw,σe
  • amplitude_th = 0.0001: blobs must be at least this prominent to be considered
  • kill_counter_th::Int = 10: after this many steps without an assigned measurement a blob will die
  • sizes::AbstractVector: vector of numbers determining the size scales at which blobs are detected. See docs for Images.blob_LoG.
  • preprocessor = ((storage, img)->nothing a function that processes img and stores the result in storage
  • correspondence::AbstractCorrespondence = HungarianCorrespondence(p=1.0, dist_th=2): Determines how blobs are assigned to measurements
  • mask = nothing: An optional boolean image that is false where you want to ignore blobs and true where you want to track them.
BlobTracker(sizes, σw, σe; kwargs...)

Helper constructor that accepts keyword arguments for the KalmanParams


  • σw: Dynamics standard deviation
  • σe: Measurement standard deviation
DiffBackground{T} <: BackgroundExtractor

Models the background of a sequence of images as the diff between two consequtive images.

This type supports background, foreground, update!


A buffer for images, this type supports efficient updating and calculation of statistics along it's third dimension.

Construct using fb = FrameBuffer(img::Matrix, n) fb = FrameBuffer{T}(w::Int,h::Int,n::Int)

where n is the buffer capacity. fb supports push!, median, mean, sum, var, std, reshape, size, length, capacity, Matrix, isready and iteration.

HungarianCorrespondence <: AbstractCorrespondence

Use the Hungarian algorithm to assign measuements to blobs. Each measurement is assigned to one blob only. Parameter p > 0 influences how eager the assignement is on a spectrum between p ≈ 0 corresponding to nearest neighbor matching, p = 1 corresponding to minimizing the earth-movers distance, p → ∞ corresponding to minimizing the maximum error. The default is p = 1.


  • p=1: the exponent of the cost matrix
  • dist_th=2: maximum allowed Mahalanobis distance between a blob and a measurement
MCCorrespondence <: AbstractCorrespondence

Assigns blobs to measurement by approximately integrating over the posterior distribution over blobs and performing the assignment using the inner assignment


  • inner: inner assignment object
  • num_samples::Int = 20 number of Monte Carlo samples to draw. The inner assignment routine will be called this many times so it can get expensive to set this too high.
MedianBackground{T} <: BackgroundExtractor

Models the background of a sequence of images as the median over a fixed time window. Supports the same constructors as a FrameBuffer

This type supports background, foreground, update!

NearestNeighborCorrespondence <: AbstractCorrespondence

Assign each blob the measurement that appears closest to the blob. This method can assign the same measurement to multiple blobs.


  • dist_th=2: maximum allowed Mahalanobis distance between a blob and a measurement

Struct used to record tracking results to a video.

Keyword Arguments:

  • filename = "trackingresult.mp4"
  • framerate = 30

A trace is a Vector{CartesianIndex{2}} and additionally supports draw!, Matrix.


This type stores a vector of blobs that were active when tracking terminated, a vector dead of blobs that died during tracking and a vector of all measurements obtained during tracking. This type supports allblobs, trace

Workspace{T1, T2}

Contains buffer arrays that can be resued to minimize allocations

Workspace(img::AbstractMatrix, n::Int)

Provide example image and length of bt.sizes

measurement = assign(c::AbstractCorrespondence, blobs, coordinates)

Assign measurements to blobs using the AbstractCorrespondence. Returns m::Measurement


  • blobs: A vector of blobs
  • coordinates: A vector of CartesianIndex
dist(blob, c)

Measure the distance between a blob and a coordinate c using the Mahalanobis distance induced the the blobs measurement covariance

kill_blobs!(result::TrackingResult, bt::BlobTracker)

Kill all blobs that have not seen a measurement for the duration bt.kill_counter_th

coordinates = measure(ws::Workspace, bt::BlobTracker, img)

Detect blobs in the image and return the coordinates


  • ws: Workspace
  • bt: BlobTracker
  • img: image
showblobs(img::AbstractMatrix{T}, result::TrackingResult, m::Measurement; rad=8, recorder=nothing, display=true) where T

Overlay found blobs on img


  • img: an image
  • result: a TrackingResult
  • m: a Measurement
  • rad: radius of blobs to draw
  • recorder: an optional Recorder
  • display = Base.display: function to display image. Use display=nothing to not display.

Get the location trace of a blob. Use tm = Matrix(t::Trace) to get an N×2 matrix. Use tm = replace(Float64.(tm), 0=>NaN) to create a matrix with NaNs where there were missing measurements, this is useful for plotting since it creates a gap where the missing measurement was.

trace(tr::TrackingResult; minlife=0)

Get all traces in a tracking result. Optionally filter based on minimum lifetime of a blob (the blob must have seen this many measurements).


Get the measurement trace of a blob. Use skipmissing(t::Trace) to filter out missing measurements. Use tm = Matrix(t::Trace) to get an N×2 matrix. Use tm = replace(Float64.(tm), 0=>NaN) to create a matrix with NaNs where there were missing measurements, this is useful for plotting since it creates a gap where the missing measurement was.

track_blobs(bt::BlobTracker, vid; display=false, recorder=nothing, threads=Threads.nthreads() > 1, ignoreempty=false)

Main entry point to tracking blobs


  • bt: a BlobTracker
  • vid: Some iterable thing that iterates images
  • display = Base.display: function to display images live. Displaying live slows things down a bit. Use display=nothing to not display anything. Consider also `c = imshow(img);

displayfun = img -> imshow!(c["gui"]["canvas"],img); `.

  • recorder: an optional Recorder that can record each frame to a video on disk. Recording things does not slow things down much and also does not affect memory usage much.
  • threads: Use threaded processing of frames? Only useful if Julia is started with multiple threads.
  • ignoreempty=false: wether or not to ignore display and recording of frames which contains no blobs and no measurements.
tune_sizes!(ws::Workspace, bt::BlobTracker, img)

Display a small GUI with a slider to help you tune the sizes

correct!(blobs, measurement::Measurement)

Correct the state of the blobs by incorporating the measurement in the Kalman filter

update!(ws::Workspace, bt::BlobTracker, coords_or_img, result::TrackingResult)

Perform one iteration of predict and correct


  • ws: a Workspace object
  • bt: the blob tracker
  • coords_or_img: vector of coordinates or an image
  • result: a TrackingResult