abstract type AbstractCorrespondence

Method for determining the correspondence matching between blobs and measurements.

Supports functions assign(c::AbstractCorrespondence, blobs, coordinates), too_far(c::AbstractCorrespondence, blob, coordinate)

Subtypes

• HungarianCorrespondence
• NearestNeighborCorrespondence

Blob

Represents a blob. Internally stores a Kalman filter, a counter that is incremented when the blob is not assigned a measurement and a trace of all locations and all seen measurements. If no measurement was seen for a particular time step, OOB = CartesianIndex(0,0) is recorded.

This type supports location, trace, tracem, lifetime, draw!

Blob(params::KalmanParams, coord::CartesianIndex)

Spawn a blob using settings from params at given coordinates.

BlobTracker{T <: AbstractCorrespondence}

Example bt = BlobTracker(sizes=3:5, σw=2.0, σe = 10.0)

Optional Keyword arguments:

• params::KalmanParams: Holds the parameters for the kalman filter σw,σe
• amplitude_th = 0.0001: blobs must be at least this prominent to be considered
• kill_counter_th::Int = 10: after this many steps without an assigned measurement a blob will die
• sizes::AbstractVector: vector of numbers determining the size scales at which blobs are detected. See docs for Images.blob_LoG.
• preprocessor = ((storage, img)->nothing a function that processes img and stores the result in storage
• correspondence::AbstractCorrespondence = HungarianCorrespondence(p=1.0, dist_th=2): Determines how blobs are assigned to measurements
• mask = nothing: An optional boolean image that is false where you want to ignore blobs and true where you want to track them.

BlobTracker(sizes, σw, σe; kwargs...)

Helper constructor that accepts keyword arguments for the KalmanParams

#Arguments:

• σw: Dynamics standard deviation
• σe: Measurement standard deviation

DiffBackground{T} <: BackgroundExtractor

Models the background of a sequence of images as the diff between two consequtive images.

This type supports background, foreground, update!

FrameBuffer{T}

A buffer for images, this type supports efficient updating and calculation of statistics along it's third dimension.

Construct using fb = FrameBuffer(img::Matrix, n) fb = FrameBuffer{T}(w::Int,h::Int,n::Int)

where n is the buffer capacity. fb supports push!, median, mean, sum, var, std, reshape, size, length, capacity, Matrix, isready and iteration.

HungarianCorrespondence <: AbstractCorrespondence

Use the Hungarian algorithm to assign measuements to blobs. Each measurement is assigned to one blob only. Parameter p > 0 influences how eager the assignement is on a spectrum between p ≈ 0 corresponding to nearest neighbor matching, p = 1 corresponding to minimizing the earth-movers distance, p → ∞ corresponding to minimizing the maximum error. The default is p = 1.

Parameters

• p=1: the exponent of the cost matrix
• dist_th=2: maximum allowed Mahalanobis distance between a blob and a measurement

MCCorrespondence <: AbstractCorrespondence

Assigns blobs to measurement by approximately integrating over the posterior distribution over blobs and performing the assignment using the inner assignment

Parameters

• inner: inner assignment object
• num_samples::Int = 20 number of Monte Carlo samples to draw. The inner assignment routine will be called this many times so it can get expensive to set this too high.

MedianBackground{T} <: BackgroundExtractor

Models the background of a sequence of images as the median over a fixed time window. Supports the same constructors as a FrameBuffer

This type supports background, foreground, update!

NearestNeighborCorrespondence <: AbstractCorrespondence

Assign each blob the measurement that appears closest to the blob. This method can assign the same measurement to multiple blobs.

Parameters

• dist_th=2: maximum allowed Mahalanobis distance between a blob and a measurement

Recorder

Struct used to record tracking results to a video.

Keyword Arguments:

• filename = "trackingresult.mp4"
• framerate = 30

A trace is a Vector{CartesianIndex{2}} and additionally supports draw!, Matrix.

TrackingResult

This type stores a vector of blobs that were active when tracking terminated, a vector dead of blobs that died during tracking and a vector of all measurements obtained during tracking. This type supports allblobs, trace

Workspace{T1, T2}

Contains buffer arrays that can be resued to minimize allocations

Workspace(img::AbstractMatrix, n::Int)

Provide example image and length of bt.sizes

allblobs(tr)

Get all blobs in a tracking result

measurement = assign(c::AbstractCorrespondence, blobs, coordinates)

Assign measurements to blobs using the AbstractCorrespondence. Returns m::Measurement

#Arguments:

• blobs: A vector of blobs
• coordinates: A vector of CartesianIndex

background(be::MedianBackground)

Extract the background of img.

background(be::DiffBackground, img)

Extract the background of img.

dist(blob, c)

Measure the distance between a blob and a coordinate c using the Mahalanobis distance induced the the blobs measurement covariance

foreground(be::DiffBackground, img)

Extract the foreground of img.

foreground(be::MedianBackground, img)

Extract the foreground of img.

kill_blobs!(result::TrackingResult, bt::BlobTracker)

Kill all blobs that have not seen a measurement for the duration bt.kill_counter_th

coordinates = measure(ws::Workspace, bt::BlobTracker, img)

Detect blobs in the image and return the coordinates

#Arguments:

• ws: Workspace
• bt: BlobTracker
• img: image

showblobs(img::AbstractMatrix{T}, result::TrackingResult, m::Measurement; rad=8, recorder=nothing, display=true) where T

Overlay found blobs on img

#Arguments:

• img: an image
• result: a TrackingResult
• m: a Measurement
• rad: radius of blobs to draw
• recorder: an optional Recorder
• display = Base.display: function to display image. Use display=nothing to not display.

trace(b::Blob)

Get the location trace of a blob. Use tm = Matrix(t::Trace) to get an N×2 matrix. Use tm = replace(Float64.(tm), 0=>NaN) to create a matrix with NaNs where there were missing measurements, this is useful for plotting since it creates a gap where the missing measurement was.

trace(tr::TrackingResult; minlife=0)

Get all traces in a tracking result. Optionally filter based on minimum lifetime of a blob (the blob must have seen this many measurements).

tracem(b::Blob)

Get the measurement trace of a blob. Use skipmissing(t::Trace) to filter out missing measurements. Use tm = Matrix(t::Trace) to get an N×2 matrix. Use tm = replace(Float64.(tm), 0=>NaN) to create a matrix with NaNs where there were missing measurements, this is useful for plotting since it creates a gap where the missing measurement was.

track_blobs(bt::BlobTracker, vid; display=false, recorder=nothing, threads=Threads.nthreads() > 1, ignoreempty=false)

Main entry point to tracking blobs

#Arguments:

• bt: a BlobTracker
• vid: Some iterable thing that iterates images
• display = Base.display: function to display images live. Displaying live slows things down a bit. Use display=nothing to not display anything. Consider also `c = imshow(img);

displayfun = img -> imshow!(c["gui"]["canvas"],img); `.

• recorder: an optional Recorder that can record each frame to a video on disk. Recording things does not slow things down much and also does not affect memory usage much.
• threads: Use threaded processing of frames? Only useful if Julia is started with multiple threads.
• ignoreempty=false: wether or not to ignore display and recording of frames which contains no blobs and no measurements.

tune_sizes!(ws::Workspace, bt::BlobTracker, img)

Display a small GUI with a slider to help you tune the sizes

location(b)

Get the location of a blob as CartesianIndex

correct!(blobs, measurement::Measurement)

Correct the state of the blobs by incorporating the measurement in the Kalman filter

predict!(blobs)

Advance all blobs by predicting their next state

update!(ws::Workspace, bt::BlobTracker, coords_or_img, result::TrackingResult)

Perform one iteration of predict and correct

#Arguments:

• ws: a Workspace object
• bt: the blob tracker
• coords_or_img: vector of coordinates or an image
• result: a TrackingResult

update!(be::BackgroundExtractor, img)

Update a background model using img.