Panel Operations

PanelStructure{R<:Signed, IP<:AbstractVector, TP<:AbstractVector}

Panel data structure defined by unique combinations of unit ids and time periods. It contains the information required for certain operations such as lag and diff. See also setpanel.

Fields

• refs::Vector{R}: reference values that allow obtaining time gaps by taking differences.
• invrefs::Dict{R, Int}: inverse map from refs to indices.
• idpool::IP: unique unit ids.
• timepool::TP: sorted unique time periods.
• laginds::Dict{Int, Vector{Int}}: a map from lag distances to vectors of indices of lagged values.

diff(panel::PanelStructure, v::AbstractArray; kwargs...)

Return the differences of v within observations for each unit in panel. By default, it calculates the first differences. See also diff!.

Keywords

• order::Integer=1: the order of differences to be taken.
• l::Integer=1: the time interval between each pair of observations.
• default=missing: default values for indices where the differences do not exist.

aligntime(col::AbstractArray, time::ScaledArrOrSub)
aligntime(col::AbstractArray, time::RotatingTimeArray)
aligntime(data, colname::Union{Symbol,Integer}, timename::Union{Symbol,Integer})

Convert a column of time values col to a ScaledArray with a pool that has the same first element and step size as the pool from the ScaledArray time. If time is a RotatingTimeArray with the time field being a ScaledArray, the returned array is also a RotatingTimeArray with the time field being the converted ScaledArray. Alternative, the arrays may be specified with a Tables.jl-compatible data table and column indices colname and timename. See also settime.

This is useful for representing all discretized time periods with the same scale so that the underlying reference values returned by DataAPI.refarray can be directly comparable across the columns.

cellrows(cols::VecColumnTable, refrows::IdDict)

A utility function for processing the object refrows returned by findcell. Unique row values from cols corresponding to the keys in refrows are sorted lexicographically and stored as rows in a new VecColumnTable. Groups of row indices from the values of refrows are permuted to match the order of row values and collected in a Vector.

Returns

• cells::VecColumnTable: unique row values from columns in cols.
• rows::Vector{Vector{Int}}: row indices for each combination.

diff!(dest::AbstractArray, panel::PanelStructure, v::AbstractArray; kwargs...)

Take the differences of v within observations for each unit in panel and store the result in dest. By default, it calculates the first differences. See also diff.

Keywords

• order::Integer=1: the order of differences to be taken.
• l::Integer=1: the time interval between each pair of observations.
• default=missing: default values for indices where the differences do not exist.

findcell(cols::VecColumnTable)
findcell(names, data, esample=Colon())

Group the row indices of a collection of data columns so that the combination of row values from these columns are the same within each group.

Instead of directly providing the relevant portions of columns as VecColumnTable`, one may specify thenamesof columns fromdataof anyTables.jl-compatible table type over selected rows indicated byesample. Note that unlessesamplecovers all rows ofdata, the row indices are those for the subsample selected based onesamplerather than those for the fulldata`.

Returns

• IdDict{Tuple, Vector{Int}}: a map from unique row values to row indices.

findlag!(panel::PanelStructure, l::Integer=1)

Construct a vector of indices of the lth lagged values for all id-time combinations of panel and save the result in panel.laginds. If a lagged value does not exist, its index is filled with 0. See also ilag!.

findlead!(panel::PanelStructure, l::Integer=1)

Construct a vector of indices of the lth lead values for all id-time combinations of panel and save the result in panel.laginds. If a lead value does not exist, its index is filled with 0. See also ilead!.

ilag!(panel::PanelStructure, l::Integer=1)

Return a vector of indices of the lth lagged values for all id-time combinations of panel. The indices are retrieved from panel if they have been collected before. Otherwise, they are created by calling findlag!. See also ilead!.

ilead!(panel::PanelStructure, l::Integer=1)

Return a vector of indices of the lth lead values for all id-time combinations of panel. The indices are retrieved from panel if they have been collected before. Otherwise, they are created by calling findlead!. See also ilag!.

setpanel(data, idname, timename; step, reftype, rotation)
setpanel(id::AbstractArray, time::AbstractArray; step, reftype, rotation)

Declare a PanelStructure which is required for certain operations such as lag and diff. Unit IDs and time values can be provided either as a table containing the relevant columns or as arrays. timestep must be specified unless the time array is a ScaledArray that is returned by settime.

Arguments

• data: a Tables.jl-compatible data table.
• idname::Union{Symbol,Integer}: the name of the column in data that contains unit IDs.
• timename::Union{Symbol,Integer}: the name of the column in data that contains time values.
• id::AbstractArray: the array containing unit IDs (only needed for the alternative method).
• time::AbstractArray: the array containing time values (only needed for the alternative method).

Keywords

• step=nothing: the length of each time interval; try step=one(eltype(time)) if not specified.
• reftype::Type{<:Signed}=Int32: the element type of the reference values for PanelStructure.
• rotation=nothing: rotation groups in a rotating sampling design; use RotatingTimeValues as reference values.

settime(time::AbstractArray, step; start, stop, reftype, rotation)

Convert a column of time values to a ScaledArray for representing discretized time periods of uniform length. If rotation is specified (time values belong to multiple rotation groups), a RotatingTimeArray is returned with the time field being a ScaledArray. The returned array ensures well-defined time intervals for operations involving relative time (such as lag and diff). See also aligntime.

Arguments

• time::AbstractArray: the array containing time values.
• step=nothing: the length of each time interval; try step=one(eltype(time)) if not specified.

Keywords

• start=nothing: the first element of the pool of the returned ScaledArray.
• stop=nothing: the last element of the pool of the returned ScaledArray.
• reftype::Type{<:Signed}=Int32: the element type of the reference values for the ScaledArray.
• rotation=nothing: rotation groups in a rotating sampling design.

lag(panel::PanelStructure, v::AbstractArray, l::Integer=1; default=missing)

Return a vector of lth lagged values of v with missing values filled with default. The panel structure is respected. See also ilag! and lead.

lead(panel::PanelStructure, v::AbstractArray, l::Integer=1; default=missing)

Return a vector of lth lead values of v with missing values filled with default. The panel structure is respected. See also ilead! and lag.

RotatingTimeArray{T<:RotatingTimeValue,N,C,I} <: AbstractArray{T,N}

Array type for RotatingTimeValues that stores the field values rotation and time in two arrays for efficiency. The two arrays that hold the field values for all elements can be accessed as properties.

RotatingTimeArray(rotation::AbstractArray, time::AbstractArray)

Construct a RotatingTimeValue from arrays of rotation and time.

RotatingTimeValue{R, T}

A wrapper around a time value for distinguishing potentially different rotation group it could belong to in a rotating sampling design. See also rotatingtime and settime.

Fields

• rotation::R: a rotation group in a rotating sampling design.
• time::T: a time value belonged to the rotation group.

rotatingtime(rotation, time)

Construct RotatingTimeValues from rotation and time. This method simply broadcasts the default constructor over the arguments.