Utilities

sparsify(x, nnzrate)

Replace AbstractMatrixx with SparseMatrixCSC if at most nnzrate fraction of elements is non-zero.

julia> n = ArrayNode([0 0; 0 0])
2×2 ArrayNode{Array{Int64,2},Nothing}:
 0  0
 0  0

julia> Mill.mapdata(i -> sparsify(i, 0.05), n)
2×2 ArrayNode{SparseMatrixCSC{Int64,Int64},Nothing}:
 0  0
 0  0

See also: Mill.mapdata.

list_lens(n)

Return a Vector of Setfield.Lenses for accessing all nodes/fields in n.

Examples

julia> n = ProductNode((BagNode(missing, bags([0:-1, 0:-1])), ArrayNode([1 2; 3 4])))
ProductNode with 2 obs
  ├── BagNode with 2 obs
  │     └── ∅
  └── ArrayNode(2×2 Array with Int64 elements) with 2 obs

julia> list_lens(n)
9-element Array{Lens,1}:
 (@lens _)
 (@lens _.data[1])
 (@lens _.data[1].data)
 (@lens _.data[1].bags)
 (@lens _.data[1].metadata)
 (@lens _.data[2])
 (@lens _.data[2].data)
 (@lens _.data[2].metadata)
 (@lens _.metadata)

See also: pred_lens, find_lens, findnonempty_lens.

find_lens(n, x)

Return a Vector of Setfield.Lenses for accessing all nodes/fields in n that return true when compared to x using Base.===.

Examples

julia> n = ProductNode((BagNode(missing, bags([0:-1, 0:-1])), ArrayNode([1 2; 3 4])))
ProductNode with 2 obs
  ├── BagNode with 2 obs
  │     └── ∅
  └── ArrayNode(2×2 Array with Int64 elements) with 2 obs

julia> find_lens(n, n.data[1])
1-element Array{Union{Setfield.ComposedLens{_A,_B} where _B where _A, Setfield.PropertyLens{_A} where _A},1}:
 (@lens _.data[1])

See also: pred_lens, list_lens, findnonempty_lens.

findnonempty_lens(n)

Return a Vector of Setfield.Lenses for accessing all nodes/fields in n that have at least one observation.

Examples

julia> n = ProductNode((BagNode(missing, bags([0:-1, 0:-1])), ArrayNode([1 2; 3 4])))
ProductNode with 2 obs
  ├── BagNode with 2 obs
  │     └── ∅
  └── ArrayNode(2×2 Array with Int64 elements) with 2 obs

julia> findnonempty_lens(n)
3-element Array{Lens,1}:
 (@lens _)
 (@lens _.data[1])
 (@lens _.data[2])

See also: pred_lens, list_lens, find_lens.

pred_lens(p, n)

Return a Vector of Setfield.Lenses for accessing all nodes/fields in n conforming to predicate p.

Examples

julia> n = ProductNode((BagNode(missing, bags([0:-1, 0:-1])), ArrayNode([1 2; 3 4])))
ProductNode with 2 obs
  ├── BagNode with 2 obs
  │     └── ∅
  └── ArrayNode(2×2 Array with Int64 elements) with 2 obs

julia> pred_lens(x -> x isa ArrayNode, n)
1-element Array{Union{Setfield.ComposedLens{_A,_B} where _B where _A, Setfield.PropertyLens{_A} where _A},1}:
 (@lens _.data[2])

See also: list_lens, find_lens, findnonempty_lens.

code2lens(n, c)

Convert code c from HierarchicalUtils.jl traversal to Setfield.Lens such that they access the same node in tree n.

Examples

julia> n = ProductNode((BagNode(missing, bags([0:-1, 0:-1])), ArrayNode([1 2; 3 4])));

julia> printtree(n; trav=true)
ProductNode with 2 obs [""]
  ├── BagNode with 2 obs ["E"]
  │     └── ∅ ["M"]
  └── ArrayNode(2×2 Array with Int64 elements) with 2 obs ["U"]

julia> code2lens(n, "U")
(@lens _.data[2])

See also: lens2code.

lens2code(n, l)

Convert Setfield.Lens l to code c from HierarchicalUtils.jl traversal such that they access the same node in tree n.

Examples

julia> n = ProductNode((BagNode(missing, bags([0:-1, 0:-1])), ArrayNode([1 2; 3 4])));

julia> printtree(n; trav=true)
ProductNode with 2 obs [""]
  ├── BagNode with 2 obs ["E"]
  │     └── ∅ ["M"]
  └── ArrayNode(2×2 Array with Int64 elements) with 2 obs ["U"]

julia> lens2code(n, (@lens _.data[2]))
"U"

See also: code2lens.

model_lens(m, l)

Convert Setfield.Lensl for a data node to a new lens for accessing the same location in model m.

Examples

julia> n = ProductNode((BagNode(ArrayNode(randn(2, 2)), bags([0:-1, 0:-1])), ArrayNode([1 2; 3 4])))
ProductNode with 2 obs
  ├── BagNode with 2 obs
  │     └── ArrayNode(2×2 Array with Float64 elements) with 2 obs
  └── ArrayNode(2×2 Array with Int64 elements) with 2 obs

julia> m = reflectinmodel(n)
ProductModel … ↦ ArrayModel(Dense(20, 10))
  ├── BagModel … ↦ ⟨SegmentedMean(10), SegmentedMax(10)⟩ ↦ ArrayModel(Dense(21, 10))
  │     └── ArrayModel(Dense(2, 10))
  └── ArrayModel(Dense(2, 10))

julia> model_lens(m, (@lens _.data[2]))
(@lens _.ms[2])

See also: data_lens.

data_lens(n, l)

Convert Setfield.Lensl for a model node to a new lens for accessing the same location in data node n.

Examples

julia> n = ProductNode((BagNode(ArrayNode(randn(2, 2)), bags([0:-1, 0:-1])), ArrayNode([1 2; 3 4])))
ProductNode with 2 obs
  ├── BagNode with 2 obs
  │     └── ArrayNode(2×2 Array with Float64 elements) with 2 obs
  └── ArrayNode(2×2 Array with Int64 elements) with 2 obs

julia> m = reflectinmodel(n)
ProductModel … ↦ ArrayModel(Dense(20, 10))
  ├── BagModel … ↦ ⟨SegmentedMean(10), SegmentedMax(10)⟩ ↦ ArrayModel(Dense(21, 10))
  │     └── ArrayModel(Dense(2, 10))
  └── ArrayModel(Dense(2, 10))

julia> data_lens(n, (@lens _.ms[2]))
(@lens _.data[2])

See also: data_lens.

replacein(n, old, new)

Replace in data node or model n each occurence of old by new.

Short description

Examples

julia> n = ProductNode((BagNode(ArrayNode(randn(2, 2)), bags([0:-1, 0:-1])), ArrayNode([1 2; 3 4])))
ProductNode with 2 obs
  ├── BagNode with 2 obs
  │     └── ArrayNode(2×2 Array with Float64 elements) with 2 obs
  └── ArrayNode(2×2 Array with Int64 elements) with 2 obs

julia> replacein(n, n.data[1], ArrayNode(maybehotbatch([1, 2], 1:2)))
ProductNode with 2 obs
  ├── ArrayNode(2×2 MaybeHotMatrix with Bool elements) with 2 obs
  └── ArrayNode(2×2 Array with Int64 elements) with 2 obs