Model nodes

AbstractMillModel

Supertype for any model defined in Mill.jl.

ArrayModel{T <: MillFunction} <: AbstractMillModel

A model node for processing ArrayNodes. It applies a (sub)model m stored in it to data of the ArrayNode.

Examples

julia> Random.seed!(0);

julia> n = ArrayNode(randn(Float32, 2, 2))
2×2 ArrayNode{Array{Float32,2},Nothing}:
 0.6791074  -0.3530074
 0.8284135  -0.13485387

julia> m = ArrayModel(Dense(2, 2))
ArrayModel(Dense(2, 2))

julia> m(n)
2×2 ArrayNode{Array{Float32,2},Nothing}:
 0.660979    -0.18795347
 0.10059327   0.27500954

See also: AbstractMillModel, IdentityModel, identity_model, ArrayNode.

identity_model()

Returns an ArrayModel realising the identity transformation.

Examples

julia> identity_model()
ArrayModel(identity)

See also: ArrayModel, IdentityModel.

IdentityModel

Alias for ArrayModel{typeof(identity)}.

See also: ArrayModel, identity_model.

BagModel{T <: AbstractMillModel, A <: Aggregation, U <: ArrayModel} <: AbstractMillModel

A model node for processing AbstractBagNodes. It first applies its "instance (sub)model" im on every instance, then performs elementwise segmented aggregation a and finally applies the final model bm on the aggregated representation of every bag in the data node.

Examples

julia> Random.seed!(0);

julia> n = BagNode(ArrayNode(randn(2, 2)), bags([0:-1, 1:2]))
BagNode with 2 obs
  └── ArrayNode(2×2 Array with Float64 elements) with 2 obs

julia> m = BagModel(ArrayModel(Dense(2, 2)), meanmax_aggregation(2), ArrayModel(Dense(5, 2)))
BagModel … ↦ ⟨SegmentedMean(2), SegmentedMax(2)⟩ ↦ ArrayModel(Dense(5, 2))
  └── ArrayModel(Dense(2, 2))

julia> m(n)
2×2 ArrayNode{Array{Float32,2},Nothing}:
 0.0  -1.1958722
 0.0   0.62269455

julia> m.bm(m.a(m.im(n.data), n.bags))
2×2 ArrayNode{Array{Float32,2},Nothing}:
 0.0  -1.1958722
 0.0   0.62269455

See also: AbstractMillModel, Aggregation, AbstractBagNode, BagNode, WeightedBagNode.

BagModel(im, a, bm=identity_model())

Construct a BagModel from the arguments. im should AbstractMillModel, aAggregation, and bmArrayModel.

It is also possible to pass any function (Flux.Dense, Flux.Chain, identity...) as im or bm. In that case, they are wrapped into an ArrayNode.

Examples

julia> m = BagModel(ArrayModel(Dense(2, 3)), max_aggregation(2), ArrayModel(Dense(3, 2)))
BagModel … ↦ ⟨SegmentedMax(2)⟩ ↦ ArrayModel(Dense(3, 2))
  └── ArrayModel(Dense(2, 3))

julia> m = BagModel(Dense(2, 3), mean_aggregation(2))
BagModel … ↦ ⟨SegmentedMean(2)⟩ ↦ ArrayModel(identity)
  └── ArrayModel(Dense(2, 3))

See also: AbstractMillModel, Aggregation, AbstractBagNode, BagNode, WeightedBagNode.

ProductModel{T <: VecOrTupOrNTup{<:AbstractMillModel}, U <: ArrayModel} <: AbstractMillModel

A model node for processing ProductNodes. For each subtree of the data node it applies one (sub)model from ms and then applies m on the concatenation of results.

Examples

julia> Random.seed!(0);

julia> n = ProductNode((a=ArrayNode([0 1; 2 3]), b=ArrayNode([4 5; 6 7])))
ProductNode with 2 obs
  ├── a: ArrayNode(2×2 Array with Int64 elements) with 2 obs
  └── b: ArrayNode(2×2 Array with Int64 elements) with 2 obs

julia> m1 = ProductModel((a=ArrayModel(Dense(2, 2)), b=ArrayModel(Dense(2, 2))))
ProductModel … ↦ ArrayModel(identity)
  ├── a: ArrayModel(Dense(2, 2))
  └── b: ArrayModel(Dense(2, 2))

julia> m1(n)
4×2 ArrayNode{Array{Float32,2},Nothing}:
 -1.2838157  -1.25368
  1.8022565   3.7105856
 -4.036125   -5.012517
  0.5873234   0.37193537

julia> m2 = ProductModel((a=identity, b=identity))
ProductModel … ↦ ArrayModel(identity)
  ├── a: ArrayModel(identity)
  └── b: ArrayModel(identity)

julia> m2(n)
4×2 ArrayNode{Array{Int64,2},Nothing}:
 0  1
 2  3
 4  5
 6  7

See also: AbstractMillModel, AbstractProductNode, ProductNode.

ProductModel(ms, m=identity_model())

Construct a ProductModel from the arguments. ms should an iterable (Tuple, NamedTuple or Vector) of one or more AbstractMillModels, and m should be an ArrayModel.

It is also possible to pass any function (Flux.Dense, Flux.Chain, identity...) as elements of ms or as m. In that case, they are wrapped into an ArrayNode.

If ms is AbstractMillModel, a one-element Tuple is constructed from it.

Examples

julia> ProductModel((a=ArrayModel(Dense(2, 2)), b=identity))
ProductModel … ↦ ArrayModel(identity)
  ├── a: ArrayModel(Dense(2, 2))
  └── b: ArrayModel(identity)

julia> ProductModel((identity_model(), BagModel(ArrayModel(Dense(2, 2)), mean_aggregation(2), identity)))
ProductModel … ↦ ArrayModel(identity)
  ├── ArrayModel(identity)
  └── BagModel … ↦ ⟨SegmentedMean(2)⟩ ↦ ArrayModel(identity)
        └── ArrayModel(Dense(2, 2))

julia> ProductModel(identity)
ProductModel … ↦ ArrayModel(identity)
  └── ArrayModel(identity)

See also: AbstractMillModel, AbstractProductNode, ProductNode.

LazyModel{Name, T} <: AbstractMillModel

A model node for processing LazyNodes. It applies a (sub)model m stored in it to data of the LazyNode after calling Mill.unpack2mill.

Examples

function Mill.unpack2mill(ds::LazyNode{:Sentence})
    s = split.(ds.data, " ")
    x = NGramMatrix(reduce(vcat, s))
    BagNode(ArrayNode(x), Mill.length2bags(length.(s)))
end

julia> Random.seed!(0);

julia> n = LazyNode{:Sentence}(["foo bar", "baz"])
LazyNode{Sentence} with 2 obs

julia> m = LazyModel{:Sentence}(BagModel(Dense(2053, 3), mean_aggregation(3), identity))
LazyModel{Sentence}
  └── BagModel … ↦ ⟨SegmentedMean(3)⟩ ↦ ArrayModel(identity)
        └── ArrayModel(Dense(2053, 3))

julia> m(n)
4×2 ArrayNode{Array{Float32,2},Nothing}:
 -0.006524003  -0.02167379
  0.033673763   0.05508352
 -0.06166087    0.07056637
  1.0986123     0.6931472

See also: AbstractMillModel, LazyNode, Mill.unpack2mill.

LazyModel([Name::Symbol], m::AbstractMillModel)
LazyModel{Name}(m::AbstractMillModel)

Construct a new LazyModel with name Name, and model m.

Examples

julia> LazyModel{:Sentence}(ArrayModel(Dense(2, 2)))
LazyModel{Sentence}
  └── ArrayModel(Dense(2, 2))

See also: AbstractMillModel, LazyNode, Mill.unpack2mill.

reflectinmodel(x::AbstractNode, fm=d -> Dense(d, 10), fa=d -> meanmax_aggregation(d);
    fsm=Dict(), fsa=Dict(), single_key_identity=true, single_scalar_identity=true)

Build a Mill.jl model capable of processing x.

All inner Dense layers are constructed using fm, a function accepting input dimension d and returning suitable model. All aggregation operators are constructed using fa in a similar manner.

More fine-grained control can be achieved with fsm and fsa keyword arguments, which should be Dicts of c => f pairs, where c is a String traversal code from HierarchicalUtils.jl and f is a function. These definitions override fm and fa.

If a ProductNode with only a single child (subtree) is encountered, its final m model is instantiated as identity instead of using fm and fsm. This can be controlled with single_key_identity.

Similarly, if an ArrayNode contains data X where size(X, 1) is 1, the corresponding model is instantiated as identity unless single_scalar_identity is false.

Examples

julia> n1 = ProductNode((; a=ArrayNode(NGramMatrix(["a", "b"]))))
ProductNode with 2 obs
  └── a: ArrayNode(2053×2 NGramMatrix with Int64 elements) with 2 obs

julia> n2 = ProductNode((ArrayNode([0 1]), BagNode(ArrayNode([0 1; 2 3]), bags([1:1, 2:2]))))
ProductNode with 2 obs
  ├── ArrayNode(1×2 Array with Int64 elements) with 2 obs
  └── BagNode with 2 obs
        └── ArrayNode(2×2 Array with Int64 elements) with 2 obs

julia> n = ProductNode((n1, n2))
ProductNode with 2 obs
  ├── ProductNode with 2 obs
  │     └── a: ArrayNode(2053×2 NGramMatrix with Int64 elements) with 2 obs
  └── ProductNode with 2 obs
        ├── ArrayNode(1×2 Array with Int64 elements) with 2 obs
        └── BagNode with 2 obs
              ⋮

julia> printtree(n; trav=true)
ProductNode with 2 obs [""]
  ├── ProductNode with 2 obs ["E"]
  │     └── a: ArrayNode(2053×2 NGramMatrix with Int64 elements) with 2 obs ["M"]
  └── ProductNode with 2 obs ["U"]
        ├── ArrayNode(1×2 Array with Int64 elements) with 2 obs ["Y"]
        └── BagNode with 2 obs ["c"]
              └── ArrayNode(2×2 Array with Int64 elements) with 2 obs ["e"]

julia> reflectinmodel(n) |> printtree
ProductModel … ↦ ArrayModel(Dense(20, 10))
  ├── ProductModel … ↦ ArrayModel(identity)
  │     └── a: ArrayModel(Dense(2053, 10))
  └── ProductModel … ↦ ArrayModel(Dense(11, 10))
        ├── ArrayModel(identity)
        └── BagModel … ↦ ⟨SegmentedMean(10), SegmentedMax(10)⟩ ↦ ArrayModel(Dense(21, 10))
              └── ArrayModel(Dense(2, 10))

julia> reflectinmodel(n, d -> Dense(d, 3), d -> mean_aggregation(d)) |> printtree
ProductModel … ↦ ArrayModel(Dense(6, 3))
  ├── ProductModel … ↦ ArrayModel(identity)
  │     └── a: ArrayModel(Dense(2053, 3))
  └── ProductModel … ↦ ArrayModel(Dense(4, 3))
        ├── ArrayModel(identity)
        └── BagModel … ↦ ⟨SegmentedMean(3)⟩ ↦ ArrayModel(Dense(4, 3))
              └── ArrayModel(Dense(2, 3))

julia> reflectinmodel(n, d -> Dense(d, 3), d -> mean_aggregation(d);
                        fsm=Dict("e" => d -> Chain(Dense(d, 2), Dense(2, 2))),
                        fsa=Dict("c" => d -> lse_aggregation(d)),
                        single_key_identity=false,
                        single_scalar_identity=false) |> printtree
ProductModel … ↦ ArrayModel(Dense(6, 3))
  ├── ProductModel … ↦ ArrayModel(Dense(3, 3))
  │     └── a: ArrayModel(Dense(2053, 3))
  └── ProductModel … ↦ ArrayModel(Dense(6, 3))
        ├── ArrayModel(Dense(1, 3))
        └── BagModel … ↦ ⟨SegmentedLSE(2)⟩ ↦ ArrayModel(Dense(3, 3))
              └── ArrayModel(Chain(Dense(2, 2), Dense(2, 2)))

See also: AbstractNode, AbstractMillModel, ProductNode, ArrayNode.