CumulantsFeatures.cumfselMethod

function cumfsel(Σ::SymmetricTensor{T,2}, c::SymmetricTensor{T, N}, f::String, k::Int = Σ.dats) where {T <: AbstractFloat, N}

Returns an Array of tuples (ind::Array{Bool}, fval::Float64, i::Int). Given k-th Array ind are marginals removed after k -steps as those with low N'th order dependency, fval, the value of the target function at step k and i, a feature removed at step k.

Uses Σ - the covariance matrix and c - the N'th cumulant tensor to measure the N'th order dependencies between marginals. Function f is the optimization function, ["hosvd", "norm", "mev"] are supported.


julia> Random.seed!(42);

julia> x = rand(12,10);

julia> c = cumulants(x, 4);

julia> cumfsel(c[2], c[4], "hosvd")
10-element Array{Any,1}:
(Bool[true, true, true, false, true, true, true, true, true, true], 27.2519, 4)
(Bool[true, true, false, false, true, true, true, true, true, true], 22.6659, 3)
(Bool[true, true, false, false, false, true, true, true, true, true], 18.1387, 5)
(Bool[false, true, false, false, false, true, true, true, true, true], 14.4492, 1)
(Bool[false, true, false, false, false, true, true, false, true, true], 11.2086, 8)
(Bool[false, true, false, false, false, true, true, false, true, false], 7.84083, 10)
(Bool[false, false, false, false, false, true, true, false, true, false], 5.15192, 2)
(Bool[false, false, false, false, false, false, true, false, true, false], 2.56748, 6)
(Bool[false, false, false, false, false, false, true, false, false, false], 0.30936, 9)
(Bool[false, false, false, false, false, false, false, false, false, false], 0.0, 7)
CumulantsFeatures.greedestepMethod

greedestep(c::Vector{Array{Float}}, maxfunction::Function, ls::Vector{Bool})

Returns vector of bools that determines bands that maximise a function. True means include a band, false exclude a band. It changes one true to false in input ls

julia> a = reshape(collect(1.:9.), 3,3);

julia> b = reshape(collect(1.: 27.), 3,3,3);

julia> testf(ar,bool)= det(ar[1][bool,bool])

julia> greedestep(ar, testf, [true, true, true])
3-element Array{Bool,1}:
true
true
false
CumulantsFeatures.hosvdapproxMethod

detoverdetfitfunction(a::Array{N}, b::Array{N})

computes the maximizing function det(Cn)/det(C2)^(n/2). It assumes, that product of singular values from HOSVD of tensor is a good approximation of hyperdeterminant of the tensor (whatever that means). Returns the value of the maximizin function

CumulantsFeatures.hosvdc4detectMethod

function hosvdc4detect(X::Matrix{T}, β::Float64 = 4.1, r::Int = 3; b::Int = 4)

Takes data in the form of matrix where first index correspond to realisations and second to feratures (marginals). Using the HOSVD of the 4'th cumulant's tensor of data returns the array of Bool that correspond to outlier realisations. β is the sensitivity parameter while r a number of specific directions, data are projected onto. Parameter b is a size of blocks in a SymmetricTensors structure

julia> Random.seed!(42);

julia> x = vcat(rand(8,2), 20*rand(2,2))
10×2 Array{Float64,2}:
0.533183    0.956916
0.454029    0.584284
0.0176868   0.937466
0.172933    0.160006
0.958926    0.422956
0.973566    0.602298
0.30387     0.363458
0.176909    0.383491
11.8582      5.25618
14.9036     10.059

julia> rxdetect(x, 0.95)
10-element Array{Bool,1}:
false
false
false
false
false
false
false
false
true
true
CumulantsFeatures.hosvdstepMethod

hosvdstep(X::Matrix{T}, ls::Vector{Bool}, β::Float64, r::Int, cc::SymmetricTensor{T,4}) where T <: AbstractFloat

Returns Vector{Bool} - outliers form an itteration step of th hosvd algorithm and Vector{Float64} vector of univariate kurtosis for data projected on specific directions

CumulantsFeatures.reducebandMethod

reduceband(ar::Array{N}, k::Vector{Bool})

Returns n-array without values at indices in ind

julia>  reshape(collect(1.:27.),(3,3,3))
3×3×3 Array{Float64,3}:
[:, :, 1] =
1.0  4.0  7.0
2.0  5.0  8.0
3.0  6.0  9.0

[:, :, 2] =
10.0  13.0  16.0
11.0  14.0  17.0
12.0  15.0  18.0

[:, :, 3] =
19.0  22.0  25.0
20.0  23.0  26.0
21.0  24.0  27.0

julia> reduceband(reshape(collect(1.:27.),(3,3,3)), [true, false, false])
1×1×1 Array{Float64,3}:
[:, :, 1] =
1.0

TODO reimplement in blocks

CumulantsFeatures.rxdetectMethod

function rxdetect(X::Matrix{T}, alpha::Float64 = 0.99)

Takes data in the form of matrix where first index correspond to realisations and second to feratures (marginals). Using the RX (Reed-Xiaoli) Anomaly Detection returns the array of Bool that correspond to outlier realisations. alpha is the sensitivity parameter of the RX detector

julia> Random.seed!(42);

julia> x = vcat(rand(8,2), 20*rand(2,2))
10×2 Array{Float64,2}:
0.533183    0.956916
0.454029    0.584284
0.0176868   0.937466
0.172933    0.160006
0.958926    0.422956
0.973566    0.602298
0.30387     0.363458
0.176909    0.383491
11.8582      5.25618
14.9036     10.059

julia> rxdetect(x, 0.95)
10-element Array{Bool,1}:
false
false
false
false
false
false
false
false
true
true
CumulantsFeatures.updatemomentsMethod

updatemoments(ma::Vector{SymmetricTensor{Float64,N} where N}, t::Int, X::Matix{T}, ls::Vector{Bool}, lsold::Vector{Bool})

Returns Array{SymmetricTensor} - an array of updated moments (after outliers removal) and t - number of realisations of data after outliers removal