CumulantsFeatures.cumfsel
— Methodfunction cumfsel(Σ::SymmetricTensor{T,2}, c::SymmetricTensor{T, N}, f::String, k::Int = Σ.dats) where {T <: AbstractFloat, N}
Returns an Array of tuples (ind::Array{Bool}, fval::Float64, i::Int). Given k-th Array ind are marginals removed after k -steps as those with low N'th order dependency, fval, the value of the target function at step k and i, a feature removed at step k.
Uses Σ - the covariance matrix and c - the N'th cumulant tensor to measure the N'th order dependencies between marginals. Function f is the optimization function, ["hosvd", "norm", "mev"] are supported.
julia> Random.seed!(42);
julia> x = rand(12,10);
julia> c = cumulants(x, 4);
julia> cumfsel(c[2], c[4], "hosvd")
10-element Array{Any,1}:
(Bool[true, true, true, false, true, true, true, true, true, true], 27.2519, 4)
(Bool[true, true, false, false, true, true, true, true, true, true], 22.6659, 3)
(Bool[true, true, false, false, false, true, true, true, true, true], 18.1387, 5)
(Bool[false, true, false, false, false, true, true, true, true, true], 14.4492, 1)
(Bool[false, true, false, false, false, true, true, false, true, true], 11.2086, 8)
(Bool[false, true, false, false, false, true, true, false, true, false], 7.84083, 10)
(Bool[false, false, false, false, false, true, true, false, true, false], 5.15192, 2)
(Bool[false, false, false, false, false, false, true, false, true, false], 2.56748, 6)
(Bool[false, false, false, false, false, false, true, false, false, false], 0.30936, 9)
(Bool[false, false, false, false, false, false, false, false, false, false], 0.0, 7)
CumulantsFeatures.cumfsel
— Methodcumfsel(Σ::Matrix{T}, k::Int = size(Σ, 1))
cumfsel that uses as default the mev method
CumulantsFeatures.greedesearchdata
— Methodgreedesearchdata(Σ::SymmetricTensor{T,2}, c::SymmetricTensor{T, N}, maxfunction::Function, k::Int)
returns array of bools that are non-outliers features
CumulantsFeatures.greedestep
— Methodgreedestep(c::Vector{Array{Float}}, maxfunction::Function, ls::Vector{Bool})
Returns vector of bools that determines bands that maximise a function. True means include a band, false exclude a band. It changes one true to false in input ls
julia> a = reshape(collect(1.:9.), 3,3);
julia> b = reshape(collect(1.: 27.), 3,3,3);
julia> testf(ar,bool)= det(ar[1][bool,bool])
julia> greedestep(ar, testf, [true, true, true])
3-element Array{Bool,1}:
true
true
false
CumulantsFeatures.hosvdapprox
— Methoddetoverdetfitfunction(a::Array{N}, b::Array{N})
computes the maximizing function det(Cn)/det(C2)^(n/2). It assumes, that product of singular values from HOSVD of tensor is a good approximation of hyperdeterminant of the tensor (whatever that means). Returns the value of the maximizin function
CumulantsFeatures.hosvdc4detect
— Methodfunction hosvdc4detect(X::Matrix{T}, β::Float64 = 4.1, r::Int = 3; b::Int = 4)
Takes data in the form of matrix where first index correspond to realisations and second to feratures (marginals). Using the HOSVD of the 4'th cumulant's tensor of data returns the array of Bool that correspond to outlier realisations. β is the sensitivity parameter while r a number of specific directions, data are projected onto. Parameter b is a size of blocks in a SymmetricTensors structure
julia> Random.seed!(42);
julia> x = vcat(rand(8,2), 20*rand(2,2))
10×2 Array{Float64,2}:
0.533183 0.956916
0.454029 0.584284
0.0176868 0.937466
0.172933 0.160006
0.958926 0.422956
0.973566 0.602298
0.30387 0.363458
0.176909 0.383491
11.8582 5.25618
14.9036 10.059
julia> rxdetect(x, 0.95)
10-element Array{Bool,1}:
false
false
false
false
false
false
false
false
true
true
CumulantsFeatures.hosvdstep
— Methodhosvdstep(X::Matrix{T}, ls::Vector{Bool}, β::Float64, r::Int, cc::SymmetricTensor{T,4}) where T <: AbstractFloat
Returns Vector{Bool} - outliers form an itteration step of th hosvd algorithm and Vector{Float64} vector of univariate kurtosis for data projected on specific directions
CumulantsFeatures.reduceband
— Methodreduceband(ar::Array{N}, k::Vector{Bool})
Returns n-array without values at indices in ind
julia> reshape(collect(1.:27.),(3,3,3))
3×3×3 Array{Float64,3}:
[:, :, 1] =
1.0 4.0 7.0
2.0 5.0 8.0
3.0 6.0 9.0
[:, :, 2] =
10.0 13.0 16.0
11.0 14.0 17.0
12.0 15.0 18.0
[:, :, 3] =
19.0 22.0 25.0
20.0 23.0 26.0
21.0 24.0 27.0
julia> reduceband(reshape(collect(1.:27.),(3,3,3)), [true, false, false])
1×1×1 Array{Float64,3}:
[:, :, 1] =
1.0
TODO reimplement in blocks
CumulantsFeatures.rxdetect
— Methodfunction rxdetect(X::Matrix{T}, alpha::Float64 = 0.99)
Takes data in the form of matrix where first index correspond to realisations and second to feratures (marginals). Using the RX (Reed-Xiaoli) Anomaly Detection returns the array of Bool that correspond to outlier realisations. alpha is the sensitivity parameter of the RX detector
julia> Random.seed!(42);
julia> x = vcat(rand(8,2), 20*rand(2,2))
10×2 Array{Float64,2}:
0.533183 0.956916
0.454029 0.584284
0.0176868 0.937466
0.172933 0.160006
0.958926 0.422956
0.973566 0.602298
0.30387 0.363458
0.176909 0.383491
11.8582 5.25618
14.9036 10.059
julia> rxdetect(x, 0.95)
10-element Array{Bool,1}:
false
false
false
false
false
false
false
false
true
true
CumulantsFeatures.unfoldsym
— Methodfunction unfoldsym{T <: Real, N}(ar::Array{T,N})
Returns a matrix of size (i, k^(N-1)) that is an unfold of symmetric array ar
CumulantsFeatures.unfoldsym
— MethodTODO reimplement in blocks
CumulantsFeatures.updatemoments
— Methodupdatemoments(ma::Vector{SymmetricTensor{Float64,N} where N}, t::Int, X::Matix{T}, ls::Vector{Bool}, lsold::Vector{Bool})
Returns Array{SymmetricTensor} - an array of updated moments (after outliers removal) and t - number of realisations of data after outliers removal