Tutorial
Necessary imports
using DifferentiableFrankWolfe: DiffFW, simplex_projection
using ForwardDiff: ForwardDiff
using FrankWolfe: UnitSimplexOracle
using Test: @test
using Zygote: Zygote
Constructing the wrapper
f(x, θ) = 0.5 * sum(abs2, x - θ) # minimizing the squared distance...
f_grad1(x, θ) = x - θ
lmo = UnitSimplexOracle(1.0) # ... to the probability simplex
dfw = DiffFW(f, f_grad1, lmo); # ... is equivalent to a simplex projection
Calling the wrapper
θ = rand(10)
10-element Vector{Float64}:
0.6578123431801371
0.3907719059508654
0.5934450116312675
0.9343851207101307
0.8576057999204917
0.9193946160936756
0.8638362569305599
0.1544875245128342
0.9472804385600486
0.19035067149221407
frank_wolfe_kwargs = (max_iteration=100, epsilon=1e-4)
y = dfw(θ; frank_wolfe_kwargs)
10-element SparseArrays.SparseVector{Float64, Int64} with 5 stored entries:
[4] = 0.229882
[5] = 0.153064
[6] = 0.214965
[7] = 0.15934
[9] = 0.242749
y_true = simplex_projection(θ)
@test Vector(y) ≈ Vector(y_true) atol = 1e-3
Test Passed
Differentiating the wrapper
J1 = Zygote.jacobian(_θ -> dfw(_θ; frank_wolfe_kwargs), θ)[1]
J1_true = Zygote.jacobian(simplex_projection, θ)[1]
@test J1 ≈ J1_true atol = 1e-3
Test Passed
J2 = ForwardDiff.jacobian(_θ -> dfw(_θ; frank_wolfe_kwargs), θ)
J2_true = ForwardDiff.jacobian(simplex_projection, θ)
@test J2 ≈ J2_true atol = 1e-3
Test Passed
This page was generated using Literate.jl.