Functions

Indicator of an affine subspace

IndAffine(A, b; iterative=false)

If A is a matrix (dense or sparse) and b is a vector, returns the indicator function of the set

\[S = \{x : Ax = b\}.\]

If A is a vector and b is a scalar, returns the indicator function of the set

\[S = \{x : \langle A, x \rangle = b\}.\]

By default, a direct method (QR factorization of matrix A') is used to evaluate prox!. If iterative=true, then prox! is evaluated approximately using an iterative method instead.

Indicator of a $L_∞$ norm ball

IndBallLinf(r=1.0)

Returns the indicator function of the set

\[S = \{ x : \max (|x_i|) \leq r \}.\]

Parameter r must be positive.

Indicator of a $L_0$ pseudo-norm ball

IndBallL0(r=1)

Returns the indicator function of the set

\[S = \{ x : \mathrm{nnz}(x) \leq r \}.\]

Parameter r must be a positive integer.

Indicator of a $L_1$ norm ball

IndBallL1(r=1.0)

Returns the indicator function of the set

\[S = \left\{ x : \sum_i |x_i| \leq r \right\}.\]

Parameter r must be positive.

Indicator of a Euclidean ball

IndBallL2(r=1.0)

Returns the indicator function of the set

\[S = \{ x : \|x\| \leq r \},\]

where $\|\cdot\|$ is the $L_2$ (Euclidean) norm. Parameter r must be positive.

Indicator of rank ball

IndBallRank(r=1)

Returns the indicator function of the set of matrices of rank at most r:

\[S = \{ X : \mathrm{rank}(X) \leq r \},\]

Parameter r must be a positive integer.

Indicator of the product of binary sets

IndBinary(low, up)

Returns the indicator function of the set

\[S = \{ x : x_i = low_i\ \text{or}\ x_i = up_i \},\]

Parameters low and up can be either scalars or arrays of the same dimension as the space.

Indicator of a box

IndBox(low, up)

Returns the indicator function of the set

\[S = \{ x : low \leq x \leq up \}.\]

Parameters low and up can be either scalars or arrays of the same dimension as the space: they must satisfy low <= up, and are allowed to take values -Inf and +Inf to indicate unbounded coordinates.

Indicator of the graph of a linear operator

IndGraph(A)

For matrix A (dense or sparse) returns the indicator function of the set

\[G_A = \{(x, y) : Ax = y\}.\]

The evaluation of prox! uses direct methods based on LDLt (LL for dense cases) matrix factorization and backsolve.

The prox! method operates on pairs (x, y) as input/output. So if f = IndGraph(A) is the indicator of the graph $G_A$, while (x, y) and (c, d) are pairs of vectors of the same sizes, then

prox!((c, d), f, (x, y))

writes to (c, d) the projection onto $G_A$ of (x, y).

Indicator of a halfspace

IndHalfspace(a, b)

For an array a and a scalar b, returns the indicator of set

\[S = \{x : \langle a,x \rangle \leq b \}.\]

Indicator of a hyperslab

IndHyperslab(low, a, upp)

For an array a and scalars low and upp, returns the indicator of set

\[S = \{x : low \leq \langle a,x \rangle \leq upp \}.\]

Indicator of a singleton

IndPoint(p=0.0)

Returns the indicator of the set

\[C = \{p \}.\]

Parameter p can be a scalar, in which case the unique element of S has uniform coefficients.

Indicator of a polyhedral set

IndPolyhedral([l,] A, [u, xmin, xmax])

Returns the indicator function of the polyhedral set:

\[S = \{ x : x_\min \leq x \leq x_\max, l \leq Ax \leq u \}.\]

Matrix A is a mandatory argument; when any of the bounds is not provided, it is assumed to be (plus or minus) infinity.

Indicator of a simplex

IndSimplex(a=1.0)

Returns the indicator of the set

\[S = \left\{ x : x \geq 0, \sum_i x_i = a \right\}.\]

By default a=1.0, therefore $S$ is the probability simplex.

Indicator of a Euclidean sphere

IndSphereL2(r=1.0)

Returns the indicator function of the set

\[S = \{ x : \|x\| = r \},\]

where $\|\cdot\|$ is the $L_2$ (Euclidean) norm. Parameter r must be positive.

Indicator of the Stiefel manifold

IndStiefel()

Returns the indicator of the Stiefel manifold

\[S_{n,p} = \left\{ X \in \mathbb{F}^{n \times p} : X^*X = I \right\}.\]

where $\mathbb{F}$ is the real or complex field, and parameters $n$ and $p$ are inferred from the matrix provided as input.

Indicator of the (primal) exponential cone

IndExpPrimal()

Returns the indicator function of the primal exponential cone, that is

\[C = \mathrm{cl} \{ (r,s,t) : s > 0, s⋅e^{r/s} \leq t \} \subset \mathbb{R}^3.\]

Indicator of the (dual) exponential cone

IndExpDual()

Returns the indicator function of the dual exponential cone, that is

\[C = \mathrm{cl} \{ (u,v,w) : u < 0, -u⋅e^{v/u} \leq w⋅e \} \subset \mathbb{R}^3.\]

Indicator of the free cone

IndFree()

Returns the indicator function of the whole space, or "free cone", i.e., a function which is identically zero.

Indicator of the nonnegative orthant

IndNonnegative()

Returns the indicator of the set

\[C = \{ x : x \geq 0 \}.\]

Indicator of the nonpositive orthant

IndNonpositive()

Returns the indicator of the set

\[C = \{ x : x \leq 0 \}.\]

Indicator of the set of positive semi-definite cone

IndPSD(;scaling=false)

Returns the indicator of the set

\[C = \{ X : X \succeq 0 \}.\]

The argument to the function can be either a Symmetric, Hermitian, or AbstractMatrix object, or an object of type AbstractVector{Float64} holding a symmetric matrix in (lower triangular) packed storage.

If scaling = true then the vectors y and x in prox!(y::AbstractVector{Float64}, f::IndPSD, x::AbstractVector{Float64}, args...) have the off-diagonal elements multiplied with √2 to preserve inner products, see Vandenberghe 2010: http://www.seas.ucla.edu/~vandenbe/publications/coneprog.pdf .

I.e. when when scaling=true, let X,Y be matrices and

x = (X_{1,1}, √2⋅X_{2,1}, ... ,√2⋅X_{n,1}, X_{2,2}, √2⋅X_{3,2}, ..., X_{n,n}),

y = (Y_{1,1}, √2⋅Y_{2,1}, ... ,√2⋅Y_{n,1}, Y_{2,2}, √2⋅Y_{3,2}, ..., Y_{n,n})

then prox!(Y, f, X) is equivalent to prox!(y, f, x).

Indicator of the second-order cone

IndSOC()

Returns the indicator of the second-order cone (also known as ice-cream cone or Lorentz cone), that is

\[C = \left\{ (t, x) : \|x\| \leq t \right\}.\]

Indicator of the rotated second-order cone

    IndRotatedSOC()

Returns the indicator of the rotated second-order cone (also known as ice-cream cone or Lorentz cone), that is

\[C = \left\{ (p, q, x) : \|x\|^2 \leq 2\cdot pq, p \geq 0, q \geq 0 \right\}.\]

Indicator of the zero cone

IndZero()

Returns the indicator function of the set containing the origin, the "zero cone".

Cubic Euclidean norm (weighted)

CubeNormL2(λ=1)

With a nonnegative scalar λ, returns the function

\[f(x) = λ\|x\|^3.\]

Elastic-net regularization

ElasticNet(μ=1, λ=1)

Returns the function

\[f(x) = μ\|x\|_1 + (λ/2)\|x\|^2,\]

for nonnegative parameters μ and λ.

$L_0$ pseudo-norm

NormL0(λ=1)

Returns the function

\[f(x) = λ\cdot\mathrm{nnz}(x)\]

for a nonnegative parameter λ.

$L_1$ norm

NormL1(λ=1)

With a nonnegative scalar parameter λ, returns the function

\[f(x) = λ\cdot∑_i|x_i|.\]

With a nonnegative array parameter λ, returns the function

\[f(x) = ∑_i λ_i|x_i|.\]

$L_2$ norm

NormL2(λ=1)

With a nonnegative scalar parameter λ, returns the function

\[f(x) = λ\cdot\sqrt{x_1^2 + … + x_n^2}.\]

Sum-of-$L_2$ norms

NormL21(λ=1, dim=1)

Returns the function

\[f(X) = λ⋅∑_i\|x_i\|\]

for a nonnegative λ, where $x_i$ is the $i$-th column of $X$ if dim == 1, and the $i$-th row of $X$ if dim == 2. In words, it is the sum of the Euclidean norms of the columns or rows.

$L_∞$ norm

NormLinf(λ=1)

Returns the function

\[f(x) = λ⋅\max\{|x_1|, …, |x_n|\},\]

for a nonnegative parameter λ.

Nuclear norm

NuclearNorm(λ=1)

Returns the function

\[f(X) = \|X\|_* = λ ∑_i σ_i(X),\]

where λ is a positive parameter and $σ_i(X)$ is $i$-th singular value of matrix $X$.

Squared Euclidean norm (weighted)

SqrNormL2(λ=1)

With a nonnegative scalar λ, returns the function

\[f(x) = \tfrac{λ}{2}\|x\|^2.\]

With a nonnegative array λ, returns the function

\[f(x) = \tfrac{1}{2}∑_i λ_i x_i^2.\]

1-dimensional Total Variation

TotalVariation1D(λ=1)

With a nonnegative scalar parameter λ, returns the function

\[f(x) = λ ∑_{i=2}^{n} |x_i - x_{i-1}|.\]

Cross Entropy loss

CrossEntropy(b)

Returns the function

\[f(x) = -\frac{1}{N} \sum_{i = 1}^{N} b_i \log (x_i)+(1-b_i) \log (1-x_i),\]

where b is an array such that 0 ≤ b ≤ 1 component-wise.

Hinge loss

HingeLoss(y, μ=1)

Returns the function

\[f(x) = μ⋅∑_i \max\{0, 1 - y_i ⋅ x_i\},\]

where y is an array and μ is a positive parameter.

Huber loss

HuberLoss(ρ=1, μ=1)

Returns the function

\[f(x) = \begin{cases} \tfrac{μ}{2}\|x\|^2 & \text{if}\ \|x\| ⩽ ρ \\ ρμ(\|x\| - \tfrac{ρ}{2}) & \text{otherwise}, \end{cases}\]

where ρ and μ are positive parameters.

Least squares penalty

LeastSquares(A, b, λ=1.0; iterative=false)

For a matrix A, a vector b and a scalar λ, returns the function

\[f(x) = \tfrac{\lambda}{2}\|Ax - b\|^2.\]

By default, a direct method (based on Cholesky factorization) is used to evaluate prox!. If iterative=true, then prox! is evaluated approximately using an iterative method instead.

Linear function

Linear(c)

Returns the function

\[f(x) = \langle c, x \rangle.\]

Logarithmic barrier

LogBarrier(a=1, b=0, μ=1)

Returns the function

\[f(x) = -μ⋅∑_i\log(a⋅x_i+b),\]

for a nonnegative parameter μ.

Logistic loss

LogisticLoss(y, μ=1)

Returns the function

\[f(x) = μ⋅∑_i log(1+exp(-y_i⋅x_i))\]

where y is an array and μ is a positive parameter.

Maximum coefficient

Maximum(λ=1)

For a nonnegative parameter λ ⩾ 0, returns the function

\[f(x) = \lambda \cdot \max \{x_i : i = 1,\ldots, n \}.\]

Quadratic function

Quadratic(Q, q; iterative=false)

For a matrix Q (dense or sparse, symmetric and positive semidefinite) and a vector q, returns the function

\[f(x) = \tfrac{1}{2}\langle Qx, x\rangle + \langle q, x \rangle.\]

By default, a direct method (based on Cholesky factorization) is used to evaluate prox!. If iterative=true, then prox! is evaluated approximately using an iterative method instead.

Squared Hinge loss

SqrHingeLoss(y, μ=1)

Returns the function

\[f(x) = μ⋅∑_i \max\{0, 1 - y_i ⋅ x_i\}^2,\]

where y is an array and μ is a positive parameter.

Sum of the positive coefficients

SumPositive()

Returns the function

\[f(x) = ∑_i \max\{0, x_i\}.\]

Distance from a convex set

DistL2(ind_S)

Given ind_S the indicator function of a convex set $S$, and an optional positive parameter λ, returns the (weighted) Euclidean distance from $S$, that is function

\[g(x) = λ\mathrm{dist}_S(x) = \min \{ λ\|y - x\| : y \in S \}.\]

Squared distance from a convex set

SqrDistL2(ind_S, λ=1)

Given ind_S the indicator function of a convex set $S$, and an optional positive parameter λ, returns the (weighted) squared Euclidean distance from $S$, that is function

\[g(x) = \tfrac{λ}{2}\mathrm{dist}_S^2(x) = \min \left\{ \tfrac{λ}{2}\|y - x\|^2 : y \in S \right\}.\]