Calculus rules

Convex conjugate

Conjugate(f)

Returns the convex conjugate (also known as Fenchel conjugate, or Fenchel-Legendre transform) of function f, that is

\[f^*(x) = \sup_y \{ \langle y, x \rangle - f(y) \}.\]

Pointwise minimum of functions

PointwiseMinimum(f_1, ..., f_k)

Given functions f_1 to f_k, returns their pointwise minimum, that is

\[g(x) = \min\{f_1(x), ..., f_k(x)\}\]

Note that g is a nonconvex function in general.

Separable sum of functions

SeparableSum(f_1, ..., f_k)

Given functions f_1 to f_k, returns their separable sum, that is

\[g(x_1, ..., x_k) = \sum_{i=1}^k f_i(x_i).\]

The object g constructed in this way can be evaluated at Tuples of length k. Likewise, the prox and prox! methods for g operate with (input and output) Tuples of length k.

Example:

f = SeparableSum(NormL1(), NuclearNorm()); # separable sum of two functions
x = randn(10); # some random vector
Y = randn(20, 30); # some random matrix
f_xY = f((x, Y)); # evaluates f at (x, Y)
(u, V), f_uV = prox(f, (x, Y), 1.3); # computes prox at (x, Y)

Sliced separable sum of functions

SlicedSeparableSum((f_1, ..., f_k), (J_1, ..., J_k))

Returns the function

\[g(x) = \sum_{i=1}^k f_i(x_{J_i}).\]

SlicedSeparableSum(f, (J_1, ..., J_k))

Analogous to the previous one, but applies the same function f to all slices of the variable x:

\[g(x) = \sum_{i=1}^k f(x_{J_i}).\]

Sum of functions

Sum(f_1, ..., f_k)

Given functions f_1 to f_k, returns their sum

\[g(x) = \sum_{i=1}^k f_i(x).\]

Moreau envelope

MoreauEnvelope(f, γ=1)

Returns the Moreau envelope (also known as Moreau-Yosida regularization) of function f with parameter γ (positive), that is

\[f^γ(x) = \min_z \left\{ f(z) + \tfrac{1}{2γ}\|z-x\|^2 \right\}.\]

If $f$ is convex, then $f^γ$ is a smooth, convex, lower approximation to $f$, having the same minima as the original function.

Regularize

Regularize(f, ρ=1.0, a=0.0)

Given function f, and optional parameters ρ (positive) and a, returns

\[g(x) = f(x) + \tfrac{ρ}{2}\|x-a\|².\]

Parameter a can be either an array or a scalar, in which case it is subtracted component-wise from x in the above expression.

Postcomposition with an affine transformation

Postcompose(f, a=1, b=0)

Returns the function

\[g(x) = a\cdot f(x) + b.\]

Precomposition with linear mapping/translation

Precompose(f, L, μ, b)

Returns the function

\[g(x) = f(Lx + b)\]

where $f$ is a convex function and $L$ is a linear mapping: this must satisfy $LL^* = μI$ for $μ > 0$. Furthermore, either $f$ is separable or parameter μ is a scalar, for the prox of $g$ to be computable.

Parameter L defines $L$ through the mul! method. Therefore L can be an AbstractMatrix for example, but not necessarily.

In this case, prox and prox! are computed according to Prop. 24.14 in Bauschke, Combettes "Convex Analysis and Monotone Operator Theory in Hilbert Spaces", 2nd edition, 2016. The same result is Prop. 23.32 in the 1st edition of the same book.

Precomposition with diagonal scaling/translation

PrecomposeDiagonal(f, a, b)

Returns the function

\[g(x) = f(\mathrm{diag}(a)x + b)\]

Function $f$ must be convex and separable, or a must be a scalar, for the prox of $g$ to be computable. Parametes a and b can be arrays of multiple dimensions, according to the shape/size of the input x that will be provided to the function: the way the above expression for $g$ should be thought of, is g(x) = f(a.*x + b).

Linear tilting

Tilt(f, a, b=0.0)

Given function f, an array a and a constant b (optional), returns function

\[g(x) = f(x) + \langle a, x \rangle + b.\]

Translation

Translate(f, b)

Returns the translated function

\[g(x) = f(x + b)\]