Dual norm

In functional analysis, the dual norm is a measure of the "size" of each continuous linear functional defined on a normed vector space.

Definition

Let $X$ be a normed vector space with norm $|\cdot |$ and let $X^{*}$ be the dual space. The dual norm of a continuous linear functional $f$ belonging to $X^{*}$ is defined to be the real number

$\left\|f\right\|:=\sup\{\left|f(x)\right|:x\in X,\left|x\right|\leq 1\}$

where $sup$ denotes the supremum.

The map $f\mapsto \|f\|$ defines a norm on $X^{*}$. (See Theorems 1 and 2 below.)

The dual norm is a special case of the operator norm defined for each (bounded) linear map between normed vector spaces.

The topology on $X^{*}$ induced by $|\cdot |$ turns out to be as strong as the weak-* topology on $X^{*}$.

If the ground field of $X$ is complete then $X^{*}$ is a Banach space.

The double dual of a normed linear space

The double dual (or second dual) $X^{**}$ of $X$ is the dual of the normed vector space $X^{*}$. There is a natural map $\varphi :X\to X^{**}$. Indeed, for each $w^{*}$ in $X^{*}$ define

$\varphi (v)(w^{*}):=w^{*}(v).$

The map $\varphi$ is linear, injective, and distance preserving. In particular, if $X$ is complete (i.e. a Banach space), then $\varphi$ is an isometry onto a closed subspace of $X^{**}$.

In general, the map $\varphi$ is not surjective. For example, if $X$ is the Banach space $L^{\infty }$ consisting of bounded functions on the real line with the supremum norm, then the map $\varphi$ is not surjective. (See $L^{p}$ space). If $\varphi$ is surjective, then $X$ is said to be a reflexive Banach space. If $1<p<\infty$, then the space $L^{p}$ is a reflexive Banach space.

Mathematical Optimization

Let $||\cdot ||$ be a norm on $\mathbb {R} ^{n}$. The associated dual norm, denoted $\|\cdot \|_{*}$, is defined as

$||z||_{*}=\sup\{z^{\intercal }x\;|\;||x||\leq 1\}.$

(This can be shown to be a norm.) The dual norm can be interpreted as the operator norm of $z^{\intercal }$, interpreted as a $1\times n$ matrix, with the norm $||\cdot ||$ on $\mathbb {R} ^{n}$, and the absolute value on $\mathbb {R}$:

$||z||_{*}=\sup\{|z^{\intercal }x|\;|\;||x||\leq 1\}.$

From the definition of dual norm we have the inequality

$z^{\intercal }x=||x||(z^{\intercal }{\frac {x}{||x||}})\leq \lVert x\rVert \lVert z\rVert _{*}$

which holds for all x and z. The dual of the dual norm is the original norm: we have $\lVert x\rVert _{**}=\lVert x\rVert$ for all x. (This need not hold in infinite-dimensional vector spaces.)

The dual of the Euclidean norm is the Euclidean norm, since

$\sup\{z^{\intercal }x\;|\;\lVert x\rVert _{2}\leq 1\}=\lVert z\rVert _{2}.$

(This follows from the Cauchy–Schwarz inequality; for nonzero z, the value of x that maximises $z^{\intercal }x$ over $\lVert x\rVert _{2}\leq 1$ is ${\frac {z}{\lVert z\rVert _{2}}}$.)

The dual of the $\ell _{1}$-norm is the $\ell _{\infty }$-norm:

$\sup\{z^{\intercal }x\;|\;\lVert x\rVert _{\infty }\leq 1\}=\sum _{i=1}^{n}|z_{i}|=\lVert z\rVert _{1},$

and the dual of the $\ell _{\infty }$-norm is the $\ell _{1}$-norm.

More generally, Hölder's inequality shows that the dual of the $\ell _{p}$-norm is the $\ell _{q}$-norm, where, q satisfies ${\frac {1}{p}}+{\frac {1}{q}}=1$, i.e., $q={\frac {p}{p-1}}.$

As another example, consider the $\ell _{2}$- or spectral norm on $\mathbb {R} ^{m\times n}$. The associated dual norm is

$\lVert Z\rVert _{2*}=\sup\{\mathrm {\bf {tr}} (Z^{\intercal }X)|\;\lVert X\rVert _{2}\leq 1\},$

which turns out to be the sum of the singular values,

$\lVert Z\rVert _{2*}=\sigma _{1}(Z)+\ldots +\sigma _{r}(Z)=\mathrm {\bf {tr}} (Z^{\intercal }Z)^{\frac {1}{2}},$

where $r=\mathrm {\bf {rank}} \;Z$. This norm is sometimes called the nuclear norm.

Examples

Dual norm for matrices

Main articles: Hilbert–Schmidt operator and Matrix norm § Frobenius norm

The Frobenius norm defined by

$\left\|A\right\|_{\text{F}}={\sqrt {\sum _{i=1}^{m}\sum _{j=1}^{n}\left|a_{ij}\right|^{2}}}={\sqrt {\operatorname {trace} (A^{{}^{*}}A)}}={\sqrt {\sum _{i=1}^{\min\{m,\,n\}}\sigma _{i}^{2}}}$

is self-dual, i.e., its dual norm is $\left\|\cdot \right\|'_{\text{F}}=\left\|\cdot \right\|_{\text{F}}$.

The spectral norm, a special case of the induced norm when $p=2$, is defined by the maximum singular values of a matrix, i.e.,

$\left\|A\right\|_{2}=\sigma _{max}(A)$,

has the nuclear norm as its dual norm, which is defined by $\|B\|'_{2}=\sum _{i}\sigma _{i}(B)$ for any matrix $B$ where $\sigma _{i}(B)$ denote the singular values.

Some basic results about the operator norm

More generally, let $X$ and $Y$ be topological vector spaces, and $L(X,Y)$ be the collection of all bounded linear mappings (or operators) of $X$ into $Y$. In the case where $X$ and $Y$ are normed vector spaces, $L(X,Y)$ can be normed in a natural way.

When $Y$ is a scalar field (i.e. $Y={\mathbb {C} }$ or $Y={\mathbb {R} }$) so that $L(X,Y)$ is the dual space $X^{*}$ of $X$.

Theorem 1: Let $X$ and $Y$ be normed spaces, and associate to each $f\in L(X,Y)$ the number:

$\left\|f\right\|=\sup\{\left|f(x)\right|:x\in X,\left\|x\right\|\leq 1\}$

We first establish that $L(X,Y)$ is bounded (using the triangle inequality), and complete (using Cauchy sequences) using our definition of $\|f\|$, thereby making $L(X,Y)$ a normed space. If $Y$ is a Banach space, so is $L(X,Y)$.

Proof:

1. A subset of a normed space is bounded if and only if it lies in some multiple of the unit sphere; thus $\lVert f\rVert <\infty$ for every $f\in L(X,Y)$

   if $\alpha$ is a scalar, then $(\alpha f)(x)=\alpha \cdot fx$ so that

   $\|\alpha f\|=|\alpha |\|f\|$

   The triangle inequality in $Y$ shows that

   ${\begin{aligned}\|(f_{1}+f_{2})x\|&=\|f_{1}x+f_{2}x\|\leq \|f_{1}x\|+\|f_{2}x\|\\&\leq (\|f_{1}\|+\|f_{2}\|)\|x\|\leq \|f_{1}\|+\|f_{2}\|\end{aligned}}$

   for every $x\in X$ with $\|x\|\leq 1$. Thus

   $\|f_{1}+f_{2}\|\leq \|f_{1}\|+\|f_{2}\|$

   If $f\neq 0$, then $fx\neq 0$ for some $x\in X$; hence $\|f\|>0$. Thus, $L(X,Y)$ is a normed space.

2. Assume now that $Y$ is complete, and that $\{f_{n}\}$ is a Cauchy sequence in $L(X,Y)$.

   Since

   $\|f_{n}x-f_{m}x\|\leq \|f_{n}-f_{m}\|\|x\|$

   and it is assumed that $\|f_{n}-f_{m}\|\to 0$ as n and m tend to $\infty$, $\{f_{n}x\}$ is a Cauchy sequence in $Y$ for every $x\in X$.

   Hence

   $fx=\lim _{n\to \infty }f_{n}x$

   exists. It is clear that $f:X\to Y$ is linear. If $\varepsilon >0$, $\|f_{n}-f_{m}\|\|x\|\leq \varepsilon \|x\|$ for sufficiently large n and m. It follows

   $\|fx-f_{m}x\|\leq \varepsilon \|x\|$

   for sufficiently large m.

   Hence $\|fx\|\leq (\|f_{m}\|+\varepsilon )\|x\|$, so that $f\in L(X,Y)$ and $\|f-f_{m}\|\leq \varepsilon$.

   Thus $f_{m}\to f$ in the norm of $L(X,Y)$. This establishes the completeness of $L(X,Y)$

Theorem 2: Now suppose $B$ is the closed unit ball of normed space $X$. Define

$\|x^{*}\|=\sup\{|\langle {x,x^{*}}\rangle |:x\in B\}$

for every $x^{*}\in X^{*}$

(a) This norm makes $X^{*}$ into a Banach space.

(b) Let $B^{*}$ be the closed unit ball of $X^{*}$. For every $x\in X$,

$\|x\|=\sup\{|\langle {x,x^{*}}\rangle |:x^{*}\in B^{*}\}.$

Consequently, $x^{*}\to \langle {x,x^{*}}\rangle$ is a bounded linear functional on $X^{*}$, of norm $\|x\|$.

(c) $B^{*}$ is weak*-compact.

Proof

Since $L(X,Y)=X^{*}$, when $Y$ is the scalar field, (a) is a corollary of Theorem 1.

Fix $x\in X$. There exists $y^{*}\in B^{*}$ such that

$\langle {x,y^{*}}\rangle =\|x\|.$

but,

$|\langle {x,x^{*}}\rangle |\leq \|x\|\|x^{*}\|\leq \|x\|$

for every $x^{*}\in B^{*}$. (b) follows from the above.

Since the open unit ball $U$ of $X$ is dense in $B$, the definition of $\|x^{*}\|$ shows that $x^{*}\in B^{*}$ if and only if $|\langle {x,x^{*}}\rangle |\leq 1$ for every $x\in U$.

The proof for (c) now follows directly.

See also

• convex conjugate
• operator norm
• Lp spaces
• Hölder's inequality

References

• Aliprantis, Charalambos D.; Border, Kim C. (2007). Infinite Dimensional Analysis: A Hitchhiker's Guide (3rd ed.). Springer. ISBN 9783540326960.
• Boyd, Stephen; Vandenberghe, Lieven (2004). Convex Optimization. Cambridge University Press. ISBN 9780521833783.
• Kolmogorov, A.N.; Fomin, S.V. (1957). Elements of the Theory of Functions and Functional Analysis, Volume 1: Metric and Normed Spaces. Rochester: Graylock Press.
• Rudin, Walter (1991), Functional analysis, McGraw-Hill Science, ISBN 978-0-07-054236-5.

External links

• Notes on the proximal mapping by Lieven Vandenberge