User:Scut723/sandbox

The singular value decomposition (SVD) is one of the most powerful tools in theoretical and numerical linear algebra. The utility comes from three basic properties:
 * Every matrix has an SVD.
 * The SVD provides an orthonormal resolution for the four invariant subspaces.
 * The SVD provides an ordered list of singular values.

The Singular Value Decomposition Theorem
The singular value decomposition is the most powerful - and most expensive - decomposition tool in linear algebra. The power comes from the resolution of the four fundamental subspaces as well as the eigenvalues.

Existence
Every matrix has a singular value decomposition. Given a matrix $$\mathbf{A} \in \mathbb{C}^{m \times n}_{\rho}$$, that is, with $$m$$ rows, $$n$$ columns, and rank $$\rho$$, the SVD can be written as
 * $$\mathbf{A} = \mathbf{U} \Sigma \mathbf{V}^{*}$$,

where
 * $$\mathbf{U} \in \mathbb{C}^{m \times m}$$ resolves the column space,
 * $$\mathbf{V} \in \mathbb{C}^{n \times n}$$ resolves the row space,
 * $$\mathbf{\Sigma} \in \mathbb{R}^{m \times n}_{\rho}$$ contains the singular values.

The domain matrices are unitary:


 * $$\mathbf{U}\mathbf{U}^{*} = \mathbf{U}^{*}\mathbf{U} = \mathbf{I}_{m}$$
 * $$\mathbf{V}\mathbf{V}^{*} = \mathbf{V}^{*}\mathbf{V} = \mathbf{I}_{n}$$

Uniqueness
The singular values are unique, therefore the matrices $$\mathbf{S}$$ and $$\Sigma$$ are unique. Typically the domain matrices are not unique. For example, there could be two different decompositions such that


 * $$\mathbf{A} = \mathbf{U}_{1} \mathbf{\Sigma} \mathbf{V}_{1}^{*} = \mathbf{U}_{2} \mathbf{\Sigma} \mathbf{V}_{2}^{*}$$

Fundamental Theorem of Linear Algebra
The [Fundamental Theorem of Linear Algebra] states that a matrix $$\mathbf{A} \in \mathbb{C}^{m \times n}_{\rho}$$ induces a row space (or domain) $$\mathbb{C}^{n}$$ and a column space (or codomain) $$\mathbb{C}^{m}$$. The row space and the column space each have an orthogonal decomposition into a range space and a null space:


 * $$\mathbb{C}^{n}$$ = $${\color{Blue}{\overline{\mathcal{R}\left( \mathbf{A}^{*} \right)}}}$$ $$\oplus$$ $${\color{Red}{\mathcal{N}\left( \mathbf{A} \right)}}$$ (domain),


 * $$\mathbb{C}^{m}$$ = $${\color{Blue}{\overline{\mathcal{R}\left( \mathbf{A} \right)}}}$$ $$\oplus$$ $${\color{Red}{\mathcal{N}\left( \mathbf{A}^{*} \right)}}$$ (codomain),

where the overbear represents the set closure required in infinite dimensional spaces.

$${\color{Blue}{x^2}}+{\color{Orange}{2x}}-{\color{LimeGreen}{1}}$$

Block structure
$$ \begin{pmatrix} x & y \\ z & v \end{pmatrix} $$

Casting the SVD in block structure emphasizes its subspace decomposition;


 * $$\mathbf{A} = \mathbf{U} \Sigma \mathbf{V}^{*}=

\left[ \begin{array}{cc} {\color{Blue}{\mathbf{U}_{\mathcal{R}}}} & {\color{Red}{\mathbf{U}_{\mathcal{N}}}} \\ \end{array} \right] \left[ \begin{array}{c|c} \mathbf{S} & \mathbf{0} \\\hline \mathbf{0} & \mathbf{0} \end{array} \right] \left[ \begin{array}{c} {\color{Blue}{\mathbf{V}_{\mathcal{R}}^{*}}} \\ {\color{Red}{\mathbf{V}_{\mathcal{N}}^{*}}} \end{array} \right] $$

Geometry of the SVD
The mapping action of a matrix demonstrates the geometry of the SVD. A matrix is an operator which maps an $$n-$$vector into an $$m-$$vector


 * $$\mathbf{A}\colon\mathbb{C}^{n}\mapsto\mathbb{C}^{m}$$

Full row and column rank

 * $$m=n=\rho=2$$

\left[ \begin{array}{cr} 1 & -1 \\ 2 & 2 \\ \end{array} \right] $$