User:Jfessler/sandbox

Proof of Eckart–Young–Mirsky theorem (for Frobenius norm)
For a given (real or complex valued) matrix $$ A $$, denote its SVD by



A = U \Sigma V^{\top} ,$$

where $$ U $$ and $$ V $$ are unitary matrices, and $$ \Sigma $$ is a (often rectangular) diagonal matrix with entries $$(\sigma_{1}, \sigma_{2}, \cdots ,\sigma_{n})$$

s.t $$(\sigma_{n} \leq \sigma_{n-1} \leq \cdots \leq \sigma_{1})$$. The ith column of $$ U $$ is $$ u_i $$ and the ith row of $$ V^{\top} $$ is $$ v_i^{\top} $$.

Claim: the rank k approximation for $$ A $$ that minimizes the Frobenius norm $$ \|A - A^k\|_F $$ is given by

A^k = \Sigma^k_{i=1} u_i \sigma_i v_i^{\top} .$$

Proof:

For any matrix $$ B $$ the same size as $$ A $$, define the (generally non-diagonal) matrix $$ D = U^{\top} B V $$, so $$ B = U D V^{\top} $$. Then

\|A - B\|_F^2 = \| U \Sigma V' - U D V' \|_F^2 = \| \Sigma - D \|_F^2 = \sum_i |\sigma_i - D_{ii}|^2 + \sum_{i \neq j} |D_{ij}|^2 $$

\geq \sum_i |\sigma_i - D_{ii}|^2 \geq \sum_{i > k} |\sigma_i - D_{ii}|^2 .$$ Thus clearly the rank k choice of $$ D $$ that minimizes $$ \|A - B\|_F^2 $$ is when $$ D $$ is diagonal with nonzero diagonal elements $$ (\sigma_{1}, \sigma_{2}, \cdots ,\sigma_{k}) $$. The corresponding $$ B $$ matches $$ A^k $$ above. QED

MIRT stuff just to save it
The Michigan Image Reconstruction Toolbox (MIRT) is an open source software collection written in the MATLAB language and largely compatible with the GNU Octave software. The software is available from.

MIRT contains algorithms for solving inverse problems in several application areas, including tomography, magnetic resonance imaging (MRI), and image denoising.

History
Initial versions of the software were developed throughout the 1990s at the University of Michigan. An early public announcement of the software was made at the 2002 IEEE Nuclear Science Symposium and Medical Imaging Conference  in the notes for an Image Reconstruction short course . The first reference to the software in a peer-reviewed journal paper was in a 2003 paper on the nonuniform Fast Fourier transform (NUFFT). . The NUFFT component of the software has been the most widely cited  .

Operators
A core component of the software is the use of operator overloading so that the code expressions match the mathematical expressions fairly closely. For example, the gradient (with respect to $$ x $$) of the least-squares cost function $$ \frac{1}{2} \| y - A x \|^2 $$ is $$ A' (A x - y) $$ and one can compute this in MIRT using the familiar expression that works both if  is a matrix and if is an overloaded operator that performs some linear operation such as a discrete Fourier transform or a discretized Radon transform. MIRT includes operators for image deblurring, tomography, non-uniform Fast Fourier transforms, among others.

Use in literature
MIRT has been used in numerous publications on image reconstruction, such as the SPIRiT method for parallel MRI and the SPIRAL-TAP method for sparse Poisson intensity reconstruction

Online discussions / citations / clones of the toolbox include:        .

The toolbox is discussed in a book on medical imaging.

Some authors have used MIRT as a benchmark  for comparing MATLAB to OpenMP and OpenCL alternatives.

Limitations
A limitation of MIRT is that it is written in the proprietary Matlab language with some functionality contained in compiled MEX files that are available only for certain software platforms. The current form of distribution is via a compressed tar file that is less convenient for updating than a git distribution.