Next: Conditioning Up: Singular Value Decomposition  J. Previous: Equivalences   Contents   Index

Decompositions

Define as the by matrix whose top rows contain and whose bottom rows are zero. Define the by matrix and the by matrix . is called the left singular vector matrix of , and is called the right singular vector matrix of . Since the are orthogonal unit vectors, we see that ; i.e., is a unitary matrix. If is real then the are real vectors, so , and we also say that is an orthogonal matrix. The same discussion applies to . The equalities and for and for may also be written and , or . The factorization

is called the SVD of . In other words, is unitarily (orthogonally) equivalent to the diagonal matrix .

There are several smaller'' versions of the SVD that are often computed. Let be an by matrix of the first left singular vectors, be an by matrix of the first right singular vectors, and be a by matrix of the first singular values. Then we can make the following definitions.

Thin SVD.
is the thin (or economy-sized) SVD of . The thin SVD is much smaller to store and faster to compute than the full SVD when .

Compact SVD.
is the compact SVD of . The compact SVD is much smaller to store and faster to compute than the thin SVD when .
Truncated SVD.
is the rank- truncated (or partial) SVD of , where . Among all rank- matrices , is the unique minimizer of and also minimizes (perhaps not uniquely) . The truncated SVD is much smaller to store and cheaper to compute than the compact SVD when , and is the most common form of the SVD computed in applications.

The thin SVD may also be written . Each is called a singular triplet. The compact and truncated SVDs may be written similarly (the sum going from to , or to , respectively).

If is by with , then its SVD is , where is by , is by with in its first columns and zeros in columns through , and is by . Its thin SVD is , and the compact SVD and truncated SVD are as above.

More generally, if we take a subset of columns of and
(say = columns 2, 3, and 5, and ), then these columns span a pair of singular subspaces of . If we take the corresponding submatrix of , then we can write the corresponding partial SVD . If the columns in and are replaced by different orthonormal vectors spanning the same invariant subspace, then we get a different partial SVD , where is a by matrix whose singular values are those of , though may not be diagonal.

Next: Conditioning Up: Singular Value Decomposition  J. Previous: Equivalences   Contents   Index
Susan Blackford 2000-11-20