Eigendecompositions

We discuss four eigendecompositions, or four matrices that are similar to
and for which it is simpler to solve eigenproblems.
The first one, *diagonal form*,
exists only when there are independent eigenvectors.
The second one, *Jordan form*, generalizes diagonal form to all
matrices. It can be very ill-conditioned (indeed, it can change discontinuously),
so for numerical purposes we typically use the third one, *Schur form*,
which is cheaper and more stable to compute. We briefly mention a fourth
one, which we call *Jordan-Schur form*,^{} that is as stable as the Schur form but computes some of
the detailed information about invariant subspaces provided by the Jordan form.

Define
.
If there are independent right eigenvectors
,
we define
.
is called an *eigenvector matrix* of .
The equalities
for may also be written
or
. The factorization

is called the

If we take a subset of columns of (say
=
columns 2, 3, and 5), these columns span an invariant subspace of .
If we take the corresponding submatrix
of , and the corresponding three rows of
(say
),
we can write the corresponding
*partial diagonal form* as
or
. If the columns in are replaced by
different vectors spanning the same invariant subspace, we get
a different partial eigendecomposition
,
where is
a by matrix whose eigenvalues are those of , though
may not be diagonal. Similar procedures for producing partial
eigendecompositions for the other eigendecompositions discussed below.

If all the are distinct, there are independent eigenvectors,
and the diagonal form
exists. This is the simplest and most common case.
For example, if one picks a matrix ``at random,''^{}the probability is 1 that the eigenvalues are distinct.

A diagonal form of the matrix in (2.3) does not exist,
since it has just one independent eigenvector. Instead, we can compute its
*Jordan form*, which is a decomposition

where is a block-diagonal matrix, i.e., a matrix of the form , where each is a square matrix called a

Unfortunately, the Jordan form is generally not suitable for numerical computation. Here are two reasons. First, it can change discontinuously as changes. For example, the matrix for with is , but for , itself. Second, the eigenvector matrix can be very ill-conditioned, i.e., hard to invert accurately. In the case of , the condition number of grows like .

So instead we use eigendecompositions
of the form

where is a unitary (or orthogonal) similarity transformation, and is triangular (or quasi-triangular). This is called the

Finally, we consider the *Jordan-Schur form*.
It is quite complicated, so we
only summarize its properties here. Like the Schur form, it only uses
unitary (orthogonal) transformation, and so can be computed stably.
Like the Jordan form, it explicitly shows what the sizes of the Jordan blocks are
and gives explicit bases for many more invariant subspaces than Schur form.

Many textbooks give explicit solutions for problems such as computing the matrix of the exponential in terms of the Jordan form of [114]. These methods are to be avoided numerically, because of the difficulty of computing the Jordan form. Nearly all these problems have alternative solutions in terms of the Schur form, or in some cases the Jordan-Schur form.