The Difference Between a Tangent and a Differential

Given a smooth point $x$ on any manifold and a smooth path $p(s)$ in the manifold such that $p(0) = x$, one can construct a differential operator on the $C^\infty$ functions defined near $x$ by the rule

$$D_p f = \frac{d}{ds} f(p(s)) \vert _{s=0}.$$

These differential operators at $x$ form a finite-dimensional vector space, and the dimension of this vector space is equal to the dimension of the manifold. This vector space is called the tangent space of $M$ at $x$, and is often denoted $T_x(M)$.

For the Stiefel manifold, and, generally, all smooth constraint manifolds, the tangent space at any point is easy to characterize. The constraint equation $Y^*Y=I$ can be thought of as $k(k+1)/2$ independent functions on $\mbox{Stief}(n,k)$ which must all have constant value. If $H$ is a tangent vector of $\mbox{Stief}(n,k)$ at $Y$, it must then be that

$$H^* Y + Y^* H = 0,$$

which is obtained by taking $\frac{d}{dt} Y(t)^* Y(t) = 0 \vert _{t=0}$ (where $\dot{Y}(0) = H$). In a constraint manifold, the tangents at $Y$ can equivalently be identified with those infinitesimal displacements of $Y$ which preserve the constraint equations to first order.

For $H$ to be a tangent, its components must satisfy $k(k+1)/2$ independent equations. These equations are all linear in the components of $H$, and thus the set of all tangents is an $(nk-k(k+1)/2)$-dimensional subspace of the vector space ${\cal R}^{n \times k}$.

A differential operator $D_H$ may be associated with an $n \times k$ matrix, $H$, given by

$$D_H f = \frac{d}{dt} f(Y+tH) \vert _{t=0}.$$

$D_H$ is the directional derivative operator in the $H$ direction. One may observe that for $H$ to be a tangent vector, the tangency condition is equivalent to $D_H (Y^*Y) = 0$.

While tangents are $n \times k$ matrices and, therefore, have associated differential operators, differentials are something else. Given a $C^\infty$ function $f$ and a point $Y$, one can consider the equation $D_H f$ for $H \in {\cal R}^{n \times k}$ ($H$ is not necessarily tangent). This expression is linear in $H$ and takes on some real value. It is thus possible to represent it as a linear function on the vector space ${\cal R}^{n \times k}$,

$$D_H f = \tr( H^* Z),$$

for some appropriate $n \times k$ matrix $Z$ whose values depend on the first-order behavior of $f$ near $Y$. We identify this $Z$ matrix with $df$, the differential of $f$ at $Y$. This is the same differential which is computed by the dF functions in the sample problems.

For any constraint manifold the differential of a smooth function $f$ can be computed without having to know anything about the manifold itself. One can simply use the differentials as computed in the ambient space (the unconstrained ${\cal R}^{n \times k}$ derivatives in our case). If one then restricts one's differential operators to be only those in tangent directions, then one can still use the unconstrained $df$ in $\tr ( H^* df )$ to compute $D_H f$ for $H \in T_Y(\mbox{Stief}(n,k))$. This is why it requires no geometric knowledge to produce the dF functions.