Singular Values

A singular value and corresponding singular vectors of a rectangular matrix A are, respectively, a scalar σ and a pair of vectors u and v that satisfy

$\begin{array}{l} A v = σ u \\ A^{H} u = σ v, \end{array}$

where $A^{H}$ is the Hermitian transpose of A. The singular vectors u and v are typically scaled to have a norm of 1. Also, if u and v are singular vectors of A, then -u and -v are singular vectors of A as well.

The singular values σ are always real and nonnegative, even if A is complex. With the singular values on the diagonal of a diagonal matrix Σ and the corresponding singular vectors forming the columns of two orthogonal matrices U and V, you obtain the equations

$\begin{array}{l} A V = U Σ \\ A^{H} U = V Σ . \end{array}$

Since U and V are unitary matrices, multiplying the first equation by $V^{H}$ on the right yields the singular value decomposition equation

$A = U Σ V^{H} .$

The full singular value decomposition of an m-by-n matrix involves an m-by-m U, an m-by-n Σ, and an n-by-n V. In other words, U and V are both square, and Σ is the same size as A. If A has many more rows than columns (m > n), then the resulting m-by-m matrix U is large. However, most of the columns in U are multiplied by zeros in Σ. In this situation, the economy-sized decomposition saves both time and storage by producing an m-by-n U, an n-by-n Σ and the same V:

The eigenvalue decomposition is the appropriate tool for analyzing a matrix when it represents a mapping from a vector space into itself, as it does for an ordinary differential equation. However, the singular value decomposition is the appropriate tool for analyzing a mapping from one vector space into another vector space, possibly with a different dimension. Most systems of simultaneous linear equations fall into this second category.

If A is square, symmetric, and positive definite, then its eigenvalue and singular value decompositions are the same. But, as A departs from symmetry and positive definiteness, the difference between the two decompositions increases. In particular, the singular value decomposition of a real matrix is always real, but the eigenvalue decomposition of a real, nonsymmetric matrix might be complex.

For the example matrix

the full singular value decomposition is

[U,S,V] = svd(A)

U =

    0.6105   -0.7174    0.3355
    0.6646    0.2336   -0.7098
    0.4308    0.6563    0.6194


S =

   14.9359         0
         0    5.1883
         0         0


V =

    0.6925   -0.7214
    0.7214    0.6925

You can verify that U*S*V' is equal to A to within round-off error. For this small problem, the economy size decomposition is only slightly smaller.

[U,S,V] = svd(A,0)

U =

    0.6105   -0.7174
    0.6646    0.2336
    0.4308    0.6563


S =

   14.9359         0
         0    5.1883


V =

    0.6925   -0.7214
    0.7214    0.6925

Again, U*S*V' is equal to A to within round-off error.

If the matrix A is large and sparse, then using svd to calculate all of the singular values and vectors is not always practical. For example, if you need to know just a few of the largest singular values, then calculating all of the singular values of a 5000-by-5000 sparse matrix is a lot of extra work. In cases where only a subset of the singular values and vectors are required, the svds function is preferred over svd.

For a 1000-by-1000 random sparse matrix with a density of about 30%,

n = 1000;
A = sprand(n,n,0.3);

the six largest singular values are

S = svds(A)

S =

  130.2184
   16.4358
   16.4119
   16.3688
   16.3242
   16.2838

Also, the six smallest singular values are

S = svds(A,6,'smallest')

S =

    0.0740
    0.0574
    0.0388
    0.0282
    0.0131
    0.0066

For smaller matrices that can fit in memory as a full matrix, full(A), using svd(full(A)) might still be quicker than svds. However, for truly large and sparse matrices, using svds becomes necessary.

Documentation

Singular Values

See Also

Related Topics

MATLAB Documentation

Support