Singular Value Decomposition

If a matrix A stretches and sends an orthonormal basis {u_1, u_2} to an orthonormal basis {v_1, v_2} then you can compute A x as a three step process.

The singular value decomposition of A writes A as a product of three matrices A = M_3M_2M_1 where matrix M_i does step i.

Three steps to compute A x .

Step 1:  Find the u_i coordinates for x .  
Since {u_1, u_2} is an orthonormal basis the u -coords for x are c_iOverscript[=, (1)] u_i  x.  
    (1) See "orthonormal coordinates" in Getting Started.
Choose M_1 = (u )         1         u         2 to be the matrix with rows u_i.
Then M_1x   = (u ) x Overscript[=, (2)] (u   x ) = (c )                       1          ...                         u   x      c                      2                         2           2.
    (2) See "row way" in Getting Started.

Step 2:  Multiply the u_i coordinates (c )   1   c   2 by the singular values s_1 and s_2.
Choose the matrix M_2 = (s      )          1   0              s        0     2.  
Then M_2 (c ) = (s      ) (c ) = (s  c )       1      1   0     1      1  1       c           s    c      s  c       2     0     2    2      2  2.

Step 3:  Use (s  c )   1  1   s  c   2  2 as the v_i coordinates of A x .
Choose M_3 = (v    v )         1    2 to be the matrix with columns v_i.
Then M_3(s  c ) = (v    v ) (s  c ) Overscript[=, (3)] s_1c_1v_1 + s_2c_2v_2      1  1      1    2    1  1      s  c                s  c      2  2                2  2.
    (3) See "column way" in Getting Started.

Combining all three steps you get
    A x = M_3M_2M_1x = (v    v ) (s      ) (u ) x                       1    2    1   0     1                                     s    u                               0     2    2.

The singular value decomposition for A is
    A = (v    v ) (s      ) (u )       1    2    1   0     1                      s    u                0     2    2.

Here's the picture of xA x for another 2  2 matrix A.

(Q19)  In terms of the vectors {z_1, z_2} and {w_1, w_2}, the singular value decomposition for A is A = (                ) (                                 &n ...                                                      .


Up

Copyright 2007 Todd Will
Created by Mathematica  (April 15, 2007)