Generalize to m  n matrices

You've seen that each 2  2 matrix A has an orthonormal basis {u_1, u_2} which it stretches and sends to an orthonormal basis {v_1, v_2}.
Analogous facts hold in higher dimensions.

A 3  3 matrix A = (v    v    v ) (s           ) (u )       1    2    3    1   0    0     1                   ...      0     2   0     2                                s    u                     0    0     3    3 has an orthonormal basis {u_1, u_2, u_3} for ^3 which it stretches and sends to an orthonormal basis {v_1, v_2, v_3} for ^3.

A 2  3 matrix A = (v    v ) (s           ) (u )       1    2    1   0    0     1                      s      ...                0     2   0     2                                u                                3 has an orthonormal basis {u_1, u_2, u_3} for ^3 which it stretches and sends to an orthonormal basis {v_1, v_2} of ^2.  (The vector u_3 has singular value 0 and is sent to the zero vector 0.)

A 3  2 matrix A = (v    v    v ) (s      ) (u )       1    2    3    1   0     1                           s    u                     0     2    2                       0    0 has an orthonormal basis {u_1, u_2} for ^2 which it stretches and sends to the first two vectors of an orthonormal basis {v_1, v_2, v_3} of ^3.

Column space and nullspace.  
For a 2  2 matrix A = (v    v ) (2   0) (u )       1    2            1                0   0                        u                         2 you saw that the column space of A was all multiples of v_1 and that the nullspace of A was all multiples of u_2.

In general for an m  n matrix A, the vectors v_i with non-zero singular values are a basis for the column space of A and the vectors u_i with zero singular values are a basis for the nullspace of A.

Suppose a 3  3 matrix A = (v    v    v ) (4   0   0) (u )       1    2    3                1                     0   ...               0   0   0    2                                  u                                  3 has singular values s_1 = 4, s_2 = 3, and s_3 = 0.
The singular value of s_3 = 0 means that A sends multiples of u_3 to the zero vector and that A sends every x to the plane spanned by v_1and v_2.
The set {v_1, v_2} is a basis for the column space and the set {u_3} is a basis for the nullspace.

Modified three steps to solving A x = y .
Step 1:  Find c_1, c_2, c_3 so that y = c_1v_1 + c_2v_2 + c_3v_3 .
    Since s_3 = 0, the v_3 portion of y is inaccessible.
    Throw away the v_3 portion and keep just Overscript[y,^] = c_1v_1 + c_2v_2.
    Continue solving A x = Overscript[y,^].
Step 2:  Divide by the singular values:  c_1/4, c_2/3.
Step 3:  Use these numbers to form x = c_1/4u_1 + c_2/3u_2.

The vector x = c_1/4u_1 + c_2/3u_2 is a solution to A x = Overscript[y,^] and a least squares solution to A x = y .

Since s_3 = 0, the general least squares solution to A x = y   is x = c_1/4u_1 + c_2/3u_2 + t u_3.

(Q42) Suppose A = (v    v    v ) (5   0   0   0) (u )       1    2    3                    1                 ...                    3                                      u                                      4.  In terms of the u_i and or v_i:
a.  Give a basis for the nullspace of A.
b.  Give a basis for the column space of A.
c.  Give the general least squares solution to A x = 10v_1 + 9v_2 + 8v_3.


Up

Copyright 2007 Todd Will
Created by Mathematica  (April 15, 2007)