Computing the SVD

In [7]: 
import numpy as np
import numpy.linalg as la
In [25]: 
np.random.seed(15)
n = 5
A = np.random.randn(n, n)

Now compute the eigenvalues and eigenvectors of $A^TA$ as eigvals and eigvecs using la.eig or la.eigh (symmetric):

In [26]: 
eigvals, eigvecs = la.eigh(A.T.dot(A))
In [27]: 
eigvals
Out[27]:
array([ 0.08637178,  0.457892  ,  2.04177547,  2.34383161,  8.37000184])

Eigenvalues are real and positive. Coincidence?

In [28]: 
eigvecs.shape
Out[28]:
(5, 5)

Check that those are in fact eigenvectors and eigenvalues:

In [29]: 
B = A.T.dot(A)
B - eigvecs.dot(np.diag(eigvals)).dot(la.inv(eigvecs))
Out[29]:
array([[  4.44089210e-16,   1.77635684e-15,   2.22044605e-15,
         -4.44089210e-16,   2.22044605e-16],
       [  2.22044605e-16,  -1.77635684e-15,  -4.44089210e-16,
         -1.11022302e-15,   1.66533454e-16],
       [ -4.44089210e-16,  -6.66133815e-16,   1.77635684e-15,
         -4.44089210e-16,  -1.66533454e-16],
       [ -4.44089210e-16,   0.00000000e+00,   1.33226763e-15,
          0.00000000e+00,  -7.77156117e-16],
       [  2.22044605e-16,   5.55111512e-17,  -5.55111512e-16,
         -8.88178420e-16,   6.66133815e-16]])

eigvecs are orthonormal! (Why?)

Check:

In [30]: 
eigvecs.T.dot(eigvecs) - np.eye(n)
Out[30]:
array([[  8.88178420e-16,  -3.19189120e-16,  -1.66533454e-16,
         -5.55111512e-17,  -1.31838984e-16],
       [ -3.19189120e-16,   1.33226763e-15,  -6.93889390e-17,
          1.66533454e-16,   4.29344060e-16],
       [ -1.66533454e-16,  -6.93889390e-17,   4.44089210e-16,
          1.11022302e-16,   1.04083409e-17],
       [ -5.55111512e-17,   1.66533454e-16,   1.11022302e-16,
          4.44089210e-16,   8.32667268e-17],
       [ -1.31838984e-16,   4.29344060e-16,   1.04083409e-17,
          8.32667268e-17,   4.44089210e-16]])

Now piece together the SVD:

In [19]: 
Sigma = np.diag(np.sqrt(eigvals))
In [20]: 
V = eigvecs
In [38]: 
U = A.dot(V).dot(la.inv(Sigma))

Check orthogonality of U:

In [39]: 
U.dot(U.T) - np.eye(n)
Out[39]:
array([[ -2.10942375e-15,  -1.08246745e-15,  -8.46545056e-16,
          8.32667268e-17,   8.25728375e-16],
       [ -1.08246745e-15,   4.44089210e-16,  -1.38777878e-16,
         -5.55111512e-17,  -1.38777878e-16],
       [ -8.46545056e-16,  -1.38777878e-16,   2.22044605e-16,
          9.85322934e-16,  -3.95516953e-16],
       [  8.32667268e-17,  -5.55111512e-17,   9.85322934e-16,
          2.22044605e-16,   4.99600361e-16],
       [  8.25728375e-16,  -1.38777878e-16,  -3.95516953e-16,
          4.99600361e-16,  -2.22044605e-16]])