# Computing the SVD

In :
import numpy as np
import numpy.linalg as la

In :
np.random.seed(15)
n = 5
A = np.random.randn(n, n)


Now compute the eigenvalues and eigenvectors of $A^TA$ as eigvals and eigvecs using la.eig or la.eigh (symmetric):

In :
eigvals, eigvecs = la.eigh(A.T.dot(A))

In :
eigvals

Out:
array([ 0.08637178,  0.457892  ,  2.04177547,  2.34383161,  8.37000184])


Eigenvalues are real and positive. Coincidence?

In :
eigvecs.shape

Out:
(5, 5)


Check that those are in fact eigenvectors and eigenvalues:

In :
B = A.T.dot(A)
B - eigvecs.dot(np.diag(eigvals)).dot(la.inv(eigvecs))

Out:
array([[  4.44089210e-16,   1.77635684e-15,   2.22044605e-15,
-4.44089210e-16,   2.22044605e-16],
[  2.22044605e-16,  -1.77635684e-15,  -4.44089210e-16,
-1.11022302e-15,   1.66533454e-16],
[ -4.44089210e-16,  -6.66133815e-16,   1.77635684e-15,
-4.44089210e-16,  -1.66533454e-16],
[ -4.44089210e-16,   0.00000000e+00,   1.33226763e-15,
0.00000000e+00,  -7.77156117e-16],
[  2.22044605e-16,   5.55111512e-17,  -5.55111512e-16,
-8.88178420e-16,   6.66133815e-16]])


eigvecs are orthonormal! (Why?)

Check:

In :
eigvecs.T.dot(eigvecs) - np.eye(n)

Out:
array([[  8.88178420e-16,  -3.19189120e-16,  -1.66533454e-16,
-5.55111512e-17,  -1.31838984e-16],
[ -3.19189120e-16,   1.33226763e-15,  -6.93889390e-17,
1.66533454e-16,   4.29344060e-16],
[ -1.66533454e-16,  -6.93889390e-17,   4.44089210e-16,
1.11022302e-16,   1.04083409e-17],
[ -5.55111512e-17,   1.66533454e-16,   1.11022302e-16,
4.44089210e-16,   8.32667268e-17],
[ -1.31838984e-16,   4.29344060e-16,   1.04083409e-17,
8.32667268e-17,   4.44089210e-16]])


Now piece together the SVD:

In :
Sigma = np.diag(np.sqrt(eigvals))

In :
V = eigvecs

In :
U = A.dot(V).dot(la.inv(Sigma))


Check orthogonality of U:

In :
U.dot(U.T) - np.eye(n)

Out:
array([[ -2.10942375e-15,  -1.08246745e-15,  -8.46545056e-16,
8.32667268e-17,   8.25728375e-16],
[ -1.08246745e-15,   4.44089210e-16,  -1.38777878e-16,
-5.55111512e-17,  -1.38777878e-16],
[ -8.46545056e-16,  -1.38777878e-16,   2.22044605e-16,
9.85322934e-16,  -3.95516953e-16],
[  8.32667268e-17,  -5.55111512e-17,   9.85322934e-16,
2.22044605e-16,   4.99600361e-16],
[  8.25728375e-16,  -1.38777878e-16,  -3.95516953e-16,
4.99600361e-16,  -2.22044605e-16]])