NumPy - Singular Value Decomposition

---

---

What is Singular Value Decomposition (SVD)?

Singular Value Decomposition, commonly abbreviated as SVD, is a matrix factorization technique in linear algebra. SVD decomposes a matrix into three other matrices, capturing important properties of the original matrix.

For instance, if you have a matrix A, the SVD is given by −

A = UVT

Here, U and V are orthogonal matrices, and is a diagonal matrix.

The columns of U are called the left singular vectors, the columns of V (or rows of V^T) are the right singular vectors, and the entries of are the singular values.

SVD in NumPy

NumPy provides the numpy.linalg.svd() function to compute the Singular Value Decomposition of a matrix. Let us see how to use this function with an example.

Example

In this example, the matrix A is decomposed into three matrices: U, (represented as the array of singular values S), and V^T −

import numpy as np

# Define a 3x3 matrix
A = np.array([[1, 2, 3],
              [4, 5, 6],
              [7, 8, 9]])

# Compute the Singular Value Decomposition
U, S, VT = np.linalg.svd(A)

print("Matrix U:\n", U)
print("Singular values:", S)
print("Matrix V^T:\n", VT)

Following is the output obtained −

Matrix U:
[[-0.21483724  0.88723069  0.40824829]
 [-0.52058739  0.24964395 -0.81649658]
 [-0.82633754 -0.38794278  0.40824829]]
Singular values: [1.68481034e+01 1.06836951e+00 4.41842475e-16]
Matrix V^T:
[[-0.47967118 -0.57236779 -0.66506441]
 [-0.77669099 -0.07568647  0.62531805]
 [-0.40824829  0.81649658 -0.40824829]]

Understanding the Components

The SVD components have specific properties and roles as shown below −

• Matrix U: The columns of U are the left singular vectors of A. These vectors form an orthogonal basis for the column space of A.
• Singular values: The diagonal entries of are the singular values of A. These values give the magnitude of the action of A along the corresponding singular vectors.
• Matrix V^T: The rows of V^T are the right singular vectors of A. These vectors form an orthogonal basis for the row space of A.

Reconstructing the Original Matrix

You can reconstruct the original matrix A from its SVD components. In NumPy, you can achieve this by using the numpy.dot() function to perform matrix multiplication.

Example

In the following example, we are reconstructing the original matrix "A" −

import numpy as np

# Define a 3x3 matrix
A = np.array([[1, 2, 3],
              [4, 5, 6],
              [7, 8, 9]])

# Compute the Singular Value Decomposition
U, S, VT = np.linalg.svd(A)

# Create the diagonal matrix  from the singular values
Sigma = np.zeros((3, 3))
np.fill_diagonal(Sigma, S)

# Reconstruct the original matrix
A_reconstructed = np.dot(U, np.dot(Sigma, VT))

print("Original matrix:\n", A)
print("Reconstructed matrix:\n", A_reconstructed)

The original matrix A is successfully reconstructed using its SVD components, demonstrating the accuracy of the decomposition −/p>

Original matrix:
[[1 2 3]
 [4 5 6]
 [7 8 9]]
Reconstructed matrix:
[[1. 2. 3.]
 [4. 5. 6.]
 [7. 8. 9.]]

Applications of SVD

SVD is a powerful tool with numerous applications, such as −

• Dimensionality Reduction: In data analysis and machine learning, SVD is used to reduce the number of dimensions while preserving important information.
• Image Compression: SVD is applied to compress images by reducing the amount of data required to store them.
• Noise Reduction: SVD can help remove noise from data by identifying and discarding small singular values.
• Signal Processing: In signal processing, SVD is used to analyze and filter signals.
• Recommendation Systems: SVD is employed in recommendation systems to predict user preferences.

Example: Image Compression using SVD

Let us see an example of how SVD can be used for image compression. We will use a grayscale image and compress it by retaining only the most significant singular values −

import numpy as np
import matplotlib.pyplot as plt
from skimage import data, color

# Load a sample image and convert it to grayscale
image = color.rgb2gray(data.astronaut())  
# Compute the Singular Value Decomposition
U, S, VT = np.linalg.svd(image, full_matrices=False)

# Retain only the first k singular values
k = 50
U_k = U[:, :k]
S_k = np.diag(S[:k])
VT_k = VT[:k, :]

# Reconstruct the compressed image
image_compressed = np.dot(U_k, np.dot(S_k, VT_k))

# Plot the original and compressed images
plt.figure(figsize=(10, 5))

plt.subplot(1, 2, 1)
plt.title("Original Image")
plt.imshow(image, cmap='gray')
plt.axis('off')

plt.subplot(1, 2, 2)
plt.title(f"Compressed Image with k={k}")
plt.imshow(image_compressed, cmap='gray')
plt.axis('off')

plt.show()

The original image and the compressed image are displayed side by side, demonstrating how SVD can reduce the size of the image while preserving its essential features −

Advantages of SVD

SVD provides several advantages, such as −

• Numerical Stability: SVD is numerically stable and can handle ill-conditioned matrices.
• Optimal Low-Rank Approximation: SVD provides the best low-rank approximation of a matrix, making it ideal for dimensionality reduction.
• Robustness: SVD is robust to small perturbations in the data.
• Versatility: SVD can be applied to any matrix, regardless of its properties.