manyspikes

Linear algebra in Python

Initialising environment...

We are approaching the end of this module, so let's recap what we covered:

  1. We learned what vectors are and we covered basic operations such as vector addition and scalar multiplication
  2. We looked at how to compute distances and similarities between vectors
  3. We also learned about the dot product and how it is often used in the context of machine learning and artificial intelligence
  4. We introduced the concept of a matrix as a linear transformation
  5. We explored a number of properties of matrices and how they relate to the underlying linear transformation
  6. We learned about the determinant and how it expresses the extent to which the corresponding transformation stretches or compresses space
  7. We looked at how eigenvectors and eigenvalues can further characterize a linear transformation, providing us with information about the extent to which space is stretched or compressed in particular directions

Throughout this process, we used either plain Python or "pen and paper" to demonstrate the calculations involved. This is helpful because it forces us to understand exactly what we are computing, which is often enlightning. However, in practice it is often better to use libraries that provide useful functionality for doing linear algebra on computers, mainly due to the following reasons:

  • We don't want to waste time reinventing the wheel;
  • Popular libraries for numerical computing are widely used and thoroughly tested, so they are less likely to have bugs;
  • Such libraries are often partly implemented in low-level programming languages, which means they are much faster compared to pure Python.

In Python, the most popular library for numerical computing is called NumPy. Among many other things, NumPy provides a set of fundamental utilities to work with vectors and matrices, as we will see in this notebook.

Using NumPy

NumPy is not part of the Python standard library. This means that if you install Python on your computer, it most likely will not include NumPy. We have pre-installed NumPy on this environment, so that's the reason you can use it without explicitly installing it.

Before using NumPy, we need to import it:

Because we will be using NumPy a lot throughout the code, and because it is a somewhat long name, we are appending as np to the import statement. This just means that we can now refer to NumPy by simply using np.

To demonstrate how to refer to NumPy in your Python code, above we are printing the version we have installed in this environment.

Now let's start by defining some vectors and matrices in NumPy. To do that, we use the np.array(...) call: NumPy will then look at the argument we passed in and transform it into a vector, a matrix or a tensor. You can think of a tensor as a generic NN-dimensional grid, as opposed to a matrix which can only have two dimensions (columns and rows). Regardless, NumPy treats all of these as arrays.

Let's start with vectors.

Now let's create another vector and perform some basic operations like vector addition and scalar multiplication. These will look much simpler than the implementation we looked at in earlier modules.

Note that the above only works because we are dealing with NumPy arrays. If you try to do w = v + u with v and u being lists, Python will concatenate the lists, which is not what we want here.

Now let's look at how to compute the Euclidean distance and the cosine similarity:

Working with matrices is equally simple. Similarly to vectors, we can pass a list of lists to np.array(...) to get a matrix, where the outer list represents the different rows of a matrix, and the inner lists contain the individual elements of a given row.

We can transpose a matrix by using the .T property on an array:

Applying linear transformations also becomes very easy by using the @ operator, which performs matrix multiplication. Let's take a look at transforming a vector using the matrix A defined above:

NumPy also provides convenient methods to calculate the determinant, rank and inverse of a matrix:

We can verify that the inverse is correctly computed by applying it to the transformed vector v_transformed and confirming that it produces the original vector v:

Last but not least, NumPy also allows us to easily compute eigenvalues and eigenvectors:

You may remember that for a diagonalizable matrix the determinant is equal to the product of the eigenvectors, so let's check that with NumPy:

Learning more

While what we covered here only scratches the surface of what NumPy has to offer, it provides you with very powerful tools for implementing machine learning models. Beyond this, the best place to learn more about what NumPy offers is their official documentation. Here are the links: