We ended the previous module by stating that a vector is an object that be scaled or added to another such object, producing another vector. In this section, we will look at what that means in practice, using simple images as examples of vectors.
Let's start by looking at the addition of 2-dimensional vectors and . The goal is to calculate . We start with the vectors and , which we plot in red and blue, respectively.
One popular approach to summing these vectors is "move" the base of one of the vectors to the tip of the other vector. The resulting vector is obtained by connecting the base of the static vector to the tip of the moving vector. Taking the vector to be the static vector (in blue) and to be the moving vector (in red), we would first move the base of to the tip of , obtaining (red dashed line). Finally, we get the vector (black) by connecting the base of the vector to the tip of vector .
An easy way to think about this is by interpreting the vectors as instructions to move around in this 2d-space. The blue vector tells us to move 3 units up and 1 unit right, while the red vector tells us to move 2 units to the right (and 0 units up). The order in which we execute these instructions is irrelevant: we will always arrive at the same place, which is given by the total horizontal and vertical displacements encoded in the vectors (3 units to the right and 3 units up). Thus, all we need to do to arrive at the resulting vector is to add the horizontal and vertical displacement values of each of the vectors, i.e. their elements.
To illustrate this in practice, let's implement a function that adds two vectors.
If you look at the function above, there is nothing in it which is specific to vectors in two dimensions. The same principle applies to -dimensional vectors. As we move towards higher dimensions, the geometric method above starts making less sense, but it is still pretty easy to think of element-wise addition.
To illustrate the effect of vector addition in higher dimensions, let's take a look at the MNIST examples we worked with in the previous section. This is the vector we saw in the previous section:
And this is a random vector with the same dimensionality:
The vector above was generated by drawing a number from a uniform random distribution for every element of the vector, which visually just looks like noise. As we have seen, adding these two vectors amounts to adding their values element-wise, which produces the following vector:
Unsurprisingly, the addition of the vectors produces a vector that looks like the original image with some noise added to it. The thing to remember is that vector addition simply boils down to element-wise addition.
Much like vector addition, scalar multiplication boils down to an element-wise operation. In this case, we multiply the scalar by every element of the vector (this is sometimes known as broadcasting, because we broadcast the scalar value over all elements of the vector), thereby producing a new vector.
Let's implement scalar multiplication in a function:
Scalar multiplication effectively scales a vector up or down and potentially mirrors it with respect to the origin. Again, this operation can be performed in vectors of any dimensionality. Here is the result of performing scalar multiplication between the scalar and the noisy vector image above.
We multiplied every element of the noisy image vector by the , which means that we made "larger" values more negative and "smaller" values more positive, resulting in a vector that is the negative image relative to the original vector image.
To wrap up, here we looked at very simple operations applied to vectors. In the next section, we will start looking at how these basic operations can be used to ask interesting questions about our images.