Inverse of a matrix

In the previous sections, we have learned how matrices relate to linear transformations and how different types of linear transformations affect their inputs. So here is one interesting question to ask: can linear transformations be undone or reversed? That is, for a given matrix $\mathbf{A}$, can we find some matrix $\mathbf{A^{-1}}$ that brings vectors transformed by $\mathbf{A}$ back to their original place?

Yes, if we satisfy two conditions:

1. The matrix $\mathbf{A}$ must be a square matrix
2. The determinant of the matrix $\mathbf{A}$ must not be 0

Under these conditions, it is possible to find a matrix $\mathbf{A^{-1}}$ such that:

which is equivalent to writing

where $I_n$ represents the $n$-by-$n$ identity matrix. The matrix $\mathbf{A^{-1}}$ is called the inverse of $\mathbf{A}$.

If a square matrix is not invertible (i.e. $\det(\mathbf{A})=0$), it is called singular or degenerate. An invertible matrix is called non-singular or nondegenerate.

But why can we not invert a transformation matrix if its determinant is 0? We have seen that a transformation matrix with zero determinant corresponds to a rank-deficient matrix, which maps the input space to a lower dimensional space. Once in the lower-dimensional space, we have lost all the information about the dimensions we discarded during the transformation. Thus, we can no longer reconstruct the original input in that space.

Some properties of inverses

Let's now go over some properties of inverse matrices and add some intuition whenever we can:

1. $(\mathbf{A^{-1}})^{-1} = \mathbf{A}$: The inverse of an inverse of a matrix is equal to the original matrix. This makes because reversing the reverse of a transformation should produce the original transformation.
2. $\det(\mathbf{A^{-1}}) = det(\mathbf{A})^-1$: The determinant of the inverse of $\mathbf{A}$ is equal to the inverse of the determinant of $\mathbf{A}$. The intuition here is that if $\mathbf{A}$ scaled the input by a factor of 3, the inverse must scale it back by one third.
3. $(k\mathbf{A})^-1 = k^{-1}\mathbf{A^{-1}}$: Multiplying the scalar $k$ by the matrix $\mathbf{A}$ scales the transformation up but does not affect any other aspect of the transformation. Thus, taking the inverse of the $k\mathbf{A}$ is equivalent to first inverting $\mathbf{A}$ and then scaling back the resulting transformation by the inverse of the scaling factor.
4. $(\mathbf{A_1} \mathbf{A_2} \cdots \mathbf{A_{k-1}} \mathbf{A_k} )^{-1} = \mathbf{A_k}^{-1} \mathbf{A_{k-1}^{-1}} \cdots \mathbf{A_{2}^{-1}} \mathbf{A_1}^{-1}$: The inverse of a sequence of linear transformations is equal to applying the inverses of the individual transformations in reverse order. This follows from the fact that matrix multiplication is not commutative, so in order to reverse a sequence of linear transformation, we need to apply the inverses of the individual transformations in reverse order.

Notes

1. Above we mentioned that non-square matrices do not have an inverse. However, a non-square, $m$-by-$n$ matrix $\mathbf{A}$ can have a left inverse or a right inverse (but not both). Left and right inverse matrices are non-square matrices $\mathbf{B}$ and $\mathbf{C}$ for which $\mathbf{B}\mathbf{A} = \mathbf{I_n}$ and $\mathbf{A}\mathbf{C} = \mathbf{I_m}$, respectively.
2. The computation of a matrix inverse can be numerically unstable in some situations, especially when the matrix has a rank much smaller than its number of columns (i.e. it has low rank). If possible, avoiding the computation of an inverse is usually a good thing to do.