Return to Computer Vision Notebooks

Linear Algebra - Matrices



Linear Algebra topic about Matrices.

1. What is a matrix?


A matrix $ A$ is a ractangular array of elements arranged in rows $m$ and columns $n$.

$$ \large A = \begin{bmatrix} a_{11} & a_{12} & \cdots & a_{1n} \\ a_{21} & a_{22} & \cdots & a_{2n} \\ a_{31} & a_{32} & \cdots & a_{3n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m1} & a_{m2} & \cdots & a_{mn} \\ \end{bmatrix} $$

A square matrix is when the number of rows is the same of the number of columns.

2. Matrix operations


All the following operations are applicable to any vector in $\large \mathbb{R}^{m \times n}$.

2.1. Matrix addition


The addition of two matrices $\large M_{3\times2}$ and $\large N_{3\times2}$ is done by the sum of their correspondent components, resulting in another matrix.

$$ \large M+N = \begin{bmatrix} m_{11} & m_{12} \\ m_{21} & m_{22} \\ m_{31} & m_{32} \\ \end{bmatrix} + \begin{bmatrix} n_{11} & n_{12} \\ n_{21} & n_{22} \\ n_{31} & n_{32} \\ \end{bmatrix} = \begin{bmatrix} m_{11}+n_{11} & m_{12}+n_{12} \\ m_{21}+n_{21} & m_{22}+n_{22} \\ m_{31}+n_{31} & m_{32}+n_{32} \\ \end{bmatrix} $$

For example:

$$ \large M = \begin{bmatrix} 2 & 5 \\ 3 & 7 \\ 8 & 6 \end{bmatrix} \quad , \quad N = \begin{bmatrix} 1 & 7 \\ 5 & 6 \\ 2 & 9 \end{bmatrix} $$$$ \large M + N = \begin{bmatrix} 2+1 & 5+7 \\ 3+5 & 7+6 \\ 8+2 & 6+9 \end{bmatrix} = \begin{bmatrix} 3 & 12 \\ 8 & 13 \\ 10 & 15 \end{bmatrix} $$

Properties of matrix addition


  1. $M + (N + P) = (M + N) + P$
  2. $M + 0 = 0 + M = M$
  3. $M + N = N + M$

2.2. Matrix subtraction


Similarly to addition, the subtraction of two matrices $\large M$ and $\large N$ is done by the subtraction of their correspondent components, resulting in another matrix.

$$ \large M-N = \begin{bmatrix} m_{11} & m_{12} \\ m_{21} & m_{22} \\ m_{31} & m_{32} \\ \end{bmatrix} - \begin{bmatrix} n_{11} & n_{12} \\ n_{21} & n_{22} \\ n_{31} & n_{32} \\ \end{bmatrix} = \begin{bmatrix} m_{11}-n_{11} & m_{12}-n_{12} \\ m_{21}-n_{21} & m_{22}-n_{22} \\ m_{31}-n_{31} & m_{32}-n_{32} \\ \end{bmatrix} $$

For example:

$$ \large M = \begin{bmatrix} 2 & 5 \\ 3 & 7 \\ 8 & 6 \end{bmatrix} \quad , \quad N = \begin{bmatrix} 1 & 7 \\ 5 & 6 \\ 2 & 9 \end{bmatrix} $$$$ \large M - N = \begin{bmatrix} 2-1 & 5-7 \\ 3-5 & 7-6 \\ 8-2 & 6-9 \end{bmatrix} = \begin{bmatrix} 1 & -2 \\ -2 & 1 \\ 6 & -3 \end{bmatrix} $$

Properties of matrix subtraction


  1. $-(-M) = M$
  2. $-M + M = M - M = 0$
  3. $M - N \neq N - M$

2.3. Scalar multiplication


The scalar multiplication is the elementwise multiplication by a scalar number $\large \alpha$. The same rule can be applied to divisions.

$$ \large \alpha M = \alpha \cdot \begin{bmatrix} m_{11} & m_{12} & \cdots & m_{1n} \\ m_{21} & m_{22} & \cdots & m_{2n} \\ m_{31} & m_{32} & \cdots & m_{3n} \\ \vdots & \vdots & \ddots & \vdots \\ m_{m1} & m_{m2} & \cdots & m_{mn} \end{bmatrix} = \begin{bmatrix} \alpha \cdot m_{11} & \alpha \cdot m_{12} & \cdots & \alpha \cdot m_{1n} \\ \alpha \cdot m_{21} & \alpha \cdot m_{22} & \cdots & \alpha \cdot m_{2n} \\ \alpha \cdot m_{31} & \alpha \cdot m_{32} & \cdots & \alpha \cdot m_{3n} \\ \vdots & \vdots & \ddots & \vdots \\ \alpha \cdot m_{m1} & \alpha \cdot m_{m2} & \cdots & \alpha \cdot m_{mn} \end{bmatrix} $$

For example:

$$ \large \alpha = 2 \quad , \quad M = \begin{bmatrix} 2 & 5 \\ 3 & 7 \\ 8 & 6 \end{bmatrix} $$$$ \large \alpha \cdot M = \begin{bmatrix} 2 \cdot 2 & 2 \cdot 5 \\ 2 \cdot 3 & 2 \cdot 7 \\ 2 \cdot 8 & 2 \cdot 6 \end{bmatrix} = \begin{bmatrix} 4 & 10 \\ 6 & 14 \\ 16 & 12 \end{bmatrix} $$

Properties of scalar multiplication


  1. $(\alpha \beta) M = \alpha (\beta M)$
  2. $(\alpha + \beta) M = \alpha M + \beta M$
  3. $\alpha(M + N) = \alpha M + \alpha N$
  4. $1M = M$

3. Matrix multiplication


The multiplication of two matrices $\large M_{m \times n}$ and $\large N_{n \times p}$ is defined under the rule that the number of columns of the first matrix (${m \times n}$) must be equal to the number of rows of the second matrix (${n \times p}$), resulting in another matrix (${m \times p}$).

$$ \large [MN]_{ij}=m_{i1} \cdot n_{1j}+m_{i2} \cdot n_{2j}+ \cdots + m_{in} \cdot n_{nj}=\sum_1^n m_{in} \cdot m_{nj} $$

For example:

$$ \large M = \begin{bmatrix} 2 & 3 & 8 \\ 5 & 7 & 6 \end{bmatrix} \quad , \quad N = \begin{bmatrix} 1 & 7 \\ 5 & 6 \\ 2 & 9 \end{bmatrix} $$$$ \large M \times N = \begin{bmatrix} 2 \cdot 1 + 3 \cdot 5 + 8 \cdot 2 & 2 \cdot 7 + 3 \cdot 6 + 8 \cdot 9 \\ 5 \cdot 1 + 7 \cdot 5 + 6 \cdot 2 & 5 \cdot 7 + 7 \cdot 6 + 6 \cdot 9 \end{bmatrix} = \begin{bmatrix} 33 & 104 \\ 52 & 131 \end{bmatrix} $$

Properties of matrix multiplication


  1. $M_{m \times n} N_{n \times m} \neq N_{n \times m} M_{m \times n}$
  2. $(M_{m \times n} N_{n \times p}) P_{p \times q} = M_{m \times n}(N_{n \times p} P_{p \times q})$
  3. $(M_{m \times n} + N_{m \times n}) P_{n \times p} = M_{m \times n}P_{n \times p} + N_{m \times n}P_{n \times p}$
  4. $P_{m \times n} (M_{n \times p} + N_{n \times p}) = P_{m \times n} M_{n \times p} + P_{m \times n} N_{n \times p}$
  5. $(\alpha M_{m \times n}) N_{n \times p} = M_{m \times n} (\alpha N_{n \times p}) = \alpha (M_{m \times n} N_{n \times p})$

4. Matrix transpose


The transposition of a matrix is the operation which converts rows into columns and vice versa, so $M_{m \times n}^T = M_{n \times m}$.

For example:

$$ \large \begin{bmatrix} 1 & 7 \\ 5 & 6 \\ 2 & 9 \end{bmatrix}^T = \begin{bmatrix} 1 & 5 & 2 \\ 7 & 6 & 9 \end{bmatrix} $$

Properties of matrix transpose


  1. $(M + N)^T = M^T + N^T$
  2. $(\alpha M)^T = \alpha M^T$
  3. $(M^T)^T = M$
  4. $(M_{m \times n} N_{n \times p})^T = N_{n \times p}^T M_{m \times n}^T$

5. Matrix determinant


The determinant of a square matrix ($\det(M)$ or $|M|$) is an operation which encodes certain properties of the linear transformation described by the matrix, resulting in a real number.

$$ \large \det (M_{2 \times 2}) = \det \begin{bmatrix} m_{11} & m_{12} \\ m_{21} & m_{22} \end{bmatrix} = m_{11} m_{22} - m_{12} m_{21} $$

Using a $3 \times 3$ matrix, we have:

$$ \large \begin{aligned} |M_{3 \times 3}| & = \det \begin{bmatrix} m_{11} & m_{12} & m_{13} \\ m_{21} & m_{22} & m_{23} \\ m_{31} & m_{32} & m_{33} \end{bmatrix} \\ |M_{3 \times 3}| & = (m_{11} m_{22} m_{33} + m_{12} m_{23} m_{31} + m_{13} m_{21} m_{32}) - (m_{13} m_{22} m_{31} + m_{12} m_{21} m_{33} + m_{11} m_{23} m_{32}) \end{aligned} $$

For example:

$$ \large \begin{aligned} M & = \begin{bmatrix} 1 & 7 & 3 \\ 5 & 6 & 8 \\ 2 & 9 & 4 \end{bmatrix} \\ |M| & = (1 \cdot 6 \cdot 4 + 7 \cdot 8 \cdot 2 + 3 \cdot 5 \cdot 9) - (3 \cdot 6 \cdot 2 + 7 \cdot 5 \cdot 4 + 1 \cdot 8 \cdot 9) \\ |M| & = (24 + 112 + 135) - (36 + 140 + 72) \\ |M| & = 271 - 248 \\ |M| & = 23 \end{aligned} $$

Properties of matrix determinant


  1. $|M^T| = |M|$
  2. $|MN| = |M| |N|$

6. Identity matrix


The identity matrix $I_n$ is a square matrix which the elements of the main diagonal is equal to 1 and all other elements is equal to 0.

$$ \large I_1 = \begin{bmatrix} 1 \end{bmatrix} \quad , \quad I_2 = \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix} \quad , \quad I_3 = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix} \quad , \quad I_n = \begin{bmatrix} 1 & 0 & \cdots & 0 \\ 0 & 1 & \cdots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \cdots & 1 \end{bmatrix} $$

Properties of identity matrix


  1. $M_{m \times n} I_n = I_m M_{m \times n} = M$
  2. $I_n^T = I_n$

7. Matrix inverse


A square matrix $M$ is invertible if it is non-singular and exists another square matrix $N$ which satisfies the following conditions:

  1. $MN = NM = I$
  2. Non-singular matrix is when $|M| \neq 0$

$N$ is inverse the inverse of $M$ and is represented by $M^{-1}$, what it means that $M M^{-1} = I$.

For example:

if:

$$ \large M = \begin{bmatrix} 2 & 1 & 0 \\ 0 & 1 & 0 \\ 1 & 2 & 1 \end{bmatrix} $$

so:

$$ \large M M^{-1} = I_3 \quad \Rightarrow \quad \begin{bmatrix} 2 & 1 & 0 \\ 0 & 1 & 0 \\ 1 & 2 & 1 \end{bmatrix} \begin{bmatrix} m_{11} & m_{12} & m_{13} \\ m_{21} & m_{22} & m_{23} \\ m_{31} & m_{32} & m_{33} \end{bmatrix} = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix} $$$$ \large \begin{bmatrix} 2 m_{11} + m_{21} & 2 m_{12} + m_{22} & 2 m_{13} + m_{23} \\ m_{21} & m_{22} & m_{23} \\ m_{11} + 2 m_{21} + m_{31} & m_{12} + 2 m_{22} + m_{32} & m_{13} + 2 m_{23} + m_{33} \end{bmatrix} = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix} $$

After solving this system of equations (we can solve this by using Cramer's rule, for example), we have:

$$ \large M^{-1} = \begin{bmatrix} \frac{1}{2} & - \frac{1}{2} & 0 \\ 0 & 1 & 0 \\ - \frac{1}{2} & - \frac{3}{2} & 1 \end{bmatrix} $$