Return to Computer Vision Notebooks

Linear Algebra - Vectors



Linear Algebra topic about Vectors.

1. What is a vector?


A vector $\large \vec{v}$ is a mathematical entity which has magnitude and direction. vectors

2. Vector operations


All the following operations are applicable to any vector in $\large \mathbb{R}^n$.

2.1. Vector addition


The addition of two vectors $\large \vec{u}$ and $\large \vec{v}$ is done by the sum of their correspondent components, resulting in another vector.

$$ \large \vec{u}+\vec{v} = \begin{bmatrix} u_1 \\ u_2 \\ \vdots \\ u_n \end{bmatrix} + \begin{bmatrix} v_1 \\ v_2 \\ \vdots \\ v_n \end{bmatrix} = \begin{bmatrix} v_1 + u_1 \\ v_2 + u_2 \\ \vdots \\ v_n + u_n \end{bmatrix} $$

vectors

For example:

$$ \large \vec{u} = \begin{bmatrix} 3 \\ 7 \end{bmatrix} \quad , \quad \vec{v} = \begin{bmatrix} 2 \\ 5 \end{bmatrix} $$$$ \large \vec{u} + \vec{v} = \begin{bmatrix} 3 + 2 \\ 7 + 5 \end{bmatrix} = \begin{bmatrix} 5 \\ 12 \end{bmatrix} $$

2.2. Vector subtraction


Similarly to addition, the subtraction of two vectors $\large \vec{u}$ and $\large \vec{v}$ is done by the subtraction of their correspondent components, resulting in another vector.

$$ \large \vec{u}-\vec{v} = \begin{bmatrix} u_1 \\ u_2 \\ \vdots \\ u_n \end{bmatrix} - \begin{bmatrix} v_1 \\ v_2 \\ \vdots \\ v_n \end{bmatrix} = \begin{bmatrix} v_1 - u_1 \\ v_2 - u_2 \\ \vdots \\ v_n - u_n \end{bmatrix} $$

vectors

For example:

$$ \large \vec{u} = \begin{bmatrix} 3 \\ 7 \end{bmatrix} \quad , \quad \vec{v} = \begin{bmatrix} 2 \\ 5 \end{bmatrix} $$$$ \large \vec{u} - \vec{v} = \begin{bmatrix} 3 - 2 \\ 7 - 5 \end{bmatrix} = \begin{bmatrix} 1 \\ 2 \end{bmatrix} $$

2.3. Scalar multiplication


The scalar multiplication is the elementwise multiplication by a scalar number $\large \alpha$. The same rule can be applied to divisions.

$$ \large \alpha\vec{u} = \alpha \cdot \begin{bmatrix} u_1 \\ u_2 \\ \vdots \\ u_n \end{bmatrix} = \begin{bmatrix} \alpha\cdot u_1 \\ \alpha\cdot u_2 \\ \vdots \\ \alpha\cdot u_n \end{bmatrix} $$

vectors

For example:

$$ \large \alpha = 2 \quad , \quad \vec{u} = \begin{bmatrix} 3 \\ 7 \end{bmatrix} $$$$ \large \alpha\cdot\vec{u} = \begin{bmatrix} 2 \cdot 3 \\ 2 \cdot 7 \end{bmatrix} = \begin{bmatrix} 6 \\ 14 \end{bmatrix} $$

3. Dot product


Dot product is an algebraic operation, which has a huge number of applications. As a result, we have a scalar value.

$$ \large \vec{u} \cdot \vec{v} = \begin{bmatrix} u_1 \\ u_2 \\ \vdots \\ u_n \end{bmatrix} \cdot \begin{bmatrix} v_1 \\ v_2 \\ \vdots \\ v_n \end{bmatrix} = \sum_i^n u_i \cdot v_i $$$$ \large \sum_i^n u_i \cdot v_i = u_1 \cdot v_1 + u_2 \cdot v_2 + ... + u_n \cdot v_n $$

For example:

$$ \large \vec{u} = \begin{bmatrix} 3 \\ 7 \end{bmatrix} \quad , \quad \vec{v} = \begin{bmatrix} 2 \\ 5 \end{bmatrix} $$$$ \large \begin{aligned} \vec{u} \cdot \vec{v} &= 3 \cdot 2 + 7 \cdot 5 \\ &= 6 + 35 \\ &= 41 \end{aligned} $$

3.1 Unit vector


Unit vector (or versor) is a vector which has the magnitude equal to 1. The magnitude of a vector is based on the euclidean norm and can be found by:

$$ \large \|\vec{u}\|_2 = \left[ \sum_i^n u_i^2 \right]^{\frac{1}{2}}=\sqrt{u_1^2+u_2^2+...+u_n^2} $$

Basically, a unit vector is a normalized vector, like:

$$ \large \hat{u}=\frac{\vec{u}}{\|\vec{u}\|} $$

vectors

For example:

$$ \large \vec{u} = [3 \quad 4] $$$$ \large \begin{aligned} \|\vec{u}\| &= \sqrt{3^2+4^2} \\ &= \sqrt{9+16} \\ &= 5 \end{aligned} $$$$ \large \frac{\vec{u}}{\|\vec{u}\|} = \left[ \frac{3}{5} \quad \frac{4}{5} \right] = [0.6 \quad 0.8] $$

3.2. Angle between vectors


Given the geometric definition of dot product:

$$ \large \vec{u} \cdot \vec{v} = \|\vec{u}\| \|\vec{v}\| \cos{\theta} $$

so,

$$ \large \theta=\cos^{-1} \frac{\vec{u} \cdot \vec{v}}{\|\vec{u}\| \|\vec{v}\|} $$

vectors

For example:

$$ \large \vec{u} = \begin{bmatrix} 2 \\ 1 \end{bmatrix} \quad , \quad \vec{v} = \begin{bmatrix} 3 \\ 3 \end{bmatrix} $$$$ \large \begin{aligned} \angle (\vec{u}, \vec{v}) = \theta &= \arccos \frac{2 \cdot 3 + 1 \cdot 3}{\sqrt{2^2+1^2} \sqrt{3^2+3^2}} \\ &= \arccos \frac{9}{\sqrt{5} \sqrt{18}} \\ &\approx 0.32 \text{rad} \quad \approx 18.43° \end{aligned} $$

3.3. Orthogonality and Parallelism


Two vectors are parallel or $\large \vec{u} // \vec{v}$ when there is a number $k$ which generalizes the relationship:

$$ \large \vec{u} = k\vec{v} $$

What it means that:

$$ \large \frac{u_1}{v_1} = \frac{u_2}{v_2} = ... = \frac{u_n}{v_n} = k $$

In other words, two vectors are parallel when their components are proportional.

vectors

For example:

$$ \large \vec{u} = \begin{bmatrix} 2 \\ 3 \end{bmatrix} \quad , \quad \vec{v} = \begin{bmatrix} 6 \\ 9 \end{bmatrix} $$$$ \large \frac{2}{6} = \frac{3}{9} = \frac{1}{3} $$

Two vectors are orthogonal or $\large \vec{u} \bot \vec{v}$ when the angle $\theta$ between of them are 90° or their dot product is equal to 0.

$$ \large \vec{u} \bot \vec{v} \Rightarrow \vec{u} \cdot \vec{v} = 0 $$

vectors

For example:

$$ \large \vec{u} = \begin{bmatrix} 2 \\ 1 \end{bmatrix} \quad , \quad \vec{v} = \begin{bmatrix} -2 \\ 4 \end{bmatrix} $$

We can find it by calculating the dot product:

$$ \large \vec{u} \cdot \vec{v} = 2 \cdot (-2) + 1 \cdot 4 = 0 $$

or by finding $\theta$:

$$ \large \begin{aligned} \angle (\vec{u}, \vec{v}) = \theta &= \arccos \frac{2 \cdot (-2) + 1 \cdot 4}{\sqrt{2^2+1^2} \sqrt{(-2)^2+4^2}} \\ &= \arccos \frac{0}{\sqrt{5} \sqrt{20}} \\ &\approx 1.57 \text{rad} \quad = 90° \end{aligned} $$