[Linear Algebra] Vector Norm, Orthogonal Vectors

Vector Norm

• For $\textbf{v} \in \mathbb{R}^n$ with entries $v_1, \cdots, v_n$, the square root of $\textbf{v} \cdot \textbf{v}$ is defined because $\textbf{v} \cdot \textbf{v}$ is non-negative
• The length (or norm) of $\textbf{v}$ is the non-negative scalar $\begin{Vmatrix} \textbf{v} \end{Vmatrix}$ defined as the square root of $\textbf{v} \cdot \textbf{v}$
  • $\begin{Vmatrix} \textbf{v} \end{Vmatrix} = \sqrt{\textbf{v} \cdot \textbf{v}} = \sqrt{v^{2}_{1} + v^{2}_{2} + \cdots + v^{2}_{n}}$ and $\begin{Vmatrix} \textbf{v} \end{Vmatrix}^2 = \textbf{v} \cdot \textbf{v}$

Geometric Meaning of Vector Norm

• $\begin{Vmatrix} \textbf{v} \end{Vmatrix}$ is the length of the line segment from the origin to $\textbf{v}$
• For any scalar $c$, the length $c\textbf{v}$ is $\begin{vmatrix} c \end{vmatrix}$ times the length of $\textbf{v}$
  • $\begin{Vmatrix} c\textbf{v} \end{Vmatrix} = \begin{vmatrix} c \end{vmatrix} \begin{Vmatrix} \textbf{v} \end{Vmatrix}$

Unit Vector

• A vector whose length is 1 is called a unit vector
• Normalizing a vector : given a non-zero vector $\textbf{v}$, divide it by its length and we obtain a unit vector $\textbf{u} = \frac{1}{\begin{Vmatrix} \textbf{v} \end{Vmatrix}} \textbf{v}$
  • $\textbf{u}$ is in the same direction as $\textbf{v}$, but its length is 1

Distance between Vectors in $\mathbb{R}^n$

• Definition : for $\textbf{u}$ and $\textbf{v}$ in $\mathbb{R}^n$, the distance between $\textbf{u}$ and $\textbf{v}$, written as $\textrm{dist}(\textbf{u}, \textbf{v})$, is the length of the vector $\textbf{u} - \textbf{v}$
  • $\textrm{dist}(\textbf{u}, \textbf{v}) = \begin{Vmatrix} \textbf{u} - \textbf{v} \end{Vmatrix}$
• The distance from $\textbf{u}$ to $\textbf{v}$ is the same as the distance from $\textbf{u} - \textbf{v}$ to $\textbf{0}$

Inner Product and Angle between Vectors

• Inner product between $\textbf{u}$ and $\textbf{v}$ can be rewritten using their norms and angle
  • $\textbf{u} \cdot \textbf{v} = \begin{Vmatrix} \textbf{u} \end{Vmatrix} \begin{Vmatrix} \textbf{v} \end{Vmatrix} \textrm{cos} \theta$

Orthogonal Vectors

• Definition : $\textbf{u} \in \mathbb{R}^n$ and $\textbf{v} \in \mathbb{R}^n$ are orthogonal (to each other) if $\textbf{u} \cdot \textbf{v} = 0$
  • That is, $\textbf{u} \cdot \textbf{v} = \begin{Vmatrix} \textbf{u} \end{Vmatrix} \begin{Vmatrix} \textbf{v} \end{Vmatrix} \textrm{cos} \theta = 0$
  • $\textbf{u}$ and $\textbf{v}$ are perpendicular each other