Foundations of Linear Algebra: Vectors and the Field Axioms
Vector
The Everyday Idea of a Vector
Definition of a vector mostly comes from our schooling: Vector is a quantity that have both Magnitude & Direction. Most of the cases, the definition stands tall: a vector is something that has both Magnitude (size or length) and Direction.
- "Walk 5 blocks (Magnitude) North-East (Direction)."
This is the standard, geometric way to think about a vector - as an arrow pointing from one place to another. In linear algebra and data science, we often use a different, but equivalent, representation. We think of a vector as an ordered list of numbers, like this:
$$u = \begin{pmatrix} u_1 \\ u_2 \\ \vdots \\ u_n \end{pmatrix}$$
- This is an n-component vector. The numbers $u_1, u_2, \dots, u_n$ are the components. Components are also called as Dimensions.
- It is also called an ordered n-tuple. "Ordered" means the sequence matters ($\begin{pmatrix} 2 \\ 3 \end{pmatrix}$ is different from $\begin{pmatrix} 3 \\ 2 \end{pmatrix}$).
Example: A 2-Component Vector
Let's look at the simplest case: a vector with 2 components.
- 2-component vector: $u = \begin{pmatrix} u_1 \\ u_2 \end{pmatrix}$.
- Specific example: $u = \begin{pmatrix} 2 \\ 3 \end{pmatrix}$.
This vector can be plotted on a 2D graph with axes $x_1$ and $x_2$. Let's plot:
The vector starts at the origin (0, 0) and ends at the point (2, 3). The arrow itself is what represents the vector.
We are able to plot this in a 2D graph, because the vector have two components. We could also possibly represent a 3D vector but no more than that (We are talking in the sense of a 2D graph). Also this is the case for the current transformer architecture. Each word is represented as a multi-dimensional vector. Plotting it will be a havoc.
Real Vectors
The vector $u = \begin{pmatrix} 2 \\ 3 \end{pmatrix}$ is a Real vector because both of its components (2 and 3) are real numbers.
Throughout the series, we will primarily deal with real vectors, but remember that components could theoretically come from other mathematical systems, like complex numbers!
Ex: Biological Parameters Example:
Now, let's step away from geometry. Imagine you are tracking a patient's health. You measure 5 different parameters. You can put all these measurements into a single 5-component vector:
$$u = \begin{pmatrix} u_1 \\ u_2 \\ u_3 \\ u_4 \\ u_5 \end{pmatrix}$$
Represented as a Table:
| Component | Biological Parameter | What is the vector representing? |
|---|---|---|
| $u_1$ | Blood pressure | A Single Patient's Health Snapshot |
| $u_2$ | Blood Glucose level | ... |
| $u_3$ | Pulse rate | ... |
| $u_4$ | Heart rate | ... |
| $u_5$ | SPO₂ levels | ... |
In this context, the vector isn't a geometric arrow, but a single, organized collection of 5 related data points. This abstraction is the foundation of modern data analysis!
Let's say we are collecting 100 patient's data.
- Collect 5 parameters ($u_1, u_2, u_3, u_4, u_5$) from 100 different patients ($P_1, \dots, P_{100}$).
- This data can be represented as a matrix.
- The matrix will have dimensions $100 \times 5$.
- Rows represent the patients.
- Columns represent the different parameters we measure.
Then the Data Representation:
$$ \begin{pmatrix} u_1^{(1)} & u_2^{(1)} & u_3^{(1)} & u_4^{(1)} & u_5^{(1)} \\ u_1^{(2)} & u_2^{(2)} & u_3^{(2)} & u_4^{(2)} & u_5^{(2)} \\ \vdots & \vdots & \vdots & \vdots & \vdots \\ u_1^{(100)} & u_2^{(100)} & u_3^{(100)} & u_4^{(100)} & u_5^{(100)}\\ \end{pmatrix} $$
Now vector becomes a powerful way to organize data.
Field $\mathbb{F}$
Theory:
- A Field 𝔽 is a non-empty collection of elements with two operations:
- Field addition (+)
- Field multiplication (⋅)
- These operations must satisfy the following properties:
Addition Properties
- (i) Additive Identity: There exists an element \(0 \in F\) such that for any element \(a \in F\), \(a + 0 = 0 + a = a\).
- (ii) Closure under addition: For any two elements \(a, b \in F\), \(a + b \in F\).
- (iii) Additive Inverse: For every element \(a \in F\), there exists an additive inverse \((-a) \in F\) such that \(a + (-a) = 0\).
Multiplication Properties
- (iv) Multiplicative Identity: There exists an element \(1 \in F\) such that for any element \(a \in F\), \(a \cdot 1 = 1 \cdot a = a\).
- (v) Closure under multiplication: For any two elements \(a, b \in F\), \(a \cdot b \in F\).
- (vi) Multiplicative Inverse: For every non-zero element \(a \in F\), there exists a unique element \(a^{-1} \in F\) such that \(a \cdot a^{-1} = a^{-1} \cdot a = 1\).
Example 1: Set of Real Numbers, $\mathbb{R}$
Consider the set of real numbers, $\mathbb{R}$. Let's check if it's a field with the standard operations of addition (+) and multiplication (⋅).
- (i) $0 \in \mathbb{R}$. (Additive identity exists)
- (ii) For any two real numbers $a, b \in \mathbb{R}, a+b \in \mathbb{R}$. (Closure under addition)
- (iii) For every $a \in \mathbb{R}, (-a) \in \mathbb{R}$ such that $a + (-a) = -a + a = 0$. (Additive inverse exists)
- (iv) $1 \in \mathbb{R}$. (Multiplicative identity exists)
- (v) For any two real numbers $a, b \in \mathbb{R}, a \cdot b \in \mathbb{R}$. (Closure under multiplication)
- (vi) For every $a \neq 0, a \in \mathbb{R}$, there exists an $a^{-1} \in \mathbb{R}$ such that $a^{-1} \cdot a = a \cdot a^{-1} = 1$. (Multiplicative inverse exists for non-zero elements)
Conclusion: $\mathbb{R}$ is a field with operations $+$ and $\cdot$ .
Example 2: Set of Integers, $\mathbb{Z}$
Consider the set of integers, $\mathbb{Z} = \{\dots, -2, -1, 0, 1, 2, \dots\}$. Let's check the field properties.
- (i) $0 \in \mathbb{Z}$. ✓ (Additive identity exists)
- (ii) $a+b \in \mathbb{Z}$ for $a, b \in \mathbb{Z}$. ✓ (Closure under addition)
- (iii) For every $a \in \mathbb{Z}$, there exists $(-a) \in \mathbb{Z}$ such that $a + (-a) = 0$. ✓ (Additive inverse exists)
- (iv) $1 \in \mathbb{Z}$. ✓ (Multiplicative identity exists)
- (v) $a \cdot b \in \mathbb{Z}$ for any $a, b \in \mathbb{Z}$. ✓ (Closure under multiplication)
- (vi) Let $a = 2 \in \mathbb{Z}$. Then the multiplicative inverse $a^{-1} = 1/2 \notin \mathbb{Z}$. ✗
Conclusion: The set of integers $\mathbb{Z}$ is not a field (fails property vi).
Example 3: Integers Modulo 5, $\mathbb{Z}_5$
Consider the set of integers modulo 5. This is the set of remainders when an integer is divided by 5.
- $\mathbb{Z}_5 = \{0, 1, 2, 3, 4\}$
- The operations are: $\oplus_5$ (add modulo 5) and $\otimes_5$ (multiply modulo 5)
Checking Field Properties for $\mathbb{Z}_5$:
- (i) $0 \in \mathbb{Z}_5$. ✓ (Additive identity)
- (ii) $a \oplus_5 b \in \mathbb{Z}_5$ for any $a, b \in \mathbb{Z}_5$. ✓ (Closed under addition)
- (iii) For every $a \in \mathbb{Z}_5$, there exists $(-a) \in \mathbb{Z}_5$ such that $a \oplus_5 (-a) = 0$. ✓ (Additive inverse)
- (iv) $1 \in \mathbb{Z}_5$. ✓ (Multiplicative identity)
- (v) $a \otimes_5 b \in \mathbb{Z}_5$ for any $a, b \in \mathbb{Z}_5$. ✓ (Closed under multiplication)
- (vi) For every $a \neq 0$, there exists $a^{-1} \in \mathbb{Z}_5$ such that $a \otimes_5 a^{-1} = 1$. ✓ (Multiplicative inverse)
Addition Table ($\oplus_5$):
| ⊕5 | 0 | 1 | 2 | 3 | 4 |
|---|---|---|---|---|---|
| 0 | 0 | 1 | 2 | 3 | 4 |
| 1 | 1 | 2 | 3 | 4 | 0 |
| 2 | 2 | 3 | 4 | 0 | 1 |
| 3 | 3 | 4 | 0 | 1 | 2 |
| 4 | 4 | 0 | 1 | 2 | 3 |
The table shows closure under addition. Additive identity is 0 (first row/column). Additive inverses: $1 \leftrightarrow 4$, $2 \leftrightarrow 3$, $0 \leftrightarrow 0$.
Multiplication Table ($\otimes_5$):
| ⊙5 | 1 | 2 | 3 | 4 |
|---|---|---|---|---|
| 1 | 1 | 2 | 3 | 4 |
| 2 | 2 | 4 | 1 | 3 |
| 3 | 3 | 1 | 4 | 2 |
| 4 | 4 | 3 | 2 | 1 |
The table shows closure under multiplication. Multiplicative identity is 1. Multiplicative inverses exist for all non-zero elements:
- $1^{-1} = 1$ since $1 \otimes_5 1 = 1$
- $2^{-1} = 3$ since $2 \otimes_5 3 = 6 \equiv 1 \pmod{5}$
- $3^{-1} = 2$ since $3 \otimes_5 2 = 6 \equiv 1 \pmod{5}$
- $4^{-1} = 4$ since $4 \otimes_5 4 = 16 \equiv 1 \pmod{5}$
Conclusion: $\mathbb{Z}_5$ is a field with operations $\oplus_5$ and $\otimes_5$.
❓ Question: Is $\mathbb{Z}_n$ a field for any integer $n$?
Answer: Only when $n$ is prime. If $n$ is prime, every non-zero element has a multiplicative inverse. If $n$ is composite (e.g., $n = 4$), some elements lack inverses (consider $2^{-1}$ in $\mathbb{Z}_4$).