← Back to linear algebra 1

Span and Linear Independence

Two fundamental concepts in linear algebra are span (what vectors can we reach?) and linear independence (are vectors redundant?).

Span of Vectors

Definition (Span):
The span of vectors v⃗1,v⃗2,…,v⃗k\vec{v}_1, \vec{v}_2, \ldots, \vec{v}_k is the set of all linear combinations: Span{v⃗1,v⃗2,…,v⃗k}={c1v⃗1+c2v⃗2+⋯+ckv⃗k:c1,c2,…,ck∈R}\text{Span}\{\vec{v}_1, \vec{v}_2, \ldots, \vec{v}_k\} = \{c_1\vec{v}_1 + c_2\vec{v}_2 + \cdots + c_k\vec{v}_k : c_1, c_2, \ldots, c_k \in \mathbb{R}\}

The span represents all vectors we can "reach" by scaling and adding v⃗1,…,v⃗k\vec{v}_1, \ldots, \vec{v}_k.

Span of One Vector

The span of a single vector v⃗≠0⃗\vec{v} \neq \vec{0} is: Span{v⃗}={cv⃗:c∈R}\text{Span}\{\vec{v}\} = \{c\vec{v} : c \in \mathbb{R}\}

Geometric interpretation: A line through the origin in the direction of v⃗\vec{v}.

Example 1: Line in R2\mathbb{R}^2

Let v⃗=[21]\vec{v} = \begin{bmatrix} 2 \\ 1 \end{bmatrix}. Then Span{v⃗}\text{Span}\{\vec{v}\} consists of all vectors of the form: c[21]=[2cc]c\begin{bmatrix} 2 \\ 1 \end{bmatrix} = \begin{bmatrix} 2c \\ c \end{bmatrix}

This is the line through the origin with slope 12\frac{1}{2}.

Points on this line:

If v⃗=0⃗\vec{v} = \vec{0}, then: Span{0⃗}={c0⃗:c∈R}={0⃗}\text{Span}\{\vec{0}\} = \{c\vec{0} : c \in \mathbb{R}\} = \{\vec{0}\}

The span of the zero vector is just the origin (a single point, not a line).

Span of Two Vectors

The span of two vectors v⃗1,v⃗2\vec{v}_1, \vec{v}_2 is: Span{v⃗1,v⃗2}={c1v⃗1+c2v⃗2:c1,c2∈R}\text{Span}\{\vec{v}_1, \vec{v}_2\} = \{c_1\vec{v}_1 + c_2\vec{v}_2 : c_1, c_2 \in \mathbb{R}\}

Geometric interpretation: Depends on whether v⃗1\vec{v}_1 and v⃗2\vec{v}_2 are parallel or not.

Case 1: Non-parallel Vectors in R2\mathbb{R}^2

If v⃗1\vec{v}_1 and v⃗2\vec{v}_2 are not parallel, Span{v⃗1,v⃗2}\text{Span}\{\vec{v}_1, \vec{v}_2\} is the entire plane R2\mathbb{R}^2.

Example 2: Spanning R2\mathbb{R}^2

Let v⃗1=[10]\vec{v}_1 = \begin{bmatrix} 1 \\ 0 \end{bmatrix} and v⃗2=[01]\vec{v}_2 = \begin{bmatrix} 0 \\ 1 \end{bmatrix}.

For any [ab]∈R2\begin{bmatrix} a \\ b \end{bmatrix} \in \mathbb{R}^2: [ab]=a[10]+b[01]\begin{bmatrix} a \\ b \end{bmatrix} = a\begin{bmatrix} 1 \\ 0 \end{bmatrix} + b\begin{bmatrix} 0 \\ 1 \end{bmatrix}

Every vector in R2\mathbb{R}^2 can be written as a linear combination, so Span{v⃗1,v⃗2}=R2\text{Span}\{\vec{v}_1, \vec{v}_2\} = \mathbb{R}^2.

Case 2: Parallel Vectors

If v⃗2=kv⃗1\vec{v}_2 = k\vec{v}_1 for some scalar kk (parallel), then: c1v⃗1+c2v⃗2=c1v⃗1+c2(kv⃗1)=(c1+kc2)v⃗1c_1\vec{v}_1 + c_2\vec{v}_2 = c_1\vec{v}_1 + c_2(k\vec{v}_1) = (c_1 + kc_2)\vec{v}_1

This is just a scalar multiple of v⃗1\vec{v}_1, so Span{v⃗1,v⃗2}=Span{v⃗1}\text{Span}\{\vec{v}_1, \vec{v}_2\} = \text{Span}\{\vec{v}_1\} (a line).

Example 3: Parallel Vectors

Let v⃗1=[12]\vec{v}_1 = \begin{bmatrix} 1 \\ 2 \end{bmatrix} and v⃗2=[24]=2v⃗1\vec{v}_2 = \begin{bmatrix} 2 \\ 4 \end{bmatrix} = 2\vec{v}_1.

Then: c1v⃗1+c2v⃗2=c1[12]+c2[24]=(c1+2c2)[12]c_1\vec{v}_1 + c_2\vec{v}_2 = c_1\begin{bmatrix} 1 \\ 2 \end{bmatrix} + c_2\begin{bmatrix} 2 \\ 4 \end{bmatrix} = (c_1 + 2c_2)\begin{bmatrix} 1 \\ 2 \end{bmatrix}

Span{v⃗1,v⃗2}\text{Span}\{\vec{v}_1, \vec{v}_2\} is just the line through the origin in direction [12]\begin{bmatrix} 1 \\ 2 \end{bmatrix}.

Two vectors in R3\mathbb{R}^3:

  • If non-parallel: Span is a plane through the origin
  • If parallel: Span is a line through the origin

Example: v⃗1=[100]\vec{v}_1 = \begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix} and v⃗2=[010]\vec{v}_2 = \begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix} span the xyxy-plane: Span{v⃗1,v⃗2}={[ab0]:a,b∈R}\text{Span}\{\vec{v}_1, \vec{v}_2\} = \left\{\begin{bmatrix} a \\ b \\ 0 \end{bmatrix} : a, b \in \mathbb{R}\right\}

Span Summary

Linear Independence

When does adding another vector to a set actually expand the span?

Definition (Linear Independence):
Vectors v⃗1,v⃗2,…,v⃗k\vec{v}_1, \vec{v}_2, \ldots, \vec{v}_k are linearly independent if the only solution to: c1v⃗1+c2v⃗2+⋯+ckv⃗k=0⃗c_1\vec{v}_1 + c_2\vec{v}_2 + \cdots + c_k\vec{v}_k = \vec{0} is c1=c2=⋯=ck=0c_1 = c_2 = \cdots = c_k = 0 (the trivial solution).

If there exist non-zero coefficients satisfying the equation, the vectors are linearly dependent.

Intuition: Vectors are linearly independent if none of them is a linear combination of the others (none is "redundant").

Testing Linear Independence

Example 4: Independent Vectors

Are v⃗1=[10]\vec{v}_1 = \begin{bmatrix} 1 \\ 0 \end{bmatrix} and v⃗2=[01]\vec{v}_2 = \begin{bmatrix} 0 \\ 1 \end{bmatrix} linearly independent?

Solution: Consider c1v⃗1+c2v⃗2=0⃗c_1\vec{v}_1 + c_2\vec{v}_2 = \vec{0}: c1[10]+c2[01]=[00]c_1\begin{bmatrix} 1 \\ 0 \end{bmatrix} + c_2\begin{bmatrix} 0 \\ 1 \end{bmatrix} = \begin{bmatrix} 0 \\ 0 \end{bmatrix}

This gives: [c1c2]=[00]\begin{bmatrix} c_1 \\ c_2 \end{bmatrix} = \begin{bmatrix} 0 \\ 0 \end{bmatrix}

Only solution: c1=0,c2=0c_1 = 0, c_2 = 0.

Answer: Yes, they are linearly independent.

Example 5: Dependent Vectors

Are v⃗1=[12]\vec{v}_1 = \begin{bmatrix} 1 \\ 2 \end{bmatrix}, v⃗2=[24]\vec{v}_2 = \begin{bmatrix} 2 \\ 4 \end{bmatrix}, and v⃗3=[36]\vec{v}_3 = \begin{bmatrix} 3 \\ 6 \end{bmatrix} linearly independent?

Solution: Notice v⃗2=2v⃗1\vec{v}_2 = 2\vec{v}_1 and v⃗3=3v⃗1\vec{v}_3 = 3\vec{v}_1. We can write: 1⋅v⃗3−32v⃗2+0⋅v⃗1=[36]−32[24]=[00]1 \cdot \vec{v}_3 - \frac{3}{2}\vec{v}_2 + 0 \cdot \vec{v}_1 = \begin{bmatrix} 3 \\ 6 \end{bmatrix} - \frac{3}{2}\begin{bmatrix} 2 \\ 4 \end{bmatrix} = \begin{bmatrix} 0 \\ 0 \end{bmatrix}

Non-zero coefficients give 0⃗\vec{0}, so they are linearly dependent.

Vectors v⃗1,…,v⃗k\vec{v}_1, \ldots, \vec{v}_k are linearly dependent if and only if:

  1. At least one vector is a linear combination of the others
  2. At least one vector can be removed without changing the span
  3. The equation c1v⃗1+⋯+ckv⃗k=0⃗c_1\vec{v}_1 + \cdots + c_k\vec{v}_k = \vec{0} has a non-trivial solution

Example: In Example 5, v⃗3=3v⃗1\vec{v}_3 = 3\vec{v}_1, so v⃗3\vec{v}_3 is redundant: Span{v⃗1,v⃗2,v⃗3}=Span{v⃗1}\text{Span}\{\vec{v}_1, \vec{v}_2, \vec{v}_3\} = \text{Span}\{\vec{v}_1\}

All three vectors lie on the same line.

Geometric Interpretation of Linear Independence

In R2\mathbb{R}^2:

In R3\mathbb{R}^3:

Example 6: Three Vectors in R3\mathbb{R}^3

Consider v⃗1=[100]\vec{v}_1 = \begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix}, v⃗2=[010]\vec{v}_2 = \begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix}, v⃗3=[001]\vec{v}_3 = \begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix}.

Are they independent? Check if c1v⃗1+c2v⃗2+c3v⃗3=0⃗c_1\vec{v}_1 + c_2\vec{v}_2 + c_3\vec{v}_3 = \vec{0} implies all coefficients are zero:

c1[100]+c2[010]+c3[001]=[c1c2c3]=[000]c_1\begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix} + c_2\begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix} + c_3\begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix} = \begin{bmatrix} c_1 \\ c_2 \\ c_3 \end{bmatrix} = \begin{bmatrix} 0 \\ 0 \\ 0 \end{bmatrix}

Only solution: c1=c2=c3=0c_1 = c_2 = c_3 = 0.

Answer: Yes, they are linearly independent. These are the standard basis vectors of R3\mathbb{R}^3.

Connection Between Span and Independence

Key Insight:

  • Linearly independent vectors: Each vector adds a new "dimension" to the span
  • Linearly dependent vectors: At least one vector is redundant (doesn't expand the span)

Example 7: Building Up the Span

Start with v⃗1=[100]\vec{v}_1 = \begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix}:

Add v⃗2=[010]\vec{v}_2 = \begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix} (independent of v⃗1\vec{v}_1):

Add v⃗3=[001]\vec{v}_3 = \begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix} (independent of v⃗1,v⃗2\vec{v}_1, \vec{v}_2):

Add v⃗4=[110]\vec{v}_4 = \begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix} (dependent on v⃗1,v⃗2,v⃗3\vec{v}_1, \vec{v}_2, \vec{v}_3):

In Rn\mathbb{R}^n, you can have at most nn linearly independent vectors.

Why?

  • In R2\mathbb{R}^2: At most 2 independent vectors (they span the plane)
  • In R3\mathbb{R}^3: At most 3 independent vectors (they span 3D space)
  • In Rn\mathbb{R}^n: At most nn independent vectors (they span nn-dimensional space)

If you have more than nn vectors in Rn\mathbb{R}^n, they must be linearly dependent.

Linear Independence and Homogeneous Systems

There's a deep connection between linear independence and homogeneous equations:

Theorem (Linear Independence via Homogeneous Systems):
Vectors v⃗1,v⃗2,…,v⃗k\vec{v}_1, \vec{v}_2, \ldots, \vec{v}_k in Rn\mathbb{R}^n are linearly independent if and only if the homogeneous system: x1v⃗1+x2v⃗2+⋯+xkv⃗k=0⃗x_1\vec{v}_1 + x_2\vec{v}_2 + \cdots + x_k\vec{v}_k = \vec{0} has only the trivial solution x1=x2=⋯=xk=0x_1 = x_2 = \cdots = x_k = 0.

Matrix form: Let A=[v⃗1v⃗2⋯v⃗k]A = \begin{bmatrix} \vec{v}_1 & \vec{v}_2 & \cdots & \vec{v}_k \end{bmatrix} be the matrix with these vectors as columns. Then:

Example 8: Testing Independence via Homogeneous System

Test if v⃗1=[121]\vec{v}_1 = \begin{bmatrix} 1 \\ 2 \\ 1 \end{bmatrix}, v⃗2=[243]\vec{v}_2 = \begin{bmatrix} 2 \\ 4 \\ 3 \end{bmatrix}, v⃗3=[122]\vec{v}_3 = \begin{bmatrix} 1 \\ 2 \\ 2 \end{bmatrix} are linearly independent.

Solution: Form the matrix and solve Ax⃗=0⃗A\vec{x} = \vec{0}:

Testing Linear Independence

Step 0 of 3

The row-reduced form shows:

Conclusion: Only the trivial solution exists, so the vectors are linearly independent.

This connection gives us a computational method for testing linear independence:

Procedure:

  1. Form matrix AA with vectors as columns
  2. Row reduce to find pivot columns
  3. If all columns are pivot columns → independent
  4. If any column is not a pivot column → dependent

Relationship to free variables:

  • Free variables in Ax⃗=0⃗A\vec{x} = \vec{0} correspond to dependent vectors
  • Each free variable gives a nontrivial solution
  • The dependent vectors can be expressed in terms of independent ones

Example insight: If v⃗3\vec{v}_3 corresponds to a free variable, then v⃗3\vec{v}_3 can be written as a linear combination of v⃗1\vec{v}_1 and v⃗2\vec{v}_2.

Example 9: Dependent Vectors via Homogeneous System

Test if v⃗1=[12]\vec{v}_1 = \begin{bmatrix} 1 \\ 2 \end{bmatrix}, v⃗2=[24]\vec{v}_2 = \begin{bmatrix} 2 \\ 4 \end{bmatrix}, v⃗3=[36]\vec{v}_3 = \begin{bmatrix} 3 \\ 6 \end{bmatrix} are linearly independent.

Solution: Form A=[123246]A = \begin{bmatrix} 1 & 2 & 3 \\ 2 & 4 & 6 \end{bmatrix} and solve Ax⃗=0⃗A\vec{x} = \vec{0}:

[12302460]→R2−2R1[12300000]\left[\begin{array}{ccc|c} 1 & 2 & 3 & 0 \\ 2 & 4 & 6 & 0 \end{array}\right] \xrightarrow{R_2 - 2R_1} \left[\begin{array}{ccc|c} 1 & 2 & 3 & 0 \\ 0 & 0 & 0 & 0 \end{array}\right]

Only one pivot column (first column). Variables x2x_2 and x3x_3 are free.

From x1+2x2+3x3=0x_1 + 2x_2 + 3x_3 = 0: x1=−2x2−3x3x_1 = -2x_2 - 3x_3

Nontrivial solution (set x2=1,x3=0x_2 = 1, x_3 = 0): x⃗=[−210]\vec{x} = \begin{bmatrix} -2 \\ 1 \\ 0 \end{bmatrix}

This means: −2v⃗1+1v⃗2+0v⃗3=0⃗-2\vec{v}_1 + 1\vec{v}_2 + 0\vec{v}_3 = \vec{0}, or v⃗2=2v⃗1\vec{v}_2 = 2\vec{v}_1.

Conclusion: The vectors are linearly dependent.

Practice Problems

Describe geometrically the span of v⃗1=[110]\vec{v}_1 = \begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix} and v⃗2=[011]\vec{v}_2 = \begin{bmatrix} 0 \\ 1 \\ 1 \end{bmatrix}.

Are these vectors parallel? If not, they span a plane through the origin.

Check if they're parallel: v⃗2=kv⃗1\vec{v}_2 = k\vec{v}_1 for some kk?

If [011]=k[110]\begin{bmatrix} 0 \\ 1 \\ 1 \end{bmatrix} = k\begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix}, then:

  • First component: 0=k⋅1  ⟹  k=00 = k \cdot 1 \implies k = 0
  • Second component: 1=k⋅1  ⟹  k=11 = k \cdot 1 \implies k = 1

Contradiction! They're not parallel.

Answer: Span{v⃗1,v⃗2}\text{Span}\{\vec{v}_1, \vec{v}_2\} is a plane through the origin in R3\mathbb{R}^3.

Specifically, it's the set of all vectors of the form: c1[110]+c2[011]=[c1c1+c2c2]c_1\begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix} + c_2\begin{bmatrix} 0 \\ 1 \\ 1 \end{bmatrix} = \begin{bmatrix} c_1 \\ c_1 + c_2 \\ c_2 \end{bmatrix}

Determine if v⃗1=[123]\vec{v}_1 = \begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix}, v⃗2=[246]\vec{v}_2 = \begin{bmatrix} 2 \\ 4 \\ 6 \end{bmatrix} are linearly independent.

Check if one is a scalar multiple of the other.

Notice v⃗2=2v⃗1\vec{v}_2 = 2\vec{v}_1: [246]=2[123]\begin{bmatrix} 2 \\ 4 \\ 6 \end{bmatrix} = 2\begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix}

We can write: 2v⃗1−1v⃗2=0⃗2\vec{v}_1 - 1\vec{v}_2 = \vec{0}

This is a non-trivial linear combination equaling 0⃗\vec{0} (coefficients c1=2,c2=−1c_1 = 2, c_2 = -1 are not both zero).

Answer: No, they are linearly dependent.

Is b⃗=[53]\vec{b} = \begin{bmatrix} 5 \\ 3 \end{bmatrix} in Span{[11],[21]}\text{Span}\left\{\begin{bmatrix} 1 \\ 1 \end{bmatrix}, \begin{bmatrix} 2 \\ 1 \end{bmatrix}\right\}?

Try to write b⃗\vec{b} as c1v⃗1+c2v⃗2c_1\vec{v}_1 + c_2\vec{v}_2. This gives a system of equations.

We need to find c1,c2c_1, c_2 such that: c1[11]+c2[21]=[53]c_1\begin{bmatrix} 1 \\ 1 \end{bmatrix} + c_2\begin{bmatrix} 2 \\ 1 \end{bmatrix} = \begin{bmatrix} 5 \\ 3 \end{bmatrix}

This gives the system: {c1+2c2=5c1+c2=3\begin{cases} c_1 + 2c_2 = 5 \\ c_1 + c_2 = 3 \end{cases}

Subtracting: (c1+2c2)−(c1+c2)=5−3(c_1 + 2c_2) - (c_1 + c_2) = 5 - 3 c2=2c_2 = 2

Then: c1=3−c2=3−2=1c_1 = 3 - c_2 = 3 - 2 = 1

Verification: 1[11]+2[21]=[11]+[42]=[53]1\begin{bmatrix} 1 \\ 1 \end{bmatrix} + 2\begin{bmatrix} 2 \\ 1 \end{bmatrix} = \begin{bmatrix} 1 \\ 1 \end{bmatrix} + \begin{bmatrix} 4 \\ 2 \end{bmatrix} = \begin{bmatrix} 5 \\ 3 \end{bmatrix} ✓

Answer: Yes, b⃗∈Span{v⃗1,v⃗2}\vec{b} \in \text{Span}\{\vec{v}_1, \vec{v}_2\}.

Can three vectors in R2\mathbb{R}^2 be linearly independent?

Think about the maximum dimension of the span in R2\mathbb{R}^2.

No. At most 2 vectors in R2\mathbb{R}^2 can be linearly independent.

Intuition: The span of any set of vectors in R2\mathbb{R}^2 is at most 2-dimensional (the entire plane). Once you have 2 independent vectors spanning the plane, any third vector must be a linear combination of the first two.

Formal argument: If v⃗1,v⃗2\vec{v}_1, \vec{v}_2 are linearly independent in R2\mathbb{R}^2, they span R2\mathbb{R}^2. Any third vector v⃗3\vec{v}_3 can be written as v⃗3=c1v⃗1+c2v⃗2\vec{v}_3 = c_1\vec{v}_1 + c_2\vec{v}_2 for some c1,c2c_1, c_2. Then: v⃗3−c1v⃗1−c2v⃗2=0⃗\vec{v}_3 - c_1\vec{v}_1 - c_2\vec{v}_2 = \vec{0}

This is a non-trivial linear combination (coefficient of v⃗3\vec{v}_3 is 1), so {v⃗1,v⃗2,v⃗3}\{\vec{v}_1, \vec{v}_2, \vec{v}_3\} is linearly dependent.

Summary