An $m \times n$ matrix is a rectangular array of numbers with $m$ rows and $n$ columns:

A = \begin{bmatrix} a_{11} & a_{12} & \cdots & a_{1n} \\ a_{21} & a_{22} & \cdots & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m1} & a_{m2} & \cdots & a_{mn} \end{bmatrix}

Notation:

$A$ is an $m \times n$ matrix means $A$ has $m$ rows and $n$ columns
We usually denote the entry in the row $i$ , column $j$ as $a_{ij}$
We can write a matrix $A$ in terms of its columns: $A = \begin{bmatrix} \vec{a}_1 & \vec{a}_2 & \cdots & \vec{a}_n \end{bmatrix},$ where each $\vec{a}_j$ is a column vector in $\mathbb{R}^m$ .

Matrix-Vector Multiplication

Definition (Matrix-Vector Multiplication):
If $A$ is an $m \times n$ matrix with columns $\vec{a}_1, \vec{a}_2, \ldots, \vec{a}_n$ and $\vec{x} = (x_1, x_2,\ldots, x_n)$ is a vector in $\mathbb{R}^n$ , then:
$A\vec{x} = \begin{bmatrix} \vec{a}_1 & \vec{a}_2 & \cdots & \vec{a}_n \end{bmatrix}\begin{bmatrix} x_1 \\ x_2 \\ \vdots \\ x_n \end{bmatrix} = x_1\vec{a}_1 + x_2\vec{a}_2 + \cdots + x_n\vec{a}_n$
The product $A\vec{x}$ is a linear combination of the columns of $A$ with weights from $\vec{x}$ , and the resulting vector lies in $\mathbb{R}^m$ .

Let $A = \begin{bmatrix} 1 & 2 \\ 3 & 4 \\ 5 & 6 \end{bmatrix}$ and $\vec{x} = \begin{bmatrix} 2 \\ 3 \end{bmatrix}$ .

Compute $A\vec{x}$ :

$A\vec{x} = 2\begin{bmatrix} 1 \\ 3 \\ 5 \end{bmatrix} + 3\begin{bmatrix} 2 \\ 4 \\ 6 \end{bmatrix} = \begin{bmatrix} 2 \\ 6 \\ 10 \end{bmatrix} + \begin{bmatrix} 6 \\ 12 \\ 18 \end{bmatrix} = \begin{bmatrix} 8 \\ 18 \\ 28 \end{bmatrix}$

There's an equivalent way to compute $A\vec{x}$ by taking dot products of rows with $\vec{x}$ :

$A\vec{x} = \begin{bmatrix} 1 & 2 \\ 3 & 4 \\ 5 & 6 \end{bmatrix}\begin{bmatrix} 2 \\ 3 \end{bmatrix} = \begin{bmatrix} 1(2) + 2(3) \\ 3(2) + 4(3) \\ 5(2) + 6(3) \end{bmatrix} = \begin{bmatrix} 8 \\ 18 \\ 28 \end{bmatrix}$

Both methods give the same result. The column perspective (linear combination) is more conceptually important for linear algebra, while the row perspective is sometimes more convenient for computation.

Let $I = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix}$ and $\vec{x} = \begin{bmatrix} a \\ b \\ c \end{bmatrix}$ .

I\vec{x} = a\begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix} + b\begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix} + c\begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix} = \begin{bmatrix} a \\ b \\ c \end{bmatrix} = \vec{x}

The identity matrix leaves any vector unchanged: $I\vec{x} = \vec{x}$ .

Properties of Matrix-Vector Multiplication

Theorem (Linearity of Matrix-Vector Multiplication):
Matrix-vector multiplication satisfies the following two fundamental linearity properties:

$A(\vec{u} + \vec{v}) = A\vec{u} + A\vec{v}$

$A(c\vec{u}) = c(A\vec{u})$

Let $A = \begin{bmatrix} \vec{a}_1 & \vec{a}_2 & \cdots & \vec{a}_n \end{bmatrix}$ , $\vec{u} = \begin{bmatrix} u_1 \\ \vdots \\ u_n \end{bmatrix}$ , and $\vec{v} = \begin{bmatrix} v_1 \\ \vdots \\ v_n \end{bmatrix}$ .

Then $\vec{u} + \vec{v} = \begin{bmatrix} u_1 + v_1 \\ \vdots \\ u_n + v_n \end{bmatrix}$ and $c\vec{u} = \begin{bmatrix} cu_1 \\ \vdots \\ cu_n \end{bmatrix}$ , so:

\begin{align*} A(\vec{u} + \vec{v}) &= (u_1 + v_1)\vec{a}_1 + \cdots + (u_n + v_n)\vec{a}_n \\ &= (u_1\vec{a}_1 + \cdots + u_n\vec{a}_n) + (v_1\vec{a}_1 + \cdots + v_n\vec{a}_n) \\ &= A\vec{u} + A\vec{v} \\ A(c\vec{u}) &= (cu_1)\vec{a}_1 + \cdots + (cu_n)\vec{a}_n \\ &= c(u_1\vec{a}_1) + \cdots + c(u_n\vec{a}_n) \\ &= c(u_1\vec{a}_1 + \cdots + u_n\vec{a}_n) \\ &= c(A\vec{u}) \end{align*}

The two properties combine into a single statement:

Corollary:
For any matrix $A$ , vectors $\vec{u}, \vec{v}$ , and scalars $c, d$ :
$A(c\vec{u} + d\vec{v}) = cA\vec{u} + dA\vec{v}$
More generally, for any linear combination, we have:
$A(c_1\vec{v}_1 + c_2\vec{v}_2 + \cdots + c_k\vec{v}_k) = c_1A\vec{v}_1 + c_2A\vec{v}_2 + \cdots + c_kA\vec{v}_k$

Introduction to Matrices

Matrix-Vector Multiplication

Properties of Matrix-Vector Multiplication