Composition of linear transformations and matrix multiplication
Notes
Given the definition of matrix-vector multiplication, we can derive the definition of AB as the product of two matrices.
If $\mathbf{b}_1, \mathbf{b}_2, \dots, \mathbf{b}_r$ are the columns of $B$, then $A\mathbf{b}_1, A\mathbf{b}_2, \dots, A\mathbf{b}_r$ are the columns of $AB$. Each column can be written as $\mathbf{b}_k = (b_{1,k}, b_{2,k}, \dots, b_{m,k})^T$. By looking at the indices we can see $(A\mathbf{b}_k)j = (AB){j,k}$. This relation gives us the key to define the matrix multiplication.
The multiplication of a matrix and a vector is given by $(A\mathbf{b})_j = \sum_l a_{j,l}b_l$. The vector is the special case where $\mathbf{b} = \mathbf{b}_1$, and thus we can also express it as $(A\mathbf{b}_1)_j = \sum_l a_{j,l} b_{l,1} = (AB)_{j,1}$. Since this works for any column $k$, we can define the product of two matrices as the matrix with entries $(AB)_{j,k} = \sum_l a_{j,l} b_{l,k}$.
Note that the size of a row of $A$ should be equal to the size of a column of $B$ in order for the multiplication to be defined. In other words, the product $AB$ is defined if and only if $A$ is an $m \times n$ matrix and $B$ is an $n \times r$ matrix and $AB$ is an $m \times r$ matrix.
If we have two linear transformations $T_1 : \mathbb{F}^n \to \mathbb{F}^m$ and $T_2 : \mathbb{F}^r \to \mathbb{F}^n$ the composition $T = T_1 \circ T_2$ is defined as:
$$ T(\mathbf{x}) = T_1(T_2(\mathbf{x})) \qquad \forall \mathbf{x} \in \mathbb{F}^r $$If $A$ is the matrix of $T_1$ and $B$ the matrix of $T_2$, we can see that:
$$ T(\mathbf{e}_k) = T_1(T_2(\mathbf{e}_k)) = T_1(B\mathbf{e}_k) = T_1(\mathbf{b}_k) = A\mathbf{b}_k $$The columns of $T$ are $A\mathbf{b}_k$, as in the definition of $AB$. From this we can conclude that the composition of two transformations is equivalent to the multiplication of two matrices.
Matrix multiplication has the following properties:
- Associativity: $A(BC) = (AB)C$
- Distributivity: $A(B + C) = AB + AC$ and $(A + B)C = AC + BC$
- Scalars can be taken out: $A(\alpha B) = (\alpha A)B = \alpha(AB) = \alpha AB$
Matrix multiplication is non-commutative, that is $AB \neq BA$.
The transposed matrix $A^T$ is defined as $(A^T){j,k} = A{k,j}$. When the matrix is a linear transformation the transpose is called the adjoint transformation. A simple analysis of the row by columns multiplication rule shows that $(AB)^T = B^TA^T$.
For a square matrix ($n \times n$) $A = (a_{j,k})$ its trace is the sum of the diagonal entries:
$$ \text{trace }A = \sum^n_{k=1} a_{k,k} $$Theorem 5.1. Let $A$ and $B$ be matrices of size $m \times n$ and $n \times m$ respectively. Then:
$$ \text{trace}(AB) = \text{trace}(BA) $$Exercises
5.1
$$ A= \begin{pmatrix} 1 & 2 \\ 3 & 1 \end{pmatrix}, B= \begin{pmatrix} 1 & 0 & 2 \\ 3 & 1 & -2 \end{pmatrix}, C= \begin{pmatrix} 1 & -2 & 3 \\ -2 & 1 & -1 \end{pmatrix}, D= \begin{pmatrix} -2 \\ 2 \\ 1 \end{pmatrix} $$a) Mark all the products that are defined, and give the dimensions of the result: $AB$, $BA$, $ABC$, $ABD$, $BC$, $BC^T$, $B^TC$, $DC$, $D^TC^T$.
b) Compute $AB$, $A(3B + C)$, $B^TA$, $A(BD)$, $(AB)D$.
a)
- $AB$ - $(2 \times 2)(2 \times 3) \to$ Defined, result is $2 \times 3$
- $BA$ - $(2 \times 3)(2 \times 2) \to$ Undefined
- $ABC$ - $(2 \times 2)(2 \times 3)(2 \times 3) \to$ Undefined
- $ABD$ - $(2 \times 2)(2 \times 3)(3 \times 1) \to$ Defined, result is $2 \times 1$
- $BC$ - $(2 \times 3)(2 \times 3) \to$ Undefined
- $BC^T$ - $(2 \times 3)(3 \times 2) \to$ Defined, result is $2 \times 2$
- $B^TC$ - $(3 \times 2)(2 \times 3) \to$ Defined, result is $3 \times 3$
- $DC$ - $(3 \times 1)(2 \times 3) \to$ Undefined
- $D^TC^T$ - $(1 \times 3)(3 \times 2) \to$ Defined, result is $1 \times 2$
b)
$$ AB = \begin{pmatrix} 7 & 2 & -2 \\ 6 & 1 & 4 \end{pmatrix} $$$$ A(3B + C) = \begin{pmatrix} 18 & 6 & -5 \\ 19 & -2 & 20 \end{pmatrix} $$$$ B^TA = \begin{pmatrix} 7 & 4 \\ 3 & 1 \\ 4 & -2 \end{pmatrix} $$$$ A(BD) = \begin{pmatrix} -12 \\ -6 \end{pmatrix} $$$$ (AB)D = \begin{pmatrix} -12 \\ -6 \end{pmatrix} $$5.2
Let $T_{\gamma}$ be the matrix of rotation by $\gamma$ in $\mathbb{R}^2$. Check by matrix multiplication that $T_{\gamma}T_{-\gamma} = T_{-\gamma}T_{\gamma} = I$.
The matrices are:
$$ T_{\gamma} = \begin{pmatrix} \cos \gamma & -\sin \gamma \\ \sin \gamma & \cos \gamma \end{pmatrix}, \quad T_{-\gamma} = \begin{pmatrix} \cos(-\gamma) & -\sin(-\gamma) \\ \sin(-\gamma) & \cos(-\gamma) \end{pmatrix} $$$$ \begin{aligned} T_{\gamma}T_{-\gamma} &= \begin{pmatrix} (\cos \gamma \cos(-\gamma)) - (\sin \gamma \sin(-\gamma)) & (-\sin \gamma \cos(-\gamma)) - (\cos \gamma \sin(-\gamma)) \\ (\cos \gamma \sin(-\gamma)) + (\sin \gamma \cos(-\gamma)) & (-\sin \gamma \sin(-\gamma)) + (\cos \gamma \cos(-\gamma)) \end{pmatrix} \\ &= \begin{pmatrix} (\cos \gamma \cos \gamma) + (\sin \gamma \sin \gamma) & (-\sin \gamma \cos \gamma) + (\cos \gamma \sin \gamma) \\ (-\cos \gamma \sin \gamma) + (\sin \gamma \cos \gamma) & (\sin \gamma \sin \gamma) + (\cos \gamma \cos \gamma) \end{pmatrix} \\ &= \begin{pmatrix} \cos^2 \gamma + \sin^2 \gamma & 0 \\ 0 & \sin^2 \gamma + \cos^2 \gamma \end{pmatrix} \\ &= \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix} \\ &= I \end{aligned} $$$$ \begin{aligned} T_{-\gamma}T_{\gamma} &= \begin{pmatrix} (\cos(-\gamma) \cos \gamma) - (\sin(-\gamma) \sin \gamma) & (-\sin(-\gamma) \cos \gamma) - (\cos(-\gamma) \sin \gamma) \\ (\cos(-\gamma) \sin \gamma) + (\sin(-\gamma) \cos \gamma) & (-\sin(-\gamma) \sin \gamma) + (\cos(-\gamma) \cos \gamma) \end{pmatrix} \\ &= \begin{pmatrix} (\cos \gamma \cos \gamma) + (\sin \gamma \sin \gamma) & (\sin \gamma \cos \gamma) - (\cos \gamma \sin \gamma) \\ (\cos \gamma \sin \gamma) - (\sin \gamma \cos \gamma) & (\sin \gamma \sin \gamma) + (\cos \gamma \cos \gamma) \end{pmatrix} \\ &= \begin{pmatrix} \cos^2 \gamma + \sin^2 \gamma & 0 \\ 0 & \sin^2 \gamma + \cos^2 \gamma \end{pmatrix} \\ &= \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix} \\ &= I \end{aligned} $$5.3
Multiply two rotation matrices $T_{\alpha}$ and $T_{\beta}$ (it is a rare case when the multiplication is commutative, i.e., $T_{\alpha}T_{\beta} = T_{\beta}T_{\alpha}$, so the order is not essential). Deduce formulas for $\sin(\alpha + \beta)$ and $\cos(\alpha + \beta)$ from here.
The matrices are:
$$ T_{\alpha} = \begin{pmatrix} \cos \alpha & -\sin \alpha \\ \sin \alpha & \cos \alpha \end{pmatrix}, \quad T_{\beta} = \begin{pmatrix} \cos \beta & -\sin \beta \\ \sin \beta & \cos \beta \end{pmatrix} $$The composition of the rotations around $\alpha$ and $\beta$ is $T = T_\alpha T_\beta = T_{\beta + \alpha}$.
$$ \begin{aligned} T_{\alpha}T_{\beta} &= \begin{pmatrix} (\cos \alpha \cos \beta) - (\sin \alpha \sin \beta) & -(\sin \alpha \cos \beta) - (\cos \alpha \sin \beta) \\ (\cos \alpha \sin \beta) + (\sin \alpha \cos \beta) & -(\sin \alpha \sin \beta) + (\cos \alpha \cos \beta) \end{pmatrix} \\ &= \begin{pmatrix} (\cos \alpha \cos \beta) - (\sin \alpha \sin \beta) & -(\sin \alpha \cos \beta + \cos \alpha \sin \beta) \\ (\cos \alpha \sin \beta) + (\sin \alpha \cos \beta) & -(\sin \alpha \sin \beta) + (\cos \alpha \cos \beta) \end{pmatrix} \\ &= \begin{pmatrix} \cos(\beta + \alpha) & -\sin(\beta + \alpha) \\ \sin(\beta + \alpha) & \cos(\beta + \alpha) \end{pmatrix} \end{aligned} $$From this we can deduce that $\cos(\alpha + \beta) = \cos \alpha\cos \beta - \sin \alpha \sin \beta$ and $\sin(\alpha + \beta) = \cos \alpha \sin \beta + \sin \alpha \cos \beta$.
5.4
Find the matrix of the orthogonal projection in $\mathbb{R}^2$ onto the line $x_1 = -2x_2$. Hint: What is the matrix of the projection onto the coordinate axis $x_1$?
First we will rotate the plane by an angle $-\gamma$, where $\cos \gamma = -2/\sqrt{5}$ and $\sin \gamma = 1/\sqrt{5}$. Then we will project to the axis $x_1$ and rotate the plane back by an angle $\gamma$.
$$ \begin{aligned} T &= \begin{pmatrix} -2/\sqrt{5} & -1/\sqrt{5} \\ 1/\sqrt{5} & -2/\sqrt{5} \end{pmatrix} \begin{pmatrix} 1 & 0 \\ 0 & 0 \end{pmatrix} \begin{pmatrix} -2/\sqrt{5} & 1/\sqrt{5} \\ -1/\sqrt{5} & -2/\sqrt{5} \end{pmatrix} \\ &= \frac{1}{5} \begin{pmatrix} -2 & -1 \\ 1 & -2 \end{pmatrix} \begin{pmatrix} 1 & 0 \\ 0 & 0 \end{pmatrix} \begin{pmatrix} -2 & 1 \\ -1 & -2 \end{pmatrix} \\ &= \frac{1}{5} \begin{pmatrix} -2 & -1 \\ 1 & -2 \end{pmatrix} \begin{pmatrix} -2 & 1 \\ 0 & 0 \end{pmatrix} \\ &= \frac{1}{5} \begin{pmatrix} 4 & -2 \\ -2 & 1 \end{pmatrix} \end{aligned} $$5.5
Find linear transformations $A, B : \mathbb{R}^2 \to \mathbb{R}^2$ such that $AB = \mathbf{0}$ but $BA \neq \mathbf{0}$.
$$ A = \begin{pmatrix} 0 & 1 \\ 0 & 0 \end{pmatrix}, \quad B = \begin{pmatrix} 1 & 0 \\ 0 & 0 \end{pmatrix} $$$$ AB = \begin{pmatrix} 0 & 0 \\ 0 & 0 \end{pmatrix}, \quad BA = \begin{pmatrix} 0 & 1 \\ 0 & 0 \end{pmatrix} $$
5.6
Prove Theorem 5.1, i.e., prove that $\text{trace}(AB) = \text{trace}(BA)$.
By definition $\text{trace}(AB) = \sum_k (AB){k,k} = \sum_k \sum_l a{k,l}b_{l,k}$. By swapping the sums $\sum_k \sum_l a_{k,l}b_{l,k} = \sum_l \sum_k b_{l,k}a_{k,l} = \sum_l (BA)_{l,l} = \text{trace}(BA)$.
From this follows that $\text{trace}(AB) = \text{trace}(BA)$.
5.7
Construct a non-zero matrix $A$ such that $A^2 = \mathbf{0}$.
$$ A = \begin{pmatrix} -1 & -1 \\ 1 & 1 \end{pmatrix} $$
5.8
Find the matrix of the reflection through the line $y = -2x/3$. Perform all the multiplications.
First we will rotate the plane by an angle $-\gamma$, where $\cos \gamma = -3/\sqrt{15}$ and $\sin \gamma = 2/\sqrt{15}$. Then we will reflect through the axis $x$ and rotate the plane back by an angle $\gamma$.
$$ \begin{aligned} T &= \begin{pmatrix} -3/\sqrt{15} & -2/\sqrt{15} \\ 2/\sqrt{15} & -3/\sqrt{15} \end{pmatrix} \begin{pmatrix} 1 & 0 \\ 0 & -1 \end{pmatrix} \begin{pmatrix} -3/\sqrt{15} & 2/\sqrt{15} \\ -2/\sqrt{15} & -3/\sqrt{15} \end{pmatrix} \\ &= \frac{1}{15} \begin{pmatrix} -3 & -2 \\ 2 & -3 \end{pmatrix} \begin{pmatrix} 1 & 0 \\ 0 & -1 \end{pmatrix} \begin{pmatrix} -3 & 2 \\ -2 & -3 \end{pmatrix} \\ &= \frac{1}{15} \begin{pmatrix} -3 & -2 \\ 2 & -3 \end{pmatrix} \begin{pmatrix} -3 & 2 \\ 2 & 3 \end{pmatrix} \\ &= \frac{1}{15} \begin{pmatrix} 5 & -12 \\ -12 & -5 \end{pmatrix} \end{aligned} $$