|
Linear Algebra
Resource Material
by Arun Ram
|
2026 |
This page curates material for teaching and learning linear algebra,
including thoughts for the teacher, for the student, problem lists,
definitions, statements of results, proofs of results, worked example solutions.
If you are teaching this material and would like tex files for pdf files
that are on this page, please contact me by email.
Some of my thoughts about teaching have been written down in the Lecture script
Teaching Math in the Next Life
Here is a guide to Proof Machine with examples.
Proof Machine
Main Topics
- Matrices and operations
- Invertible matrices, kernels and images and solving linear equations
- Factoring Matrices (i.e. row reduction)
- Eigenvalues and eigenvectors
- Vector spaces, linear transformations, bases, kernels and images
- Inner products and orthogonality
A focused mathematically rigourous account of these topics (with proofs)
is found in the following notes.
Mathematically tight row reduction and Solving Linear Systems notes
The factoring algorithm in these notes is supported by the following
Sage code which computes factorizations via this specific row reduction algorithm.
Sage Jupyter notebook for factoring matrices with the factorization algorithm
You may need to right-click to download this file
(filename: SageMatrixFactorizationDemo260831.ipynb).
If you need to learn how to run this notebook on your computer ask AI.
Here is a pdf file version (
pdf file).
Assessment
The exam is 12 randomly chosen questions from the following list
(pdf file).
Assignments:
Lectures and Lecture slides
The lecture by lecture schedule is as follows. The following is
lecture by lecture exposition in Lecture note format.
Arun Ram's Lecture notes for MAST10007
Here is the same material in a document camera prepared slide format.
Arun Ram's Slide deck for MAST10007
- Lecture 1: Column vectors ℝn
- Lecture 2: Linear combinations, lines and planes
- Lecture 3: Cross products
- Lecture 4: Matrices and operations
- Lecture 5: Finding inverse
- Lecture 6: Factoring and the rank theorem
- Lecture 7: The factoring algorithm
- Lecture 8: Solutions of linear systems
- Lecture 9: Kernels and Images
- Lecture 10: Computing kernels and images of matrices
- Lecture 11: Eigenvalues and eigenvectors
- Lecture 12: Symmetric, Hermitian, unitary and orthogonal matrices
- Lecture 13: Singular value decomposition
- Lecture 14: Traces and determinants
- Lecture 15: Applications to graphs and networks
- Lecture 16: Application of diagonalization to dynamics
- Lecture 17: Vector spaces and linear transformations
- Lecture 18: Linear transformations
- Lecture 19: span, linear independence and bases
- Lecture 20: kernel and image of a linear transformation
- Lecture 21: With respect to a basis
- Lecture 22: Picturing linear transformations
- Lecture 23: Inner product spaces
- Lecture 24: Gram matrices, orthogonality and projections
- Lecture 25: Projections and orthgonalisation
- Lecture 26: Learning to do proofs -- Orthogonality and linear independence
- Lecture 27: Learning to do proofs -- Linear transformations and subspaces
- Lecture 28: Learning to do proofs -- The minimax basis theorem
- Lecture 29: Learning to do proofs -- Invertible mastrices are square
- Lecture 30: Applications to data analysis
- Lecture 31: Review -- Subspace examples
- Lecture 32: Review -- Subspace examples
- Lecture 33: Review -- Linear independence examples
- Lecture 34: Review -- Basis examples
Tutorials
- Week 1: 3d-space-time and cross products
(pdf file)
- Week 2: Generator matrices and row and column operations
(pdf file)
- Week 3: Inverse and normal form for generic 2x2 and 3x3 matrices
(pdf file)
- Week 4: Solving problems with an unknown parameter
(pdf file)
- Week 5: Finding bases -- the row space, column space and null space
(pdf file)
- Week 6: Fibonacci numbers and diagonalisation
(pdf file)
- Week 7: Diagonalization and Jordan form
(pdf file)
- Week 8: Gram-Schmidt and orthogonal polynomials
(pdf file)
- Week 9: Conic sections
(pdf file)
- Week 10: Volumes of parallelipipeds
(pdf file)
- Week 11: Comparing formulas for the determinant
(pdf file)
- Week 12: A past exam
(pdf file)
Vocabulary lists and software
- Vocabulary list: Linear algebra
(pdf file)
- Vocabulary list: Sets, functions, number systems
(pdf file)
Notes written by Arun Ram
Matrices and operations
Normal forms
Solving systems of linear equations
Matrix groups
Flag varieties
Determinants
Eigenvalues, eigenvectors and diagonalization
Vector Geometry
Vector spaces and Linear transformations
𝔽-modules with bilinear form
Introduction
- Chapter 1: Matrices and matrix operations
- The first point that is often glossed over is, what is the definition
of a matrix? Of course, the student should think of a matrix as a table
of numbers and, when I teach I usually begin with this statement, and
give a couple of examples, one square matrix, and one not square matrix.
But then, very importantly, the precise definition of an mxn matrix is
that it is a function from {1,...m}x{1,...,n} to 𝔽. The
entries of the matrix are the values A(i,j) of this function.
- At this point, one realizes, if one is actually teaching, that
m and n are bad in the classroom, because it is difficult for the
students to tell the difference when listening. So it is desirable to
use different letters in place of m and n. I choose t and s, which are,
secretly a premonition of the fact that a matrix encodes a linear transformation
and the dimension of the source of the linear transformation is s and the
dimension of the target of the linear transformation is t.
- Thus, the careful definition of a t by s matrix A is that it is a
function A:{1,...,t}x{1,...,s} -> 𝔽
- Next one must introduce matrix addition and scalar multiplication
by 𝕗 and matrix multiplication and
claim that they make the set of txs matrices into an 𝔽-modules and
that they make the set of sxs matrices into a 𝔽-algebra.
In other words, one must at least claim, if not prove, that the
associative and distributive laws hold for matrix addition and
scalar multiplication and matrix multiplication.
- I find the matrix units Eij to be indispensible tools.
I introduce them at this point and at least claim that every matrix can
be written
as a unique combination of the Eij using scalar multiplication and
addition (i.e. that the Eij form a basis of Mtxs(𝔽)).
- The multiplication law for the Eij is so wonderfully elegant
that I am compelled to state it and do a couple of examples. This also
gives me a chance to formally introduce the Kronecker delta, another tool
that is indispensible for the working mathematician.
- At this point one might cover transpose that the fact that transpose
is a linear transformation and an involution and
(AB)T=BTAT.
One must be careful not to muddy the waters by wanting to talk about the beautiful stuctures of bilinear forms, dual vector spaces, adjoint linear transformations. Such a torrent of structures, in spite of their beauty, is too much
for the student and teacher who are just learning to process the structures
of matrix addition, scalar multiplication and matrix multiplication.
Sometimes I find it provides better focus just to put these facts as
questions on the homework assignment. The definition of transpose is
short enough (i.e. AT(i,j) = A(j,i)) that it can be covered
in too minutes when it is need, later in the course, when it appears naturally.
- One of the difficulties of transpose is notation: should one use
At (which conflicts with the t in txs matrices and
the t in problems where t is the natural variable for time) or
should one use AT (which conflicts with T for
linear transformations)? Neither feels exactly right.
Since the transpose is really the matrix of the adjoint linear transformation,
a better notation would be A*, but this raises challenges of
the difference between symmetric and Hermitian forms and gets us into a
morass of decisions about whether we should be using real or
complex numbers at any given point -- an issue which, in reality, is
not (and should not be) part of the structure of linear algebra,
since linear algebra (including bilinear forms) works over any field.
I still don't feel that I have found the right notation for transpose, and
I have noticed that one actually need to use it rather infrequently;
infrequently enough that it is not difficult and is just fine to
introduce and specify/clarify the notation in each subsection, or
exercise that uses transpose.
- Chapter 2: Orbit decompositions and normal form
- The first goal here is to introduce the existence of an oracle -- an oracle
that produces a factorization of a matrix A as P1rQ, where
P and Q are invertible. It would be nice to have the computer be the oracle
and to check, in class, that the oracle isn't lying.
- Of course, the first job, before introducing the oracle, is to
define invertible matrices and GLn.
Finding the inverses of the elementary
matrices (secretly the generators of the group GLn)
is the ideal way to drive in what an invertible matrices are.
- Though the P and the Q are not unique, the r is. The number r
is the rank of A. It would be cute to design the software so that it
sometimes spits out a different choice for P and Q (and one could talk about
how to say what all possible choices of P and Q might be).
- Then one can talk about making the R=1rQ unique (i.e.
reduced row echelon form). Actually, 'reduced row echelon form' is a
terrible and archaic terminology, it would be much better to say
'left GL representative'.
- The main point of all this is that there are orbit decompositions
of Mtxs(𝔽) with respect to the action of
GLtxGLs,
and with respect to the action of GLt and we know exactly
what the preferred representatives of these orbits are.
- At this point, the question is vivid: "How does the oracle produce
the P, the Q and the r?". The answer is row reduction and normal form.
It is healthy to view this as the task of factoring a matrix, and
to do this factorization step by step. In a manner analogous to the
way that one factors a positive integers by first dividing out all the factors of 2, and then dividing out all the factors of 3, and then dividing out
all the factors of 5, ..., the matrix can be factored by first
dividing out the row reducers matrices (in order) and then
dividing out the diagonal generators (in order) and then
dividing out 1r and then dividing out
the root generators (in order).
Perhaps a good way to teach this algorithm is to write the code
for it in class.
- Structurally, the normal form algorithm is fundamentally important.
By setting it up in the right order
(first row reducers, then diagonal generators, then
1r, then root generators) one has established a number
of extremely powerful structural results:
(a) the Bruhat decomposition of GLn,
(b) the affine coordinatization of the Bruhat cells,
(c) that GLn is generated by row reducers, diagonal generators and
root generators (every invertible matrix can be factored as a product
of elementary matrices)
- At this point, one is only step away from esablishing a presentation
of GLn by generators and relations. The hard part has been
done by the row reduction algorithm.
- Chapter 3: Kernels and images
- With the result that every matrix A can be factored as A=P1rQ
in hand, it is an easy matter to determine the kernel and the
image of A in terms of the matrices P and Q. This gives a wonderful
introduction to subspaces and bases by executing it on examples.
There is no need to introduce abstract notions of vector spaces
and subspaces and bases at this point, one can simply make the definitions
for what is needed right at this juncture, and keep the focus
on finding ker(A) and im(A).
- Of course the first step is to make the definitions: the
definition of ker(A), the definition of im(A), the definition
of 𝔽n, and the definition of subspace of &Fn.
The definition of a basis of a subspace can simply be: a collection
of vectors such that every vector in the subspace has a
unique expression in terms of the elements of B by using addition and
scalar mutliplication. This is not so surprising, because in week
one we have already pointed out that the matrix units Eij
are such that every matrix A in Mtxs has a unqie expression
in terms of the Eij by using addition and scalar multiplication.
- The next step is to prove that
ker(P1rQ) = Q-1ker(1r) and
im(P1rQ)=Pim(1r).
This is easy, and the corollaries (a basis for ker(A) and a basis for
im(A) and the rank-nullity theorem) are striking.
- Chapter 4: Application 1: Solving systems of linear equations
and
Application 2: Eigenvectors and diagonalization
- At this point it is a short route to an explicit formula
for solutions of linear systems. The steps are: (a) note that
a linear system can be written as Ax=b, (b) note that
if Ap=b then Sol(Ax=b) = p+ker(A), (c) note that if
Ax=b has a solution and A = P1rQ then p=Q-1b
is a solution to Ax=b. When a solution exists then
Sol(Ax=b) = Q-1b+ker(A), and this is the explicit formula
for solutions of a linear system.
- Since the equation ker(A)=ker(P1r Q)
=Q-1ker(1r)$ spits out a basis of ker(A)
(anmely, the first r columns of Q-1).
So we know how to find a basis of ker(A-λ), which is the
set of eigenvectors of A of eigenvalue λ.
- If it happens that the union of the bases of ker(A-λ) form
a basis 𝔽n (i.e. there are n linearly independent vectors)
then PAP-1 is diagonal, where P is the matrix that has the n
linearly independent eigenvectors as its columns.
- Chapter 5: Matrix groups
- This chapter provides presentations by generators and relations
of the following groups:
- The symmetric group of permutation matrices
(generated by special row reducers)
- The invertible diagonal matrices (generated by diagonal generators)
- The invertible upper triangular matrices (generated by
diagonal matrices and root generators)
- The group GLn of invertibel matrices
- The group On of orthogonal matrices
- The group Sp2n of symplectic matrices
- The group Un of unitary matrices
In each case the goal is to specify a row reduction algorithm that
gives a normal form for the elements of that group.
- Chapter 6: Determinants
- The determinant is determined by saying that det(AB)=det(A)det(B)
and that its value on the row reducers is -1, its value on the root generators
is 1, and its value on the diagonal generator di(c)=c.
It follows that det(A)=det(P)det(1r)det(Q) is 0 if and only if
r≠n (and we already know that r≠n if and only if ker(A)≠0).
- A direct consequence of this definition of the determinant
is that the determinant
of a permutation matrix is 1 or -1. The proof of the permutation formula
for the determinant
is by showing that the permutation formula satisfies the defining conditions
of of det.
- The proof of Laplace expansion (along a single row or a signle column)
is then a decomposition of the symmetric group Sn into
cosets with respect to the subgroup Sn-1.
- The other favorite uses of determinants are to compute
inverses by determinants (adjugate matrices) and to solve
systems of linear equations by determinants (Cramer's rule).
- The factorization of the Vandermonde determinant is one of the most
useful results to have in one's toolkit.
- It is possible to give a fairly easy computational proof of
the Cayley-Hamilton theorem theorem that is valud over any commutative ring.
- With the advent of the exterior algebra
(sometimes called the Grassmann algebra) and its relation
to minors of a matrix, treating determinants in the above way does
feel rather clumsy and outdated. It might be more sensible option to treat
determinants in a later course more focused on results like the
Cauchy-Binet theorem, invariant factors, conjugacy invariants,
Smith normal form, Jordan normal forms and the like.
Computationally, determinants are more efficient than row reduction
only when order n! is smaller than than order n2, so if
n is greater than 3 or 4 it is much more efficient to use row reduction
than to be computing determinants to solve problems that amount to
solving linear systems.
- Chapter 7: Vector spaces and linear transformations
- The definition of an 𝔽-module (or 𝔽-vector space)
as a set with addition and scalar multiplication following the study
of the favorite example Mtxs(𝔽). A linear transformation
is just a function that respects addition and scalar multiplication.
- The definition of subspaces and bases is hardly shocking
given that the special cases of ker(A) and im(A) have been dealt
with on the way to solving linear systems. The favorite results
showing that span(B) is a subspace, that linear independence and
spannning are equivalent to every vector having a unique expression in
the elements of B by addition and scalar multiplication
that ker(T)=0 is equivalent to injectivity, that im(A)=W is equivalent
to surjectivty,
are so simple, and beautifully striking, that this section inroducing
these tools is always a pleasure to present.
- The result that choosing bases allows for
a linear transformation to be coded as a matrix (and that this process
of converting linear transformations to matrices converts composition
of linear transformations to matrix multiplication) is so
'obvious' on examples, that student rarely have difficulties with this
part of the game if the focus is placed on illustrative
stated examples where the language in the statement of the example
is carefully presented to accurately define
the linear transformation as a function with a precisely specifed source and
target.
- The fact that dimension (the number of elements in a basis)
is well defined is probably the deepest result in the whole course.
A good proof uses a minimax characterization of a basis,
and an exchange lemma. This result
is essentially equivalent to the statement that invertible
matrices are square and that the left inverse is equal to the right inverse.
- Chapter 8: Bilinear forms, orthogonality and Gram-Schmidt
- Doing bilinear forms over ℝ or ℂ is a major cheat and
hampers the progression of a mathematics student. Nondegenerate
bilinear forms over an arbitrary field 𝔽 are ubiquitous and
extremely powerful tools. The "positive definiteness" in a
Hermitian form is an axiom which acheives nondegeneracy.
With a nondegernate form, the constructs of orthogonality and
projection all go through and so it is wiser to work with
nondegenerate forms and use positive definiteness as a step
in checking nondegeneracy in the cases that positive definitenss applies.
- Choosing a basis converts a bilinear form to a matrix, the corresponding
matrix is the Gram matrix of the bilinear form. The Gram matrix is
invertible if and only if the bilinear form is nondegenerate
and the bilinear form is nondegenerate if and only if dual bases with respect
to the bilinear form exist.
- The bilinear form is a tool for constructing orthogonal complements
to subspaces. The projections onto subspaces have a nice formula
in terms of dual bases.
- The analog of row reduction for bilinear forms (Gram matrices)
is the Gram-Schmidt algorithm. It provides P such that PAP^t is diagonal.
Gram-Schmidt actually even works (and is immensely useful)
over a general commutative ring. Many modern applications,
such as Topological data analysis and integer programming
and Hodge theory, heavily rely on this fact.
A good proof of the Cauchy-Schwarz inequality
|<x,y>|≤|x|.|y|
is to apply the Gram-Schmidt process to {x,y}.