Prerequisites - Linear Algebra

Before the workshop: this deck checks the linear algebra that Tensors for Machine Learning builds on: vectors and matrices, products and transposes, inverses and linear independence, norms, special matrices, eigenvalues, the SVD, the pseudoinverse, the trace, the determinant and PCA.

It follows Chapter 2 of Deep Learning by Ian Goodfellow, Yoshua Bengio and Aaron Courville, free to read at deeplearningbook.org. Read the chapter there and come here to check what stuck: every card cites the page its answer comes from.

The same questions make up the workshop’s intake test, so your instructor can see where the group starts.

23 topics

Your path through this unit

Step 1 of 3

Initial review

Take a first look at each topic and tick Got it once you have the general idea.

Scalars, vectors, matrices and tensors

The transpose

Adding matrices, and broadcasting

The matrix product

The dot product

Properties of matrix multiplication

The identity matrix and the inverse

Solving Ax = b by computing A⁻¹

Linear combinations, span and column space

Linear independence and singular matrices

Vector norms: L¹, L², max

The “L⁰ norm” is not a norm

The Frobenius norm

Diagonal and symmetric matrices

Orthogonal vectors and orthogonal matrices

Eigenvectors and eigenvalues

Eigendecomposition of a real symmetric matrix

Positive definite and semidefinite matrices

Singular value decomposition (SVD)

The Moore-Penrose pseudoinverse

The trace

The determinant

Principal components analysis (PCA)

Companion questions for Chapter 2 of Deep Learning by Ian Goodfellow, Yoshua Bengio and Aaron Courville (MIT Press, 2016), deeplearningbook.org. Summaries and questions are original; quoted passages are cited to the book.