M 375T — Mathematical Foundations of Machine Learning — Fall 2026

  • M 375T, Unique Number 59335 — Tuesday & Thursday, 3:30–5:00 PM, PMA 5.122
  • Instructor: Matias G. Delgadino — matias.delgadino@utexas.edu
  • The complete syllabus (grading, office hours, university policies) is on the course Canvas site.

References

  • Calder, J., & Olver, P. J. (2025). Linear Algebra, Data Science, and Machine Learning. Springer. (Primary reference, cited as C&O below)
  • Gordan Zitkovic. Lecture notes for “Introduction to Stochastic Processes”. Zitkovic lecture notes
  • Cover, T. M., & Thomas, J. A. (2006). Elements of Information Theory (2nd ed.). Wiley. (Entropy and relative entropy: Ch. 2; Sanov's theorem via the method of types: §11.1–11.4. Cited as C&T below)
  • MacKay, D. J. C. (2003). Information Theory, Inference, and Learning Algorithms. Cambridge University Press. Free PDF from the author's site
  • Shalev-Shwartz, S., & Ben-David, S. (2014). Understanding Machine Learning: From Theory to Algorithms. Cambridge University Press.
  • Strang, G. (2016). Introduction to Linear Algebra. Wellesley-Cambridge Press.

Course Schedule

Course schedule: dates, lecture topics, lecture notes, quizzes and practice notebooks, and suggested reading
Date Lecture Topic Lecture Notes Quiz / Practice Suggested Reading
Tue, Aug 25 Introduction. Data as samples from a distribution. Python/NumPy basics Notes Practice 1 (Python) Syllabus
Thu, Aug 27 Sampling from distributions: uniform, Gaussian; histograms vs. densities; empirical distribution Notes Quiz 0 (ungraded) · solutions C&O §7.1
Tue, Sep 1 Multivariate Gaussians and mixtures, operationally: sampling, mean, covariance Notes Practice 2 (NumPy + sampling) C&O §7.1
Thu, Sep 3 Matrices that stretch Gaussians; is the best fit a good model? Mixtures of Gaussians by gradient descent Notes Quiz 1 · solutions C&O §7.1
Tue, Sep 8 Maximum likelihood; the two-bin test by hand; sizing the experiment; Sanov's theorem exactly Notes Practice 3 (Gaussian fit + Sanov) C&T §11.1–11.4
Thu, Sep 10 Relative entropy for densities; Gibbs’ inequality and maximum likelihood as D-minimization; generative models: mixtures, pushforwards, GANs, diffusion Notes Quiz 2 · solutions C&T Ch. 2; MacKay Ch. 2
Tue, Sep 15 From samples to graphs: similarity graphs, random walks, the Markov property, and probability distributions on a graph Notes Practice 4 (model testing) C&O §9.1, §9.1.2, §9.6 (pp. 410–412); Zitkovic Ch. 5
Thu, Sep 17 The linear algebra of a random walk: the law moves by matrix multiplication; the lazy walk; invariant measure; eigenvalues and the spectral theorem; the graph Laplacian, connected components, and spectral clustering Notes Quiz 3 · solutions C&O §§5.1–5.3, §9.3, §9.4, §9.6; Zitkovic Ch. 5, 8
Tue, Sep 22 Two clusters: why the sign of one Laplacian eigenvector cuts a graph; cuts, the normalized cut and its relaxation; a weak bridge by hand; three species and the limits of a sign; k-means Notes Practice 5 (graphs and the Markov property) C&O §9.3, §9.4, §9.7
Thu, Sep 24 Reset: from a graph to a vector worth looking at. All the notation on one graph; connected components are not clusters; the spectral theorem, Rayleigh quotient and power method; the identity D−1L = I − P; the power method on the lazy walk and the plateau as the second eigenvector Notes Quiz 4 · solutions C&O §5.3, §9.1, §9.3, §9.6; von Luxburg §§1–3
Tue, Sep 29 Eigenvalues without symmetry: diagonalizable matrices and powers; directed graphs and Jordan blocks; the cut problem, its relaxation, and why the answer is an eigenvector; convergence of the lazy walk and relative entropy Notes Midterm 1 practice C&O §5.1, §9.4; Axler Ch. 8
Thu, Oct 1 Entropy decay along random walks; eigenvalues set the rate; Google's PageRank; review Notes Quiz 5 (optional) · solutions Zitkovic Ch. 8; C&O §9.6.1
Tue, Oct 6 Midterm 1: random walks, linear systems, eigenvector fixed points Midterm 1 · solutions —
Thu, Oct 8 Coordinates for data and distributions: vector spaces, bases, orthogonality Practice 6 (linear algebra) C&O Ch. 1–2
Tue, Oct 13 Projections and best approximation of data by a subspace; Gram–Schmidt, QR — C&O §2.4–2.5, §4.7
Thu, Oct 15 Spectral theorem: covariance matrices of data; reversible chains and reversing a random walk Quiz 6 C&O §5.3
Tue, Oct 20 Rayleigh quotients: max-variance directions in data; Perron–Frobenius, spectral gap, convergence rate — C&O §5.4–5.6
Thu, Oct 22 SVD of the data matrix; low-rank approximation Quiz 7 C&O §5.7
Thu, Oct 29 Lab: k-means clustering hands-on — C&O §7.5
Tue, Nov 3 Spectral clustering via the graph Laplacian Practice 7 (SVD/eigen) C&O §9.4, §9.7.2
Thu, Nov 5 PCA via SVD; the covariance matrix Quiz 8 C&O §8.1
Tue, Nov 10 PCA compression; best approximating subspace Practice 8 (PCA) C&O §8.2–8.3
Thu, Nov 12 The multivariate Gaussian, formally: affine image of N(0, I); sampling via Σ1/2 Quiz 9; Midterm 2 practice C&O §7.1
Tue, Nov 17 Mixtures of Gaussians: density, latent variables; GMM as soft k-means Practice 9 (Gaussian sampling) —
Thu, Nov 19 Midterm 2: eigen-theory, spectral theorem, SVD, spectral clustering, PCA — —
Nov 24 & 26 Fall break / Thanksgiving — no classes — —
Tue, Dec 1 Generative models: forward diffusion as a noising Markov chain; reversing the chain; closed-form scores for Gaussian mixtures Final practice Lecture notes
Thu, Dec 3 Review for final — —
Sat, Dec 12 Final Exam, 1:00–3:00 PM (room TBD): cumulative; emphasis on spectral methods, PCA, Gaussians, generative models — —