A New Textbook on the Math Behind Machine Learning

Three prominent researchers — Afonso S. Bandeira (ETH Zurich), Amit Singer (Princeton), and Thomas Strohmer (UC Davis) — have posted a new textbook on arXiv titled Mathematics of Data Science. It's a rare beast: a rigorous, self-contained treatment of the mathematical ideas that underpin modern machine learning and data analysis, written for graduate students and working engineers who want to go beyond black-box usage.

The book covers 16 chapters, starting with the geometry of high-dimensional spaces (the "curse and blessing of dimensionality") and moving through SVD/PCA, linear regression with regularization, graph theory and clustering, nonlinear dimension reduction (diffusion maps), random projections, optimization for data science, classification, and a mathematical introduction to deep learning. Later chapters tackle concentration of measure, matrix concentration inequalities, compressive sensing, and low-rank matrix recovery.

What sets this apart from typical ML textbooks is the depth of the mathematical treatment. The authors don't shy away from proofs and rigorous derivations, but they also connect each topic to practical data science workflows. The chapter on deep learning, for instance, builds up from basic neural network architectures to the theory of why overparameterized models generalize — a topic that remains poorly understood even by many practitioners.

For engineers working with high-dimensional data, the chapters on random projections and concentration inequalities are particularly valuable. These tools are directly applicable to understanding embeddings, vector search, and the behavior of large-scale ML systems. The section on matrix concentration inequalities alone could save a lot of head-scratching when debugging why a particular randomized algorithm works (or doesn't).

The book is available as a free PDF on arXiv, and the authors have released the TeX source as well — a nice touch for anyone who wants to remix or adapt parts for teaching. It's not a tutorial for beginners; you'll want comfort with linear algebra, probability, and real analysis. But for the target audience — engineers who need to understand the math behind the tools they use daily — this is a welcome addition to the literature.