Mathematics for Machine Learning
A collection of resources to learn and review mathematics for machine learning.
Mathematics For Machine Learning
The book is not intended to cover advanced machine learning techniques because there are already plenty of books doing this. Instead, we aim to provide the necessary mathematical skills to read those other books.
Book: https://github.com/mml-book/mml-book.github.io (opens in a new tab)
Algebra, Topology, Differential Calculus, and Optimization Theory For Computer Science and Machine Learning
by Jean Gallier and Jocelyn Quaintance
Includes mathematical concepts for machine learning and computer science.
Book: https://www.cis.upenn.edu/~jean/math-deep.pdf (opens in a new tab)
Applied Math and Machine Learning Basics
by Ian Goodfellow and Yoshua Bengio and Aaron Courville
This includes the math basics for deep learning from the Deep Learning (opens in a new tab) book.
Chapter: https://www.deeplearningbook.org/contents/part_basics.html (opens in a new tab)
Mathematics for Machine Learning
by Marc Peter Deisenroth, A. Aldo Faisal, and Cheng Soon Ong
This is probably the place you want to start. Start slowly and work on some examples. Pay close attention to the notation and get comfortable with it.
Book: https://mml-book.github.io (opens in a new tab)
Probabilistic Machine Learning: An Introduction
by Kevin Patrick Murphy
This book contains a comprehensive overview of classical machine learning methods and the principles explaining them.
Book: https://probml.github.io/pml-book/book1.html (opens in a new tab)
Mathematics for Deep Learning
by Brent Werness, Rachel Hu et al.
This reference contains some mathematical concepts to help build a better understanding of deep learning.
Chapter: https://d2l.ai/chapter_appendix-mathematics-for-deep-learning/index.html (opens in a new tab)
The Mathematical Engineering of Deep Learning
by Benoit Liquet, Sarat Moka and Yoni Nazarathy
This book provides a complete and concise overview of the mathematical engineering of deep learning. In addition to overviewing deep learning foundations, the treatment includes convolutional neural networks, recurrent neural networks, transformers, generative adversarial networks, reinforcement learning, and multiple tricks of the trade. The focus is on the basic mathematical description of deep learning models, algorithms and methods.
Book: https://deeplearningmath.org (opens in a new tab)
Bayes Rules! An Introduction to Applied Bayesian Modeling
by Alicia A. Johnson, Miles Q. Ott, Mine Dogucu
Great online book covering Bayesian approaches.
Book: https://www.bayesrulesbook.com/index.html (opens in a new tab)
📄 Papers
The Matrix Calculus You Need For Deep Learning
by Terence Parr & Jeremy Howard
In deep learning, you need to understand a bunch of fundamental matrix operations. If you want to dive deep into the math of matrix calculus this is your guide.
Paper: https://arxiv.org/abs/1802.01528 (opens in a new tab)
The Mathematics of AI
by Gitta Kutyniok
An article summarising the importance of mathematics in deep learning research and how it’s helping to advance the field.
Paper: https://arxiv.org/pdf/2203.08890.pdf (opens in a new tab)
🎥 Video Lectures
Multivariate Calculus by Imperial College London
by Dr. Sam Cooper & Dr. David Dye
Backpropagation is a key algorithm for training deep neural nets that rely on Calculus. Get familiar with concepts like chain rule, Jacobian, gradient descent.
Video Playlist: https://www.youtube.com/playlist?list=PLiiljHvN6z193BBzS0Ln8NnqQmzimTW23 (opens in a new tab)
Mathematics for Machine Learning - Linear Algebra
by Dr. Sam Cooper & Dr. David Dye
A great companion to the previous video lectures. Neural networks perform transformations on data and you need linear algebra to get better intuitions of how that is done.
Video Playlist: https://www.youtube.com/playlist?list=PLiiljHvN6z1_o1ztXTKWPrShrMrBLo5P3 (opens in a new tab)
CS229: Machine Learning
by Anand Avati
Lectures containing mathematical explanations to many concepts in machine learning.
Course: https://www.youtube.com/playlist?list=PLoROMvodv4rNH7qL6-efu_q2_bPuy0adh (opens in a new tab)
🧮 Math Basics
The Elements of Statistical Learning
by Jerome H. Friedman, Robert Tibshirani, and Trevor Hastie
Machine learning deals with data and in turn uncertainty which is what statistics aims to teach. Get comfortable with topics like estimators, statistical significance, etc.
Book: https://hastie.su.domains/ElemStatLearn/ (opens in a new tab)
If you are interested in an introduction to statistical learning, then you might want to check out "An Introduction to Statistical Learning" (opens in a new tab).
Probability Theory: The Logic of Science
by E. T. Jaynes
In machine learning, we are interested in building probabilistic models and thus you will come across concepts from probability theory like conditional probability and different probability distributions.
Source: https://bayes.wustl.edu/etj/prob/book.pdf (opens in a new tab)
Information Theory, Inference and Learning Algorithms
by David J. C. MacKay
When you are applying machine learning you are dealing with information processing which in essence relies on ideas from information theory such as entropy and KL Divergence,...
Book: https://www.inference.org.uk/itprnn/book.html (opens in a new tab)
Statistics and probability
by Khan Academy
A complete overview of statistics and probability required for machine learning.
Course: https://www.khanacademy.org/math/statistics-probability (opens in a new tab)
Linear Algebra Done Right
by Sheldon Axler
Slides and video lectures on the popular linear algebra book Linear Algebra Done Right.
Lecture and Slides: https://linear.axler.net/LADRvideos.html (opens in a new tab)
Linear Algebra
by Khan Academy
Vectors, matrices, operations on them, dot & cross product, matrix multiplication etc. is essential for the most basic understanding of ML maths.
Course: https://www.khanacademy.org/math/linear-algebra (opens in a new tab)
Calculus
by Khan Academy
Precalculus, Differential Calculus, Integral Calculus, Multivariate Calculus
Course: https://www.khanacademy.org/math/calculus-home (opens in a new tab)
This collection is far from exhaustive but it should provide a good foundation to start learning some of the mathematical concepts used in machine learning. Reach out on Twitter (opens in a new tab) if you have any questions.
Reference
https://github.com/dair-ai/Mathematics-for-ML (opens in a new tab)