Machine Learning and Physics

Machine learning is becoming a transformational force in our society, and will profoundly impact humanity in ways both good and bad. On the good side, recent scientific breakthroughs such as having solved the protein folding problem will dramatically accelerate the development of new drugs and vaccines; on the bad side, we are facing a future in which deep fakes are used for manipulation, autonomous weapons roam our skies and machine learning supports surveillance in totalitarian states.

On the scientific side, machine learning is making its weight felt across all disciplines. Some go so far as to postulate a fifth paradigm of scientific discovery, fueled by machine learning.

In view of the role that machine learning is starting to play also in physics, the faculty of Physics and Astronomy has decided to establish a new MSc core course: Machine Learning and Physics.

Contents

1. If, after browsing the FAQ below, you believe this course is for you: Then please register here.
2. The course starts with a python refresher in the tutorial on Oct 17th or 18th (identical content). Unless you are familiar with python and it's basic scientific stack (jupyter, numpy, matplotlib, scipy), please take part to help you solve the computational exercises.
3. The main lectures are on Tuesdays and Thursdays from 09h15-11h00 in Großer Hörsaal, Philosophenweg 12.
4. Identical plenary tutorials are offered on Mondays and Tuesdays, from 16h15-18h00. Pick whichever day suits you best. Tutorials are held in KIP, INF 227 HS2.

FAQ

• Q: Do I need prior knowledge in machine learning?

• A: No.

• Q: I just want to learn the basics. Is this the right course?

• A: This course will have a steep learning curve; if you only want to cover the basics, you probably find easier alternatives.

• Q: Is this course about deep learning?

• A: Neural networks will play an important role; but this course is more about principles. For sure we will not discuss details of the latest architectures.

• Q: Will this course be repeated next year?

• A: Yes, like every MSc core course. In winter semester 2023, the course will be given by Jan Pawlowski and Tilman Plehn.

• Q: I do not find the course listed as MSc core course in the MSc module handbook. Why?

• A: Because the version on the departmental's website has not been updated yet; this will happen before start of the teaching term.

• Q: Is there a text book?

• A: The book with the biggest overlap is the soon-to-be-published Murphy, Probabilistic Machine Learning: Advanced Topics. Full pdf available here.

• Q: Exam modalities?
• A: To be admitted to the written exam at the end of the semester, you need to gain 50% of the points in the exercise sheets.

Preliminary curriculum

1. Introduction & linear dimension reduction
2. Nonlinear dimension reduction: connection to stat. mechanics
3. Nonparametric density estimation
4. Basic clustering techniques, review of information theory
5. Classification, take 1: discriminative
6. Review: Multivariate distributions, Bayes theorem, conjugate priors
7. Classification, take 2: parametric / generative
8. Regression
9. Regularization, linear differential operators
10. Gaussian processes
11. Classification, take 3: logistic regression, generalized linear models
12. Multi-layer perceptrons
13. Multi-layer perceptrons: capacity
14. Deep neural networks
15. Architectures
16. Directed Probabilistic Graphical Models
17. Hidden Markov Models
18. Kalman filter
19. Markov decision processes, Reinforcement learning
20. Gaussian Mixture Model (GMM), variational methods, mean field
21. Variational auto-encoders
22. Markov Chain Monte Carlo, Hamiltonian Monte Carlo
23. Geometric Machine Learning: symmetries, groups, graph neural networks
24. Attention / transformers
25. Diffusion models, normalizing flow
26. Nonlinear association measures
27. Optimal transport
28. Graph partitioning and network analysis
29. Ethics of ML
30. Q&A