Professor for Image Analysis and Learning Heidelberg Collaboratory for Image Processing (HCI) Interdisciplinary Center for Scientific Computing (IWR) and Department of Physics and Astronomy Heidelberg University Im Neuenheimer Feld 205, 04.312; D-69120 Heidelberg Tel: +49 6221 5414803; Lab Manager Barbara Werner: +49 6221 5414833 Scientific matters: fred.hamprecht@iwr.uni-heidelberg.de Dean of Students affairs: studiendekan@physik.uni-heidelberg.de |

**Hiring:**

- Geometric machine learning in quantum chemistry: learning kinetic energy density functionals. If successful, this project will enable a breakthrough in the computational cost of quantum chemical calculations. A solid understanding of quantum mechanics and machine learning are required. We work on this exciting project together with the labs of Andreas Dreuw and Pascal Friederich in the simplaix consortium.
- Transformer-based cell tracking in virology. The aim is to develop software that biologists will really be able to use. The project entails a large software component, and the ideal candidate brings experience in full stack software development, a deep understanding of machine learning, and a passion to help address some of humanity's greatest challenges by working closely with virologists from SFB 1129. For our prior software work in the field of bioimage analysis, check out ilastik and plantseg.

Hi!

I develop machine learning algorithms for image analysis. More specifically, I am interested in principled methods that ingest graphs or images or video, and output "structured" predictions, such as the partitioning of an image into its constituent parts, or the tracking of all objects in a video. Most of our applications contribute to a better understanding of the physics of life; for instance, we try and segment all cells in a brain or a plant, or track virus components before assembly.

My focus is on methods that have a sound mathematical background, such as combinatorial optimization or algebraic graph theory, while being widely applicable and useful in practice. If this is what you like, please do apply! I attach particular importance to methods that can be trained with few examples, or by weak supervision, to empower colleagues from other disciplines to analyze their own data, conveniently. To the same end, I have launched the ilastik program for interactive machine learning (paper) that is now headed by Anna Kreshuk.

I greatly profit from the interaction with my colleagues in the STRUCTURES cluster of excellence and in ELLIS Life Heidelberg.

I enjoy, and feel most privileged, to be able to work on things unknown, and to teach the next generation of scientists and engineers.

I am proud that several former PhD students continue to serve in research and education, including

- Anna Kreshuk, Group Leader at EMBL
- Bernhard Renard, Professor at HPI, Potsdam Universty
- Bjoern Andres, Professor at TU Dresden
- Bjoern Menze, Professor at Zurich University
- Melih Kandemir , Assoc. Professor at University of Southern Denmark

*and*nice.

## News

05.2022 Philipp Nazari and Sebastian Damrich have an upcoming paper at ICML, showing how to analyze graphically what your auto-encoder is really doing. Congratulations to Philipp -- not so many BSc students end up as first authors on an ICML paper!

07.2022 Congratulations to former intern Quentin Garrido (now doing PhD with Yann LeCun and Laurent Najman)! He has won the Outstanding Student Paper award at ISMB 2022 for his work on "Visualizing hierarchies in scRNA-seq data using a density tree-biased autoencoder".

12.2021 Dimension reduction method UMAP still has not been published "properly" -- but Sebastian's Damrich lucent analysis of it is! Now out at NeurIPS (along with Enric Fita's work on Gibbs distributions over arborescences in directed graphs, a.k.a. "Directed Probabilistic Watershed").

07.2021 First time that a BSc student from the lab is first author on an ICCV paper: Erik Jenner studies Karger-type algorithms for the s-t mincut problem: blog

06.2020 Lab members have made crucial contributions to analysis of the "Kinderstudie" that has led to reopening of schools in the state of Baden-Wuerttemberg: press release.

10.2019 After a decennial effort, ilastik has become a potent tool for (bio-) image analysis. Development, including a "deep" version, is now headed by Anna Kreshuk. A new Perspective in Nature Methods summarizes the status quo.

09.2019 We have held the top spots in the ISBI and CREMI connectomics segmentation challenges almost continuously for three years. Kudos to Constantin Pape, Steffen Wolf, Thorsten Beier, Nasim Rahaman & co!

08.2019 Back from a fantastic sabbatical at Uppsala University. Thanks to Carolina Wahlby for hosting me!

09.2018 Thomas Hehn and I have won a best paper award in GCPR 2018 for our work on "Differentiable Decision Trees". Update: An extended version is now published by IJCV.

06.2018 Ullrich Koethe has been promoted to Adjunct Professor. Congratulations!

Older news## Publications

List of publications by me (Google Scholar) or the entire lab (hosted locally).Selected publications:

**The Mutex Watershed and its Objective: Efficient, Parameter-Free Graph Partitioning**Wolf, S., Bailoni, A., Pape, C., Rahaman, N., Kreshuk, A., Köthe, U. and Hamprecht, F. A.

*IEEE PAMI*(2020)[10.1109/TPAMI.2020.2980827]**Multicut brings automated neurite segmentation closer to human performance.**Beier, T, Pape, C, Rahaman, N, Prange, T, Berg, S, Bock, D, Cardona, A, Knott, G W, Plaza, S M, Scheffer, L K, Köthe, U, Kreshuk, A and Hamprecht, F A.*Nature Methods*(2017) [10.1038/nmeth.4151]**A Comparative Study of Modern Inference Techniques for Structured Discrete Energy Minimization Problems**J. H. Kappes, B. Andres, F. A. Hamprecht, C. Schnörr, S. Nowozin, D. Batra, S. Kim, B. X. Kausler, T. Kröger, J. Lellmann, N. Komodakis, B. Savchynskyy, C. Rother.*IJCV 2015*[10.1007/s11263-015-0809-x | Technical Report | BibTeX]**Graphical Model for Joint Segmentation and Tracking of Multiple Dividing Cell**M. Schiegg, P. Hanslovsky, C. Haubold, U. Köthe, L. Hufnagel, F. A. Hamprecht.*Bioinformatics 2015*[10.1093/bioinformatics/btu764 | Technical Report | URL | BibTeX]**Globally Optimal Closed-Surface Segmentation for Connectomics**B. Andres, T. Kröger, K. L. Briggmann, W. Denk, N. Norogod, G. Knott, U. Köthe, F. A. Hamprecht*ECCV 2012*[10.1007/978-3-642-33712-3_56 | Technical Report | BibTeX]**Optimal lattices for sampling**H. R. Künsch, E. Agrell, F. A. Hamprecht*IEEE Transactions on Information Theory, (2005)*[10.1109/TIT.2004.840864]**Development and assessment of new exchange-correlation functionals**F. A. Hamprecht, A. J. Cohen, D. J. Tozer, N. C. Handy*Journal of Chemical Physics, (1998)*[10.1063/1.477267]

## Some video lectures

### Computer Vision: Foundations (Summer Term 2020)

- 1.1 Scope of the Lecture | Introduction
- 1.2 Linear filters | Convolution
- 1.3 Interactive Semantic Segmentation with ilastik
- 2.1 Human Vision | Imaging
- 2.2 Downsampling an Image
- 2.3 Upsampling | Image Interpolation
- 3.1 Shallow vs Deep Learning
- 3.2 Training of a Neural Network (Introduction)
- 4.1 Convolutional Neural Networks | Image Classification
- 4.2 Training of a Neural Network | Optimization
- 4.3 U-net architecture | Semantic Segmentation
- 5.1 Fundamentals of Instance Segmentation
- 5.2 Proposal-based Instance Segmentation
- 5.3 Hough-transform
- 5.4 Instance Segmentation by Similarity Learning
- 6.1 Efficiently Solvable Graph Problems (Introduction)
- 6.2 Shortest Paths | Dijkstra Algorithm
- 6.3 Shortest Paths | 1D Labeling Problems
- 6.4 Shortest Paths | Segmented Least Squares
- 7.1 Dynamic Programming on Trees (Introduction)
- 7.2 Dynamic Programming on Trees | Message Passing
- 7.3 Dynamic Programming on Trees | Applications in Computer Vision
- 8.1 Watershed Algorithm | Clinical Application
- 8.2 Shortest Path vs. Widest Path | Seeded Segmentation
- 8.3 Minimax Paths | Prim's Algorithm
- 8.4 All-pairs Minimax Paths | Minimum Spanning Tree
- 8.5 Seeded Watershed Segmentation | ilastik Demo
- 8.6 Watershed Segmentation | Connection to Deep Learning
- 9.1 Recap of Shortest Path Algorithm
- 9.2 All-pairs Shortest Paths | Distance Product
- 9.3 The Algebraic Path Problem
- 9.4 Infimal Convolution | Euclidean Distance Transform
- 10.1 Tracking: Introduction and Overview
- 10.2 Tracking by Assignment | Min-Cost Flow
- 10.3 (Integer) Linear Programming | Polyhedral Geometry
- 10.4 Total Unimodularity
- 11.1 Optimal Transport: Introduction and Motivation
- 11.2 Discrete Optimal Transport
- 11.3 Discrete Optimal Transport (cont.) | Sinkhorn Iterations
- 11.4 Wasserstein Generative Adversarial Networks

### Machine Learning (Winter Term 2019)

- 1. Introduction and PCA
- 2. SVD and KDE
- 3. Mean-Shift and k-Means
- 4. Classification, k-NN, Cross-Validation, and Decision Trees
- 5. Decision Trees and Random Forests
- 6. Bayes Theorem, Statistical Decision Theory, and Quadratic Discriminant Analysis
- 7. Linear Regression
- 8. Regularized Linear Regression (Ridge, Lasso, ...)
- 9. Gaussian Process Regression
- 10. Logistic Regression and Generalized Linear Models
- 11. Perceptron and Multi-Layer Perceptron
- 12. Projection Trick and Function Counting Theorem
- 13. Backpropagation and Neural Network Training
- 14. CNNs and Deep Learning Tricks
- 15. Bayesian Networks/Probabilistic Graphical Models
- 16. Hidden Markov Models
- 17. Kalman Filter
- 18. Guest Lecture (not recorded)
- 19. Multicut and Correlation Clustering
- 20. Cluster Analysis
- 21. Dimension Reduction

### Machine Learning for Computer Vision (Winter Term 2017)

- 1. Introduction
- 2. Undirected Probabilistic Graphical Models
- 2.1 MAP & Priors
- 2.2 Markov Random Fields
- 2.3 Gibbs Sampling
- 2.4 MRF as Integer Linear Program (I)
- 2.5 MRF as Integer Linear Program (II)
- 2.6 Tree-Shaped MRF
- 2.7 Belief Propagation
- 2.8 Gaussian MRF (I)
- 2.9 Gaussian MRF (II)
- 3. Neural Networks
- 3.1 Perceptrons
- 3.2 Back Propagation
- 3.3 Introduction to Deep Learning
- 3.4 Deep Learning Architectures
- 3.5 Natural Gradient Optimization (I)
- 3.6 Natural Gradient Optimization (II)
- 3.7 Combining Graphical Models & Neural Networks (I)
- 3.8 Combining Graphical Models & Neural Networks (II)
- 4. Directed Probabilistic Graphical Models
- 4.1 Reinforcement Learning
- 4.2 Policy Gradient
- 6.3 Robotics

### Pattern Recognition Video Lectures (Summer Term 2012)

Full playlist here.- 1.1 Applications of Pattern Recognition
- 1.2 k-Nearest Neighbors Classification
- 1.3 Probability Theory
- 1.4 Statistical Decision Theory
- 2.1 Pearson Correlation
- 2.2 Alternative Correl. Measures
- 2.3 Gaussian Graphical Models
- 2.4 Discriminant Analysis
- 3.1 Regularized LDA/QDA
- 3.2 Principal Component Analysis (PCA)
- 3.3 Bilinear Decompositions
- 4.1 History of Neural Networks
- 4.2 Perceptrons
- 4.3 Multilayer Perceptrons
- 4.4 The Projection Trick
- 4.5 Radial Basis Function Networks
- 5.1 Loss Functions
- 5.2 Linear Soft-Margin SVM
- 5.3 Nonlinear SVM
- 6.1 Kernels
- 6.2 One Class SVM
- 6.3 Random Forest
- 6.4 Random Forest Feature Importance
- 7.1 Least-Squares Regression
- 7.2 Optimum Experimental Design
- 7.3 Case Study: Functional MRI
- 7.4 Case Study: CT
- 7.5 Regularized Regression
- 8.1 Gaussian Process Regression
- 8.2 GP Regression: Interpretation
- 8.3 Gaussian Stochastic Processes
- 8.4 Covariance Function
- 9.1 Kernel Density Estimation
- 9.2 Cluster Analysis
- 9.3 Expectation Maximization
- 9.4 Gaussian Mixture Models
- 10.1 Bayesian Networks
- 10.2 Variable Elimination
- 10.3 Message Passing
- 10.4 State Space Models
- 11.1 The Lagrangian Method
- 11.2 Constraint Qualifications
- 11.3 Linear Programming
- 11.4 The Simplex Algorithm
- 12.1 StructSVM
- 12.2 Cutting Planes

**1 Introduction**

**2 Correlation measures, Gaussian Models**

**3 Dimensionality Reduction**

**4 Neural Networks**

**5 Support Vector Machines**

**6 Kernels, Random Forest**

**7 Regression**

**8 Gaussian Processes**

**9 Unsupervised Learning**

**10 Directed Graphical Models**

**11 Optimization**

**12 Structured Learning**

### Image Analysis Video Lectures (Summer Term 2013 / 2015).

Full playlist here, including some added material (scroll to last entries).- Introduction
- Human early vision
- Image representations
- Block Matching
- Texture Synthesis
- Non-Local Means for Image Denoising
- BM3D for image denoising
- Unitary transformations
- The Fourier Transform
- The Discrete Fourier Transform (DFT)
- 2D-DFT: Application to Images
- Fourier Transform
- Time Frequency Decompositions
- The Wavelet Transform
- Watershed
- Maximally Stable Extremal Regions (MSER)
- Mathematical Morphology
- Minkowski functionals
- Markov Random Fields (MRFs)
- Gaussian Markov Random Fields (GMRF)
- Intrinsic GMRFs (IGMRF)
- Factor Graphs
- Fields of Experts
- Discrete-Valued MRFs
- MAP inference via Integer Linear Programming (ILP)
- Integer Linear Programs (continued)
- Pseudo Boolean Functions (PBFs)
- Quadratic PBFs with submodular terms
- Max-Flow / Min-Cut
- Graph Cuts
- Introduction
- Example model: Tracking by assignment
- Structured Support Vector Machine (structSVM)
- Structured Learning: Applications
- Light Fields
- Coded Aperture Imaging
- Compressive Sensing

**1 Introduction**

**2 Patches in Image Analysis**

**3 Fourier Transformation**

**4 Wavelets**

**5 Images as Topographic Maps**

**6 Gaussian Random Fields**

**7 Fields of Experts, Discrete MRFs**

**8 Binary pairwise MRFs and Graph Cut**

**9 Structured learning**

**10 Light Fields and Compressive Sensing**