News:

• We did some work on Early-Stopping without a Validation Set. Code will be released in the near future.
• An extended version of our NIPS '15 conference paper on probabilistic line searches is now on arXiv. It also contains a detailed pseudocode.
• I gave a talk at Amazon Research -- Cambridge -- 03/2017.
• I co-organized the workshop `Optimizing the Optimizers' @NIPS 2016
• I gave a talk at the Workshop on Uncertainty Quantification @GPSS (Gaussian Process Summer School) – Sheffield, UK – 09/2016
• I did a summer internship at the Amazon Development Center in Berlin, Germany – 07/2016 - 10/2016
• Our work on probabilistic line-searches is selected for a full oral at NIPS 2015.
• I gave a talk at Microsoft Research – Cambridge – 09/2015
• I gave a talk at the Workshop on Probabilistic Numerics @DALI 2015

Research Overview:

Stochastic optimization methods have become an increasingly important tool in minimizing nonlinear high dimensional objectives where only noisy function and gradient evaluations are available.

Probabilistic numerics provides a framework to address these challenges by explicitly modeling noise and uncertainty.

We see optimization as an inference problem where the unknown gradient (e.g. for gradient decent) or the unknown Hessian (e.g. for quasi-Newton methods) is inferred through previously collected noisy gradient evaluations.

In fact Hennig and Kiefel [ ] showed that quasi-Newton methods, such as the BFGS rule,
arise as the mean of a Gaussian distribution over the elements of the Hessian matrix of an optimization objective.

Aside search directions, probabilistic numerics can help to adjust step sizes (learning rates) by explicitly modeling noise and informing the optimizer about it [ ].

The goal is to have a smart machine (optimizer) that needs little to no human expert knowledge to perform its task, even communicating important information about the optimization progress to us.

For more information see The Independent Max Planck Research Group on Probabilistic Numerics

About me: I am a physicist by training and I love models, math and abstract thinking. Simulations and testing own theories on data (by writing a computer program) is an essential part of science for me.