Workshop on Themes across Control and Reinforcement Learning

Following our Spring School 2025 of the research semester programme on Control Theory and Reinforcement Learning, we have a general workshop on Themes across Control and Reinforcement Learning.

Share this page

When

24 Mar 2025 from 9:15 a.m. to 25 Mar 2025 6 p.m. CET (GMT+0100)

Where

Science Park 125, Turingzaal

Add

Add event to calendar

You are welcome to bring along a poster. We also invite abstracts for talks (limited slots). Attendance without presenting is welcome as well.

Speakers information

Ann Nowé, Vrije Universiteit Brussel, Belgium

Ann Nowé graduated from the University of Ghent in 1987, where she studied mathematics with optional courses in computer science. She then became research assistant at the University of Brussels where she finished her PhD in 1994 in collaboration with Queen Mary and Westfield College, University of London. The subject of her PhD is located in the intersection of Computer Science (A.I.), Control Theory (Fuzzy Control) and Mathematics (Numerical Analysis, Stochastic Approximation). After a period of 3 years as senior research assistant at the Vrije Universiteit Brussel (VUB), she became a Postdoctoral Fellow of the Fund for Scientific Research-Flanders (F.W.O.). Nowadays, She is a professor both in the Computer Science Department of the faculty of Sciences as in the Computer Science group of the Engineering Faculty. She coordinates the project CTRLXAI, on automated controller generation at the intersection of control engineering and machine learning.

Bert Kappen, Radboud University, Nijmegen, Netherlands

Bert Kappen completed his PhD in theoretical particle physics in 1987 at the Rockefeller University in New York. From 1987 until 1989 he worked as a scientist at the Philips Research Laboratories in Eindhoven, the Netherlands. Since 2004 he is full professor on machine learning and neural networks at the science faculty of the Radboud University. In 1998, he co-founded the company Smart Research that commercializes applications of neural networks and machine learning.

Bert Kappen conducts research on neural networks, Bayesian machine learning, stochastic control theory and computational neuroscience. Currently, he is investigating ways to use quantum mechanics for a new generation of quantum machine learning algorithms and control methods for quantum computing.

Talk details

Title: Path integral control for open quantum systems

We consider the generic problem of state preparation for open quantum systems. As is well known, open quantum systems can be simulated by quantum trajectories described by a stochastic Schr\"odinger equation. In this context, the state preparation problem becomes a stochastic optimal control (SOC) problem. The SOC problem requires the solution of the Hamilton-Jacobi-Bellman equation, which is generally challenging to solve. A notable exception are the so-called path integral (PI) control problems for which one can estimate the optimal control solution by sampling. We derive a class of quantum state preparation problems that can be solved with PI control. Since our method only requires the propagation of state vectors $\psi$, it presenting a quadratic advantage over density-centered approaches, such as PMP. Unlike most conventional quantum control algorithms, it does not require computing gradients of the cost function to determine the optimal controls. We illustrate the practicality of our algorithm through multiple examples for single- and multi-qubit systems.

Davide Grossi, University of Amsterdam, University of Groningen, Netherlands

Davide Grossi is an associate professor (adjunct hoogleraar) in Multi-Agent Decision Making, affiliated to the Bernoulli Institute for Mathematics, Computer Science and Artificial Intelligence at the University of Groningen . He is a member of the Multi-Agent Systems Group, and of the Groningen Cognitive Systems and Materials Center (CogniGron).

At the University of Amsterdam he is an associate professor affiliated to the Amsterdam Center for Law and Economics (ACLE), and the Institute for Logic, Language and Computation (ILLC).

Talk details

Title: Learning to Cooperate under Uncertain Incentive Alignment

We apply a multi-agent reinforcement learning approach to study the emergence of cooperation among reinforcement learning agents that interact in a game. Crucially, agents are however uncertain about the extent to which their incentives are aligned, or misaligned, in the game. In this framework, we explore through computational experiments under what conditions further features of the model (like communication, reputation mechanisms, varying risk attitudes) may support learning of more cooperative policies.

This is joint work with: Nicole Orzan (University of Groningen), Erman Acar (University of Amsterdam) and Roxana Radulescu (Utrecht University)

Frans Oliehoek, TU Delft, Netherlands

Frans A. Oliehoek is Associate Professor at Delft University of Technology, where he is a leader of the sequential decision making group, a scientific director of the Mercury machine learning lab, and director and co-founder of the ELLIS Unit Delft. He received his Ph.D. in Computer Science (2010) from the University of Amsterdam (UvA), and held positions at various universities including MIT, Maastricht University and the University of Liverpool. Frans' research interests revolve around intelligent systems that learn about their environment via interaction, building on techniques from machine learning, AI and game theory. He has served as PC/SPC/AC at top-tier venues in AI and machine learning, and currently serves as associate editor for JAIR and AIJ. He is a Senior Member of AAAI, and was awarded a number of personal research grants, including a prestigious ERC Starting Grant.

Talk details

Title: Learning and using models for controlling complex environments

In reinforcement learning (RL), we develop techniques to learn to control complex systems, and over the last decade we have seen impressive successes ranging from beating grand masters in the game of Go, to real-world applications like chip design, power grid control, and drug design. However, nearly all applications of RL require access to an accurate and lightweight model or simulator from which huge numbers of trials can be sampled. In this talk, I will discuss some topics in model-based RL, in which this is not the case, but where we instead try to learn such models.

Harri Lähdesmäki, Aalto University, Finland

Harri Lähdesmäki is an Associate Professor (tenured) in Department of Computer Science at Aalto University, where he leads the Computational Systems Biology research group. He works on probabilistic machine learning with applications in biomedicine and molecular biology. His current research interests include deep generative models, dynamical systems, longitudinal analysis, single-cell biology, and health applications.

Talk details

Title: Learning latent dynamics models from complex and high-dimensional data

There is an abundance of dynamical systems around us, including physical systems, biological networks, industrial processes, population dynamics, and social graphs, which we would like to model and control. While models in some applications can be derived analytically, there are many systems whose governing equations cannot be derived from the first principles because their behavior is too complex and poorly understood or dimensionality far too high. In this talk, I will present our recent work on developing methods to learn latent continuous-time dynamics models from complex and high-dimensional (spatio)temporal data. Our approach builds on formulating these models as neural net parameterized generative differential equation systems that can be learned using efficient amortized variational inference methods and used for long-term predictions as well control approaches.

This is joint work with Valerii Iakovlev, Cagatay Yildiz, and Markus Heinonen.

Jens Kober, TU Delft, Netherlands

Jens Kober is an associate professor at TU Delft, Netherlands. He is member of the Cognitive Robotics department (CoR) and the TU Delft Robotics Institute.

Jens is the recipient of the Robotics: Science and Systems Early Career Award 2022 and the IEEE-RAS Early Academic Career Award in Robotics and Automation 2018. His Ph.D. thesis has won the 2013 Georges Giralt PhD Award as the best Robotics PhD thesis in Europe in 2012.

Sean Meyn, University of Florida, USA

Sean Meyn was raised by the beach in Southern California. Following his BA in mathematics at UCLA, he moved on to pursue a PhD with Peter Caines at McGill University. After about 20 years as a professor of ECE at the University of Illinois, in 2012 he moved to beautiful Gainesville. He is now Professor and Robert C. Pittman Eminent Scholar Chair in the Department of Electrical and Computer Engineering at the University of Florida, and director of the Laboratory for Cognition and Control. He also holds an Inria International Chair to support research with colleagues in France. His interests span many aspects of stochastic control, stochastic processes, information theory, and optimization. For the past decade, his applied research has focused on engineering, markets, and policy in energy systems.

Sofie Haesaert, Eindhoven University of Technology, Netherlands

Sofie Haesaert is an Associate Professor in the Control Systems group of the Department of Electrical Engineering at Eindhoven University of Technology. Her research interests lie in the development of verification and control synthesis methods for cyber-physical systems. She holds a bachelor's degree in Mechanical Engineering and a master's degree in Systems and Control from Delft University of Technology, both cum laude. As a master's degree student, Haesaert spent a summer doing research at the University of Texas Arlington (USA). Haesaert obtained her PhD from Eindhoven University of Technology (TU/e) in 2017 and then went to Caltech (USA) for 17 months to do a postdoc. In 2018, Haesaert returned to TU/e to become Assistant Professor in the Control Systems group.