Fall 2025 - Reinforcement Learning
September 2025 - December 2025, University of St. Gallen
The Method Reading Group takes place (approximately) bi-weekly with sessions of 1h15m, and we aim to devote at least one hour to topic discussion.
This semester, we meet on Tuesdays, 17:15-18:30, alternating with the Quantitative Methods and Learning Research Seminar. Please see the schedule section below for updated room locations, dates and topics.
The group is open to anyone: if you would like to participate in the discussion too, let us know!
- Contact:
- Giovanni Ballarin (giovanni.ballarin[-at-]unisg.ch)
- Erik-Jan Senn (erik-jan.senn[-at-]unisg.ch)
- Files:
- See below for materials discussed
- Some additional references are also listed
Locations
Room C 83-2233, Haus Washington
Rosenbergstrasse 20/22, St. Gallen
Link to MazeMap
Join Online
Teams Meeting Link
Note: the Teams link will be the same for all meetings.
Schedule
Please check this website for regular updates
| # | Date | Room | Paper/Topic | Presenter | Discussant |
|---|---|---|---|---|---|
| 1) | 16.09* | C 83-2233 | RL Foundations: MDPs / Dynamic Optimization / Q-Learning | Tobias | --- |
| 2) | 14.10 | C 83-2233 | RL Foundations: Q-Learning from a Statistical Viewpoint | Giovanni | Tobias |
| 3) | 21.10 | C 83-2233 | A Two-Timescale Primal-Dual Framework for Reinforcement Learning via Online Dual Variable Guidance (2025) | Axel Wolter | Erik |
| 🍂 SEMESTER BREAK 🍂 | |||||
| 4) | 25.11 | C 83-2233 | A Review of Reinforcement Learning in Financial Applications (2025) | Jonas | Erik |
| 5) | 09.12 | C 83-2233 | TBD | TBD | TBD |
Schedule Notices:
- Please note that on Tuesday 16.09 the meeting will run for an extended time to account for more material & one less session!
Materials
- Sessions 1
- Session 2
- Handwritten Notes
- Main reference: Clifton and Laber (2020)
- For regularized two-period Q-Learning, see also Soing et al. (2015)
- For nonparametric sieve estimation of conditional moment restriction models, which include (infinite horizon) Q-Learning, see Chen et al. (2025)
Additional References
-
Meyn - Control Systems and Reinforcement Learning (2022)
(Free) book on RL, mostly from the viewpoint of optimal control theory and dynamical systems. -
Murphy - Reinforcement Learning: An Overview (2025)
Broad introductory discussion of (almost all) RL methods. Current version also includes chapter on RL applied to LLM training. -
Foster and Rakhlin - Foundations of Reinforcement Learning and Interactive Decision Making (2023)
MIT lecture notes that cover especially the bandit setting (multi-armed, structured, etc.)
Members
- Giovanni Ballarin
- Luca Fiumana
- Prof. Lyudmila Grigoryeva
- Christoph Hirt
- Jonas Huwyler
- Prof. Jana Mareckova
- Angelo Mimmo
- Linda Odermatt
- Erik-Jan Senn
- Prof. Tobias Sutter
- Lion Szlagowski
How To
The general guidelines and "house rules" we follow are much inspired by those of e.g. the TS&ML Reading Group at the University of Southampton.
We will alternate over time so that each person can try and fulfill the two main roles at least once:
-
Presenter: the person who presents (with annotated paper, whiteboard, slides...) the meeting's topic -- they act as the 'key expert' on the chosen paper/chapter/topic, and go over the core points of the material, step by step. The presenter should be well-prepared in the content presented, and be able to provide a compact but transparent and technically accessible presentation of the material.
-
Discussant: the person that, having read in some detail the material pertaining to the the meeting, can help steer the discussion: what are some key issues or questions? What things may be unclear? What are the strengths and weaknesses? Your role is to help everyone in the session navigate and understand the main ideas discussed.
The following (total) preparation times are suggested:
- Presenter: approx. 10 hours
- Discussant: approx. 5 hours
- General audience: approx. 2 hours
If you encounter any issues with the materials, do not hesitate to contact us!