University of Cambridge > Talks.cam > Isaac Newton Institute Seminar Series > Convergence of the actor-critic gradient flow for entropy regularised MDPs in general action spaces

Log in

University Account

External (via Google)

Information on

Subscribing to talks details

Finding a talk details

Adding a talk details

Disseminating talks details

Help and Documentation details

Convergence of the actor-critic gradient flow for entropy regularised MDPs in general action spaces

Download to your calendar using vCal

David Siska (University of Edinburgh)
Thursday 13 November 2025, 16:30-17:10
Seminar Room 1, Newton Institute.

If you have a question about this talk, please contact nobody.

SCLW01 - Bridging Stochastic Control And Reinforcement Learning: Theories and Applications

We prove the stability and global convergence of a coupled actor-critic gradient flow for infinite-horizon and entropy-regularised Markov decision processes (MDPs) in continuous state and action space with linear function approximation under Q-function realisability. We consider a version of the actor critic gradient flow where the critic is updated using temporal difference (TD) learning while the policy is updated using a policy mirror descent method on a separate timescale. We demonstrate stability and exponential convergence of the actor critic flow to the optimal policy. Finally, we address the interplay of the timescale separation and entropy regularisation and its effect on stability and convergence. This is joint work with Denis Zorba and Lukasz Szpruch.

This talk is part of the Isaac Newton Institute Seminar Series series.

This talk is included in these lists:

Note that ex-directory lists are not shown.

Convergence of the actor-critic gradient flow for entropy regularised MDPs in general action spaces

📅 Download to calendar (vCal)

⚠️ Important: SCLW01 - Bridging Stochastic Control And Reinforcement Learning: Theories and Applications

👤 Speaker: David Siska (University of Edinburgh)
📅 Date & Time: Thursday 13 November 2025, 16:30 - 17:10
📍 Venue: Seminar Room 1, Newton Institute

Questions? Contact the organiser

Abstract

Series This talk is part of the Isaac Newton Institute Seminar Series series.

Included in Lists

Note: Ex-directory lists are not shown.

Log in

🔐 Log In

Information on

ℹ️ Information

Convergence of the actor-critic gradient flow for entropy regularised MDPs in general action spaces

This talk is included in these lists:

Convergence of the actor-critic gradient flow for entropy regularised MDPs in general action spaces

Abstract

Included in Lists

Log in

🔐 Log In

Information on

ℹ️ Information

Convergence of the actor-critic gradient flow for entropy regularised MDPs in general action spaces

This talk is included in these lists:

Other lists

Other talks

Convergence of the actor-critic gradient flow for entropy regularised MDPs in general action spaces

Abstract

Included in Lists