BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Talks.cam//talks.cam.ac.uk//
X-WR-CALNAME:Talks.cam
BEGIN:VEVENT
SUMMARY:The Curious Price of Distributional Robustness in Reinforcement Le
 arning with a Generative Model - Yuting Wei (University of Pennsylvania)
DTSTART:20251110T113000Z
DTEND:20251110T121000Z
UID:TALK238423@talks.cam.ac.uk
DESCRIPTION:In this talk\, we investigate model robustness in reinforcemen
 t learning (RL) to reduce the sim-to-real gap in practice. We adopt the fr
 amework of distributionally robust Markov decision processes (RMDPs)\, aim
 ed at learning a policy that optimizes the worst-case performance when the
  deployed environment falls within a prescribed uncertainty set around the
  nominal MDP. Despite recent efforts\, the sample complexity of RMDPs rema
 ined mostly unsettled regardless of the uncertainty set in use. It was unc
 lear if distributional robustness bears any statistical consequences when 
 benchmarked against standard RL. Assuming access to a generative model tha
 t draws samples based on the nominal MDP\, we provide a near-optimal chara
 cterization of the sample complexity of RMDPs when the uncertainty set is 
 specified via either the total variation (TV) distance or &chi\;2 divergen
 ce. The algorithm studied here is a model-based method called distribution
 ally robust value iteration\, which is shown to be near-optimal for the fu
 ll range of uncertainty levels.&nbsp\;
LOCATION:Seminar Room 1\, Newton Institute
END:VEVENT
END:VCALENDAR