BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Talks.cam//talks.cam.ac.uk//
X-WR-CALNAME:Talks.cam
BEGIN:VEVENT
SUMMARY:Fokker-Planck-based Inverse Reinforcement Learning --- A Physics-C
 onstrained Approach to Markov Decision Process Models of Cell Dynamics - K
 rishna Garikipati (University of Michigan)
DTSTART:20230802T080000Z
DTEND:20230802T090000Z
UID:TALK202420@talks.cam.ac.uk
DESCRIPTION:Inverse Reinforcement Learning (IRL) is a compelling technique
  for revealing the rationale underlying the behavior of autonomous agents.
 &nbsp\; IRL seeks to estimate the unknown reward function of a Markov deci
 sion process (MDP) from observed agent trajectories. However\, IRL needs a
  transition function\, and most algorithms assume it is known or can be es
 timated in advance from data. It therefore becomes even more challenging w
 hen such transition dynamics is not known a-priori\, since it enters the e
 stimation of the policy in addition to determining the system's evolution.
  When the dynamics of these agents in the state-action space is described 
 by stochastic differential equations (SDE) in It\\^{o} calculus\, these tr
 ansitions can be inferred from the mean-field theory described by the Fokk
 er-Planck (FP) equation. We conjecture there exists an isomorphism between
  the time-discrete FP and MDP that extends beyond the minimization of free
  energy (in FP) and maximization of the reward (in MDP). We identify speci
 fic manifestations of this isomorphism and use them to create a novel phys
 ics-aware IRL algorithm\, FP-IRL\, which can simultaneously infer the tran
 sition and reward functions using only observed trajectories. &nbsp\;We em
 ploy variational system identification to infer the potential function in 
 FP\, which consequently allows the evaluation of reward\, transition\, and
  policy by leveraging the conjecture.&nbsp\;We demonstrate the effectivene
 ss of FP-IRL by applying it to a synthetic benchmark and a biological prob
 lem of cancer cell dynamics\, where the transition function is inaccessibl
 e.\nThis is joint work with Changyang Huang\, Siddhartha Srivastava\, Kenn
 eth Ho\, Kathryn Luker\, Gary Luker and Xun Huan.
LOCATION:Seminar Room 1\, Newton Institute
END:VEVENT
END:VCALENDAR