Abstract

This talk discusses a class of non-linear stochastic control problems that can be efficiently solved using a path integral. In this control formalism, the central concept of cost-to-go or value function becomes a free energy and methods and concepts from statistical physics can be readily applied, such as Monte Carlo sampling or the Laplace approximation. Qualitatively different optimal control strategies for different noise levels can be understood as a result of spontaneous symmetry breaking. When applied to a receding horizon problem in a stationary environment, the solution resembles the one obtained by traditional reinforcement learning with discounted reward. An advantage of the path integral control method over RL is that the control can be computed for the current state, without considering all other states and 2) that it can be easily generalized to time-dependent tasks. It is therefore a suitable approach for time dependent control. We further discuss exploration and an how agents can approximately compute their coordination using belief propagation.