Abstract

We develop an exhaustive study of Markov decision process (MDP) under mean field interaction both on states and actions in the presence of common noise, and when optimization is performed over open-loop controls on infinite horizon.  We highlight the crucial role of relaxed controls for this class of models, called CMKV -MDP for conditional McKean-Vlasov MDP , with respect to classical MDP theory. We prove the correspondence between CMKV -MDP and a general lifted MDP on the space of probability measures, and establish the dynamic programming Bellman fixed point equation satisfied by the value function, as well as the existence of ε-optimal randomized feedback controls.  We obtain the propagation of chaos of the optimal value functions of the N-agent MDP to the CMKVMDP when N → +∞, with some convergence rate, denoted by O(MNγ ).  We finally provide examples of application of the propagation of chaos result, by approximately solving several toy models for N-agent targeted advertising problem with social influence via the resolution of the associated CMKV -MDP. Based on joint work with Médéric Motte (LPSM).