Abstract

Please join us for our fortnightly Computational Neuroscience journal club on Tuesday 28th March at 2pm UK time in the CBL seminar room, or online on zoom. The title is ‘Linearly solvable control problems: formalism and application to decision making’, presented by Marine Schimel and Guillaume Hennequin (CBL), with a guest presentation by Alfonso Renart (Champalimaud) in the second half.

Zoom information: https://eng-cam.zoom.us/j/84204498431?pwd=Um1oU284b1YxWThObGw4ZU9XZitWdz09 Meeting ID: 842 0449 8431 Passcode: 684140

Summary In the mid 2000’s, Bert Kappen and Emo Todorov independently discovered a class of non-trivial control problems for which the Bellman equation becomes linear. These problems ─ which include both (discrete-time) Markov decision problems and (continuous-time) nonlinear diffusive dynamics ─ can therefore be solved very efficiently. The tractability of these ‘linearly solvable control’ problems has fostered advances in a variety of other domains, including compositional control, risk-sensitive control, inverse RL, and game-theory.

Marine and Guillaume will give a 45 min overview of the ‘KL control’ framework, summarising the main derivation of the linearised Bellman equation and going through several concrete examples. In the remaining 45 min, Alfonso Renart will present his group’s latest preprint where they extend these ideas to (i) formalise the cost of control incurred by an animal having to modify its default behaviour, and (ii) study the properties of such control-limited behaviour. Their new framework ‘provides an organizing principle for how different task manipulations shape the phenomenology of perceptual decision making, including the nature and consequence of decision lapses and sequential dependencies’.