Abstract

Most current theories of agency deal only with dualistic or Cartesian agents. That is, the agent that makes the decisions is itself outside of the world that it takes decisions in, and is not affected by it.

But technically this is not the case, as the algorithm that makes decisions is implemented using something (a brain, a computer) in the world, and modifications to that something can change the algorithm.

According to some views, understanding the theory behind embedded, non-Cartesian agency, is key to solving some important problems in AI safety (provably aligned self-modification, wireheading). Others think it’s not so important. We shall learn about current attempts to build embedded agency theories and discuss how important it is to continue work in that area.

Reading list:

- Embedded Agency sequence from MIRI : https://www.lesswrong.com/s/Rm6oQRJJmhGCcLvxh/p/i3BTagvt3HbPMx6PN