BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Talks.cam//talks.cam.ac.uk//
X-WR-CALNAME:Talks.cam
BEGIN:VEVENT
SUMMARY:Incomplete Contracting and AI Alignment - Paolo Bova (University o
 f Cambridge)
DTSTART:20181128T170000Z
DTEND:20181128T190000Z
UID:TALK115687@talks.cam.ac.uk
CONTACT:Adrià Garriga Alonso
DESCRIPTION:This week\, Paolo Bova will lead the discussion on "Incomplete
  Contracting and AI Alignment'\nLink:https://arxiv.org/pdf/1804.04268.pdf\
 n\n\nThe authors argue that we can frame the design of the reward function
  for a learning agent as a contract. However\, sources of incompleteness r
 esult in a misalignment of the rewards functions of the agent and the valu
 es of the human. Incomplete Contract Theory helps us think about economic 
 contracts when facing bounded rationality\, planned renegotiation\, and st
 rategic behaviour and all have analogues in the AI case. Therefore\, Incom
 plete Contract Theory provides insight into tackling AI Contracts.\n\nThe 
 paper even argues that Incomplete Contract Theory has relevance for Super 
 Intelligent AI. Finally\, they discuss the legal scholars' insight\, calle
 d 'normative social order'\, as a prerequisite for AI to absorb human valu
 es as we do.\n\n\nThis week we're mixing up the format to include question
 s now to better understand the paper and what we should take away from it:
 \n\nComprehension Questions:\n1. What are the main sources of incompletene
 ss in designing contracts (Section 3)?\n2. How does incomplete contract th
 eory address these problems? Which AI applications do the author's mention
  use this theory?\n(try to describe at least one - Section 4)\n3. What are
  the extra challenges faced by 'Strongly Strategic' AI? Which solutions do
  the authors suggest in these cases?\n(try to describe at least one - Sect
 ion 4)\n4. Describe the concept of 'Normative Social Order' (Section 5)? W
 hy do the authors believe it matters for AI Safety?\n\n\nWe will also spli
 t into smaller groups at times to focus on learning parts of the paper and
  then teach the content back to the whole group.\n\n--\n\nThere will be fr
 ee pizza. At 17:00\, we will start reading the paper\, mostly individually
 . At 17:30\, the discussion leader will start going through the paper\, ma
 king sure everyone understands\, and encouraging discussion about its cont
 ents and implications.\n\nEven if you think you cannot contribute to the c
 onversation\, you should give it a try. Last year we had several people fr
 om non-computer-y backgrounds\, and others who hadn't thought about alignm
 ent before\, that ended up being essential. If you have already read the p
 aper in your own time you can come in time for the discussion.\n\nA basic 
 understanding of machine learning is helpful\, but detailed knowledge of t
 he latest techniques is not required. Each session will have a brief recap
  of immediate necessary knowledge. The goal of this series is to get peopl
 e to know more about the existing work in AI research\, and eventually con
 tribute to the field.\n\nInvite your friends to join the mailing list (htt
 ps://lists.cam.ac.uk/mailman/listinfo/eng-safe-ai)\, the Facebook group (h
 ttps://www.facebook.com/groups/1070763633063871) or the talks.cam page (ht
 tps://talks.cam.ac.uk/show/index/80932). Details about the next meeting\, 
 the week's topic and other events will be advertised in these places.\n\n
LOCATION:Cambridge University Engineering Department\, CBL Seminar room BE
 4-38
END:VEVENT
END:VCALENDAR