University of Cambridge > Talks.cam > Machine Learning @ CUED > Observation and Intervention Incentives in Causal Influence Diagrams: Towards an Understanding of Powerful Machine Learning Systems

  Browse A-Z

Log in

University Account External (via Google)

🔐 Log In

Information on

Subscribing to talksdetails Finding a talkdetails Adding a talkdetails Disseminating talksdetails Help and Documentationdetails

ℹ️ Information

Observation and Intervention Incentives in Causal Influence Diagrams: Towards an Understanding of Powerful Machine Learning Systems

Download to your calendar using vCal

If you have a question about this talk, please contact Adrià Garriga Alonso .

As machine learning systems gain in capability and complexity, understanding their incentives will become increasingly important. In this paper, we model their objectives and environment interaction in graphical models called influence diagrams. This allows us to answer two fundamental questions about the incentives of a machine learning system directly from the graphical representation: (1) which nodes would the system like to observe in addition to its observations, and (2) which nodes would the system like to control in addition to its actions? The answers tell us which information and influence points need extra protection, and have applications to fairness and reward tampering. For example, we may want a classifier for job applications to not use the race of the candidate, and a reinforcement learning agent not to take direct control of its reward mechanism. Different algorithms and training paradigms can lead to different influence diagrams, so our results can help designing algorithms with less problematic observation and intervention incentives.

This talk is part of the Machine Learning @ CUED series.

This talk is included in these lists:

Note that ex-directory lists are not shown.

Other lists

Small Lecture Theatre, Department of Geography, University of Cambridge CMS seminars Babbage Lecture Theatre, David Attenborough Building

Other talks

Spectral rigidity of random covers of compact hyperbolic surfaces TBC Secrets of the Surface: The Mathematical Vision of Maryam Mirzakhani ( EDI Film Screening) Group Work Cultural evolution creates language-like structure: from humans to humpback whales and beyond Think Before you Speak: Next Gen LLMs with Global Reasoning and External Memory
 

© 2006-2025 Talks.cam, University of Cambridge. Contact Us | Help and Documentation | Privacy and Publicity