BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Talks.cam//talks.cam.ac.uk//
X-WR-CALNAME:Talks.cam
BEGIN:VEVENT
SUMMARY:Security and Privacy in Machine Learning  - Nicolas Papernot\, Pen
 nsylvania State University
DTSTART:20170331T121500Z
DTEND:20170331T131500Z
UID:TALK71863@talks.cam.ac.uk
CONTACT:Microsoft Research Cambridge Talks Admins
DESCRIPTION:There is growing recognition that machine learning exposes new
  security and privacy issues in software systems. In this talk\, we first 
 articulate a comprehensive threat model for machine learning\, then presen
 t an attack against model prediction integrity\, and finally discuss a fra
 mework for learning privately.\n\nMachine learning models were shown to be
  vulnerable to adversarial examples--subtly modified malicious inputs craf
 ted to compromise the integrity of their outputs. Furthermore\, adversaria
 l examples that affect one model often affect another model\, even if the 
 two models have different architectures\, so long as both models were trai
 ned to perform the same task. An attacker may therefore conduct an attack 
 with very little information about the victim by training their own substi
 tute model to craft adversarial examples\, and then transferring them to a
  victim model. The attacker need not even collect a training set to mount 
 the attack. Indeed\, we demonstrate how adversaries may use the victim mod
 el as an oracle to label a synthetic training set for the substitute. We c
 onclude this first part of the talk by formally showing that there are (po
 ssibly unavoidable) tensions between model complexity\, accuracy\, and res
 ilience that must be calibrated for the environments in which they will be
  used.\n\nIn addition\, some machine learning applications involve trainin
 g data that is sensitive\, such as the medical histories of patients in a 
 clinical trial. A model may inadvertently and implicitly store some of its
  training data\; careful analysis of the model may therefore reveal sensit
 ive information. To address this problem\, we demonstrate a generally appl
 icable approach to providing strong privacy guarantees for training data. 
 The approach combines\, in a black-box fashion\, multiple models trained w
 ith disjoint datasets\, such as records from different subsets of users. B
 ecause they rely directly on sensitive data\, these models are not publish
 ed\, but instead used as "teachers" for a "student" model. The student lea
 rns to predict an output chosen by noisy voting among all of the teachers\
 , and cannot directly access an individual teacher or the underlying data 
 or parameters. The student's privacy properties can be understood both int
 uitively (since no single teacher and thus no single dataset dictates the 
 student's training) and formally\, in terms of differential privacy. \n
LOCATION:Small Lecture Room \, Microsoft Research Ltd\, 21 Station Road\, 
 Cambridge\, CB1 2FB
END:VEVENT
END:VCALENDAR