Variational Bayes In Private Settings

Add to your list(s) Download to your calendar using vCal

• Mijung Park, University of Amsterdam
• Wednesday 25 January 2017, 15:00-16:00
• CBL Room BE-438, Department of Engineering.

If you have a question about this talk, please contact Alessandro Davide Ialongo.

Bayesian methods are frequently used for analysing privacy-sensitive datasets, including medical records, emails, and educational data, and there is a growing need for practical Bayesian inference algorithms that protect the privacy of individuals’ data. To this end, we provide a general framework for privacy-preserving variational Bayes (VB) for a large class of probabilistic models, called the conjugate exponential (CE) family. Our primary observation is that when models are in the CE family, we can privatise the variational posterior distributions simply by perturbing the expected sufficient statistics of the complete-data likelihood. For widely used non-CE models with binomial likelihoods (e.g., logistic regression), we exploit the Polya-Gamma data augmentation scheme to bring such models into the CE family, such that inferences in the modified model resemble the original (private) variational Bayes algorithm as closely as possible. The iterative nature of variational Bayes presents a further challenge for privacy preservation, as each iteration increases the amount of noise needed. We overcome this challenge by combining: (1) a relaxed notion of differential privacy, called concentrated differential privacy, which provides a tight bound on the privacy cost of multiple VB iterations and thus significantly decreases the amount of additive noise; and (2) the privacy amplification effect of subsampling mini-batches from large-scale data in stochastic learning. We empirically demonstrate the effectiveness of our method in CE and non-CE models including latent Dirichlet allocation (LDA), Bayesian logistic regression, and Sigmoid Belief Networks (SBNs), evaluated on real-world datasets.

Speaker Bio:

Mijung Park completed her Ph.D. in the department of Electrical and Computer Engineering under the supervision of Prof. Jonathan Pillow (now at Princeton University) and Prof. Alan Bovik at The University of Texas at Austin. She worked with Prof. Maneesh Sahani as a postdoc at the Gatsby computational neuroscience unit at University College London. Currently, she works with Prof. Max Welling as a postdoc in the informatics institute at University of Amsterdam. Her research focuses on developing practical algorithms for privacy preserving data analysis. Previously, she worked on a broad range of topics including approximate Bayesian computation (ABC), probabilistic manifold learning, active learning for drug combinations and neurophysiology experiments, and Bayesian structure learning for sparse and smooth high dimensional parameters.

This talk is part of the Machine Learning @ CUED series.

This talk is included in these lists:

• Seminar
• All Talks (aka the CURE list)
• Biology
• CBL Room BE-438, Department of Engineering
• CBL important
• Cambridge Centre for Data-Driven Discovery (C2D3)
• Cambridge Forum of Science and Humanities
• Cambridge Language Sciences
• Cambridge Neuroscience Seminars
• Cambridge talks
• Chris Davis' list
• Creating transparent intact animal organs for high-resolution 3D deep-tissue imaging
• Featured lists
• Guy Emerson's list
• Hanchen DaDaDash
• Inference Group Summary
• Information Engineering Division seminar list
• Interested Talks
• Joint Machine Learning Seminars
• Life Science
• Life Sciences
• ML
• Machine Learning @ CUED
• Machine Learning Summary
• Neuroscience
• Neuroscience Seminars
• Neuroscience Seminars
• Required lists for MLG
• Simon Baker's List
• Stem Cells & Regenerative Medicine
• Trust & Technology Initiative - interesting events
• bld31
• dh539
• kt532's list
• ndk22's list
• ob366-ai4er
• rp587
• yk373's list

Note that ex-directory lists are not shown.