Abstract

Differential privacy is a recent privacy definition that permits only indirect observation of data held in a database through noisy measurement. I will show that there is a strong connection between differential privacy and probabilistic inference. Previous research on learning from data protected by differential privacy has been driven by researchers inventing sophisticated learning algorithms which are applied directly to the data and output model parameters which can be proven to respect the privacy of the data set. Proving these privacy properties requires an intricate analysis of each algorithm on a case-by-case basis. While this does result in many valuable algorithms and results, it is not a scalable solution for two reasons: first, to solve a new learning problem, one must invent and analyze a new privacy-preserving algorithm; second, one must then convince the owner of the data to run this algorithm. Both of these steps are challenging. In contrast, I will consider the potential of applying probabilistic inference to the measurements and measurement process to derive posterior distributions over the data sets and model parameters thereof, showing that for the models investigated, probabilistic inference can improve accuracy, integrate multiple observations, measure uncertainty, and even provide posterior distributions over quantities that were not directly measured.