Abstract

Latent variables are an important component of many statistical models. Most latent variable models, such as mixture models, factor analysis, and independent components analysis (ICA), assume a finite, usually small number of latent variables. However, it may be statistically undesirable to constrain the number of latent variables a priori. Here we show how a more flexible nonparametric approach is possible in which the number of latent variables is unbounded. To do this, we describe a probability distribution over equivalence classes of binary matrices with a finite number of rows, corresponding to the data points, and an unbounded number of columns, corresponding to the latent variables. Each data point can be associated with a subset of the possible latent variables, which we refer to as the latent features of that data point. The binary variables in the matrix indicate which latent feature is possessed by which data point, and there is a potentially infinite array of features. We derive the distribution over unbounded binary matrices by taking the limit of a distribution over $N \times K$ binary matrices as $K \rightarrow \infty$, a strategy inspired by the derivation of the Chinese restaurant process (Aldous, 1985; Pitman, 2002) which preserves exchangeability of the rows. We define a simple generative processes for this distribution which we call the Indian buffet process (IBP; Griffiths and Ghahramani, 2005). We describe recent extensions of this model, Markov chain Monte Carlo algorithms for inference, and a number of applications to collaborative filtering, independent components analysis, bioinformatics, cognitive modelling, and causal discovery.

Joint work with Thomas L. Griffiths (UC Berkeley).