Abstract

For this journal club we will look at Principled Hybrids of Generative and Discriminative Models by Bishop and Laserre (2006)

A shortened abstract: When labelled training data is plentiful, discriminative techniques are widely used since they give excellent generalization performance. However, for large-scale applications such as object recognition, hand labelling of data is expensive, and there is much interest in semi-supervised techniques based on generative models in which the training data is unlabelled. In an attempt to gain the benefit of both generative and discriminative approaches, heuristic procedure have been proposed [2, 3] which interpolate between these two extremes.

In this paper we adopt a new perspective which says that there is only one correct way to train a given model, and that a `discriminatively trained’ generative model is fundamentally a new model [7]. From this viewpoint, generative and discriminative models correspond to specific choices for the prior over parameters.

Another related article is Generative or Discriminative? Getting the Best of Both Worlds

The journal club will be preceded by a 5-minute talk by Ignas about his undergraduate project