Abstract

For many tasks in NLP , choosing the best internal representation, which encodes the meaning of each word in some mathematical object, is crucial. In this talk I’ll begin by briefly introducing a couple of simple naïve representation schemes and show how Vector Space Models (VSMs) can be used to address their shortcomings. I will then demonstrate how to learn those vector representations, focusing on a particular method called Global Vectors (GloVe). I will derive its mathematical formulation from the desired properties and compare its performance to other models. GloVe word embeddings, and VSMs in general, are an easy yet effective way to encode meaning, given a large enough training corpus. They are very versatile: embeddings make it easy to measure similarity between two words, and they are also useful feature vectors for other NLP systems. Furthermore, they can be used as the building blocks for sentence or document embeddings, which can then be tested for similarity in the same way. Finally, the same vectors can be used for many projects, and pre-trained models are available, eliminating the cost and effort of training.