Abstract

Structured representations are a powerful tool in machine learning, in particular for natural language: The discrete, compositional nature of words and sentences leads to natural combinatorial representations such as trees, sequences, segments, or alignments, among others. Such representations are at odds with deep neural networks, which conventionally perform smooth, soft computations, learning dense, inscrutable hidden representations.

We present SparseMAP, a strategy for inferring differentiable combinatorial latent structures, alleviating the tension between discrete and continuous representations through sparsity. SparseMAP computes a globally-optimal combination of a very small number of structures, and can be extended to arbitrary factor graphs (LP-SparseMAP), only requiring access to local maximization oracles. Our strategy is fully deterministic and compatible with familiar gradient-based methods for training neural networks. We demonstrate sparse and structured neural hidden layers, with successful empirical results and visualization properties.

Bio: Vlad Niculae is an assistant professor in the Language Technology Lab, at the Informatics Institute (IvI), University of Amsterdam. His research is on machine learning for natural language processing, focusing on sparsity, structure, and optimization. He obtained his PhD in 2018 from Cornell University, advised by Claire Cardie, and did a postdoc at the Instituto de Telecomunicações in Lisbon. Vlad is an ELLIS Scholar and a member of the Amsterdam ELLIS unit.

Georgi Karadzhov is inviting you to a scheduled Zoom meeting.

Topic: NLIP Seminars 26.11.2021 Time: Nov 26, 2021 12:00 PM London

Join Zoom Meeting https://cl-cam-ac-uk.zoom.us/j/92488988422?pwd=QWRKekxqRU52Vm1BR3NtWE9uT1dBUT09

Meeting ID: 924 8898 8422 Passcode: 681744 One tap mobile +13126266799,,92488988422# US (Chicago) +13462487799,,92488988422# US (Houston)

Dial by your location +1 312 626 6799 US (Chicago) +1 346 248 7799 US (Houston) +1 669 900 6833 US (San Jose) +1 929 205 6099 US (New York) +1 253 215 8782 US (Tacoma) +1 301 715 8592 US (Washington DC) Meeting ID: 924 8898 8422 Find your local number: https://cl-cam-ac-uk.zoom.us/u/aenYipJC2k