RESEARCH
NLP
Cooperative Learning of Disjoint Syntax and Semantics
June 2, 2019
Abstract
There has been considerable attention devoted to models that learn to jointly infer an expression’s syntactic structure and its semantics. Yet, Nangia and Bowman (2018) has recently shown that the current best systems fail to learn the correct parsing strategy on mathematical expressions generated from a simple context-free grammar. In this work, we present a recursive model inspired by Choi et al. (2018) that reaches near perfect accuracy on this task. Our model is composed of two separated modules for syntax and semantics. They are cooperatively trained with standard continuous and discrete optimization schemes. Our model does not require any linguistic structure for supervision and its recursive nature allows for out-of-domain generalization with little loss in performance. Additionally, our approach performs competitively on several natural language tasks, such as Natural Language Inference or Sentiment Analysis.
AUTHORS
Research Topics
Natural Language ProcessingRelated Publications
June 03, 2019
NLP
FAIRSEQ: A Fast, Extensible Toolkit for Sequence Modeling | Facebook AI Research
FAIRSEQ is an open-source sequence modeling toolkit that allows researchers and developers to train custom models for translation, summarization, language modeling, and other text generation tasks. The toolkit is based on PyTorch and supports…
Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, Michael Auli
June 03, 2019
June 02, 2019
NLP
Cooperative Learning of Disjoint Syntax and Semantics | Facebook AI Research
There has been considerable attention devoted to models that learn to jointly infer an expression’s syntactic structure and its semantics. Yet, Nangia and Bowman (2018) has recently shown that the current best systems fail to learn the correct…
Serhii Havrylov, Germán Kruszewski, Armand Joulin
June 02, 2019
October 30, 2018
NLP
Loss in Translation: Learning Bilingual Word Mapping with a Retrieval Criterion | Facebook AI Research
Continuous word representations learned separately on distinct languages can be aligned so that their words become comparable in a common space. Existing works typically solve a least-square regression problem to learn a rotation aligning a…
Armand Joulin, Piotr Bojanowski, Tomas Mikolov, Hervé Jégou, Edouard Grave
October 30, 2018
October 31, 2018
NLP
Understanding Back-Translation at Scale | Facebook AI Research
An effective method to improve neural machine translation with monolingual data is to augment the parallel training corpus with back-translations of target language sentences. This work broadens the understanding of back-translation and…
Sergey Edunov, Myle Ott, Michael Auli, David Grangier
October 31, 2018
