RESEARCH

NLP

Finding Generalizable Evidence by Learning to Convince Q&A Models

November 4, 2019

Abstract

We propose a system that finds the strongest supporting evidence for a given answer to a question, using passage-based question-answering (QA) as a testbed. We train evidence agents to select the passage sentences that most convince a pretrained QA model of a given answer, if the QA model received those sentences instead of the full passage. Rather than finding evidence that convinces one model alone, we find that agents select evidence that generalizes; agent-chosen evidence increases the plausibility of the supported answer, as judged by other QA models and humans. Given its general nature, this approach improves QA in a robust manner: using agent-selected evidence (i) humans can correctly answer questions with only ∼20% of the full passage and (ii) QA models can generalize to longer passages and harder questions.

Download the Paper

Related Publications

October 08, 2016

COMPUTER VISION

NLP

Learning Visual Features from Large Weakly Supervised Data | Facebook AI Research

Convolutional networks trained on large supervised datasets produce visual features which form the basis for the state-of-the-art in many computer-vision problems. Further improvements of these visual features will likely require even larger…

Armand Joulin, Laurens van der Maaten, Allan Jabri, Nicolas Vasilache

October 08, 2016

September 15, 2019

COMPUTER VISION

NLP

Sequence-to-Sequence Speech Recognition with Time-Depth Separable Convolutions | Facebook AI Research

We propose a fully convolutional sequence-to-sequence encoder architecture with a simple and efficient decoder. Our model improves WER on LibriSpeech while being an order of magnitude more efficient than a strong RNN baseline. Key to our…

Awni Hannun, Ann Lee, Qiantong Xu, Ronan Collobert

September 15, 2019

September 10, 2019

NLP

Bridging the Gap Between Relevance Matching and Semantic Matching for Short Text Similarity Modeling | Facebook AI Research

A core problem of information retrieval (IR) is relevance matching, which is to rank documents by relevance to a user’s query. On the other hand, many NLP problems, such as question answering and paraphrase identification, can be considered…

Jinfeng Rao, Linqing Liu, Yi Tay, Wei Yang, Peng Shi, Jimmy Lin

September 10, 2019

June 16, 2019

NLP

On the Idiosyncrasies of the Mandarin Chinese Classifier System | Facebook AI Research

While idiosyncrasies of the Chinese classifier system have been a richly studied topic among linguists (Adams and Conklin, 1973; Erbaugh, 1986; Lakoff, 1986), not much work has been done to quantify them with statistical methods. In this paper,…

Shijia Liu, Hongyuan Mei, Adina Williams, Ryan Cotterell

June 16, 2019

Help Us Pioneer The Future of AI

We share our open source frameworks, tools, libraries, and models for everything from research exploration to large-scale production deployment.