June 29, 2020
We introduce a new large-scale NLI benchmark dataset, collected via an iterative, adversarial human-and-model-in-the-loop procedure. We show that training models on this new dataset leads to state-of-the-art performance on a variety of popular NLI benchmarks, while posing a more difficult challenge with its new test set. Our analysis sheds light on the shortcomings of current state-of-the-art models, and shows that non-expert annotators are successful at finding their weaknesses. The data collection method can be applied in a never-ending learning scenario, becoming a moving target for NLU, rather than a static benchmark that will quickly saturate.
Publisher
ACL
Research Topics
December 15, 2021
Akash Bharadwaj, Graham Cormode
December 15, 2021
December 06, 2021
Hongyu Gong, Yun Tang, Juan Miguel Pino, Xian Li
December 06, 2021
November 16, 2021
Rahma Chaabouni, Roberto Dessì, Evgeny Kharitonov
November 16, 2021
November 08, 2021
Baptiste Rozière, Marie-Anne Lachaux, Marc Szafraniec, Guillaume Lample
November 08, 2021