Sergey Edunov

Sergey is a Research Engineering Manager at Facebook AI Research. He works on natural language processing and speech. Previously he worked on distributed graph analytics and was a part of the Apache Giraph team. He earned a MS degree in Physics in 2005 in the Moscow Institute of Physics and Technology.

Sergey's Publications

November 11, 2020

RESEARCH

NLP

Beyond English-Centric Multilingual Machine Translation

Existing work in translation demonstrated the potential of massively multilingual machine translation by training a single model able to translate between any pair of languages. However, much of this work is English-Centric by training only on data which was translated from or to English.

Angela Fan, Shruti Bhosale, Holger Schwenk, Zhiyi Ma, Ahmed El-Kishky, Siddharth Goyal, Mandeep Baines, Onur Celebi, Guillaume Wenzek, Vishrav Chaudhary, Naman Goyal, Tom Birch, Vitaliy Liptchinsky, Sergey Edunov, Edouard Grave, Michael Auli, Armand Joulin

November 11, 2020

November 11, 2020

RESEARCH

NLP

On The Evaluation of Machine Translation Systems Trained With Back-Translation

Back-translation is a widely used data augmentation technique which leverages target monolingual data. However, its effectiveness has been challenged since automatic…

Sergey Edunov, Myle Ott, Marc’Aurelio Ranzato, Michael Auli

November 11, 2020

November 11, 2020

RESEARCH

Classical Structured Prediction Losses for Sequence to Sequence Learning

There has been much recent work on training neural attention models at the sequence-level using either reinforcement learning-style methods or by optimizing the beam. In this paper, we survey a range of classical objective functions that have…

Sergey Edunov, Myle Ott, Michael Auli, David Grangier, Marc'Aurelio Ranzato

November 11, 2020

November 11, 2020

RESEARCH

Scaling Neural Machine Translation

Sequence to sequence learning models still require several days to reach state of the art performance on large benchmark datasets using a single machine. This paper shows that reduced precision and large batch training can speedup training by nearly 5x on a single 8-GPU machine with careful tuning and implementation…

Myle Ott, Sergey Edunov, David Grangier, Michael Auli

November 11, 2020

November 11, 2020

RESEARCH

NLP

Understanding Back-Translation at Scale

An effective method to improve neural machine translation with monolingual data is to augment the parallel training corpus with back-translations of target language sentences. This work broadens the understanding of back-translation and…

Sergey Edunov, Myle Ott, Michael Auli, David Grangier

November 11, 2020

November 11, 2020

RESEARCH

NLP

FAIRSEQ: A Fast, Extensible Toolkit for Sequence Modeling

FAIRSEQ is an open-source sequence modeling toolkit that allows researchers and developers to train custom models for translation, summarization, language modeling, and other text generation tasks. The toolkit is based on PyTorch and supports…

Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, Michael Auli

November 11, 2020

November 11, 2020

RESEARCH

NLP

Cloze-driven Pretraining of Self-attention Networks

We present a new approach for pretraining a bi-directional transformer model that provides significant performance gains across a variety of language understanding problems. Our model solves a cloze-style word reconstruction task, where each…

Alexei Baevski, Sergey Edunov, Yinhan Liu, Luke Zettlemoyer, Michael Auli

November 11, 2020