Myle Ott

Myle Ott is a Research Engineer in Facebook’s AI Research group (FAIR), focusing on projects in dialogue, machine translation and text generation. Prior to Facebook, Myle completed his PhD at Cornell University in their NLP group, where his research explored approaches for identifying deception and sentiment in social media text.

Myle's Publications

October 31, 2019

RESEARCH

ML APPLICATIONS

The FLORES Evaluation Datasets for Low-Resource Machine Translation: Nepali–English and Sinhala–English

For machine translation, a vast majority of language pairs in the world are considered low-resource because they have little parallel data available. Besides the technical challenges of learning with limited supervision, it is difficult to evaluate methods…

Francisco (Paco) Guzmán, Peng-Jen Chen, Myle Ott, Juan Pino, Guillaume Lample, Philipp Koehn, Vishrav Chaudhary, Marc'Aurelio Ranzato

October 31, 2019

June 10, 2019

RESEARCH

ML APPLICATIONS

Mixture Models for Diverse Machine Translation: Tricks of the Trade

Mixture models trained via EM are among the simplest, most widely used and well understood latent variable models in the machine learning literature. Surprisingly, these models have been hardly explored in text generation applications such as…

Tianxiao Shen, Myle Ott, Michael Auli, Marc'Aurelio Ranzato

June 10, 2019

June 03, 2019

RESEARCH

ML APPLICATIONS

FAIRSEQ: A Fast, Extensible Toolkit for Sequence Modeling

FAIRSEQ is an open-source sequence modeling toolkit that allows researchers and developers to train custom models for translation, summarization, language modeling, and other text generation tasks. The toolkit is based on PyTorch and supports distributed training across multiple GPUs and machines. We also support fast mixed-precision training and inference on modern…

Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, Michael Auli

June 03, 2019

October 30, 2018

RESEARCH

ML APPLICATIONS

Scaling Neural Machine Translation

Sequence to sequence learning models still require several days to reach state of the art performance on large benchmark datasets using a single machine. This paper shows that reduced precision and large batch training can speedup training by nearly 5x on a single 8-GPU machine with careful tuning and implementation.1 On WMT’14 English-German…

Myle Ott, Sergey Edunov, David Grangier, Michael Auli

October 30, 2018

October 31, 2018

RESEARCH

ML APPLICATIONS

Understanding Back-Translation at Scale

An effective method to improve neural machine translation with monolingual data is to augment the parallel training corpus with back-translations of target language sentences. This work broadens the understanding of back-translation and investigates a number of methods to generate synthetic source sentences. We find that in all but resource poor settings back-translations…

Sergey Edunov, Myle Ott, Michael Auli, David Grangier

October 31, 2018

October 31, 2018

RESEARCH

ML APPLICATIONS

Phrase-Based & Neural Unsupervised Machine Translation

Machine translation systems achieve near human-level performance on some languages, yet their effectiveness strongly relies on the availability of large amounts of bitexts, which hinders their applicability to the majority of language pairs. This work investigates how to learn to translate when having access to only large monolingual corpora in each language. We propose…

Guillaume Lample, Myle Ott, Alexis Conneau, Ludovic Denoyer, Marc'Aurelio Ranzato

October 31, 2018

July 13, 2018

RESEARCH

ML APPLICATIONS

Analyzing Uncertainty in Neural Machine Translation

Machine translation is a popular test bed for research in neural sequence-to-sequence models but despite much recent research, there is still a lack of understanding of these models. Practitioners report performance degradation with large beams, the under-estimation of rare words and a lack of diversity in the final translations. Our study relates some of these issues…

Myle Ott, Michael Auli, David Grangier, Marc'Aurelio Ranzato

July 13, 2018

June 01, 2018

RESEARCH

ML APPLICATIONS

Classical Structured Prediction Losses for Sequence to Sequence Learning

There has been much recent work on training neural attention models at the sequence-level using either reinforcement learning-style methods or by optimizing the beam. In this paper, we survey a range of classical objective functions that have been widely used to train linear models for structured prediction and apply them to neural sequence to sequence models. Our…

Sergey Edunov, Myle Ott, Michael Auli, David Grangier, Marc'Aurelio Ranzato

June 01, 2018