Rohan Varma

Rohan is an engineer on the PyTorch team in Facebook AI, where he primarily works on distributed training for ranking, recommendation, and content understanding use cases. His research interests lie in exploring various hybrid parallelism techniques to enable efficient training of large-scale models and developing efficient algorithms for cross-machine communication.

Rohan's Work

Rohan's Publications

August 31, 2020

SYSTEMS RESEARCH

PyTorch Distributed: Experiences on Accelerating Data Parallel Training

This paper presents the design, implementation, and evaluation of the PyTorch distributed data parallel module. PyTorch is a widely-adopted scientific computing package used in deep learning research and applications…

Shen li, Yanli Zao, Rohan Varma, Omkar Salpekar, Pieter Noordhuis, Teng Li, Adam Paszke, Jeff Smith, Brian Vaughan, Pritam Damania, Soumith Chintala

August 31, 2020