At our first-ever Glow Summit, we brought together leading hardware organizations to discuss the future of Glow, a machine learning (ML) compiler for neural network hardware accelerators. Glow is being developed to drive hardware accelerators across different hardware companies.
During the event, Habana and Facebook announced the first experimental back end for the Glow compiler and runtime to target Habana’s existing Goya inference accelerator. The back-end code is now available as part of the Glow repository, and Habana customers can use PyTorch and Glow to drive their hardware accelerators.
We presented our roadmap for the next development iterations, including Glow runtime, the software component that queues and executes inference requests across multiple acceleration cards on server-grade machines. We also shared the current Caffe2 integration and presented our plans to support the non-Caffe interface of PyTorch 1.0. ML experts presented their work on quantization, the process of converting neural networks from floating point arithmetic to integer arithmetic, and PyTorch engineers presented PyText and language-based models.
Hardware accelerators for ML are designed to solve a range of distinct problems. Some focus on inference, while others focus on training. Glow is designed to target a wide range of hardware accelerators. The hardware-independent parts of the compiler focus on math-related optimizations that are not tied to a specific hardware model. In addition to the target-independent optimizations, Glow contains a number of utilities and building blocks that can be configured to support multiple hardware targets. For example, the memory allocator of the compiler is used to generate efficient code for a variety of hardware accelerators, each with a different memory configuration. These capabilities include a powerful linear algebra optimizer, an extensive test suite, and a CPU-based reference implementation for testing the accuracy of hardware accelerators, as well as the memory allocator, an instruction scheduler, etc.
Hardware partners that use Glow can reduce the time it takes to bring their products to market. Relying on the existing optimizations and capabilities reduces development time, and the extensive test suite improves a hardware provider’s confidence in the accuracy of the compiler and its conformance to the PyTorch specification.
The Glow and PyTorch teams are working on enabling the first generation of inference hardware accelerators with industry partners. The work involves building new hardware back ends for each accelerator target, and verifying and optimizing the generated code. We also plan to improve the interoperability with PyTorch as PyTorch continues to evolve.
We are excited to be working on Glow in the open on GitHub. We look forward to improving Glow with feedback from the community and working with hardware makers on supporting new hardware accelerators.
Get started at the Glow GitHub project.
Technical Program Manager, Facebook
Engineering Manager, Facebook