PyTorch 2.0 has been released with fundamental changes to the way it works at the compiler level, faster performance, and support for Dynamic Shapes and Distributed.

PyTorch is an optimized tensor library for deep learning using GPUs and CPUs that until now has mainly been developed by Meta AI. It aims to offer a replacement for NumPy that makes use of the power of GPUs, while providing a deep learning research platform that provides maximum flexibility and speed.

The new release includes a stable version of Accelerated Transformers (formerly called Better Transformers); and torch.compile, a feature that improves PyTorch performance and starts the move for parts of PyTorch from C++ back into Python. torch.compile is an optional and additive feature. Torch.compile makes use of several new technologies – TorchDynamo, AOTAutograd, PrimTorch and TorchInductor.

TorchDynamo captures PyTorch programs safely using Python Frame Evaluation Hooks; AOTAutograd overloads PyTorch’s autograd engine as a tracing autodiff for generating ahead-of-time backward traces.
PrimTorch simplifies around 2000 PyTorch operators into a closed set of around 250 primitive operators that developers can target to build a complete PyTorch backend. The developers say this substantially lowers the barrier of writing a PyTorch feature or backend. TorchInductor is a deep learning compiler that generates fast code for multiple accelerators and backends. For NVIDIA GPUs, it uses OpenAI Triton as a key building block.

TorchDynamo, AOTAutograd, PrimTorch and TorchInductor are written in Python and support dynamic shapes (the ability to send in Tensors of different sizes without inducing a recompilation).

Alongside the new release the team has released a series of beta updates to the PyTorch domain libraries, and separate libraries including TorchAudio, TorchVision, and TorchText. An update for TorchX is also being released as it moves to community supported mode.

PyTorch is available now on the PyTorch Foundation website and on GitHub.

More Information

PyTorch Foundation

PyTorch On GitHub

Related Articles

PyTorch Joins Linux Foundation

PyTorch Adds New APIs

PyTorch Scholarship Challenge

PyTorch Developer Day Updates

PyTorch Adds TorchScript API

PyTorch 1.5 Updates C++ API

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Comments

Make a Comment or View Existing Comments Using Disqus

or email your comment to: comments@i-programmer.info