More Pythonic PyTorch 2 Released
Written by Alex Denham   
Thursday, 30 March 2023

PyTorch 2.0 has been released with fundamental changes to the way it works at the compiler level, faster performance, and support for Dynamic Shapes and Distributed.

PyTorch is an optimized tensor library for deep learning using GPUs and CPUs that until now has mainly been developed by Meta AI. It aims to offer a replacement for NumPy that makes use of the power of GPUs, while providing a deep learning research platform that provides maximum flexibility and speed.

pytorch

The new release includes a stable version of Accelerated Transformers (formerly called Better Transformers); and torch.compile, a feature that improves PyTorch performance and starts the move for parts of PyTorch from C++ back into Python. torch.compile is an optional and additive feature. Torch.compile makes use of several new technologies – TorchDynamo, AOTAutograd, PrimTorch and TorchInductor.

TorchDynamo captures PyTorch programs safely using Python Frame Evaluation Hooks; AOTAutograd overloads PyTorch’s autograd engine as a tracing autodiff for generating ahead-of-time backward traces.
PrimTorch simplifies around 2000 PyTorch operators into a closed set of around 250 primitive operators that developers can target to build a complete PyTorch backend. The developers say this substantially lowers the barrier of writing a PyTorch feature or backend. TorchInductor is a deep learning compiler that generates fast code for multiple accelerators and backends. For NVIDIA GPUs, it uses OpenAI Triton as a key building block.

TorchDynamo, AOTAutograd, PrimTorch and TorchInductor are written in Python and support dynamic shapes (the ability to send in Tensors of different sizes without inducing a recompilation).

Alongside the new release the team has released a series of beta updates to the PyTorch domain libraries, and separate libraries including TorchAudio, TorchVision, and TorchText. An update for TorchX is also being released as it moves to community supported mode.

PyTorch is available now on the PyTorch Foundation website and on GitHub.

pytorch

More Information

PyTorch Foundation

PyTorch On GitHub

Related Articles

PyTorch Joins Linux Foundation

PyTorch Adds New APIs

PyTorch Scholarship Challenge

PyTorch Developer Day Updates

PyTorch Adds TorchScript API

PyTorch 1.5 Updates C++ API

 

 

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Banner


A Turing Machine In LEGO
06/10/2024

A Turing Machine is one of those abstract ideas that is much easier to undertand if you can see it working. Now there's a proposal on the LEGO Ideas site for a Working Turing Machine. Lend it your sup [ ... ]



Raspberry Pi Beaten To It With Pico Plus 2
02/10/2024

RPi recently announced the Pico 2, but while we are all waiting for the more useful Pico 2W, with WiFi, the tiny Pimoroni outfit has produced something better - the Pico Plus 2 with WiFi. What is goin [ ... ]


More News

espbook

 

Comments




or email your comment to: comments@i-programmer.info

Last Updated ( Thursday, 30 March 2023 )