PyTorch has been updated with improved support for FFTs, better distributed model training, new APIs, library updates, and support for ways to improve and scale your code for performance at both inference and training time.

PyTorch is an optimized tensor library for deep learning using GPUs and CPUs. It aims to offer a replacement for NumPy that makes use of the power of GPUs, while providing a deep learning research platform that provides maximum flexibility and speed.

Possibly the most important improvement to the new version is the inclusion of the torch.fft module. This module implements the same functions as NumPy’s np.fft module, but with support for hardware acceleration and autograd. Alongside this comes improved support for fast Fourier transforms (FFT) in the Torch.fft library to add support for 2D FFT functions, alongside the ability to use new FFT operators in stft, added helper functions and a fuzzing benchmark.

Other API improvements include new Linear Algebra functions, added support for autograd for complex tensors, and updates to improve performance for calculating hessians and jacobians.

The developers say there have been "significant updates and improvements to distributed training". These updates improve NCCL reliability, add support for Pipeline parallelism and RPC profiling; and also add support for communication hooks adding gradient compression.

Support has been added for doing python to python functional transformations via torch.fx, adding the ability to set up transformations where you can feed in a Module instance and get a transformed Module instance out of it.

A wide range of functions have been updated, mainly to improve NumPy compatibility. The torch.linalg module, modeled after NumPy’s np.linalg module, brings NumPy-style support for common linear algebra operations including Cholesky decompositions, determinants, eigenvalues and many others. 

Alongside the new version, the team is also releasing major updates to PyTorch libraries including TorchCSPRNG, TorchVision, TorchText and TorchAudio.

More Information

PyTorch Website

PyTorch On GitHub

Related Articles

PyTorch Adds New APIs

PyTorch Scholarship Challenge

PyTorch Developer Day Updates

PyTorch Adds TorchScript API

PyTorch 1.5 Updates C++ API 

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Comments

Make a Comment or View Existing Comments Using Disqus

or email your comment to: comments@i-programmer.info