PyTorch 2.15, released on 28 October, moves every training job that uses the nccl backend onto a rewritten version of it. Teams that hit problems can switch back by setting TORCH_DIST_USE_NCCL2=0 or by asking for the nccl-legacy backend by name, according to the release blog and the v2.15.0 release notes on GitHub.
NCCL is the library NVIDIA GPUs use to swap data during multi-GPU training, so the change reaches almost every multi-GPU PyTorch job on NVIDIA hardware. The release also drops Python 3.10, drops the older CUDA 12.6 and 13.0 wheels and adds new features for Apple, AMD, Intel and NVIDIA hardware. The project says it is made up of 3,117 commits from 583 contributors since PyTorch 2.14.
In brief
- The rewritten NCCL backend is now the default;
TORCH_DIST_USE_NCCL2=0ornccl-legacyrestores the old one. - Python 3.10, CUDA 12.6 and CUDA 13.0 wheels are gone, and MPS on Macs now needs macOS 15.
- Apple Silicon gains FP8, and PyTorch reports 1.32–1.34x faster MoE decoding there.
What to check before upgrading
The communication backend. PyTorch previewed the new backend, ported from its torchcomms project, as an opt-in called nccl2 in version 2.14. In 2.15 it backs every nccl process group. Communicators are now created as soon as the process group is, and a new flight recorder writes a report to disk when a collective operation hangs, so engineers can see what each GPU was doing at the time. The project labels the feature “API Unstable”.
Wheels and drivers. The default PyPI wheel now uses CUDA 13.2, with CUDA 13.4 wheels alongside, which add experimental support for NVIDIA’s Rubin GPUs. CUDA 12.6 and 13.0 wheels are no longer built. The release notes say CUDA 13 wheels need NVIDIA driver 580 or newer and a Turing-generation GPU or later, so machines with Maxwell, Pascal or Volta GPUs, or a CUDA 12 driver, should stay on 2.14.
Python and macOS. PyTorch 2.15 supports Python 3.11 to 3.15 and no longer publishes Python 3.10 wheels. Apple Silicon GPU acceleration (MPS) now needs macOS 15; on macOS 14, the notes say, MPS reports itself as unavailable.
FSDP2 gradients. For sharded training, a MixedPrecisionPolicy with reduce_dtype=None now reduces gradients in each parameter’s own data type rather than the parameter precision setting, a change the notes flag as backwards-incompatible.
New hardware features
For distributed jobs, torch.distributed gains one-sided transports, with NIXL as the first backend, that move data between peers without needing a process group. FSDP2 can keep precision-sensitive parameters such as MoE router gates and norm weights in fp32 while the rest of a model trains in bf16.
On NVIDIA Blackwell GPUs, PyTorch says its NVGEMM backend speeds up 4-bit NVFP4 decoding by cutting and overlapping kernel launches. On Apple Silicon, MPS adds the float8_e4m3fn FP8 type, _scaled_mm and a new grouped_mm; PyTorch says its decode path gives 1.32–1.34x higher MoE decode throughput, measured on an M3 Ultra.
AMD support adds FlyDSL kernels on gfx950 chips and preview Tensor Data Mover support for gfx1250 (MI450), along with ROCm 10.0 wheels built on AMD’s TheRock packages. Intel GPUs get a simpler way to write SYCL kernels and multi-GPU profiling. torch.compile now works on Python 3.15.
The upgrade risk sits in the defaults
Most of the new features in 2.15 are opt-in and marked unstable, so a team can ignore them until it needs them. The changes that will bite are the ones that happen without anyone asking. A multi-GPU job that ran on 2.14 will, on 2.15, run its communication through different code, and the first sign of a problem may be a hang or a slowdown deep into a long training run rather than an error at start-up.
That is why the rollback switch matters as much as the rewrite. PyTorch has given teams two ways back, and the flight recorder gives them evidence to file a bug instead of a guess. A sensible upgrade path is to run a short job on the new backend, compare it with the same job using TORCH_DIST_USE_NCCL2=0, and only then move long runs across.
The wheel changes are blunter. Clusters on older drivers or pre-Turing GPUs will not get 2.15 from pip at all, which is fine as long as installs are pinned, and confusing when they are not. The same goes for Python 3.10 environments, where pip simply finds no 2.15 wheel.
The speed-ups, meanwhile, are PyTorch’s own measurements on specific hardware and models. The Apple Silicon figure, for example, comes from MoE decoding on one high-end Mac chip, and users should test their own workloads before counting on it.
A wider hardware race
The release comes as chipmakers compete to be a native PyTorch target. Google open-sourced TorchTPU this month, AWS released TorchNeuron for Trainium, and IBM moved its Spyre compiler onto PyTorch’s Inductor. AMD, which contributed much of the ROCm work in 2.15, said on 27 October that its Helios racks are being tested by Meta.
PyTorch will take questions on the release in a webinar on Thursday 5 November at 11am Pacific time (19:00 GMT).
Sources
- PyTorch, “PyTorch 2.15 Release Blog”, 28 October 2026
- pytorch/pytorch on GitHub, “PyTorch 2.15.0 Release” notes (v2.15.0), 28 October 2026
Related stories
- Google Open-Sources TorchTPU, Making Its TPUs a Native Option for PyTorch
- TorchNeuron Brings a Native PyTorch Backend to AWS Trainium
- IBM Moves Its Spyre AI Chip’s Compiler Onto PyTorch’s Open-Source Inductor
- AMD to Invest $1bn in Meta Share Sale as Helios Racks Start Testing
Follow @tsnmediahub on X for breaking AI news as it happens.
Follow on X
