CONFIRMED: This is based on a post on the official PyTorch blog by IBM engineers. All benchmark and code-size numbers are IBM’s own figures from its example tests.
IBM has rebuilt the compiler for Spyre, its AI accelerator chip, on top of Inductor, the open-source compiler built into PyTorch [1].
PyTorch is the most widely used framework for building AI models. A compiler turns a model’s operations into instructions a chip can run. Spyre’s PyTorch software, called torch-spyre, used to rely on a separate compiler built from scratch called sendnn; it now uses “backend=inductor” [1].
What is Spyre?
IBM describes Spyre as a “75-watt, PCIe-attached accelerator” for running AI models in businesses, built for power efficiency [1]. It has 32 small cores linked in a ring, each with its own 2 MB local memory (“scratchpad”), and moves data in fixed chunks called sticks of 128 bytes [1].
That design is different from a graphics card (GPU), which is what Inductor was originally shaped around. IBM says most of Inductor “just worked”, but it had to add substantial extensions for how Spyre moves data, manages on-chip memory and splits work across cores [1].
What IBM says it gained (IBM’S FIGURES)
- Less code. The old sendnn stack was roughly 273,000 lines of C++ and Python. The new Inductor extension is roughly 38,600 lines of Python, “about 7x fewer lines” [1]. IBM notes this is not a like-for-like comparison, because the new code leans on Inductor’s own machinery.
- Faster kernels in examples. In a softmax example, better use of on-chip memory cut median kernel time “from 32.51 µs to 23.67 µs — a 27% reduction” [1].
- Layout matters. On a batched matrix multiplication, changing how tensors are laid out cut kernel time by 29-69% across a range of sizes [1].
- Better coordination. On one MLP-style kernel, kernel time fell by around 17%, which IBM says is roughly a 2x improvement over the same kernel compiled with sendnn [1].
IBM says the fuller performance picture “is still being assembled” [1].
What comes next
IBM says it is working with the PyTorch project to formalise the extension points it relied on, so other chipmakers building “out-of-tree” hardware backends can use them too [1]. A second, proprietary IBM compiler called DeepTools still turns the output into the final instructions for the chip [1].
What this means
- Open tools for new chips. Building on PyTorch’s own compiler can save a chip designer a lot of work.
- The numbers are early. They come from IBM’s example tests, not full model benchmarks.
- Others may benefit. If the extension points are formalised upstream, other non-GPU accelerators could plug in more easily.
Sources
- PyTorch blog (Dave Grove and IBM Spyre team), “Extending Inductor for the IBM Spyre Accelerator”, 13 October 2026. https://pytorch.org/blog/extending-inductor-for-the-ibm-spyre-accelerator/
