Google Open-Sources TorchTPU and Ships llm-d 1.0

Google has released TorchTPU, its native PyTorch stack for its Tensor Processing Unit (TPU) chips, as open source on GitHub, and launched version 1.0 of llm-d, an open-source system for running AI inference across large fleets of chips. Both were announced at PyTorch Conference 2026 on 20 October in a Google Cloud blog post on TorchTPU and a second on llm-d.

The move matters because PyTorch is the framework many AI developers already write in, and Google’s own chips have not been a native target for it. “With TorchTPU, TPUs join GPUs and CPUs as a native option for PyTorch,” wrote Mark Lohmeyer, Google’s VP and general manager of AI and computing infrastructure, and Bill Jia, VP of engineering.

In brief

Google has open-sourced TorchTPU, co-developed with Meta’s PyTorch team, and released llm-d 1.0, a Kubernetes-native stack for distributed inference co-founded with Red Hat, IBM and others.
It makes Google’s TPUs easier to use for teams whose code is written in PyTorch, widening the choice of hardware beyond GPUs.
How many PyTorch developers will actually move workloads to TPUs is unknown, and Google’s claims have not been independently tested.

What does TorchTPU actually do?

Google says TorchTPU, “co-developed with the PyTorch team at Meta”, lets developers run existing PyTorch workloads on TPUs “with minimal code changes”, using familiar APIs. Models can be debugged in eager mode, which runs code step by step, and then compiled with torch.compile for production.

The stack was already known. In April, Google’s Next ’26 infrastructure post said TorchTPU was “in preview with select customers”. What is new is the open-source release.

Meta, whose PyTorch team co-developed the stack, is the headline supporter. “We cannot be constrained by a single hardware architecture. TorchTPU is a massive step forward,” said Jana van Greunen, Meta’s director of PyTorch engineering, in Google’s post.

Who else is plugging in?

Google lined up partners across the software people already use to train and serve models. Hugging Face says that in its Transformers v5 library, deploying models to TPUs “is now as simple as passing device_map=”tpu””. Jeff Boudier, its VP of product, said “hardware diversity is essential to making open-source AI accessible to everyone”.

For serving models, Google built TorchTPU backends for two popular open-source engines. The vLLM backend was “developed in partnership with Inferact” and the SGLang backend “developed in partnership with RadixArk”, according to the blog.

Release	What it is	Built with
TorchTPU	Native PyTorch stack for TPUs	Meta’s PyTorch team
vLLM backend	Serving engine support on TPUs	Inferact
SGLang backend	Serving engine support on TPUs	RadixArk
Transformers v5	TPU deployment via one setting	Hugging Face
llm-d 1.0	Kubernetes stack for inference	Red Hat, IBM, others

Source: Google Cloud blog posts, 20 October 2026.

Google also said LiteRT, its on-device inference tool formerly known as TensorFlow Lite, is joining the PyTorch ecosystem, and that a TPU agent suite to help developers set up and tune PyTorch models on TPUs is in preview on GitHub.

Where does llm-d fit in?

TorchTPU gets a model running on a chip; llm-d is about running many copies of models across a cluster. Google describes it as “an open-source, Kubernetes-native stack for distributed LLM inference”, co-founded with Red Hat, IBM and others, and built with “a community of more than 500 contributors over 25 organizations”. It is a Cloud Native Computing Foundation (CNCF) Sandbox project.

In practice, llm-d sits in front of model servers such as vLLM and SGLang and decides where each request goes. Google says its router sends requests to the copy of a model that already holds the relevant context, and separates the compute-heavy and memory-heavy stages of generating an answer.

Version 1.0 adds multi-tenant flow control, so urgent and background requests can share hardware, as well as router high availability and stable v1 APIs, according to Google. On its Kubernetes service, customers can put GPUs and TPUs behind one llm-d endpoint. At the conference keynote, Google demonstrated 1,000 sandboxed agents hitting a single llm-d endpoint across a mixed fleet of TPUs and GPUs.

Our read

This is TSN’s analysis. The pattern is now clear: chipmakers are competing to be a native PyTorch target rather than asking developers to learn new tools, as AWS did with TorchNeuron for Trainium and IBM did with its Spyre compiler. Open-sourcing TorchTPU, with Meta’s public backing, removes one obvious excuse not to try TPUs. Whether developers switch will depend on results they can check for themselves, not on launch-day partner quotes.