Tensormesh, the company behind the open-source tool LMCache, has launched Tensormesh Platform, software that stores and reuses a large language model’s “KV cache” across a company’s servers. Tensormesh announced it in a press release on 14 October 2026 and says it is available now.
Tensormesh calls the product “the first enterprise-ready KV cache management offering that works with any LLM, inference orchestration software, and storage vendor”, and describes LMCache as “the world’s number one open-source KV caching technology”. Both are the company’s own claims.
What is a KV cache?
When a large language model (LLM) reads a prompt, it turns every token, roughly every word or part of a word, into two sets of numbers called keys and values. The model uses these to work out how each part of the text relates to the rest. Storing them is called the KV (key-value) cache.
Building that cache is the expensive first step before the model can write anything. The longer the prompt, the more work it takes. The delay before the first word appears is known as time-to-first-token.
Why reuse it?
Many requests repeat the same material: a long system prompt, a shared document, or the earlier turns of a chat. Without reuse, the model rebuilds the keys and values for that text every time.
“LLM workloads recompute the same context endlessly. That wastes GPU time and inflates time-to-first-token,” said Junchen Jiang, Tensormesh’s chief executive. If the cache is saved and reused, the model can skip that repeated work, so answers start sooner and each GPU, the expensive chips that run AI, can handle more requests.
What the platform does
According to Tensormesh, the software:
- stores the cache in tiers, moving it automatically between the GPU’s own fast memory (HBM), ordinary server memory and storage;
- reuses the cache for the start of a prompt (prefix caching) and, through a feature called CacheBlend, for matching chunks anywhere in a prompt, which Tensormesh says suits RAG, where documents are pulled into prompts;
- lets servers fetch cache directly from one another (peer-to-peer transfer);
- adds the controls companies need to run it in production: security, monitoring, high availability, rules for how cache is stored, shared and removed, and separation between teams or customers; and
- lets operators pin, look up, move, compress and clear cache entries through APIs.
It runs on Kubernetes in a company’s own data centres.
Who backs it
Tensormesh was founded by faculty, PhD researchers and alumni from the University of Chicago, UC Berkeley and Carnegie Mellon. Jiang is a University of Chicago faculty member and co-creator of LMCache. Its backers include NVentures, AMD Ventures and CoreWeave. The release gives two different totals for the money raised, $28.5 million and $24.5 million.
The release includes supporting quotes from Nebius and SUSE. Patrick Lundquist, a senior software engineer at Nebius, said: “Tensormesh delivered strong results in our P2P KV cache transfer and resiliency benchmarks across multiple models.” Tensormesh did not publish performance figures for the platform.
“GPU utilization, not GPU ownership, determines the economics of AI,” said Paul Nashawaty, principal analyst with theCUBE and ECI Research, in the release.
What this means
- Inference cost is the new battleground. Getting more answers out of each GPU matters as AI use grows.
- Caching is moving from research to product. Tensormesh is turning its open-source tool into paid enterprise software.
- No numbers yet. The company has not published speed or cost savings for the platform, so the gains are untested in public.
Sources
- Tensormesh, “Tensormesh Launches Tensormesh Platform to Cut LLM Inference Costs by Reusing KV Cache at Enterprise Scale”, 14 October 2026. https://tensormesh.ai/newsroom/tensormesh-platform-launch-llm-inference-kv-cache/
