Amazon Web Services has released Infinity, an agent runtime that suspends an AI agent every time it calls a tool and wakes it only when the result comes back, under the Apache 2.0 open-source licence. AWS calls it “an early research release” and says it is sharing the code to get feedback.
Infinity is part of Hydro, a project that AWS describes as “a collaborative effort between AWS and UC Berkeley”. It was announced on 23 October in a post on the AWS Open Source Blog by Shadaj Laddad, research lead for the Hydro team at AWS, and the code, written in Rust, is on GitHub.
It arrives a day after Google Cloud unveiled Agent Compute Engine, which gives each agent its own microVM and pauses idle ones, and as Amazon chief executive Andy Jassy says AWS capacity is sold out for 2026.
Infinity treats every tool call like an await in async code
The starting point is that agents spend much of their time waiting. Laddad gives the examples of a coding agent whose build runs for twenty minutes and an assistant that waits until the next morning for an approval. In most agent harnesses, he writes, the agent “holds a process and its memory” while it waits and cannot respond to anything else that happens in the meantime.
Infinity borrows from the way programming languages suspend asynchronous functions. When an agent calls a tool, the runtime dispatches the call, saves the conversation and suspends the agent. The conversation history holds everything needed to carry on, so the runtime can drop all in-memory state and any process with access to the store can resume the agent later.
Each agent runs as a series of short slices: load the history, run one model completion, save the output and dispatch at most one tool call. Results, user messages and other events arrive on a shared input queue. Messages for one conversation are handled in order, while different conversations run in parallel.
The serverless version keeps nothing running while an agent waits
Because no slice waits on anything, Infinity can run on serverless platforms, where a function starts for each piece of work and exits when it is done. AWS’s reference deployment uses an SQS FIFO queue as the input queue, Lambda to run each step, Aurora DSQL to store conversation histories and DynamoDB to track which messages have been processed. Timers use SQS delivery delays and EventBridge Scheduler, and the repository includes CDK templates to set them up.
Laddad says that means “an agent waiting on a three-day CI pipeline uses no compute until the pipeline reports its result.”
The README also describes a Reactive Agent Protocol for serving tools over a network without holding a connection open, and says “existing MCP servers run unchanged through a compatibility layer”. A coding agent, Infinity Code, is built on the runtime.
Synthetic tool calls carry events and messages between agents
Model APIs expect every tool call to get exactly one result, which leaves no clean way to tell an agent that something new has happened, such as a pull request being opened. Infinity’s answer is to insert a “synthetic” tool call that the model never made, named receive_event__injected, followed by a result containing the event.
The same mechanism links agents. A parent creates a child with a built-in spawn_thread tool, carries on with other work, and receives the child’s reports later as synthetic events tied to that call.
AWS’s 75,000-agent figure comes from a simulated model
To measure memory, AWS ran a benchmark with a simulated model rather than a real one, using in-memory stores on a single thread of an AMD EPYC 9R14 server processor. It launched 80,000 agents, each running twenty turns, for 3.2 million completions in total. Memory grew at about 100 KB per idle agent, which AWS says means “about 75,000 agents fit in 8 GB, which is the memory of a Raspberry Pi 5”. The Pi is a comparison of memory size; the test did not run on one. The figures are AWS’s own and have not been independently checked.
Laddad lists open questions, among them how models trained mostly on step-by-step tool use cope with events arriving out of order, and how a runtime can confirm that an incoming event is genuine.
Infinity bets that waiting, not thinking, is the expensive part
Most of the recent effort to make agents cheaper to run has gone into the machines that host them. Google’s microVM service is the latest example; Infinity starts from the other end, asking how little an idle agent needs to keep at all. Its answer, a conversation history and nothing else, is simple enough to explain in a sentence, which is part of its appeal.
The serverless design is the most practical piece. An agent that costs nothing while a long build or an approval is pending changes the economics of agents that run for days, at a time when AWS says its capacity for this year is sold out.
But the headline number needs care. It measures memory, with a simulated model, on a server, and real agents spend money and time on model calls that the test left out. AWS is also clear that this is an early research release, not a finished product, and the hard questions it lists, from event authenticity to how models handle interleaved events, are unsolved. As with the Agent Skills format Anthropic gave to the Agentic AI Foundation, and AWS’s earlier agent toolkit work, the test is whether other developers pick it up. For now, Infinity’s 75,000 idle agents are a measurement of what waiting costs, taken without a real model in the loop.
Sources
- AWS Open Source Blog: “Introducing Infinity: a runtime for extreme agent concurrency”, Shadaj Laddad, 23 October 2026
- GitHub: hydro-project/infinity (Apache 2.0)
Related stories
- Google Cloud Unveils Agent Compute Engine to Run AI Agents in MicroVMs
- Anthropic Gives Agent Skills to AAIF, Saying 70+ Platforms Use It
Follow @tsnmediahub on X for breaking AI news as it happens.
Follow on X
