Rime Launches Prism Speech-to-Speech Model

Rime, a voice AI company that says its voices answer calls for some of the world’s leading brands, has launched Prism, a speech-to-speech model that writes out what it is about to say before it says it. Lily Clifford announced the model in a launch post on 28 October, saying it “responds to user audio in about 500ms”, with “tool calls happening in parallel to speech, and an inspectable text layer to run guardrails over before the model speaks.”

The launch is a reversal for Clifford. “Eighteen months ago I said, very publicly and very confidently, that speech-to-speech models were a decade out from real use,” she writes. Her argument now is that businesses no longer have to choose between models that hear tone of voice and pipelines they can control. That pitch lands in a crowded month, after SpaceXSI’s Grok Voice 3.0, ElevenLabs’ Eleven Realtime and Mistral’s Voxtral 3.

In brief

Rime has launched Prism, a speech-to-speech model that produces a text version of each reply that developers can check before it is spoken.
On Rime’s own run of Artificial Analysis’s τ-Voice test, Prism scores 56.6 and starts speaking in 0.57 seconds; the figures are Rime’s, not a published leaderboard entry.
Prism is English only, and self-serve fine-tuning is still being built with beta customers.

The text layer is the point

Most voice agents today use a cascade: speech-to-text, then a language model, then text-to-speech. That is easy to control, because every word passes through text, but Clifford says it strips out “hesitation, anger, laughter, or the dog barking in the background”. The big labs’ fused speech models keep that audio, she writes, but “you can’t inspect what it’s about to say, you can’t use it in a composable system, and you can’t own it.”

Prism is Rime’s attempt at both. It is trained end to end to listen, write what it will say and keep hearing the caller while it speaks. “In Prism the text is a checkpoint,” Clifford writes. Rules can read every word before it becomes speech, and the same layer handles pronunciation of names and account terms and the management of tool calls.

The developer documentation describes one WebSocket connection doing the work of speech-to-text, a language model and text-to-speech. Prism decides when the caller has finished, stops when interrupted and sends text with every spoken reply. A demo in the docs shows it handling a lost debit card: finding the card, freezing it, reading recent charges and filing a dispute.

Benchmarks Rime ran itself

Rime says it “put Prism on the same harness Artificial Analysis uses for its speech-to-speech leaderboard”, with no reasoning step before speaking. On τ-Voice, a test of simulated customer-service calls in airline, retail and telecom, Prism scores 56.6. Rime says that is 17 to 30 points ahead of every OpenAI and Google realtime model that does not reason first, and ahead of GPT-Realtime-2.1 even with its reasoning turned up. In telecom, Rime reports 82.5 against 45.6 for OpenAI’s GPT-Live-1, the model behind ChatGPT’s Live voice.

Rime puts Prism’s wait before speaking at 0.57 seconds, “about half the wait of GPT-Live-1, GPT-Realtime-2.1 and Gemini 3.8 Live”. The two models that beat it on τ-Voice reason before answering and take 1.24 to 1.35 seconds to start talking, it says. Prism did not appear on Artificial Analysis’s public leaderboard as of 28 October.

Rime also reports a customer trial. It trained a version of Prism on 10 hours of calls from Revmo, which runs an ordering agent on OpenAI. On Revmo’s calls, Rime says, 94% of orders were placed only after the caller agreed, compared with 57% for OpenAI.

Fine-tuning is the business case, and it is not self-serve yet

Clifford calls Prism “the first speech-to-speech model anyone can fine-tune”. The same post says Rime is building self-serve reinforcement learning and supervised fine-tuning APIs “with our beta customers”, so general access is not yet available. The docs say only that “enterprises can fine-tune Prism to their calls”.

There is a clear logic to the bet. Clifford argues that chain-of-thought reasoning “blows the latency budget of a live call”, so the only way to get a better voice agent in a given domain is to own its weights. That is a direct challenge to the big labs, whose realtime models customers can use but not retrain. The Revmo result is the best evidence Rime offers. It is a single customer, measured by Rime, on one behaviour: asking before acting. Buyers will want the same test on their own calls.

The comparison also has gaps. Rime’s figures name only OpenAI and Google models. SpaceXSI’s own published figures put Grok Voice 3.0 at 71.6% on τ-Voice at 0.81 seconds, and ElevenLabs reports 0.57 seconds to start speaking for Eleven Realtime v1 Max. Those runs differ in date and set-up, so they cannot be ranked against Prism directly. Rime’s latency edge is therefore against its chosen rivals, not the whole field. An independent listing on Artificial Analysis would settle more than any vendor chart.

What developers get on day one

The docs set out the limits plainly. Prism speaks English only, with four voices of its own; Rime’s text-to-speech voices are not compatible. Audio is 16 kHz in and 24 kHz out, and phone calls need a bridge because Prism does not accept the standard mu-law telephone format. The endpoint is listed in US East (us-east-2), and no European region is given.

SDKs for Python, Node.js, Go and Rust are in alpha versions, with quickstarts for LiveKit and Pipecat. Prism uses the event names of OpenAI’s Realtime beta interface, and a migration guide says most existing agent code stays as it is. A conversation is capped at 65,536 tokens of context, a caller’s turn ends after 55 seconds and sessions close after five minutes idle. Prism does not offer text-only replies, temperature settings or a choice of language model.

Prism joins a busy field of voice models that also includes Amazon’s Nova 2.5 Sonic and Block’s Dessa. Clifford’s own test is public: she predicts that in 18 months, cascade-based agents “will look like pale, ghastly imitations” of agents built on Prism.