NVIDIA Nemotron 3.5 Bolt E4B: A 4GB Open Model

NVIDIA released Nemotron 3.5 Bolt E4B on 23 October, an open small language model that it says can run conversational AI agents in as little as about 4GB of memory. The model is built on Google’s Gemma 4 E4B and then retrained by NVIDIA, and it is aimed at voice assistants, game characters and robots that need to answer quickly on local hardware rather than in the cloud.

NVIDIA’s research blog post describes it as “our latest open small language model optimized for speed and intelligence per GB in conversational agents”. It is a different model from the much larger Nemotron 3.5 Bolt 74B, which NVIDIA set out earlier this month for its RTX Spark laptops. The new model shares the Bolt name and the focus on local agents, but it is sized for far smaller devices.

In brief

What’s new: An open model from NVIDIA, retrained from Gemma 4 E4B, for local voice, device and game-character agents.
Why it matters: NVIDIA says it fits in about 4GB, so it can run on a mid-range graphics card or a small Jetson board.
What’s unknown: How it performs in independent tests; every figure so far comes from NVIDIA.

A Gemma 4 base with NVIDIA’s training on top

According to the Hugging Face model card, the model has 8 billion parameters in total, of which 4 billion are “effective”, and is published in a compressed 4-bit format known as Q4_K_M. It takes text in and gives text out, with a context window of up to 64,000 tokens, which NVIDIA says leaves room for conversation history, instructions, tool definitions and retrieved information.

NVIDIA says it supports 35 languages, on a base model that was pre-trained on more than 140. The card lists English, Spanish, French, German, Italian and Japanese among them. The model is released under the OpenMDW 1.1 licence and, the card says, is “ready for commercial or non-commercial use”. The underlying Gemma 4 model is under Apache 2.0.

NVIDIA says it added several features for agents. The model can call tools over several turns of a conversation, switch into a conversational mode, and reason step by step with an option to cap how much “thinking” it does so that replies stay fast. NVIDIA trained it with reinforcement learning using its NeMo-RL and NeMo-Gym tools.

Where NVIDIA wants it to run

The company names GeForce RTX graphics cards, Jetson boards for robots and embedded devices, and its RTX Spark and DGX Spark machines as targets. Its demonstrations include a Reachy Mini robot running on a Jetson Orin Nano 8GB and a blacksmith trader character in a game.

NVIDIA’s own speed tests, run with the llama.cpp software and a single user, show the model using between 3.1GB and 5.0GB of graphics memory depending on the machine and how much earlier conversation it holds. On an RTX 5070 12GB card it reports 133.4 tokens a second and a first response in 548 milliseconds. On a Jetson Orin Nano 8GB, the figure falls to 15.5 tokens a second, with nearly six seconds before the first token. NVIDIA adds that “results may vary”.

The model is available through Hugging Face, Ollama, LM Studio, Unsloth, Canonical and NVIDIA’s Jetson AI Lab, and through its In-Game Inferencing SDK for game developers. NVIDIA has also released a separate “DFlash” drafter model that it says gives “up to 21% higher throughput” than the drafter shipped with Gemma 4 E4B.

NVIDIA’s benchmarks put Qwen ahead on several tests

NVIDIA says the model scores roughly 40% higher on average than its previous small model, Nemotron Nano 4B, and roughly 10% higher than other peers it evaluated, while using up to 80% fewer tokens when reasoning is switched on. All of these numbers come from NVIDIA’s own testing, and two of the benchmarks it uses, Brevity Bench and When2Call-MT, are NVIDIA’s and have not yet been released.

NVIDIA’s own table also shows where the model falls short. Alibaba’s Qwen 3.5 4B scores 75.1 on Tau2-Bench, a test of agents completing customer-service tasks, against 55.1 for Bolt E4B. Qwen also leads on the long-context tests LongBench v2 (59.9 to 50.5) and RULER at 64,000 tokens (95.1 to 87.0). In the compressed 4-bit versions listed on the model card, the Tau2 gap is 61.74 to 52.32.

Bolt E4B leads clearly on tests of following instructions, deciding when to call a tool, role-play and grounded answers. On IHEval it scores 84.8 against Qwen’s 60.6, and on RoleBench 67.0 against 53.7. The model card notes that NVIDIA ran every comparison “under a consistent harness” and that results “may differ from vendors’ self-reported numbers”.

Intelligence per gigabyte is the trade NVIDIA is making

NVIDIA is not claiming the best small model overall. It is claiming the best fit for a narrow job: a character or assistant that talks, calls tools and keeps answers short on cheap hardware. The benchmarks it leads on, such as instruction-following, role-play and knowing when to call a tool, are the ones that job needs, and the tests it loses, such as long documents and multi-step customer-service tasks, matter less to a game character or a desk robot.

That framing is sensible, but it is also self-selected. NVIDIA chose the benchmarks, ran them, and built two of them itself, and those two have not been published for others to check. Until independent testers run the model, the 40% and 10% figures are best read as NVIDIA’s view of its own product.

The open licence and the wide spread of runtimes on day one count for more. A developer can try the model in Ollama or LM Studio today and see whether it suits their own task, which is the test NVIDIA’s model card itself recommends. NVIDIA has now released a run of open small models this month, including the looped Dory 4B model and five retrieval models for agents, and Bolt E4B extends that push to the smallest devices it sells.

The real proof will be in the Jetson Orin Nano figures: if a robot built on an 8GB board can hold a useful conversation at 15 tokens a second, NVIDIA has a case; if it waits six seconds to start each reply, developers will notice.