NVIDIA Releases Nemotron Labs Dory1.0-4B

NVIDIA has published NVIDIA-Nemotron-Labs-Dory1.0-4B on Hugging Face — a 4-billion-parameter language model whose inference compute can be turned up or down by looping a shared block of layers. The BF16 model card lists Release Date: October 16, 2026. Companion checkpoints cover MXFP8, NVFP4 and a Q5_K_M GGUF export.

The weights are released under the OpenMDW License Agreement, version 1.1. Say open-weight under OpenMDW-1.1, not unqualified “open source.” The card states the model is “ready for commercial or non-commercial use.”

All benchmark numbers below are NVIDIA’s own measurements “under a consistent harness (NeMo Gym / Nemo Evaluator SDK).” They are not independently verified here.

What “looped” means here

Most transformers run each layer once per token. A looped (or latent-recurrent) model reuses a shared block of layers several times for the same token before moving on — spending more compute inside the network instead of only generating a longer answer.

Dory’s stack, per NVIDIA’s diagram and text, has three groups:

Input block: 8 layers
Recurrent block: 20 layers (shared; applied multiple times)
Output block: 8 layers

The input is “reinjected into every loop.” At inference you choose how many times the recurrent block runs — NVIDIA documents 1, 2, 3 or 4 loops in its llama.cpp notes.

One checkpoint, several compute budgets

NVIDIA’s highlight line: “A single checkpoint serves different inference compute budgets. By changing how many times the shared block of layers is applied, you can trade inference compute for accuracy to suit each task, with no retraining and no model swap.”

Further card claims:

“We recommend two loops as the default starting point”; “Most of the accuracy gain from looping comes from the second loop.”
Context: “Up to 256K tokens”; language: English.
Last-loop KV caching can keep the KV-cache footprint at the one-loop size for any loop count (optional mode).
Pre-training: “pre-trained natively in NVFP4, NVIDIA’s 4-bit floating-point format, before continuing in BF16,” on 6T tokens plus 300B for context extension.
Formats: “BF16, MXFP8, NVFP4 and Q5_K_M.”

Architecture summary on the card: “Looped (latent-recurrent) normalized-GPT with interleaved sliding-window attention (SWA) and full-attention layers.”

NVIDIA’s SWE-bench Verified table (its harness)

Selected coding / agentic rows from NVIDIA’s comparison table:

Model / setting	SWE-bench Verified
Dory1.0-4B, 1 loop	50.3
Dory1.0-4B, 2 loops	59.5
Dory1.0-4B, 4 loops	61.8
Qwen3.5-4B	54.7
Qwen3.5-9B	61

On that same NVIDIA table, Dory at four loops is slightly above Qwen3.5-9B on SWE-bench Verified (61.8 vs 61) but behind on other rows — for example GPQA Diamond (no tools) 72.16 (4 loops) vs Qwen3.5-9B 80.93, and Terminal-Bench 2.1 22.2 vs 24. Do not read the release as an overall win over Qwen3.5-9B.

Other NVIDIA-measured Dory scores on the card (1 / 2 / 4 loops): Terminal-Bench 2.1 12.5 / 18.1 / 22.2; GPQA Diamond 69.19 / 72.41 / 72.16; IFBench 74.29 / 77.94 / 79.

Runtime support is still in pull requests

vLLM: “Dory support in vLLM is under review in vllm-project/vllm#60149.” Until a release includes it, NVIDIA points users at installing from that PR.
llama.cpp: Support is in ggml-org/llama.cpp#30306 / a pinned fork revision on the card — “Until it is included in a llama.cpp release, build from the Dory-enabled source.”

Do not say vLLM or llama.cpp support has already shipped in a stable release.

Why it matters

A 4B checkpoint that can act like a cheaper or a dearer model by changing loop count is useful for local and edge agent stacks — the same Nemotron lane TSN has covered with Nemotron 3.5 Super VL and RTX Spark / Nemotron 3.5 Bolt local agents. For enterprise open-weight risk framing, see Microsoft and NVIDIA’s open-weight risk guide. The useful discipline on Dory is to keep NVIDIA’s harness labels on every score and to remember OpenMDW-1.1 is not a blanket “open source” claim.

What’s still unknown

Independent SWE-bench / Terminal-Bench reproductions
When (or whether) the vLLM and llama.cpp PRs land in releases
How last-loop KV caching behaves outside NVIDIA’s reported experiments
Non-English quality (card: English)

What this means

NVIDIA’s Nemotron Labs Dory1.0-4B is a 4B open-weight (OpenMDW-1.1) looped model: an 8 / 20 / 8 layer stack where the shared 20-layer block can run 1–4 times per token from one checkpoint. NVIDIA’s own harness puts SWE-bench Verified at 50.3 / 59.5 / 61.8 for 1 / 2 / 4 loops — interesting next to Qwen3.5-4B/9B on that row, not proof of an overall lead — while vLLM and llama.cpp support remain open pull requests.