NVIDIA releases Dory 4B, a looped reasoning model with adjustable inference compute and a 256K-token context

NVIDIA has released Nemotron Labs Dory 1.0 4B, a compact language model that introduces a different approach to scaling reasoning performance: instead of increasing model parameters or generating longer chains of thought, Dory can repeatedly execute the same internal transformer layers before producing each token.

Released on October 16, 2026, the model contains approximately four billion parameters, supports context lengths up to 256,000 tokens and allows developers to adjust computational depth during inference without retraining or switching checkpoints.

NVIDIA reports 61.8% on SWE-bench Verified with four recurrent loops, compared with 50.3% using one loop. The 11.5-percentage-point improvement comes from increasing internal computation using the same weights. Terminal-Bench 2.1 rises from 12.5% to 22.2%, and GDPval-AA-V2 from 815 to 967 Elo.

Dory is based on a latent-recurrent normalized GPT, combining shared transformer layers, sliding-window attention and full-attention layers. It is not a Mixture-of-Experts model: its defining feature is repeated application of the same learned parameters.

NVIDIA has published BF16, MXFP8, NVFP4 and GGUF checkpoints under the OpenMDW 1.1 license, supporting commercial and non-commercial use subject to its terms. Developers can trade processing time and compute for accuracy with a single four-billion-parameter checkpoint.

··········

DORY 1.0 INTRODUCES ADJUSTABLE REASONING DEPTH WITH A SINGLE MODEL CHECKPOINT.

Most language models have a fixed computational structure during inference. Each generated token passes through the same transformer layers; extra reasoning typically means generating more tokens or using tools.

Dory adds another variable: how many times a recurrent transformer block processes an internal representation before output. Developers can select one, two or four passes while using identical model weights.

........

Specification	NVIDIA Nemotron Labs Dory 1.0 4B
Developer	NVIDIA
Release date	October 16, 2026
Parameters	Approximately 4 billion
Architecture	Looped latent-recurrent normalized GPT
Input transformer layers	8
Shared recurrent layers	20
Output transformer layers	8
Maximum context	256K tokens
Attention	Sliding-window and full attention
Adjustable computation	Recurrent loop count
Available formats	BF16, MXFP8, NVFP4, Q5_K_M GGUF
Primary language	English
Input and output	Text
License	OpenMDW 1.1
Intended workloads	Reasoning, coding, agents, tool use

........

NVIDIA recommends two loops as the default starting point, a compromise between cost and accuracy. One loop prioritizes lower inference computation, while four loops can improve demanding coding and agentic tasks. The optimal choice depends on workload.

··········

THE ARCHITECTURE REUSES A 20-LAYER RECURRENT BLOCK INSTEAD OF ADDING PARAMETERS.

Dory divides its transformer stack into an eight-layer input block, twenty shared recurrent layers and an eight-layer output block. The input representation is reinjected during recurrent processing, preserving original information while refining the intermediate state.

Input layers → Recurrent block × N → Output layers, where N is the configured number of recurrent passes.

........

Configuration	Input layers	Recurrent layer executions	Output layers	Total layer executions
1 loop	8	20	8	36
2 loops	8	40	8	56
4 loops	8	80	8	96

........

These counts represent layer executions, not distinct learned layers. Four loops execute eighty recurrent layer operations while reusing the same twenty sets of weights, leaving the checkpoint at approximately four billion parameters. The extra computation occurs in hidden representations rather than requiring proportionally more parameters.

··········

ELASTIC INFERENCE ALLOWS DEVELOPERS TO CHANGE COMPUTE WITHOUT RETRAINING.

A developer can use fewer loops for straightforward requests and more for difficult engineering problems without loading separate checkpoints. However, every additional loop executes the recurrent block again and increases work per generated token.

Four loops involve approximately 2.67 times the transformer-layer executions of one loop, a structural calculation rather than a measured FLOP or latency ratio. Actual runtime depends on attention, kernels, caching, hardware and output length.

NVIDIA also reports that additional looping can reduce the number of generated reasoning tokens. Thus, total task cost depends on both greater computation per token and the number of tokens needed for a correct answer.

··········

SWE-BENCH VERIFIED IMPROVES FROM 50.3% TO 61.8% WITH FOUR LOOPS.

NVIDIA's evaluations show significant improvements in coding and agentic tasks as recurrent depth increases.

........

Coding and agentic benchmark	1 loop	2 loops	4 loops
SWE-bench Verified	50.3%	59.5%	61.8%
Terminal-Bench 2.1	12.5%	18.1%	22.2%
SciCode	27.30%	34.94%	35.11%
τ³-bench Banking	11.60%	12.89%	14.69%
GDPval-AA-V2 Elo	815	946	967

........

SWE-bench Verified improves by 9.2 percentage points from one to two loops, then another 2.3 points at four. The overall 11.5-point gain is approximately 22.9% relative improvement from the one-loop result.

Terminal-Bench 2.1 rises by 9.7 points from one to four loops. SciCode shows diminishing returns: the large improvement occurs between one and two loops, with little additional benefit at four. These task-specific differences support NVIDIA's recommendation to start with two loops rather than always selecting the maximum.

··········

COMPARISONS WITH QWEN AND GEMMA SHOW WHERE DORY IS COMPETITIVE.

NVIDIA evaluated Dory against several open models, including Qwen3.5-4B, Qwen3.5-9B, Gemma 4 26B A4B and Qwen3.6-35B-A3B.

........

Model	SWE-bench Verified	Terminal-Bench 2.1
Dory 4B, 1 loop	50.3%	12.5%
Dory 4B, 2 loops	59.5%	18.1%
Dory 4B, 4 loops	61.8%	22.2%
Qwen3.5-4B	54.7%	15.0%
Qwen3.5-9B	61.0%	24.0%
Gemma 4 26B A4B	57.4%	35.0%
Qwen3.6-35B-A3B	73.4%	43.0%

........

At four loops, Dory exceeds Qwen3.5-9B on SWE-bench Verified by 0.8 percentage points, despite fewer stored parameters. However, Qwen3.5-9B performs better on Terminal-Bench 2.1 and Qwen3.6-35B-A3B substantially exceeds Dory on both.

Some comparator figures are reported rather than internally measured under identical conditions. These results demonstrate competitive compact-model performance, not universal superiority in accuracy, cost or speed.

··········

ADDITIONAL LOOPS DO NOT IMPROVE EVERY REASONING BENCHMARK.

The benefits of recurrence vary across reasoning tasks.

........

Benchmark	1 loop	2 loops	4 loops
GPQA Diamond	69.19%	72.41%	72.16%
Humanity's Last Exam, no tools	8.25%	10.89%	10.43%
IFBench	74.29%	77.94%	79.00%
AA-LCR long-context benchmark	63	71	73
AA-Omniscience accuracy	10.03%	10.08%	10.03%

........

GPQA Diamond and Humanity's Last Exam peak at two loops before declining slightly at four. Instruction following and long-context reasoning improve through four loops, while AA-Omniscience remains essentially unchanged.

Additional processing can help iterative reasoning but does not automatically remedy factual knowledge limitations. Deployments should choose loop counts based on measured results for specific tasks.

··········

DORY MOVES PART OF THE REASONING PROCESS INTO LATENT SPACE.

Conventional reasoning models often allocate compute by generating intermediate text, each token requiring autoregressive processing. Dory adds a second mechanism: repeated computation in hidden representations before token generation.

This does not mean Dory never generates reasoning tokens. It allows internal recurrent computation alongside ordinary text generation.

Consider a hypothetical task requiring 10,000 output tokens at one loop. If two loops reduce output length to 6,000 tokens, the simplified layer-execution proxy is:

One loop: 10,000 × 36 = 360,000 layer-token executions.

Two loops: 6,000 × 56 = 336,000 layer-token executions.

Under these hypothetical assumptions, two loops use about 6.7% fewer layer-token executions despite more work per token. This is not an NVIDIA benchmark and excludes attention, caching and hardware effects. It illustrates why end-to-end cost cannot be inferred from loop count alone.

··········

LAST-LOOP KV CACHING ADDRESSES THE MEMORY COST OF RECURRENCE.

Autoregressive transformers cache attention keys and values for previous tokens. A straightforward recurrent implementation can retain separate KV caches for each loop, increasing memory requirements with recurrent depth.

Dory offers a last-loop KV-cache mode that retains only the final pass's cached keys and values for recurrent attention layers, overwriting earlier entries.

........

Cache strategy	Behavior	Memory implication
Per-loop caching	Separate KV cache for each recurrent pass	Memory increases with loop count
Last-loop caching	Reuses one cache per recurrent attention layer	Recurrent KV footprint stays at one-loop level

........

NVIDIA reports that last-loop caching maintains accuracy close to per-loop caching in its tests. The method matters for long contexts and concurrent inference, although it does not eliminate KV caches or the computational cost of attention.

··········

THE 256K CONTEXT WINDOW SUPPORTS LONG DOCUMENTS AND CODING WORKFLOWS.

Dory supports a 256,000-token context. NVIDIA reports context extension during training with an additional 300 billion tokens after principal pretraining, using sliding-window and full-attention layers.

Long contexts can support repository analysis, technical documentation and extended agent sessions. However, advertised capacity does not guarantee uniform retrieval accuracy: information position, irrelevant content and prompt structure affect results.

NVIDIA reports AA-LCR improvements from 63 at one loop to 73 at four, suggesting recurrent computation benefits the tested long-context workload. Organizations should still evaluate retrieval accuracy, memory and latency at their target context lengths, often combining long context with selective retrieval.

··········

NVIDIA PRETRAINED DORY ON SIX TRILLION TOKENS USING NVFP4 AND BF16.

NVIDIA reports pretraining on approximately six trillion tokens covering code, mathematics, science and general knowledge. Approximately 4.5 trillion tokens used NVFP4 low-precision arithmetic before training continued in BF16.

........

Training stage	Reported configuration
Main pretraining	6 trillion tokens
Initial precision	NVFP4
NVFP4 training portion	Approximately 4.5 trillion tokens
Later precision	BF16
Context-extension training	300 billion tokens
Supervised fine-tuning	NVIDIA-curated dataset
Reinforcement learning	Multi-environment GRPO
Pretraining framework	Megatron-LM
RL infrastructure	NeMo RL and NeMo Gym

........

Supervised fine-tuning and reinforcement learning targeted mathematics, coding, science, instruction following, tools, conversations and structured outputs. NVIDIA used Group Relative Policy Optimization and asynchronous RL infrastructure separating training from inference execution.

The results reflect both recurrent architecture and extensive task-oriented post-training. Public information does not isolate exactly how much of the performance comes from each training component.

··········

FOUR RELEASED PRECISION FORMATS SUPPORT DIFFERENT DEPLOYMENT ENVIRONMENTS.

NVIDIA offers BF16, MXFP8, NVFP4 and Q5_K_M GGUF checkpoints, addressing numerical precision, storage footprint and local inference compatibility.

........

Format	Approximate weight precision	Deployment consideration
BF16	16-bit	Higher precision, larger memory footprint
MXFP8	8-bit floating point	Reduced weight storage
NVFP4	4-bit floating point	Compact NVIDIA-optimized inference
Q5_K_M GGUF	Approximately 5-bit quantization	Local llama.cpp-compatible deployment

........

Four billion parameters theoretically require approximately 8 GB at 16 bits, 4 GB at 8 bits and 2 GB at 4 bits before metadata and runtime overhead. NVIDIA's BF16 repository is approximately 7.97 GB, MXFP8 approximately 4.77 GB and GGUF approximately 3.67 GB.

Runtime memory also includes attention caches, activations and engine overhead. Practical requirements depend on context length, concurrent requests, loop count and cache mode; maximum-context inference can need much more memory than a short local chat.

··········

VLLM AND LLAMA.CPP SUPPORT REQUIRE ARCHITECTURE-SPECIFIC INTEGRATIONS.

Dory's recurrent execution and cache behavior require explicit inference-framework support. NVIDIA documents vLLM and llama.cpp integration paths, including a Dory-enabled llama.cpp implementation for local use on Apple Silicon.

At release, vLLM support was under review through a dedicated pull request, rather than universally available in stable builds. The loop count can be configured with n_recurrent_loops and caching through recurrent_kv_cache_mode, with values per_loop or last_loop.

An illustrative vLLM invocation is:

`vllm serve nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-BF16 --hf-overrides '{"n_recurrent_loops": 2, "recurrent_kv_cache_mode": "last_loop"}' --max-model-len 262144`

This requires a build with Dory support and adequate hardware. NVIDIA recommends benchmarking one, two and four loops on actual workloads.

··········

THE ECONOMICS OF LOOPED REASONING DEPEND ON COST PER COMPLETED TASK.

Elastic inference allows providers to spend less compute on easy requests and more on difficult ones. Its economics depend on cost per attempt, generated tokens and probability of success.

Expected cost per successful task = Average execution cost ÷ Success probability.

Consider an illustrative comparison using NVIDIA's SWE-bench Verified success rates but hypothetical costs.

........

Configuration	Assumed cost per attempt	Reported success rate	Illustrative cost per success
1 loop	$0.10	50.3%	$0.199
2 loops	$0.14	59.5%	$0.235
4 loops	$0.18	61.8%	$0.291

........

These are invented cost assumptions, not NVIDIA measurements. Under them, one loop has the lowest cost per success despite lower accuracy.

If four loops reduced its cost per attempt below approximately $0.123, it would beat the one-loop configuration on this simplified metric, because cost at four loops ÷ 0.618 = $0.10 ÷ 0.503.

Real operations must include retries, tool calls, latency, failure severity and task difficulty. Higher accuracy may justify extra inference spending on expensive software engineering errors, while lower loops may suit high-volume, low-risk work.

··········

DORY'S RESULTS ESTABLISH A USEFUL TRADE-OFF, NOT UNIVERSAL MODEL SUPERIORITY.

The strongest evidence for Dory is comparison of different loop counts using the same checkpoint. SWE-bench Verified increases from 50.3% to 61.8%, Terminal-Bench 2.1 from 12.5% to 22.2%, and selected long-context results also improve.

However, GPQA Diamond and Humanity's Last Exam peak at two loops, while general-knowledge accuracy is largely unchanged. Several larger competitors also outperform Dory on coding and agentic benchmarks.

Its principal innovation is adjustable computational depth without changing weights, not universal benchmark leadership. Independent testing must determine how reliably it improves production tasks and whether output-token savings offset added internal computation.

··········

DORY MAKES REASONING COMPUTE AN ADJUSTABLE PROPERTY OF SMALL LANGUAGE MODELS.

Dory 1.0 demonstrates an alternative to scaling through additional parameters or longer generated reasoning traces. Its recurrent transformer reuses twenty learned layers, allowing a single four-billion-parameter checkpoint to operate at different computational depths.

The measurable improvement is 61.8% SWE-bench Verified at four loops versus 50.3% at one, alongside gains in terminal-based engineering and selected long-context tests. Its last-loop KV-cache design addresses recurrent memory scaling, and multiple quantization formats broaden deployment options.

Dory separates model size from reasoning depth: developers can allocate more computation to difficult tasks without loading a larger model. Its lasting value will depend on whether that flexibility reduces the total cost of correct, completed work in production.

··········

FOLLOW US FOR MORE.

DATA STUDIOS

datastudios.org