Run 1
0:46 min:stime
4,209output tokens
76.2%thinking
91.64tok/s
114lines of code
Effort requested: medium; received: on · differs from requested effort · source: metrics.json. Effort is not comparable across models.
nemotron-3-nano_bare_r1
Nemotron 3 Nano spends 76% of its output thinking, and 2 of 6 pages broke
nvidia/nemotron-3-nano
Same model, same prompt, same rig. 3 runs with nothing added, 3 with one Karpathy rules file.
nvidia/nemotron-3-nano
Ranges overlap (counted effects)Effects count CSS/JS constructs, not visual quality.
Repeated ranges
5–8
KARPATHY · n=36–12
0:37–0:59 min:s
KARPATHY0:36–0:47 min:s
3,672–5,851
KARPATHY4,032–4,407
91.64–99.43 tok/s
KARPATHY93.03–111.19 tok/s
53.2%–77.5%
KARPATHY57.4%–75.6%
114–212
KARPATHY122–188
BARENO RULES, NO EXTRAS
0:59 min:stime
5,851output tokens
77.5%thinking
162lines of code
Effort requested: medium; received: on · differs from requested effort · source: metrics.json. Effort is not comparable across models.
KARPATHYONE RULES FILE
0:42 min:stime
4,057output tokens
75.6%thinking
122lines of code
Effort requested: medium; received: on · differs from requested effort · source: metrics.json. Effort is not comparable across models.
Every run. Every result.
BARE
NO RULES, NO EXTRAS · n=3
Run 2
0:59 min:stime
5,851output tokens
77.5%thinking
98.91tok/s
162lines of code
Effort requested: medium; received: on · differs from requested effort · source: metrics.json. Effort is not comparable across models.
nemotron-3-nano_bare_r2
Run 3
0:37 min:stime
3,672output tokens
53.2%thinking
99.43tok/s
212lines of code
Effort requested: medium; received: on · differs from requested effort · source: metrics.json. Effort is not comparable across models.
nemotron-3-nano_bare_r3
KARPATHY
ONE RULES FILE · n=3Run 1
6effects
0:36 min:stime
4,032output tokens
68.9%thinking
111.19tok/s
161lines of code
Effort requested: medium; received: on · differs from requested effort · source: metrics.json. Effort is not comparable across models.
nemotron-3-nano_karpathy_r1
Run 2
12effects
0:47 min:stime
4,407output tokens
57.4%thinking
93.03tok/s
188lines of code
Effort requested: medium; received: on · differs from requested effort · source: metrics.json. Effort is not comparable across models.
nemotron-3-nano_karpathy_r2
Run 3
6effects
0:42 min:stime
4,057output tokens
75.6%thinking
96.27tok/s
122lines of code
Effort requested: medium; received: on · differs from requested effort · source: metrics.json. Effort is not comparable across models.
nemotron-3-nano_karpathy_r3
output = thinking + final code. Lines of code show the page size the model chose, not the cost of equivalent work.
How to reproduce
Settings recorded for these runs. Multiple values show differences between runs. Effort is requested, not necessarily received; it is not comparable across models. Hover over quantization, K/V cache or engine values for their sources.
nvidia/nemotron-3-nano
| Model | Quantization | Context window | K/V cache | max_tokens | Engine | Harness | Effort requested |
|---|---|---|---|---|---|---|---|
| nvidia/nemotron-3-nano | Q3_K_L | 262,144 | K f16 / V f16 (LM Studio default) | 131,072 | [email protected] | pi 0.85.1 | medium |
Watch the experiment.
REPRODUCE THE EXPERIMENT
One prompt.
All the evidence.
Compare runs within the same model. Inspect the shared prompt, settings and raw measurements in the repository.