Run 1
3:57 min:stime
8,896output tokens
4.6%thinking
37.47tok/s
563lines of code
Effort requested: medium; received: on · differs from requested effort · source: metrics.json. Effort is not comparable across models.
qwen3.6-27b_bare_r1
Qwen 3.6 27B builds a web page in 4 minutes. Do rules make it leaner?
qwen/qwen3.6-27b
Same model, same prompt, same rig. 3 runs with nothing added, 3 with one Karpathy rules file.
qwen/qwen3.6-27b
Ranges overlap (counted effects)Effects count CSS/JS constructs, not visual quality.
Repeated ranges
54–73
KARPATHY · n=348–69
3:57–5:19 min:s
KARPATHY4:28–5:37 min:s
8,896–11,787
KARPATHY9,861–12,415
35.59–37.47 tok/s
KARPATHY36.78–37.30 tok/s
3.6%–5.2%
KARPATHY3.0%–6.2%
548–603
KARPATHY479–717
BARENO RULES, NO EXTRAS
4:19 min:stime
9,228output tokens
5.2%thinking
548lines of code
Effort requested: medium; received: on · differs from requested effort · source: metrics.json. Effort is not comparable across models.
KARPATHYONE RULES FILE
4:40 min:stime
10,439output tokens
3.0%thinking
479lines of code
Effort requested: medium; received: on · differs from requested effort · source: metrics.json. Effort is not comparable across models.
Every run. Every result.
BARE
NO RULES, NO EXTRAS · n=3
Run 2
4:19 min:stime
9,228output tokens
5.2%thinking
35.59tok/s
548lines of code
Effort requested: medium; received: on · differs from requested effort · source: metrics.json. Effort is not comparable across models.
qwen3.6-27b_bare_r2
Run 3
5:19 min:stime
11,787output tokens
3.6%thinking
36.96tok/s
603lines of code
Effort requested: medium; received: on · differs from requested effort · source: metrics.json. Effort is not comparable across models.
qwen3.6-27b_bare_r3
KARPATHY
ONE RULES FILE · n=3Run 1
69effects
4:28 min:stime
9,861output tokens
6.2%thinking
36.78tok/s
698lines of code
Effort requested: medium; received: on · differs from requested effort · source: metrics.json. Effort is not comparable across models.
qwen3.6-27b_karpathy_r1
Run 2
48effects
4:40 min:stime
10,439output tokens
3.0%thinking
37.3tok/s
479lines of code
Effort requested: medium; received: on · differs from requested effort · source: metrics.json. Effort is not comparable across models.
qwen3.6-27b_karpathy_r2
Run 3
67effects
5:37 min:stime
12,415output tokens
3.3%thinking
36.84tok/s
717lines of code
Effort requested: medium; received: on · differs from requested effort · source: metrics.json. Effort is not comparable across models.
qwen3.6-27b_karpathy_r3
output = thinking + final code. Lines of code show the page size the model chose, not the cost of equivalent work.
How to reproduce
Settings recorded for these runs. Multiple values show differences between runs. Effort is requested, not necessarily received; it is not comparable across models. Hover over quantization, K/V cache or engine values for their sources.
qwen/qwen3.6-27b
| Model | Quantization | Context window | K/V cache | max_tokens | Engine | Harness | Effort requested |
|---|---|---|---|---|---|---|---|
| qwen/qwen3.6-27b | Q4_K_M | 262,144 | K q4_0 / V q4_0 | 131,072 | [email protected] | pi 0.85.1 | medium |
Watch the experiment.
REPRODUCE THE EXPERIMENT
One prompt.
All the evidence.
Compare runs within the same model. Inspect the shared prompt, settings and raw measurements in the repository.