Run 1
1:39 min:stime
9,511output tokens
5.5%thinking
96.14tok/s
987lines of code
Effort requested: medium; received: on · differs from requested effort · source: metrics.json. Effort is not comparable across models.
glm-4.7-flash_bare_r1
GLM 4.7 Flash builds a full web page in under 2 minutes. Do rules slow it down?
zai-org/glm-4.7-flash
Same model, same prompt, same rig. 3 runs with nothing added, 3 with one Karpathy rules file.
zai-org/glm-4.7-flash
Ranges overlap (counted effects)Effects count CSS/JS constructs, not visual quality.
Repeated ranges
54–73
KARPATHY · n=356–73
1:24–1:39 min:s
KARPATHY1:19–1:44 min:s
8,404–9,511
KARPATHY7,826–9,934
96.14–102.00 tok/s
KARPATHY95.35–99.06 tok/s
5.5%–6.0%
KARPATHY5.3%–8.5%
775–987
KARPATHY762–1,080
BARENO RULES, NO EXTRAS
1:27 min:stime
8,404output tokens
5.7%thinking
869lines of code
Effort requested: medium; received: on · differs from requested effort · source: metrics.json. Effort is not comparable across models.
KARPATHYONE RULES FILE
1:34 min:stime
9,152output tokens
5.7%thinking
876lines of code
Effort requested: medium; received: on · differs from requested effort · source: metrics.json. Effort is not comparable across models.
Every run. Every result.
BARE
NO RULES, NO EXTRAS · n=3
Run 2
1:24 min:stime
8,538output tokens
6.0%thinking
102.0tok/s
775lines of code
Effort requested: medium; received: on · differs from requested effort · source: metrics.json. Effort is not comparable across models.
glm-4.7-flash_bare_r2
Run 3
1:27 min:stime
8,404output tokens
5.7%thinking
97.13tok/s
869lines of code
Effort requested: medium; received: on · differs from requested effort · source: metrics.json. Effort is not comparable across models.
glm-4.7-flash_bare_r3
KARPATHY
ONE RULES FILE · n=3Run 1
56effects
1:34 min:stime
9,152output tokens
5.7%thinking
97.32tok/s
876lines of code
Effort requested: medium; received: on · differs from requested effort · source: metrics.json. Effort is not comparable across models.
glm-4.7-flash_karpathy_r1
Run 2
57effects
1:19 min:stime
7,826output tokens
8.5%thinking
99.06tok/s
762lines of code
Effort requested: medium; received: on · differs from requested effort · source: metrics.json. Effort is not comparable across models.
glm-4.7-flash_karpathy_r2
Run 3
73effects
1:44 min:stime
9,934output tokens
5.3%thinking
95.35tok/s
1,080lines of code
Effort requested: medium; received: on · differs from requested effort · source: metrics.json. Effort is not comparable across models.
glm-4.7-flash_karpathy_r3
output = thinking + final code. Lines of code show the page size the model chose, not the cost of equivalent work.
How to reproduce
Settings recorded for these runs. Multiple values show differences between runs. Effort is requested, not necessarily received; it is not comparable across models. Hover over quantization, K/V cache or engine values for their sources.
zai-org/glm-4.7-flash
| Model | Quantization | Context window | K/V cache | max_tokens | Engine | Harness | Effort requested |
|---|---|---|---|---|---|---|---|
| zai-org/glm-4.7-flash | Q4_K_M | 202,752 | K q8_0 / V q8_0 | 131,072 | [email protected] | pi 0.85.1 | medium |
Watch the experiment.
REPRODUCE THE EXPERIMENT
One prompt.
All the evidence.
Compare runs within the same model. Inspect the shared prompt, settings and raw measurements in the repository.