Worlds: modular tanh lattices — collective bistability,
hysteresis, per-region timescales, slow fatigue (D4). One collective switch per
region; the agent must calibrate scrambled sensors, find the switches, map
thresholds and memory. World guide.
1. Difficulty tracks ability (chat tier, M0 grid)
model (null harness)
D0
D1
D2
D3
google/gemini-3.5-flash
0.72
0.58
0.54
0.35
deepseek/deepseek-v4-flash
0.64
0.56
0.52
0.38
openai/gpt-5-nano
0.12
0.11
0.13
0.04
scripted reference
0.72
0.72
0.65
0.67
Pooled difficulty↔reward Spearman ρ=−0.50 (p=0.003, n=32 rollouts).
gpt-5-nano sits at the null floor everywhere — mostly protocol failure,
which motivated the coding-harness tier.
2. The D4 frontier (tools tier, 5 seeds)
pairing
prediction
budget used
claude-fable-5 + claude_code
0.349 ± 0.04
0.32
claude-opus-5 + claude_code
0.324 ± 0.11
0.41
gpt-5.6-sol + codex
0.244 ± 0.07
0.54
gemini-3.1-pro + codex
0.194 ± 0.12
0.01
scripted reference / tail / null
0.274 / 0.211 / 0.133
0.11 / — / —
Budget spent ≠ score: Spearman ρ=0.14 (p=0.55) across 20 rollouts —
what you measure beats how much. sol burns the most ticks for mid scores.
Everyone fails the same stratum: S1 (weak-push relaxation, the most
sensitive probe of the slow fatigue variable) ≤0.5 for all pairings. Nobody
has characterized the adaptation dynamics.
Conduct differences: gemini classifies the task as low-value and
answers after ~12 turns / 1% budget with honest-but-useless wide intervals
(coverage 0.96, accuracy ~null). fable runs 60–180-turn research programs.
3. Preparation and theory (M2/M3 grid, D4, 3 seeds, n_prep=3)
pairing
prediction
preparation
theory
claude-fable-5
0.305
0.73
0.29 (3/3 submitted)
claude-opus-5
0.269
0.67
0.31 (2/3)
gpt-5.2
0.219
0.47
0/3 submitted
gpt-5.6-sol
0.214
0.51
0.21 (1/3)
prep_pi certifier / null
0.34 / 0.145
0.24 / 0.0
—
Preparation ≫ prediction — with a caveat discovered later: the
v0.1 prep sampler let ~half of D-track bands include the do-nothing outcome.
With honest bands (v0.3.2+), fable-5's D4 preparation drops to 0/1 (seeds
5–6); the original 0.73 mixed real steering with trivial bands. Honest D4
preparation is now an open problem, like D4 prediction.
Prep outcomes are bimodal (20/36 policies hit 5/5 clones, 13 hit
0/5, only 8% flaky): the 5-clone scoring measures understanding, not luck.
Failure audit: every 0/5 contract was solvable; one required
inverted-sign feedback (naive P-control latches the wrong branch) — models
failed exactly where control depth crosses from open-loop to
feedback-with-identification.
Theory tracks prediction (r=0.72, n=6). Anthropic agents submit
theories; codex agents skip the optional tool entirely.
4. Best artifacts
fable-5's D4 MODEL.md: identifies "6 bistable
relaxation-oscillator units", maps ports→units with polarities, measures
release schedules and mutual entrainment (period ~390–430), ships a
per-contract prediction procedure — and its own failure mode ("jitter grows
~2%/cycle") explains its S4 losses. Gallery.
gpt-5.2's D0 model.py: two-basin affine model with switching
threshold and time constant; S4=0.96 where chat-tier models got ~0.2 —
the coding harness converts budget into long-horizon accuracy.
5. Cost accounting (tools tier, D4)
pairing
model calls
cached prompt
completion
wall time
gpt-5.6-sol + codex
66–98
4–5M
12–17k
8–12 min
claude-opus-5 + claude_code
67–93
4–6.5M
111–125k
25–36 min
claude-fable-5 + claude_code
113–173
22–51M
191–241k
~1 h
fable's edge rides a 5–10× cached-context bill and hour-long rollouts —
a long-context research strategy the others don't attempt.
Generated by python -m physim.results; numbers from REPORT.md (the running lab log with full methodology).