Factual stability

5 prompts observed across 9 models over 25 weeks.

Median response length — model × week

Median response length in tokens, averaged across the prompts in this axis. Sustained shifts often accompany a model update. Cell shade uses a viridis (colorblind-safe) palette, anchored at zero and scaled to 115; darker = lower, brighter = higher. Numeric values are in each cell for programmatic access. A · means the model was not sampled that week and is not a measurement of zero. Frontier models alternate on a biweekly cadence, so roughly half of their cells are unsampled by design. 2026-W30, 2026-W31 are a different case: the audit did not run at all, so those columns are empty for every model. A lost week is a hole in the record rather than a cadence skip, and it keeps its column instead of being closed up. Why these weeks are missing. This axis leads with median response length because that is the measure that moves on it. How this is computed.
Model 2026-W172026-W182026-W192026-W202026-W212026-W222026-W232026-W242026-W252026-W262026-W272026-W282026-W292026-W302026-W312026-W322026-W332026-W342026-W352026-W362026-W372026-W382026-W392026-W402026-W412026-W422026-W43
gpt-6-astra · · · · · · · · · · · · · · · · · · · · · · · · median response length 15 · median response length 15
llama3.2:3b · · median response length 34 median response length 35 median response length 34 median response length 34 median response length 34 median response length 32 median response length 34 median response length 31 median response length 34 median response length 30 median response length 34 · · median response length 32 median response length 34 median response length 34 median response length 34 median response length 33 median response length 30 median response length 32 median response length 35 median response length 34 median response length 34 median response length 34 median response length 34
qwen3.6:27b · · · · · · · · · · · · · · · · · · · · · · · · · · median response length 71
claude-opus-4-8 · · · · · · · · · · · median response length 65 · · · median response length 67 · median response length 68 · · · · · median response length 64 · median response length 63 ·
claude-opus-5 · · · · · · · · · · · · · · · · · · · · · · · median response length 115 · median response length 111 ·
claude-opus-5-5 · · · · · · · · · · · · · · · · · · · · · · · · · median response length 47 ·
gpt-5.5 · · · · · · · · · · median response length 12 · median response length 12 · · · median response length 12 · median response length 12 · median response length 12 · median response length 12 · median response length 12 · ·
claude-opus-4-7 · median response length 44 · median response length 44 · median response length 45 · median response length 55 · median response length 58 · · · · · · · · · · · · · · · · ·
gpt-5.1 median response length 12 · median response length 13 · median response length 12 · median response length 12 · median response length 12 · · · · · · · · · · · · · · · · · ·

Prompts in this axis