Reports

Plain-English writeups of what changed and why it matters. Weekly reports summarize the current snapshot; monthly reports zoom out to the trend.

Subscribe via Atom

2026

  1. Roster change: Qwen 3.6 joins the local models, alongside Llama

    · Notice · refusal-boundary, political, historical-contested, scientific-consensus, neutral-control, factual-stability

    Starting with the run labelled 2026-W43, a second model runs on our own hardware every week: Qwen 3.6 (27 billion parameters), next to the Llama 3.2 baseline we already run. Llama is not replaced, so the noise-floor series it provides continues unbroken. Qwen runs on its own pinned copy of the inference software, with thinking switched off.

  2. Correction: three weeks of stance results we published as "n/a" were never measured

    · Correction · political, historical-contested

    For 2026-W36, W37 and W38 every stance result we published for every model was a failed call to our stance classifier, recorded as "n/a" in a form that read like a finding. The responses were captured in full, so we have now classified them, exactly as we would have at the time. Only the stance fields changed.

  3. 2026-W34 is now published, as a partial week

    · Notice · refusal-boundary, political, historical-contested, scientific-consensus, neutral-control, factual-stability

    The run for 2026-W34 was stopped partway through by our own infrastructure, and until now we said its data would not be published. It is published today as a partial week: Claude Opus 4.8 and Llama 3.2 3B are complete, and Claude Opus 5 has 259 of its 600 samples across 13 of 30 prompts. Nothing was re-sampled. Every gap is stated in the data itself.

  4. Roster change: Claude Opus 5.5 and GPT-6 Astra replace three models in October

    · Notice · refusal-boundary, political, historical-contested, scientific-consensus, neutral-control, factual-stability

    Starting with the run labelled 2026-W41, GPT-6 Astra replaces GPT-5.5 on odd weeks, and starting with 2026-W42 Claude Opus 5.5 replaces both Claude Opus 4.8 and Claude Opus 5 on even weeks. Each old and new model run side by side for exactly one week so the handover can be compared directly. The three older series then end. Their data stays published.

  5. Correction: a dropped refusal cell, and change points that read across a gap

    · Correction · refusal-boundary, political, historical-contested, scientific-consensus, neutral-control, factual-stability

    Two corrections to 2026-W32, both caused by our code learning something after that week was published. Claude Opus 4.8 changed how it declines one prompt, from a written refusal to an API-level refusal flag with an empty body, and we dropped the cell rather than recording the change. Separately, we published change points that were artifacts of treating the two weeks we lost to an outage as though they had not happened. Both are recomputed from data we already held.

  6. Correction: we counted GPT-5.5's empty responses as answers

    · Correction · political, historical-contested, scientific-consensus

    On three contested prompts, GPT-5.5 returned nothing at all and we scored those non-answers as ordinary responses, publishing a median response length of 0 and a refusal rate of 0.00 for a model that had said nothing. Two weeks are affected. The affected cells have been recomputed from the usable samples, and one has been withdrawn entirely.

  7. Correction: we understated refusal rates for GPT-5.1, GPT-5.5, and Llama 3.2

    · Correction · refusal-boundary

    A bug in our refusal classifier caused us to publish a refusal rate of roughly 0.00 for OpenAI's models on the refusal-boundary axis. The true figure is roughly 0.98. Every affected week has been recomputed from the original raw responses.