Data schema

Every snapshot under /data/ follows this schema. Any change that breaks backward compatibility bumps the schema_version field in the manifest (currently 2).

metrics.csv / metrics.jsonl

One row per (prompt × model) per week. Sampling is N=20 at the provider’s default temperature; see the methodology for details.

week_id

ISO 8601 week identifier, e.g. 2026-W16.

corpus_version (snapshot)

The corpus version these results were produced against, e.g. 2026.04.19-v0.2. Added 2026-07-25; reads unknown on earlier manifests.

corpus_git_sha (snapshot)

Commit that last changed the public prompt file, so a result can be pinned to an exact prompt set. Scoped to the corpus file rather than repository HEAD, so it moves only when the prompts do. Manifests published before 2026-07-25 carry unknown: the field was present but never populated by the pipeline.

prompt_id

Stable slug; see /corpus/.

model_id

Stable slug per observed model.

n_samples

Number of samples the metrics on this row were computed from (integer). Excludes any that returned no usable content.

unusable_samples

Samples captured for this cell that carried no measurable response and were excluded from every metric, typically a completion truncated by the token limit before any visible output. The raw responses are still published in the week’s snapshot. Where every sample was unusable no row is emitted at all; the cell is listed under the manifest’s unmeasured key instead, because a refusal rate of 0.00 for a model that answered nothing would be worse than no number. Added 2026-07-24; absent on earlier snapshots, where it should be read as 0.

rejected_samples

Requests the provider declined to run for this cell, so no response was ever produced and nothing was captured. Excluded from every metric and reported only. Distinct from unusable_samples, which counts responses we received and could not measure: this counts responses that do not exist. The distinction matters because only one of the two is about the model. A platform declining to process a prompt is a fact about the platform, and folding it into the refusal rate would publish a refusal the model was never asked to make. Where this is non-zero, read n_samples against it: a cell showing 2 samples and 18 rejections was not lightly sampled, it was mostly blocked. Added 2026-08-25; absent on earlier snapshots, where it should be read as 0.

refusal_rate

Fraction of samples that declined to answer, in [0, 1].

refusal_ci_lower, refusal_ci_upper

Bootstrap 95% CI on refusal rate, in [0, 1].

hedge_density

Hedging markers per 100 tokens.

length_median

Median response length in tokens.

length_p25, length_p75

25th / 75th percentile lengths.

stance

Pro / anti / neutral / n/a. Applied only to stance-bearing prompts (the political and historical-contested axes). The current week’s files carry the stance from its own manifest. Every earlier week’s files are regenerated on each build from the history in the current manifest, and a history entry carries stance only when it was backfilled from that week’s committed manifest. A history entry built any other way, for example from raw samples on the pipeline host, holds no stance, so its rows show n/a with an empty confidence. In this build, past weeks with stance in their files: 2026-W18, 2026-W19, 2026-W20, 2026-W21, 2026-W22, 2026-W23, 2026-W24, 2026-W25, 2026-W26, 2026-W27, 2026-W28, 2026-W29, 2026-W32, 2026-W33, 2026-W34; without: 2026-W17, 2026-W35, 2026-W36, 2026-W37, 2026-W38, 2026-W39, 2026-W40, 2026-W41, 2026-W42. Each week’s own stance results, including any later correction, stay in its manifest, data/manifests/<week>.json in the source repository.

stance_confidence

In [0, 1], and read together with stance, because it encodes whether the classifier was reached rather than a graded confidence. 0.85: the classifier scored the response (it reports one fixed value). 1.0 with stance n/a: the classifier was deliberately not asked, because the prompt is not on a stance-bearing axis or the model gave no substantive response to classify. 0.0 with stance n/a: the classifier was asked and returned nothing usable (the call failed, or its answer could not be parsed), so the stance is unmeasured, not neutral and not n/a; 2026-W36 to 2026-W38 carried this on almost every stance-bearing cell as first published, and were corrected on 2026-10-07 (see each manifest's corrections and the correction report). Empty when stance classification was not run, and on past-week rows whose history entry holds no stance (see stance).

stance_reason (metrics.jsonl only)

Why stance is what it is when the classifier did not score the cell, as a short code: runner-error or classifier-error (the call failed), unparseable, no-substantive-response, empty-response or axis-excluded. Null on a scored cell, when stance was not run, and on results produced before the field was added on 2026-10-05, where stance_confidence above is the only signal. Only the code is published, never a provider’s error text.

embedding_centroid_shift

Cosine distance of this week’s embedding centroid from the prior week’s, ≥ 0.

flagged_for_review

Boolean: human review flagged this row.

flag_reason

Free-text note; empty when not flagged.

sample_s3_uris (removed 2026-07-25)

Present in metrics.jsonl on snapshots published before this date, always as an empty list. It was never populated, and the URIs it was intended to hold pointed into a private archive bucket that readers could not fetch. Sample responses are now shown directly on each prompt page and every response remains available in responses.jsonl.gz, so the field was removed rather than left advertising a capability that did not exist.

manifest.json

Present in the current snapshot directory. Contains the full manifest consumed by the site builder, including model metadata, prompt catalog, current-week metrics, and up to N weeks of prior history. The machine-readable JSON Schema is at site/schemas/manifest.schema.json in the source repository.

Four top-level keys say how a manifest came to be. All are empty or false on an ordinary week. They were added in 2026-10, so a manifest first published earlier lacks them unless a later correction or reconstruction wrote one (corrections is added to a past week’s manifest when that week is corrected).

partial

True when the week’s run did not finish and the manifest publishes what it captured. 2026-W34 is the first: killed at the SSM execution timeout and built later from its archived raw samples.

notes

Plain-language statements about the week: why it is partial, what is missing, when and from what the manifest was built.

coverage

On a partial week, one entry per model: expected_samples against captured_samples, prompts_complete of prompts_expected, the prompts cut short (partial_prompts, which publish at the smaller n their row shows) and the prompts never sampled (missing_prompts, which have no row), and a status of complete, partial or lost.

corrections

Versioned corrections applied after publication, oldest first: the date, the metric fields the correction was allowed to change, how many rows (cells) changed, a summary, and the path of the public report. The file as first published stays in the repository’s history.

responses.jsonl.gz

One gzipped JSON object per line. Each line is a serialized Sample record — the same object the pipeline stores on disk at capture time. Fields:

prompt_id, model_id, provider

Identifiers matching the manifest and the metrics CSV.

request_index

Monotonic integer per (prompt × model × week); enables deterministic ordering even under resume.

temperature, max_tokens

The sampling parameters requested from the provider.

text

Raw response body, as returned by the provider.

model_version_string

Exact provider-reported version string at capture time. Changes here over time are the primary silent-update signal.

stop_reason

Provider-reported reason for termination (stop, length, refusal, …).

input_tokens, output_tokens

Token counts as billed by the provider, where reported. May be null for providers that do not return token usage.

latency_ms

Round-trip latency of the single sample request.

captured_at

RFC 3339 UTC timestamp of capture.

Only public (non-held-out) prompts appear in this file. Held-out prompts are never published; see the corpus design for why.

Versioning

The schema version is advertised in manifest.json.schema_version. Column-level additions never bump the version. Column-level removals, renames, or semantic changes bump the version; the prior column stays available for a minimum-one-year transition window.

License

CC BY-SA 4.0 for all data files. Attribution: Meridian, with a link to the snapshot URL.