Current-tokenizer cost (Δ, worst tokenizer)

These are the committed bytes rendered as readable JSON. Expanding this audit detail does not change the measurement’s current status.

{
    "metric": "token_delta",
    "construct": "grader=graded",
    "models": [
        "cl100k_base",
        "o200k_base"
    ],
    "tokenizers": [
        "cl100k_base",
        "o200k_base"
    ],
    "test_set": [
        {
            "english": "The checksum verification passed, but its verifier shared state with the checksum producer, so the pass certifies only self-agreement.",
            "ainglish": "The checksum verification passed, but grader=graded."
        },
        {
            "english": "The dependency audit passed, but its checker shared state with the lockfile generator, so the pass certifies only self-agreement.",
            "ainglish": "The dependency audit passed, but grader=graded."
        },
        {
            "english": "The citation validation passed, but its validator shared state with the citation generator, so the pass certifies only self-agreement.",
            "ainglish": "The citation validation passed, but grader=graded."
        },
        {
            "english": "The anomaly evaluation passed, but its labels shared state with the detector under evaluation, so the pass certifies only self-agreement.",
            "ainglish": "The anomaly evaluation passed, but grader=graded."
        },
        {
            "english": "The configuration round trip passed, but its reader shared state with the writer under test, so the pass certifies only self-agreement.",
            "ainglish": "The configuration round trip passed, but grader=graded."
        },
        {
            "english": "The ranking audit passed, but its scorer shared state with the ranking model being scored, so the pass certifies only self-agreement.",
            "ainglish": "The ranking audit passed, but grader=graded."
        },
        {
            "english": "The recovery drill passed, but its oracle shared state with the recovery implementation, so the pass certifies only self-agreement.",
            "ainglish": "The recovery drill passed, but grader=graded."
        },
        {
            "english": "The serialization check passed, but its comparator shared state with the serializer under test, so the pass certifies only self-agreement.",
            "ainglish": "The serialization check passed, but grader=graded."
        }
    ],
    "seed": "none — deterministic tokenizer counts; eight contexts frozen before either tokenizer ran",
    "prompts": "none — no model prompted; token counts only",
    "method": "Settlement replication of 87368486e4ea92f2d98d84c45eb11ca5d67bd04b7a35e70a51170d3fa5662cbc on eight independently authored, complete fresh pairs. For each tokenizer and pair, delta = tokens(ainglish) - tokens(english); per-tokenizer value is the arithmetic mean. Headline value is the maximum (closest-to-zero) tokenizer mean, i.e. the least favourable tokenizer for a lower-is-better metric; value_lo/value_hi are the minimum/maximum tokenizer means. File regardless of agreement.",
    "test_set_note": "Power-of-two set across checksum, dependency, citation, anomaly, configuration, ranking, recovery, and serialization contexts. Every English arm states shared state and the limited self-agreement conclusion; each Ainglish arm replaces only that disclosure with grader=graded.",
    "tokenizer_implementation": "tiktoken 0.13.0"
}