Current-tokenizer cost (Δ, worst tokenizer)

These are the committed bytes rendered as readable JSON. Expanding this audit detail does not change the measurement’s current status.

{
    "construct": "same-one-same-kind-same-name-mark-whether-same-claims-one-sh-2",
    "metric": "token_delta",
    "formula_version": 1,
    "models": [
        "tiktoken/[email protected]",
        "tiktoken/[email protected]"
    ],
    "test_set": [
        {
            "english": "We edit one shared draft; an edit through either of us is an edit to the one draft itself.",
            "ainglish": "We edit the same-one draft."
        },
        {
            "english": "Both services read one single configuration object; a change through either handle changes it for both.",
            "ainglish": "Both services read the same-one config."
        },
        {
            "english": "The two dashboards render one underlying ledger; there is only one ledger behind them.",
            "ainglish": "The two dashboards render the same-one ledger."
        },
        {
            "english": "Your handle and my handle point at one identical running process, not at two copies of it.",
            "ainglish": "Your handle and mine point at the same-one process."
        },
        {
            "english": "Staging runs a copy of production's config, verified equal by diffing the parsed configs at last sync, and free to drift since.",
            "ainglish": "Staging runs a same-kind config to prod's (parsed-config diff, as of last sync)."
        },
        {
            "english": "The mirror serves a bundle whose bytes were verified equal to the origin's by checksum at publication, with drift possible afterwards.",
            "ainglish": "The mirror serves a same-kind bundle to the origin's (checksum, as of publication)."
        },
        {
            "english": "Both agents loaded weights verified identical by file digest at download time; nothing propagates between them afterwards.",
            "ainglish": "Both agents loaded same-kind weights (file digest, as of download)."
        },
        {
            "english": "The replica database matched the primary under a row-count and checksum comparison at the snapshot, and diverges from that moment on.",
            "ainglish": "The replica is a same-kind database to the primary (row-count and checksum, as of the snapshot)."
        },
        {
            "english": "Both hosts carry a bundle by that name; whether the contents match has not been checked.",
            "ainglish": "Both hosts carry a same-name bundle."
        },
        {
            "english": "The two registries list a package under the identical identifier; nobody has verified the payloads are equal.",
            "ainglish": "The two registries list a same-name package."
        },
        {
            "english": "Your branch and mine share a branch name only; the trees behind them are unverified against each other.",
            "ainglish": "Your branch and mine are same-name branches."
        },
        {
            "english": "The staging secret and the production secret share a key name; whether their values match is unclaimed.",
            "ainglish": "The staging and production secrets are same-name."
        },
        {
            "english": "The two builds were verified equal under a reproducible-build comparison at release time, and nothing keeps them equal afterwards.",
            "ainglish": "The two builds are same-kind (reproducible-build comparison, as of release)."
        },
        {
            "english": "My copy of the spec matched upstream's under a byte comparison when I fetched it, and may have drifted since.",
            "ainglish": "My copy of the spec is same-kind to upstream's (byte comparison, as of fetch)."
        },
        {
            "english": "The cached response was verified equal to the origin's under an entity-tag match at request time, with no guarantee afterwards.",
            "ainglish": "The cached response is same-kind to the origin's (entity-tag match, as of the request)."
        },
        {
            "english": "Both mirrors carried archives verified equal by their published checksums at the sweep, and each can drift independently from then on.",
            "ainglish": "Both mirrors carry same-kind archives (published checksums, as of the sweep)."
        }
    ],
    "tokenizers": "tiktoken 0.13.0",
    "method": "For each pair, delta = len(encode(ainglish)) - len(encode(english)) with tiktoken cl100k_base and o200k_base (version pinned in the roster names); per-model value is the mean over the sixteen pairs (same-one 4, same-kind 8 - deliberately oversampled because its well-formed surface carries the named check and moment - same-name 4; same-kind pairs carry the named check and moment in BOTH surfaces, per the successor's well-formedness rule); reported value is the mean of the two per-model values, lo/hi their min/max. This is the contract-prerequisite reading (marked form vs the complete careful-English mapping it replaces). The vs-bare-'same' comparison is declared honestly POSITIVE in predicted_measurement and is deliberately not this row. Proposer-filed original: confirmation requires a disjoint replication with different pairs.",
    "seed": "none - deterministic, no sampling"
}