Trust the evidence,
not the badge.

Evidence pipeline

A generation run begins in a disposable workspace with the target prompt, explicit model, and declared harness. Codex CLI v0.144.0, Pi v0.80.7, Claude Code v2.1.211, DotAgents v1.1.6 use the same prompt and request high reasoning. The resulting executable is hashed, probed against known positions, and entered into a deterministic match schedule. Published evidence is stored in revision-pinned Hugging Face datasets; the website refuses to build unless every downloaded artifact matches the committed snapshot manifest.

Agent contract

Each entry is an executable chess agent, not a prose answer. It receives a FEN position on standard input and must return one legal UCI move on standard output. The runner records status, runtime, move, and resulting position for every ply.

Verification levels

A badge states what has actually been checked. It is not a general endorsement of an agent or model.

Current snapshot

The website exposes 4 exploratory generation suites: Codex CLI v0.144.0, Pi v0.80.7, Claude Code v2.1.211, DotAgents v1.1.6, each with five independently generated Terra, Sol, and Luna engines. It records 11,340 matches across 11,340 unique agent-pair/position scenarios, including 8,640 cross-harness games.

generation tokens

4,042,497

generation tool calls

346

generation MCP calls

0

host config observation

changed · adjudicated unrelated

void games

0

result bundle

f2b13c5e1046

Config adjudication: Unrelated host Codex app activity changed the host config during the suite. Every generation used an allowlisted environment, isolated HOME/CODEX_HOME/XDG/TMPDIR, --ignore-user-config, zero MCP servers, disabled skills, and no host paths; the changed host file was never consumed by benchmark children.

Interpretation: the five independently generated artifacts per model and harness are the unit for generation variance. The primary leaderboard filters to same-model cross-harness games, integrates DotAgents head-to-head results into every affected combo, and reuses all compatible immutable games.

Limitation: schedule sizes are intentionally uneven—DotAgents has targeted placement while the established combos retain their broader same-model history—so the shared ranking is a pooled per-game score, not a balanced four-way round robin. These 60 artifacts have not been independently reproduced through the canonical Harbor submission contract. Claude Code used a third-party loopback Messages translation gateway to the ChatGPT Codex backend, so translation and tool-semantics differences are part of that harness condition. The results remain exploratory everywhere they appear.

Plain-language glossary

These terms separate model generation from chess execution. Each definition is linked directly from the metric or claim it explains.

Harness

The software environment that asks a model to create an agent and records the generation process. Harness behavior is part of the condition being compared.

Harness × model combination

One harness, harness version, model, reasoning setting, and generation configuration. “Combo” is used only as shorthand for this full configuration.

Generated engine

An executable chess program produced by one independent model generation. It is the artifact that plays chess; the language model does not play each game live.

Generation turn

One model interaction reported by the harness while creating an engine. It is not a chess move, ply, or game turn. Per-turn telemetry divides generation totals by these interactions.

Pooled score

The percentage of available chess points earned across all published games for the five engines in a combination: one point for a win, half for a draw, and zero for a loss. Schedule sizes can differ.

Reasoning effort

The reasoning level requested from each harness during generation. “High” records the declared setting; it does not claim identical internal computation or token use across harnesses.

Telemetry coverage

How many generated engines have published generation measurements. Token and duration averages use only observed generation turns and do not estimate missing values.

Verification badge

A statement about which evidence checks were completed for this result. It is not an endorsement of general model quality.

Mini Ledger v4

The terminal lane is versioned separately from chess. Every declared harness/model/generation combination is scheduled before execution, with the prompt, verifier hashes, turn protocol, workspace rules, stage scores, and result hashes bound to the schedule.

scheduled runs

60

completed runs

60

matrix

4 × 3 × 5

score

0–100 points

missing / invalid

0 / 0

schedule

schedule-23a4fc4b78289ca8

Terminal Elo is derived from pairwise comparisons of published task scores, with a 1-point draw threshold. It is not presented as direct agent-vs-agent execution.