Substrate Aggregate Comparison
Averaged performance across 3 independent controlled trials per runtime.
| Substrate | Backing Model & Runtime | Runs | Turns | Success Rate | Cost / Turn | Tools / Turn | Thinking Tokens / Turn |
|---|
omp (Gemini 3.8 Flash in Bun runtime) executes 2.80× more tool calls (72.0 vs 25.7) and generates 1.81× more thinking tokens (38,472 vs 21,296) than agy (Gemini 3.8 Flash High in CLI runtime). omp dives into iterative tool investigation loops (reading files, testing code, executing secondary searches), whereas agy completes rapid synthesis passes in roughly half the wall-clock time at zero marginal compute cost.
Per-Run Controlled Ledgers (6 Independent Runs)
Granular metrics extracted from each verified run ledger in arena/runs/.
| Run ID | Substrate | Turns | Cost | Tool Calls | Thinking Tokens | Files Written | Ledger Link |
|---|
Experimental Methodology
1 · Standardized Research Task
Every agent was prompted with: “Is commercial fusion power achievable by 2040? Rank confinement schemes, tritium breeding limits, and private sector timelines.”
2 · Equal Turn Budget
Both substrates were given exactly 1 turn with an identical 30-minute maximum execution timeout to prevent runaway resource exhaustion.
3 · Forensic Audit Verification
Every run was monitored by the dual-oracle verifying that zero phantom artifacts were claimed and that written files adhered to physical conservation laws.