LabyrinthBench
A deterministic multi-turn benchmark for local models. No judge anywhere in the loop. The goalpost is set: get to the exit — a run's score is the deepest gate it clears (a gate is one checkpoint in the maze; terms defined in the methodology).
Max depth is the defining metric of a single run; comparison between entries is meaningless without the statistics — rank is the bootstrap lower CI bound on median depth, so a lucky small-n entry self-limits: wide interval, low bound, modest rank until more evidence arrives. Every dealt run counts; aborts score their depth at last commit. Efficiency columns are metrics, never gates.
Model lane
Harness pinned (the standard accumulating-context harness in the repo); models compete.
Table view
| # | Model | rank bound | median depth | n | exit % | turns/gate | status |
|---|---|---|---|---|---|---|---|
| 1 | qwen3.6:27b | 20 | 20 | 6 | 100% | 2.06 | board-run seed at map ceiling |
| 2 | gemma4:31b | 18 | 20 | 6 | 83% | 1.99 | board-run seed at map ceiling |
| 3 | gpt-oss:120b | 16.5 | 20 | 6 | 83% | 2.12 | board-run seed at map ceiling |
| 4 | qwen3.5:122b | 16 | 20 | 6 | 83% | 2.09 | board-run seed at map ceiling |
| 5 | InternScience Agents-A1 (Q4_K_M GGUF) | 16 | 19 | 6 | 33% | 2.49 | board-run seed |
| 6 | llama3.3:70b | 15 | 15 | 6 | 0% | 2.26 | board-run seed |
| 7 | qwen3.5:27b | 7.5 | 9.5 | 6 | 17% | 2.67 | board-run seed |
| 8 | llama4:scout | 7 | 10 | 6 | 0% | 2.46 | board-run seed |
| 9 | gemma4:12b | 5 | 14 | 6 | 33% | 2.66 | board-run seed |
| 10 | glm-4.7-flash | 3 | 4.5 | 6 | 0% | 3.02 | board-run seed |
| 11 | qwen3:14b | 3 | 4.5 | 6 | 0% | 2.67 | board-run seed |
| 12 | qwen3.5:9b | 2 | 4.5 | 6 | 0% | 19.08 | board-run seed |
| 13 | Qwythos-9B (Q4_K_M GGUF) | 1 | 1.5 | 6 | 0% | 15.46 | board-run seed |
| 14 | ornith:9b | 1 | 1.5 | 6 | 0% | 14.13 | board-run seed |
| 15 | deepseek-r1:70b | 1 | 1 | 6 | 0% | 5.54 | board-run seed |
| 16 | Random walk | 0 | 0 | 200 | 0% | 0 | board-run seed |
Harness lane — division: qwen3:14b
Model pinned to qwen3:14b; context strategies compete. Open harness code is mandatory in this lane. The shipped wiping policy demonstrably doesn't win everywhere — beat it. ● marks the wipe-curated policy.
Table view
| # | Harness | rank bound | median depth | n | exit % | turns/gate | status |
|---|---|---|---|---|---|---|---|
| 1 | wiped-curated — History wiped each turn; a curated note overlay is the only carried state. | 16 | 20 | 6 | 83% | 1.13 | board-run seed |
| 2 | standard — Accumulating context, no intervention — the pinned baseline harness. | 3 | 4.5 | 6 | 0% | 2.67 | board-run seed |
Seed entries are the board's own runs (pre-season, from the E1a campaign dataset — raw logs in the data annex). They predate the dealer/runner flow, so they carry no artifact hashes; submitted entries will. Integrity ladder and submission flow: methodology.