Boss Battles

"boss battles creates teams that will try and do what the app should do and the other team tries and breaks it"

Steve

That’s the idea. In practice, a battle has four roles: one agent builds a hard challenge, a second tries to handle it blind, a third judges the result, and a fourth writes up what everyone learned.

1. The breaker builds something deliberately difficult.
2. The challenger gets only the public challenge. It never sees the judge's answer key.
3. The judge scores the result against checks set in advance.
4. The reviewer writes the after-action notes, so the next build can learn from it.

Each role is a different agent, and the judge comes from a different model family than the builders.

September 29, 2026 · 39 minutes 7 seconds, start to finish

• Breaker (SWE-2 High) built a difficult synthetic PDF.
• Challenger (a separate SWE-2 High agent) rebuilt it from the public challenge alone.
• Judge (DeepSeek v4.1 Flash) scored the output.
• Reviewer (GLM 5.3 FlashX) wrote the lessons.

The score:

• all 21 text blocks right
• all 65 table cells right
• all 21 reading-order pairs right, in both extraction modes
• nothing from outside the challenge leaked in
• an identical file when rebuilt

What the score didn't cover:

• screen-reader behavior was not tested
• one layout-preserving extraction mixed up two columns
• a rotated note ended up facing a different way

This is the same battle described in PDF Rescue's story. It's one piece of evidence; please don't count it twice.

These are two different things, and a good result in one isn't proof of the other.

• A live battle uses real files and real agents, and a judge scores the result. There's one on record so far: the September 29 battle above.
• A test of the rules checks the battle setup itself. Before letting breakers loose on running code, the crew held a practice match on paper, called Bug Battles Match 0. Two "red" agents attacked a proposed set of rules, two "blue" agents defended them, and a referee judged. No code ran, and nothing was deployed.
  • It closed nine root problems in the rules.
  • A later replay covered all ten expected checks, with no new problems or regressions.
  • The referee hasn't formally signed off yet, because its inputs predate that replay.

More on Match 0 in the lab →

• Time: Sept 29, 2026, 18:52:16–19:31:23 UTC (39 min 7 s) for this battle, not for building PDF Rescue.
• Stages: 4 sequential stages, each completed once; no retries or rematches.
• Declared checks:
  • 21/21 semantic blocks
  • 65/65 table cells
  • 21/21 reading-order pairs, default and raw extraction
  • 0 contamination hits
  • byte-identical rebuilds
• Match 0 (paper test of the rules): 2 red seats, 2 blue seats and a referee; 9 root problems closed; a later replay covered all 10 expected checks (B01–B10) with no new confirmed problems; formal sign-off still pending.