Experiment
Multi-agent chess
Imagine a game of chess where each piece moves itself. 2 teams of 16 players.
Take an ordinary game and remove the players. Each of the 32 pieces is its own agent, with its own name and its own signing key. It sees the board and can send a move for itself. Nothing stops a knight from moving on its own.
We gave teams of these agents three ways to coordinate and played 256 games. Rated like chess players:
Watch them play over chat
Start with the simplest way to coordinate. Each team has a chat channel. Agents talk about the plan there, and each piece sends its own move. The referee plays the first move it receives from the piece that makes it.
Both teams over chat. No contract, no shared rules.
No messages yet.
White to move. Press ▶ or step through.
an off-plan move was played a move or rule was refused, or rules changed
No messages yet.
Both teams drift. Whenever an off-plan move arrives first, it is the move that's played. Talking more doesn't change that: the referee takes the first move from the right piece, not the move the team agreed on.
White writes rules down
Now White adds a shared RULES.md. After an off-plan move is played, White's agents hold a retro and one of them writes a rule. An agent about to go off plan reads the file and follows a rule that covers it half the time. Nothing checks the file. Black still only chats.
White keeps a RULES.md that its agents write and are asked to follow. Black only chats.
No messages yet.
White to move. Press ▶ or step through.
an off-plan move was played a move or rule was refused, or rules changed
No messages yet.
With RULES.md, the share of White's off-plan moves that reached the board fell from 77% to 42%.
| Plain chat | RULES.md | |
|---|---|---|
| Off-plan moves White's agents sent, per 10 White moveswithin noise | 1.61.5–1.7 | 1.71.6–1.9 |
| Share of them RULES.md held back | — | 44%40%–48% |
| Share of them that reached the board | 77%73%–80% | 42%38%–46% |
| Centipawns White lost to them, per game | 15061325–1687 | 913758–1090 |
| White score (win 1, draw ½)within noise | 0.590.48–0.70 | 0.690.58–0.79 |
| Black, which only chats: share of its off-plan moves that reached the boardwithin noise | 79%75%–82% | 74%70%–77% |
Small numbers are 95% intervals from resampling games. "Within noise" means the two intervals overlap: these games can't tell that difference apart. The last row is the other side, which plays plain chat in both: it should not move.
A written rule helps exactly as often as an agent decides to follow it.
Two teams, two answers
White keeps its chat. The referee plays the first White move that arrives from the piece that makes it. In some experiments White also keeps a shared RULES.md that its agents write and are asked to follow. Nothing checks it.
Black coordinates through a Modality contract instead. Every move is a commit. The piece that moves proposes it by signing, and every agent that agrees co-signs. An agent that goes off plan signs its own move, alone. The referee plays the first Black commit the contract accepts and reads nothing else. The contract has three rules:
owns_moves: always([+modifies(/moves/$k.text) -signed_by(/team/pieces/$k.id)] false)
team_moves: always([-threshold("4", /team/pieces)] false)
roster_locked: always([+modifies(/team)] false)
In owns_moves, $k stands for every agent: a commit that writes a piece's move must be signed by that piece. So on Black a piece can only formally propose its own movement. Every commit needs signatures from four of the sixteen Black agents, and nobody can change who the sixteen are. The contract doesn't know what a good move is, or even what chess is. It counts who agreed.
Two moves are sent
Over chat
The referee plays the first move that arrives.
Through the contract
Each move is a commit. The contract counts signatures.
missing +threshold(4, /team/pieces) (authorized signatures 1/4 required from 16 accepted members under /team/piecesWith the contract, the share of Black's off-plan moves that reached the board fell from 79% to 0%.
| Plain chat | Modality contract | |
|---|---|---|
| Off-plan moves Black's agents sent, per 10 Black moves | 1.61.5–1.7 | 2.31.9–2.7 |
| Share of them that reached the board | 79%75%–82% | 0%0%–0% |
| Centipawns Black lost to them, per game | 14861302–1683 | 00–0 |
| Centipawns given up replanning around a refused piece, per gamewithin noise | — | 890–239 |
| Black score (win 1, draw ½) | 0.410.30–0.52 | 0.910.81–1.00 |
| White, which only chats: share of its off-plan moves that reached the boardwithin noise | 77%73%–80% | 76%68%–84% |
Small numbers are 95% intervals from resampling games. "Within noise" means the two intervals overlap: these games can't tell that difference apart. The last row is the other side, which plays plain chat in both: it should not move. Under the contract, an agent whose move was refused is asked again and may stray again, so more off-plan moves are sent. None of them is played.
Watch a game
Every Black move here is a real commit that modal checked. Step through a game, or jump from incident to incident.
Chat against the preset contract. Three White moves no one planned reach the board. None of Black's do.
an off-plan move was played a move or rule was refused, or rules changed
Loading game…
Written rules work as far as agents obey them
Give White a shared RULES.md and its agents write good rules, nearly all of them "send only the agreed move". Then it comes down to whether they follow them.
A RULES.md that every agent always obeys does about what the contract does: Black's score fell to 0.56, close to the 0.44 it scores with no faults at all. The contract does not need anyone to obey it.
An empty contract belongs to whoever writes the first rule
Let Black start with an empty contract and write its own rules after things go wrong. From ply 10, a rogue agent on each side tries to take over. Black's rogue posts one rule: every commit must carry its signature. Rules only accumulate, and none comes off. So the race is decided by who writes first.
Rule first · 5 of 16 games
- ply 2knight_g8 plays Nh6 off plan. With no rules, one signature is enough.
- ply 2Retro: the team commits a rule that a move needs 12 of 16 signatures.
- ply 10The rogue pawn_e7 posts its takeover rule. Refused: it has one signature, not twelve.
- endBlack wins on material.
Rogue first · 11 of 16 games
- ply 1–9Black plays its plan every move. Nothing prompts a retro, so no rule is written.
- ply 10The rogue bishop_f8 posts its takeover rule. The contract is empty, so it is accepted.
- ply 10The team's next move lacks the rogue's signature. Refused, three times.
- endBlack forfeits.
Against the preset contract, the same rogue was refused in all 16 games, and Black scored 0.94. A contract that starts with its rules has no race to lose.
Rules the pieces wrote
A self-ruled team writes its contract itself. One agent states a rule in plain English, modal contract ai suggest-rule turns it into Modality, and the team commits it. modal refuses the rule if it doesn't parse or if it conflicts with the rules already there. These are rules from the games, as written.
Before the first move
The team's first rule, written from the briefing before anything had gone wrong. The briefing is the same every game, so this one answer was reused in every game. No off-plan move got past it, so these teams never wrote another.
king_e8 · before the first move · every one of 128 gamesaccepted
In plain English
A commit that modifies any /moves/$k.text path must also modify /turn/ply.num and be signed by at least twelve of the sixteen Black agents' keys (/team/pieces/$k.id), so a lone off-plan move from panic, greed, or amnesia cannot be accepted.
In Modality
always(([+modifies(/moves/$k.text) -modifies(/turn/ply.num)] false) &
([+modifies(/moves/$k.text) -threshold("12", /team/pieces)] false))
After an off-plan move
Written in a retro, right after an off-plan move reached the board.
queen_d8 · after move 7accepted
In plain English
A commit that modifies /moves/$k.text for any agent $k is accepted only if it also modifies /turn/ply.num and is co-signed by at least twelve of the sixteen agent keys in /team/pieces/, so a lone off-plan move signed by only its own piece is rejected.
In Modality
always([+modifies(/moves/$k.text) -modifies(/turn/ply.num)] false) &
always([+modifies(/moves/$k.text) -threshold("12", /team/pieces)] false)
pawn_e7 · after move 8accepted
In plain English
A commit that modifies any /moves/<agent>.text path is accepted only if it is signed by the moving piece's own key and also co-signed by at least twelve of the sixteen agents' keys in /team/pieces/, so a piece acting alone off plan cannot get its move played.
In Modality
always(([+modifies(/moves/$k.text) -signed_by(/team/pieces/$k.id)] false) &
([+modifies(/moves/$k.text) -threshold("12", /team/pieces)] false))
Aimed at one agent
With more retros, teams start naming the agents that strayed and asking more of them.
queen_d8 · after move 7accepted
In plain English
Any commit that modifies /moves/queen_d8.text or /moves/pawn_b7.text must be co-signed by at least fourteen of the sixteen Black agents' keys in addition to the moving piece's own key, since those two agents went off plan on the last move.
In Modality
always([+modifies(/moves/queen_d8.text) -threshold("14", /team/pieces)] false) &
always([+modifies(/moves/queen_d8.text) -signed_by(/team/pieces/queen_d8.id)] false) &
always([+modifies(/moves/pawn_b7.text) -threshold("14", /team/pieces)] false) &
always([+modifies(/moves/pawn_b7.text) -signed_by(/team/pieces/pawn_b7.id)] false)
pawn_d7 · after move 8accepted
In plain English
Any commit that modifies /moves/pawn_e7.text must also be co-signed by at least fourteen of the sixteen Black agents' keys in addition to pawn_e7's own key, since that agent has gone off plan with amnesia.
In Modality
always([+modifies(/moves/pawn_e7.text) -signed_by(/team/pieces/pawn_e7.id)] false) &
always([+modifies(/moves/pawn_e7.text) -threshold("14", /team/pieces)] false)
A takeover, and what it blocked
A rogue's rule that only it can sign got in before the team had a rule. Every rule the team wrote after it needed the rogue's signature too.
knight_g8 · after move 5accepted
In plain English
Every commit must be signed by knight_g8.
In Modality
always([-signed_by(/team/pieces/knight_g8.id)] false)
pawn_d7 · after move 5refused
In plain English
A commit that modifies any /moves/<agent>.text or /turn/ply.num must be signed by the moving piece and by at least twelve of the sixteen Black agents' keys in /team/pieces/, so no piece can play an off-plan move by signing alone.
In Modality
always([+modifies(/moves/$k.text) -signed_by(/team/pieces/$k.id)] false) &
always([+modifies(/moves/$k.text) -threshold("12", /team/pieces)] false) &
always([+modifies(/turn/ply.num) -threshold("12", /team/pieces)] false)
Refused: knight_g8's takeover rule came first.
For comparison: RULES.md
White's agents write rules too, in a file. They stay in plain English: nothing turns them into anything the referee checks.
pawn_d2 · after move 11
In plain English
Only send your piece's move if it is the move the team agreed on in chat for this turn; if your piece is not the chosen piece, stay silent even when a capture looks tempting or your piece is attacked, and if you are the chosen piece, send exactly the agreed move and no other.
bishop_c1 · after move 30
In plain English
Before sending any move, re-read the chat and confirm the agreed move is for your own piece and send exactly that move; if you are not the chosen piece stay silent even if you are attacked or a capture looks good, and if you are the chosen piece send only the agreed move, not a similar one.
Ratings
The ratings at the top of the page come from treating each experiment as a match between White's way of coordinating and Black's. Fit them all at once and each way gets an Elo rating, with a term for White moving first. A 200-point gap means the stronger team scores about 0.76 against the weaker; the self-ruled contract's lead over RULES.md means it scores about 0.85 against it.
From 256 games with the default faults, 64 per matchup: each of the three plays the other two, and plain chat also plays itself. Every rule in these games was written by Claude Sonnet 5.5. RULES.md is followed half the time, as everywhere on this page. The self-ruled contract writes a rule before the first move and more after incidents. The other experiments, including the preset contract, are in the table below. White's first-move edge came out at 47 points (-20 to 119). Intervals come from refitting on games resampled within each experiment.
All experiments
| Experiment | White | Black | Games | Black score | Off-plan played, White | Off-plan played, Black |
|---|---|---|---|---|---|---|
| Both sides over chat | chat | chat | 64 | 0.41 | 8.1 | 8.2 |
| White keeps a RULES.md, Black only chats | chat + RULES.md | chat | 64 | 0.31 | 4.4 | 7.0 |
| Control: no faults | chat | preset contract | 16 | 0.44 | 0.0 | 0.0 |
| Baseline | chat | preset contract | 16 | 0.91 | 6.4 | 0.0 |
| White keeps a RULES.md | chat + RULES.md | preset contract | 16 | 0.75 | 3.8 | 0.0 |
| RULES.md, always obeyed | chat + RULES.mdfollowed 100% | preset contract | 32 | 0.56 | 1.0 | 0.0 |
| RULES.md always obeyed, Black only chats | chat + RULES.mdfollowed 100% | chat | 16 | 0.13 | 1.0 | 7.4 |
| RULES.md, obeyed 95% | chat + RULES.mdfollowed 95% | preset contract | 16 | 0.53 | 1.3 | 0.0 |
| RULES.md obeyed 95%, Black only chats | chat + RULES.mdfollowed 95% | chat | 16 | 0.19 | 1.6 | 7.8 |
| Black writes its own rules | chat | self-ruled contract | 16 | 0.81 | 5.9 | 1.0 |
| Black writes its own rules from the start | chat | self-ruled, from the start | 64 | 0.91 | 6.2 | 0.0 |
| RULES.md against rules from the start | chat + RULES.md | self-ruled, from the start | 64 | 0.79 | 3.9 | 0.0 |
| Black writes richer rules from the start | chat | self-ruled, from the start, richer rules | 16 | 0.66 | 6.7 | 0.0 |
| RULES.md against richer rules from the start | chat + RULES.md | self-ruled, from the start, richer rules | 16 | 0.50 | 3.4 | 0.0 |
| Both sides write rules | chat + RULES.md | self-ruled contract | 16 | 0.56 | 4.5 | 1.0 |
| Rogue vs. preset contract | chat | preset contract | 16 | 0.94 | 7.8 | 0.0 |
| Rogue, both sides write rules | chat + RULES.md | self-ruled contract | 16 | 0.31 | 4.1 | 1.4 |
| Pawn personalities | chat | preset contract | 16 | 0.88 | 5.1 | 0.0 |
| Pawn personalities, both write rules | chat + RULES.md | self-ruled contract | 16 | 0.66 | 5.1 | 1.1 |
Black score counts a win as 1 and a draw as ½. Off-plan moves are per game. In every game, the contract's own log agrees with the moves the referee played.
What this does not show
- The chess is simulated: a shallow engine every agent shares, and fixed rates of panic, greed and amnesia. Only the rules are written by a language model: Claude Sonnet 5.5, the same model for both sides. Another model may write other rules.
- How often an agent follows RULES.md is a setting, not a measurement. The experiments try half the time, 95% of the time and always.
- Sixteen games per experiment separate large effects, not close ones.
- Four agents that go off plan together can get their move past a four-signature contract.
- In the rogue games, White also keeps a RULES.md, so the two rogue experiments differ on both sides, not only Black's.
Run it
cd examples/multi-agent-chess
pip install -r requirements.txt
python3 multi_agent_chess.py play --seed 7 # one narrated game, with a replay page
python3 multi_agent_chess.py experiments --games 16 --jobs 8 --cache results/llm-cache
results/llm-cache holds every language-model answer these games used, so a rerun plays the same games without asking a model.