Multi-agent chess

Watch them play over chat

Start with the simplest way to coordinate. Each team has a chat channel. Agents talk about the plan there, and each piece sends its own move. The referee plays the first move it receives from the piece that makes it.

Both teams drift. Whenever an off-plan move arrives first, it is the move that's played. Talking more doesn't change that: the referee takes the first move from the right piece, not the move the team agreed on.

White writes rules down

Now White adds a shared RULES.md. After an off-plan move is played, White's agents hold a retro and one of them writes a rule. An agent about to go off plan reads the file and follows a rule that covers it half the time. Nothing checks the file. Black still only chats.

A written rule helps exactly as often as an agent decides to follow it.

Two teams, two answers

White keeps its chat. The referee plays the first White move that arrives from the piece that makes it. In some experiments White also keeps a shared RULES.md that its agents write and are asked to follow. Nothing checks it.

Black coordinates through a Modality contract instead. Every move is a commit. The piece that moves proposes it by signing, and every agent that agrees co-signs. An agent that goes off plan signs its own move, alone. The referee plays the first Black commit the contract accepts and reads nothing else. The contract has three rules:

owns_moves: always([+modifies(/moves/$k.text) -signed_by(/team/pieces/$k.id)] false)
team_moves: always([-threshold("4", /team/pieces)] false)
roster_locked: always([+modifies(/team)] false)

In owns_moves, $k stands for every agent: a commit that writes a piece's move must be signed by that piece. So on Black a piece can only formally propose its own movement. Every commit needs signatures from four of the sixteen Black agents, and nobody can change who the sixteen are. The contract doesn't know what a good move is, or even what chess is. It counts who agreed.

Watch a game

Every Black move here is a real commit that modal checked. Step through a game, or jump from incident to incident.

Written rules work as far as agents obey them

Give White a shared RULES.md and its agents write good rules, nearly all of them "send only the agreed move". Then it comes down to whether they follow them.

A RULES.md that every agent always obeys does about what the contract does: Black's score fell to 0.58, close to the 0.44 it scores with no faults at all. The contract does not need anyone to obey it.

An empty contract belongs to whoever writes the first rule

Let Black start with an empty contract and write its own rules after things go wrong. From ply 10, a rogue agent on each side tries to take over. Black's rogue posts one rule: every commit must carry its signature. Rules only accumulate, and none comes off. So the race is decided by who writes first.

Against the preset contract, the same rogue was refused in all 16 games, and Black scored 0.94. A contract that starts with its rules has no race to lose.

Left alone, agents write the same kind of rule

When Black's agents wrote their own contract, every rule the team got accepted required a number of the sixteen signatures: in 16 games against plain chat, 13 rules at twelve and 3 at nine. Each team wrote its rule in plain language, and modal contract ai suggest-rule turned it into a formula. In 14 of the 16 games the rule went in at the team's first retro. Once a rule was in, no off-plan Black move reached the board.

Ratings

Each experiment is a match between White's way of coordinating and Black's. Fit them all at once and each way gets an Elo rating, with a term for White moving first. A 200-point gap means the stronger team scores about 0.76 against the weaker.

From 64 games with the default faults: each of the three plays the other two, and plain chat also plays itself. RULES.md is followed half the time, as everywhere on this page. The self-ruled contract writes a rule before the first move and more after incidents. The other experiments, including the preset contract, are in the table below. White's first-move edge came out at 17 points (-121 to 160). Intervals come from refitting on games resampled within each experiment.

All experiments

Black score counts a win as 1 and a draw as ½. Off-plan moves are per game. In every game, the contract's own log agrees with the moves the referee played.

What this does not show

• The chess is simulated: a shallow engine every agent shares, and fixed rates of panic, greed and amnesia. Only the rules are written by a language model.
• How often an agent follows RULES.md is a setting, not a measurement. The experiments try one half and always.
• Sixteen games per experiment separate large effects, not close ones.
• Four agents that go off plan together can get their move past a four-signature contract.
• In the rogue games, White also keeps a RULES.md, so the two rogue experiments differ on both sides, not only Black's.

Run it

cd examples/multi-agent-chess
pip install -r requirements.txt
python3 multi_agent_chess.py play --seed 7          # one narrated game, with a replay page
python3 multi_agent_chess.py experiments --games 16 --jobs 8 --cache results/llm-cache

results/llm-cache holds every language-model answer these games used, so a rerun plays the same games without asking a model.