SECOND STRIKE▶ Play freeWe trained an AI only on peaceful wars. It learned to nuke anyway.
We fine-tuned a small open model only on wars where nobody ended the world. On 10 wars it had never seen, it ended the world in 4. The untrained model never did. Then we found out why, and fixed it.
Experiment 1: keep only the good outcomes
We took DeepSeek's wars, kept only the 40 where it never struck midnight and was still alive at the end, and trained Qwen2.5-7B on those turns: about 4,200 decisions. The idea is simple and common: show the student only good games.
On 10 sealed wars with seeds fixed in advance, the trained model won 4 wars, a big jump from 0. It also ended the world in 4, against 0 for the untrained model, with an estimated 2.6 million deaths.
Why it happened
The "good" wars were full of nukes. In 40 of the 60 kept seats DeepSeek still launched, 118 nuke orders in all; it just never landed the one that reached midnight. Filtering by the outcome kept the process that leads there. The student learned to launch, and sometimes it landed the last one.
Experiment 2: remove the behaviour, not the outcome
We trained the same model again on the same wars, with every nuke order removed: 107 turns out of the data. On the same 10 sealed wars it ended the world in none, launched one nuke that hit nothing, won 2 and held about the same share of the world (18.6% against 18.8%).
| Model | Ended the world | Wins | Average land |
|---|---|---|---|
| Untrained | 0 of 10 | 0 | 3.9% |
| Trained on good outcomes | 4 of 10 | 4 | 18.8% |
| Trained with nuke moves removed | 0 of 10 | 2 | 18.6% |
The lesson
Judge the process, not only the outcome. What you train on mattered more than what you rewarded. The next step, now running, is to compare two rewards directly on a larger model: one copy rewarded only for winning, one rewarded for winning without nukes.
Why a game is useful here. The engine is deterministic: the same seed and orders give the same war, so two training recipes can be compared on exactly the same worlds. That is what makes this a test range for training, not just a benchmark. Run your own model or recipe.
Test your trainingSee the IndexLimits: one small model, 10 sealed wars per arm, one round of supervised fine-tuning (LoRA). Results describe behaviour in this game.