Studios tune level difficulty by watching bots play, because waiting for real players means shipping the wall before you know it is there. So I built the game, trained an agent on it, and asked how hard each level is. The answer was not a number. It was a number per player.
Greedy, level 20
42 tiles in 21 moves
Ten times the training moved the agent from 56% to 59%. A twenty-line greedy rule, which simply takes whichever swap clears the most target tiles right now, still reaches 83%. That gap is the honest result: PPO learns, more training helps, and in this environment a one-move heuristic is still ahead of it.
Every player below is dealt the identical starting board, so what differs is the choices, not the luck. One deal is still one deal: on levels 3 and 20 every player clears it, and on level 31 every player runs out, with the random player collecting more tiles than greedy. What the boards show is how each one plays, not who wins. The curves above are the measurement; these are one episode inside them.
42 tiles in 21 moves, same starting board for all three
Random
picks any legal swap
PPO 2M
trained 2,000,000 steps
Greedy
one-move lookahead
Subtracting the random player’s clear rate from the greedy player’s gives a number per level: how much skill is worth there. It peaks at 63% on level 27. On 8 levels it is under five points. Those levels cannot tell a careful player from a careless one, which is worth knowing before shipping them as a tutorial.
A spike detector looks for a level-to-level drop too large to be chance. My first version used one fixed threshold for every curve, and flagged three walls on the greedy curve. They were measured with sixty episodes against the others’ two hundred; re-measured with five times as many, those drops fell from twenty-odd points to single digits. They were never levels. They were the sample size.
The threshold now scales with the measurement behind it: 3 standard errors of a difference in proportions, so a curve built from fewer episodes has to clear a higher bar.
a drop must exceed20.8 pts
With the bar scaled to the measurement, 2 drops clear it: Random from level 18 to 19 (15.0 points against a bar of 15.0), Greedy from level 29 to 30 (9.7 points against a bar of 9.3). The level table was built as a straight line, so anything flagged is worth opening before it is believed.