Check a Chance-Based TV Game by Enumeration Before Trusting a Simulation

A rules card says: draw two tokens, add them, that's your score. Five tokens, marked 0 through 4. Everyone nods. Nobody asks whether the first token goes back.

That unasked question is the game. It decides whether a contestant can draw the same token twice, whether the ceiling is 8 or 7, and how often the biggest scores land. It does not change the average score at all. That last fact is precisely why you enumerate before you simulate: a sampler will answer a question you have not finished asking, and the most obvious number it hands you is blind to the one thing you changed.

Five tokens and two draws is small enough to write down in full. Do that first — not because exact arithmetic is nobler than Monte Carlo, but because you cannot sample a rule you have not finished writing, and twenty-five rows is a specification review with a proof attached.

Turn the rule into states and transitions

Before any code, fix the pieces.

Objects. Five distinct tokens, one each marked 0, 1, 2, 3, 4. Distinct matters. If the props actually contain two tokens marked 3, the population below is not the one on your table.

The transition. A draw takes one token from whatever is currently available, each available token equally likely. That equality is an assumption about the physical procedure — a blind pull from a mixed bag — not a discovered fact. Write it down as an assumption so you can attack it later.

The state. After one draw you are holding a token, and the pool of available tokens has either changed or it hasn't. That is the whole state. After two draws you have a score.

The variant. Either the drawn token returns to the pool before the second draw, or it is spent. Call these with replacement and without replacement. They are ordinary probability terms and also, unfortunately, not words a production team says out loud. "Draw two tokens" covers both.

The stopping rule. Two draws, then score. Fixed. Nothing here is a decision. If the format lets a contestant stop after the first token or trade it in, that is a participant decision, and it needs a separate model with a stated policy for what contestants tend to do — which is a claim about people, not about tokens.

Now the reason to be pedantic. A simulator does not resolve ambiguity; it encodes whichever reading you typed. Running it for a million rounds instead of ten thousand shrinks sampling noise and leaves the assumption exactly where it was. And in this fixture the usual smoke test makes things worse: the average score is the same under both rules, so more rounds produce a more precise version of a number that cannot tell the two formats apart.

Enumerate the small model exactly

List every permitted ordered draw and give it a probability.

With replacement, the first draw is one of five tokens and the second is one of five, so there are 25 ordered pairs at 1/25 each. Without replacement, the second draw comes from the four remaining tokens, so there are 20 ordered pairs at 1/20 each.

Then group by score. The word ordered is doing real work, so decide it once and stay consistent: (2,3) and (3,2) are two outcomes, not one.

Three checks, done out loud.

The columns sum to 25 and 20, so the probabilities sum to one.

A score of 8 is possible only as (4,4). That requires the 4 to come back, which makes 8 impossible without replacement. A score of 0 is possible only as (0,0), gone for the same reason. The extremes at both ends are products of the same rule.

The nine possible sums are not equally likely. The with-replacement column runs 1, 2, 3, 4, 5, 4, 3, 2, 1. Uniform tokens do not produce uniform sums. That mistake is easy to make and it usually survives a simulator, because the simulator was written by the same person holding the same wrong belief.

Enumeration also catches the denominator. Under without replacement, the ten unordered pairs — {0,1} through {3,4} — each carry probability 1/10, and the twenty ordered pairs each carry 1/20. Count unordered pairs but divide by 25, or count ordered pairs and divide by 10, and your probabilities are wrong before the first sample is drawn. Choose the unit, then divide by the number of units.

Change one rule and compare more than the mean

Hold the tokens, the two draws and the score definition fixed. Change replacement and nothing else.

Mean: 4 either way. Not a coincidence of this token set. The second draw without replacement is uniform over the four remaining tokens, but as a marginal over all five it is still uniform — each token is equally likely to be the second one, by symmetry. Something has to be drawn second, and no token has a stronger claim than any other. So E[sum] = E[X₁] + E[X₂] = 2 + 2 = 4 under both rules. Removing replacement does not move the average. It only makes the two draws depend on each other.

Variance: 4 with replacement, 3 without. With independent draws the variances simply add: 2 + 2. Without replacement, the first draw tells you something about the second — take the 4 and you have weakened the pool for the second pull — so the two draws covary negatively and the variance of the sum drops. If the population variance (dividing by N) is σ², the covariance between two draws is −σ²/(N−1) = −2/4 = −0.5, and the sum's variance is 2 + 2 + 2(−0.5) = 3.

The size of that drop is a property of the population, not of the phrase "without replacement." Two draws from five lose a full point of variance; two draws from five hundred would barely notice. Take the same rule to a large-token format and the prediction is that the variance gap becomes negligible while the support still changes. That is a checkable consequence of the arithmetic, not a claim about your show.

Support: [0, 8] versus [1, 7]. The ceiling and the floor move together.

Tail: P(score ≥ 7) is 3/25 = 12% with replacement and 1/10 = 10% without. Higher with replacement, which is worth pausing on. Removing replacement sounds like it should make the round wilder. Here it makes big scores slightly rarer, because the only path to 8 was the double (4,4), and (4,4) is the first thing the rule change deletes.

Shape: the middle flattens. With replacement the single most likely score is 4, at 20%. Without, 4 is still 20% — but 3 and 5 join it, giving three scores tied for most likely. If the round's design leans on a recognizable typical score, without replacement dissolves the peak into a plateau. Nothing in the words "draw two tokens" predicts that.

So: same average, different ceiling, slightly different tails, different shape. If the format sells the chance of a perfect 8, without replacement quietly deletes it. If it sells the possibility of a humiliating 0, the same rule deletes that too. Neither fact is visible in the mean, which is identical.

One extension worth keeping in mind. For a sum score, only the distinctness of the two tokens matters, not their order — but that does not make unordered pairs equally likely under both rules. Without replacement, the ten unordered pairs of distinct tokens each carry 1/10. With replacement, each of those ten pairs carries 2/25, while the five doubles — {0,0} through {4,4} — carry 1/25 apiece, so the unordered list is not uniform. The moment a rule reads the order — a bonus when the second token is larger than the first — ordered enumeration becomes mandatory, because the order now carries value. Match the state space to the rules that actually pay out.

Use the simulation as a cross-check

Now the sampler earns its place, because you finally have something to check it against.

The standard library draws the line between these two operations and puts it in the function name. random.choices samples with replacement. random.sample samples without replacement. There is no separate flag; the name is the rule. A one-word slip silently swaps one format for the other.

import random
from itertools import product
from collections import Counter

tokens = [0, 1, 2, 3, 4]

# exact enumeration: 25 ordered pairs with replacement, 20 without
with_rep = Counter(a + b for a, b in product(tokens, repeat=2))
without  = Counter(a + b for a, b in product(tokens, repeat=2) if a != b)

# seeded sampling from an isolated generator
rng = random.Random(9595)
sample_with = [sum(rng.choices(tokens, k=2)) for _ in range(10_000)]
rng = random.Random(9595)
sample_without = [sum(rng.sample(tokens, k=2)) for _ in range(10_000)]

A few details decide whether the run is auditable at all.

Seed the generator, not the module. random.Random(9595) keeps a private stream, so nothing else in your session can consume it. random.seed() works too and makes the seed a global property of the program.

Record four things: the version, the seed, the round count, and the exact call for each variant. The documentation's notes on reproducibility make the version non-optional — a seed reproduces a run on an implementation, and identical output across every version is not promised. And the same seed only reproduces the same numbers if the sequence of calls that consumes the stream is also the same. Reorder your loop and the "same" seed produces a different sample.

Know what sample removes. It removes entries from the sequence you passed it, not duplicate values. random.sample([0, 0, 1], 2) can legitimately return [0, 0]. If the real props contain two tokens marked 3, model them as two entries and remember that sampling still treats them as separate objects. And sample returns tokens in the order it picked them — irrelevant for a sum, decisive for an order-sensitive bonus.

Now, what does the run actually tell you? This is where a fixture this small pays off.

A recorded run of the fixture — Python 3.13.5, seed 9595, 10,000 rounds per variant — produced sample means of 3.9987 and 4.0119. The exact mean is 4. Those numbers look reassuring until you notice that the exact mean is 4 for both rules. A sample mean near 4 is evidence that the sampler is drawing tokens from roughly the right range. It is not evidence that the replacement rule is right, because the quantity you measured does not distinguish the two rules. On this fixture, the most natural number to check is the one number that was never in question.

Put the mean aside and check the quantities the rule change actually moves:

• A score of 0 or 8 in the without-replacement sample is a bug, not noise. Those outcomes are structurally impossible; no run length makes them real.
• A repeated token inside a without-replacement round is the same class of failure.
• The frequency of exactly 7 should sit near 10% under the without-replacement model and 8% under the with-replacement model.
• The 0-and-8 counts run in opposite directions across the two samples. Without replacement they should be zero, not small. With replacement they should be present at roughly 1/25 each — about 400 apiece in 10,000 rounds — and a zero there is the bug, not the reassurance.

How far is "near"? The model answers that too. For a proportion p over n rounds the standard error is √(p(1−p)/n). At p = 0.10 and n = 10,000 that is about 0.3 percentage points. Sampling will move those frequencies by a few tenths of a point across seeds, so hitting 10.0% exactly is not the goal.

The same arithmetic sets the resolution of the whole exercise. The gap between the two tail probabilities is 12% versus 10%, about four and a half standard errors of the difference at 10,000 rounds and about one at 500. So the tail difference is real and modest: roughly a few thousand rounds per variant to see it reliably against noise. That is itself a design fact — a rule change with a small consequence needs a lot of evidence before you could call it a big one.

And keep the two failure categories apart. Noise can make a 10% event appear 9% of the time. Noise cannot make an 8 appear where 8 is impossible. That distinction is the real value of running the sampler at all: it tests your implementation against your stated model, and when the model is impossible to violate, it fails loudly instead of quietly. The exact enumeration remains the source of truth. Agreement at 10,000 rounds contradicts nothing; it is not a proof.

What this model deliberately leaves out

Physical uniformity. The model assumes every available token is equally likely. Real draws depend on bag geometry, mixing, token wear and handling. If that assumption is wrong, every number above is wrong in a way no amount of enumeration repairs.

Participants. Uniform chance is not a model of people. Which token a contestant reaches for, when they stop, whether they take a trade — all of that sits outside the state space. Predicting it is a different job with different evidence.

The rest of the format. Bonuses, steals, host discretion and multi-round aggregation each change the state space and need their own enumeration. This one does not extend.

The audience. Nothing here speaks to whether a round is fun, tense, clear or fair to the people in it. A correct toy program is evidence about its written assumptions and nothing else.

There is a complementary method that answers a different question, and the two are worth keeping apart. MIT OpenCourseWare's paper-prototyping lecture for a game-design course teaches the same kind of narrowing: ask a small question, represent the rules lightly, keep facilitation, operation and observation in separate hands, and revise the prototype after watching play. That pedagogy is about how a rule behaves when people touch it, and it takes the rules as given. The enumeration above does the reverse, taking the rules as text and asking what they imply with nobody in the room. Neither substitutes for the other. What that lecture supports here is only the broader distinction between a bounded model and play — it is not a trial, an audience study, or a finding about fairness.

Where the arithmetic stops

One rule change moves four things at once, and only one of them is invisible to a casual check. Replacement on or off shifts the variance from 4 to 3, the support from [0, 8] to [1, 7], the repeat rate from one round in five to none, and the upper tail from 12% to 10%. It leaves the mean at 4. Compare the two formats by average score and they look identical; you would have picked one by accident.

The next question is not arithmetic. Whether the round is better because the perfect score exists, or better because the possible scores are flatter and the disaster outcome is gone, is a question about the show. Enumeration can tell you the rule deletes the 8. It cannot tell you whether anyone was watching for it. Take the table to a paper run with people, put the tokens in a bag, and see which rule makes the room lean forward — and bring the enumeration with you, not as proof of anything, but as a written record of what you believe the rule says. When the play surprises you, that record is how you tell whether the rules were wrong or your model of them was.

Frequently asked questions

Why enumerate the token game before running a simulation?

Enumeration fixes the rule you are sampling. A simulator encodes whichever reading you typed; more rounds shrink sampling noise but leave the replacement assumption unchanged, and the mean cannot distinguish the two formats.

What changes when the first token is not replaced?

Variance falls from 4 to 3, support shrinks from [0, 8] to [1, 7], P(score >= 7) goes from 3/25 (12%) to 1/10 (10%), and the most likely score changes from 4 alone at 20% to a tie among 3, 4, and 5 at 20%. The mean stays 4.

Which simulation outcomes are bugs rather than noise?

Under without replacement, a score of 0 or 8 or a repeated token inside a round is structurally impossible, so it is a bug regardless of run length. Under with replacement, a zero count of 0 or 8 in a long enough run is the suspicious result.

How should a sampler be used once enumeration is done?

As a cross-check against the stated model. Check quantities the rule change moves, seed the generator, and record the version, seed, round count, and exact call for each variant. A sample mean near 4 only checks the range, not replacement, because both rules have mean 4.

What does the token model leave out?

Physical uniformity of draws, contestant behavior and stopping or trading decisions, bonuses and other format rules, and audience questions about fun, tension, clarity, or fairness. Enumeration is evidence about its written assumptions only.

Further reading

• Replay a Format Trial Under Revised Rules—or Run a New Trial? — Read this after a rule changes: it separates recomputing the old record, replaying it as a diagnostic, and running new evidence, and insists you reproduce the original first.
• When Your Competition Rules Reward the Wrong Kind of Play — The enumeration stops at the rules; this audit picks up the other half, asking what a clause pays players to do when clarity and incentive are different tests.