Replay a Format Trial Under Revised Rules—or Run a New Trial?

A small trial gets run. A room, a rule set, a score. It goes well enough that somebody proposes one change: the small build should cost four, not two. Now the question is whether the old run can be recomputed under the new number instead of assembling the room again.

Part of it can. Recorded actions are data, and data can be recalculated. What cannot be recalculated is the decision each action came from. The moment a new rule changes what a participant would have known or wanted, the replay's authority over choices is gone. You can keep computing, but the output stops being a trial result and becomes a labeled arithmetic exercise.

That leaves three instruments rather than two — a rescore, a partial diagnostic replay, and a fresh trial — and most of the skill is knowing which one is in your hands. The order of operations is what makes the difference: preserve the record, reproduce the original result, then revise. Skipping straight to the revised numbers is the common failure, and it looks exactly like rigor.

Preserve the record and define the revised question

Two things need settling before anything is recomputed: what the record actually contains, and what the revised question is really asking. Get either wrong and the rest of the work looks careful while resting on nothing.

The record needs the starting state exactly as participants met it; ordered event IDs; the rule version in force at each event; the recorded action; the information that was consequential — what a participant was told, shown, or could see, and when; and the operator interventions, including rulings, pauses and restarts. Each event should point back to its source: the scoresheet row, the video timecode, the payment record, the facilitator's note. That information field is the likeliest to be dropped, and it is the one the fixture below carries as a column.

A final scoreboard is not a record. It is a state, not a history. Two runs can finish on the same total — one through three single moves, one through a single triple — and a revised rule will treat those two histories very differently. The scoreboard cannot tell them apart, so it cannot answer a question about either.

Then the question itself. In a meeting, these two sound identical:

• What would the totals have been if the small build cost four?
• What would have happened if the small build cost four?

The first is about arithmetic. The second is about people. The first can sometimes be answered from the log. The second needs a trial. The test that separates them is whether the change ever reached the participant — and holding that test firmly is most of the work in this article.

One discipline to fix now: missing facts stay missing. Do not restore a gap in the log using today's rulebook, today's values, or your sense of what obviously should have happened. If the record does not show a credit, you do not know there was one. This becomes concrete shortly.

Reconstruct the original result in isolation

The replay engine is small. It takes a state, an event and a rule version, and returns either the next state or an explicit error.

next_state(state, event, rules):
    if event.rule_version != rules.version:
        return ERROR("version mismatch", event.id)
    if not rules.legal(state, event.action):
        return ERROR("illegal action", event.id)
    return rules.apply(state, event.action)

Then a driver walks the recorded events in order at the original rule version:

replay(events, rules):
    state = start_state
    for event in events:        # recorded order, not sorted
        state = next_state(state, event, rules)
    return state

The isolation matters as much as the logic. This interpreter must not tell a host what to say, tell a contestant what they may do next, advance a visible clock, release a graphic, or pay a prize. Those are the format equivalent of what Martin Fowler's Event Sourcing — a software-architecture article dated 12 December 2005 — treats under external updates and external queries: a replay that reissues its own effects has changed the world it was supposed to be describing. Keep the model a pure function and the problem disappears. It is worth saying plainly that the article is an architecture account from 2005, not a tested television-format method, and it says nothing about whether people repeat decisions.

Now the fixture. It is hand-worked, deliberately tiny, and constructed for this explanation — no interpreter was run for it and no participants played it. The arithmetic is the entire check.

Original rule version, call it v1: small build costs 2, large build costs 3. The last column is the one a usable record has to carry and a scoreboard never does: what the participant could see at that moment. Here the small build's price is public, on the board in front of the player, so it reads 2 all the way down.

Recorded end state: 1 token, one small build, one large build. The replay reproduces every row, including the intermediates. That is what a baseline looks like. The engine derives the state columns and no others; visibility is an assertion the record makes, and it is the only thing that can later tell you whether a revised price was something the participant saw or something done to them. Only now is the revised rule worth trying.

Before that, watch the baseline fail on purpose. Delete E2 from the log and rerun. The driver produces 5 → 3 → 0, ending with one small and one large build but zero tokens. The recorded end state says 1 token. Reproduction fails, so stop — the mismatch is a finding about the record, not about the game.

The wrong move here is topping the replay up with a credit because small builds obviously pay one, or because the current rulebook says so. That is inventing a historical input, and every later number inherits the invention. The right move is to find E2's source: the operator's note, the payment record, the counter on the board. If the source exists, it goes back into the log with its link. If it is gone, you do not have a reproducible baseline, and everything built on top of it would be built on a gap. A one-token discrepancy is small; the habit it protects is not.

There is a close relative of this failure. If an event is stamped with a rule version the log says was not in force at that moment, the driver returns a version mismatch and stops before computing anything. Do not assume which version "must" have applied. Unexpected versions are usually the trace of a mid-trial adjustment that nobody wrote down, and that adjustment is itself part of the record.

Apply revised logic without rewriting old actions

Now v2: the small build costs 4, the large build still costs 3. Same preserved sequence, same event order, a separate rule version. At every step, two questions — would the action still be legal, and would the participant have met different information?

There are two distinct "firsts" here, and conflating them is how teams misread a replay. The first state divergence is after E1: three tokens versus one. The first absolutely invalid action is E3. If you only read the final outcome, you see "cannot afford the large build" and conclude the problem sits at E3. The cause was set two events earlier.

Then the substitution trap. Do not have the participant buy something cheaper at E3. Do not have them skip the credit, save the token, or wait. Do not quietly reroute them to a small build because it is affordable. Each of those is a decision, and you do not have it. The replay's job is to name the wall, not to walk around it.

The information question settles what the prefix is worth, and the visibility column already answers it: 2 at every event in the v1 record, 4 from E1 under the revision. The price of a small build is public — it is on the board in front of the player — so a participant meeting the revised format would have seen "4" before E1, not after E3. That makes the whole prefix a legality diagnostic and nothing more. Even E1, legal and landing on a legal state, is a different decision: at five tokens, spending four leaves one, and a large build costing three is no longer reachable from there at all. A player who did that arithmetic might have bought nothing, waited a turn, or gone for the large build first. You do not know. The log cannot tell you.

The contrast sharpens the rule. Suppose the revision were to something participants never saw — a rounding convention on the operator's scoring sheet, a backstage weighting applied after the fact. The recorded actions remain the same actions taken under the same information, and a rescore is defensible. The test is not "did the rule change?" It is "did anyone at the table experience the change?" That single question sorts the whole job into one instrument or another.

Choose what the available replay can answer

Three instruments, compared under the same conditions:

These are not tiers of effort with better answers at the bottom. A rescore is not a cheap fresh trial; it answers a different question, and answers it completely within its stated assumption that nobody's information changed. A fresh trial does not validate the rescore — it answers a question the rescore never asked. Choosing wrongly in either direction wastes real money: a rescore where a trial was needed produces confident numbers with no behavioral content, and a trial where a rescore would do spends a room to learn arithmetic.

There is one more limit worth holding. Even a fully legal revised replay proves only reachability: that the recorded path was available under the new rules. It says nothing about whether anyone would take it. Under changed incentives, the interesting cases are exactly the ones where the arithmetic is fine and the decision is not — where every step is permitted and the path is no longer attractive. Deterministic output is not behavioral evidence, and no amount of clean computation converts one into the other.

Keep the actual and modeled accounts separate

The final artifact is a ledger, not a result. Four fields: the original state sequence, the revised diagnostic prefix, the first stopping reason, and the next question.

For the fixture:

• Original (v1): 5 → 3 → 4 → 1, ending with one small and one large build, once the credit record is restored and linked.
• Revised diagnostic prefix (v2): 5 → 1 → 2, then stop before E3.
• Stopping reason: illegal action at E3 — balance 2 against a cost of 3 — with the first state divergence after E1. The prefix is a legality check only: the visibility column puts the revised price on the board from E1, so the change reached the participant before the first action rather than partway through the run.
• Next question: would a participant who knows the small build costs four still buy it first? That is a fresh-trial question.

Nothing in that list overwrites anything else. The historical result stays the historical result; modeled numbers stay labeled wherever they travel, including into the deck where somebody will inevitably screenshot them.

Two recovery paths belong in the ledger too. An incomplete log means stopping at the gap, going to the source first, and only then — if the source is genuinely gone — running a clearly labeled counterfactual from the gap forward. A labeled counterfactual is a legitimate answer to "what if there had been a credit here," and it is not an answer to "what happened." Separately, never connect the research interpreter to anything live. A diagnostic loop that can pay a prize, advance a clock, or light a cue is not a model; it is a second production, and its mistakes are real mistakes with real consequences. Record which version of the interpreter produced a set of numbers, because the interpreter is a rule too, and a revised interpreter produces revised output.

For this fixture you end with both halves the situation actually offers: the original reproduced, one bounded modeled consequence — under v2 the recorded path stays legal through E2 while the stored state after E1 already differs, and E3 cannot be paid — and an explicit reason to rerun, because the record shows the revised price visible from the first decision.

Keep the first unsupported step visible where it happens, marked in the table, not in a footnote. That boundary is the whole product. A replay does not rescue an old trial by making it answer a new question. It earns its keep by telling you precisely where the old run stops being evidence about the new rule — and that edge is what you take into the design of the next one.

Frequently asked questions

When can a recorded trial be recomputed under a revised rule, and when can't it?

Recorded actions can be recalculated, but the decision each action came from cannot. If a new rule changes what a participant would have known or wanted, the replay loses authority over choices; the output becomes a labeled arithmetic exercise. The sorting test is whether the change ever reached the participant.

What must a usable trial record contain?

The starting state as participants met it; ordered event IDs; the rule version in force at each event; the recorded action; the consequential information—what a participant was told, shown, or could see, and when—and operator interventions such as rulings, pauses, and restarts. Each event should point to its source, such as a scoresheet row, video timecode, payment record, or facilitator's note. A final scoreboard is only a state, not a history.

What is the difference between a rescore, a diagnostic replay, and a fresh trial?

A rescore applies new values to the same fixed actions and answers what those actions cost, with the fixed-action assumption stated; the change does not reach participants. A diagnostic replay is for a change that does reach participants and answers where recorded actions stop being legal and where state first diverges, producing a first-divergence index and stopping reason. A fresh trial is also for a change that reaches participants and answers what people do under the new rule, producing new actions and a new log.

In the fixture, why does E3 fail under v2, and why is E1 already important?

Under v2 the small build costs 4 and the large build still costs 3. After E1 the v2 balance is 1; E2's credit makes it 2; E3 tries to buy a large build for 3, so it is illegal because 2 < 3. But the first state divergence is after E1—v1 leaves 3, v2 leaves 1. Reading only the final outcome points to E3, while the cause was set earlier. The visibility column shows the revised price 4 from E1, so the change reached the participant before the first action; the whole prefix is only a legality diagnostic.

What does even a fully legal revised replay not prove, and how should modeled numbers be kept?

It proves only reachability: the recorded path was available under the new rules. It says nothing about whether anyone would take that path. Deterministic output is not behavioral evidence. The historical result stays historical, and modeled numbers stay labeled wherever they travel. If a log is incomplete, stop at the gap and go to the source first; only if the source is genuinely gone run a clearly labeled counterfactual from the gap forward, which answers 'what if there had been a credit here,' not 'what happened.' Never connect the research interpreter to anything live, and record which interpreter version produced a set of numbers.

Further reading

• Build a No-Repeat Question Draw for a Small TV-Format Trial — When the answer is a fresh trial, this covers keeping trial identity, draw order and request records separate so a second pass cannot contaminate the first run.