Why claims about tournament format and competitive balance are untestable

The single-sample problem

A tournament happens once, with one set of participants, one draw and one sequence of results that could easily have gone differently. Judging a format by that single realisation is like judging a coin by one flip, except that the coin has many more faces. A format that produced a dull edition may be perfectly sound, and one that produced a memorable edition may be badly flawed.

Distinguishing the format from the realisation would require many editions under identical conditions, which the calendar cannot supply. Everything said about a format after a single tournament is therefore constrained by a sample of one.

Two goals that get confused

Competitive balance means the outcome is genuinely uncertain across many participants, which is a statistical property of the field. Excitement means individual matches feel consequential and unpredictable, which is a property of how the schedule distributes stakes. A format can improve one while damaging the other, and formats that spread stakes evenly often reduce the number of decisive occasions.

Arguments about format usually slide between the two without noticing, which allows any change to be defended on whichever ground it satisfies. Fixing which goal is being pursued would resolve a substantial share of the disagreement before any evidence was needed. Most format disputes are therefore disputes about what a tournament is for, conducted in the vocabulary of what a tournament produces.

What format genuinely controls

A format determines how many matches a weaker participant must survive, how early the strongest can meet and how much a single result matters. Those are structural facts that can be reasoned about in advance without waiting for a tournament to be played. Increasing the number of matches before elimination reduces the influence of any one result, which mechanically favours stronger participants.

Shortening the path increases variance, which produces more surprises and also more paths where the best participant never appears late. Both consequences follow from the structure and neither requires an empirical claim about a particular edition.

Where the reasoning stops

What structural reasoning cannot deliver is the magnitude of the effect, because that depends on how large the true differences between participants are. If the field is closely matched, path length barely matters, and if it is stratified, path length matters a great deal. Estimating the stratification requires ranking participants, and every ranking system carries its own assumptions and its own errors.

So the qualitative direction is knowable and the quantitative answer depends on an input that is itself contested. That is an honest description of an unresolved question rather than a failure to reach a verdict.

How the argument should be conducted

The productive version states the goal, derives the structural consequence and accepts that the size of the effect is uncertain. The unproductive version points at a memorable edition and treats it as proof that a format works or does not. Editions are evidence of what can happen under a format, which is worth knowing and is not the same as evidence about what tends to happen.

Holding that distinction is difficult when a tournament has just finished and everyone has a vivid impression to defend. It is also the only way any of these arguments could ever be resolved. Resolution would require patience across many editions, and the discussion is structured to deliver a conclusion within days of a final.