Variant pools: why answers don't travel
In a self-paced course, a static question seen by one learner is effectively published to everyone behind them. Maestro's answer is the variant pool — and a strict split between what an item tests and how it is dressed.
The answer-bank problem
Learners reach the same Evidence Check weeks apart, and cohorts repeat year on year. A static item, once seen, circulates: screenshots travel, answer banks accumulate, and by the second semester the check measures memory of the item, not mastery of the concept.
So no official Evidence Check in Maestro is a single static item. Each one is a family of variants over the same tested substance.
Radicals and incidentals
Every template's form schema declares its fields in exactly two groups — the radical/incidental split from automatic item generation research — a design decision made once, reviewed, and inherited by every instance.
- Radicals — what the item tests. The concept, the misconception trap, the evidence criteria, the difficulty-bearing structure. Locked after SME approval. Byte-identical across every variant of a pool — the validator rejects a pool if even one radical drifts.
- Incidentals — how the item is dressed. Scenario dressing, entity names, option ordering, context flavor. These vary freely across variants and carry no cognitive demand.
How a pool works
At publish time, Studio pre-generates a pool of five to twenty variants per check, varying only incidentals. Every variant is individually SME-reviewable, and a pool is publishable only when every variant is approved — nothing reaches a learner unreviewed.
Delivery is deterministic: a hash of the learner and the item selects the variant, so the same learner always sees the same version of a given check — there is no re-rolling for a friendlier item. When a concept resurfaces later as a spaced re-check, the learner gets a fresh pool over the same radicals: a genuine re-test of the concept, not a memory test of the item.
What variation does and does not defend
Variant pools defend against answers circulating between people over time. They do not stop a learner pointing an AI at their own item in real time — that is the job of task design and template classification. Maestro is precise about which layer carries which threat, because a defense you overstate is a defense you stop maintaining.