The Concept

Mastery & the Lesson's Check

How Maestro decides how well an idea has landed. Every lesson ends in a short check — a diagnostic moment, not a graded test and not a gate. It reads how a learner thinks, gives them one of five plain results, and updates one evolving picture of what they know.

Why not a threshold model

The instinct is to treat mastery as a score crossing a cutoff. Maestro does not, for a precise reason: a single measurement collapses cases that must be handled differently.

• A right answer with flawed reasoning looks identical to a right answer with sound reasoning.
• A wrong answer from a slip looks identical to a wrong answer from a genuine wrong model.

Any threshold-based system adapts wrongly on both. Maestro treats mastery as a structured result that separates them, and as a belief that updates over time rather than a state set once.

The lesson's check

Every lesson follows one Lesson Page Standard: an opening case, parts with Watch (or Read instead) and Try it, Pause moments, then the lesson's check, then the close. The check is written by the Lead Designer in Studio and tests specific lines of the lesson's cards. It comes in three shapes:

Check type	What the learner does
Compare and decide	Reads two attempts at the same task and decides which is sound — A, B, both or neither — then says why.
Select and justify	Picks the best option and justifies the choice in their own words.
Misconception confrontation	Predicts what happens in a case where one tempting prediction embodies a known mix-up, then explains the prediction.

Try it activities earlier in the lesson are practice with instant feedback; the check is the one moment that writes evidence. The learner's first answer is kept exactly as written, and nothing on the check screen shows right or wrong before they submit.

The four signals

To find out what went wrong and why, the check gathers four signals:

Signal	What it is
Response	What the learner chose — a verdict or an option, including any designed tempting error.
Reasoning	Why they chose it, in their own words, with a minimum length. This turns a wrong answer into a diagnosable one; it is judged sound, partial or unsound against the check's hidden rubric.
Confidence	A four-step self-rating. It separates a confident wrong model from a simple gap.
Process signals	Light traces (time per step, focus changes, pasting, revisions) used only to weight evidence — supportive, never a verdict and never shown to the learner.

"I'm not sure"

Every check offers "I'm not sure", styled as an invitation, never as a failure. Choosing it is honest evidence in its own right: it resolves straight to the Not sure result, skips the confidence rating, and defers the belief write — no mastery is recorded and nothing is lost. If the learner is still not sure on a second reading, it is treated as Not yet so they get the support they need. The learner is offered a warm "let's make sure together" path instead. Guessing is never cheaper than admitting uncertainty.

Internal results & the five words

A deterministic band map crosses the response category, reasoning quality and confidence to produce one internal result. A wrong answer that matches a tempting error the check declared becomes a specific mix-up, tied to that named misconception. Internal names stay in code, in Studio and in staff views; learners only ever see the five words:

Internal result	Learner sees	Headline on the card
secure	Solid	"Solid."
fragile	Nearly there	"Nearly there."
knowledge_gap	Not yet	"Not yet. Let's see it another way."
not_sure	Not sure	"Thanks for being honest. Let's make sure together."
misconception:<id>	A mix-up	"There's a mix-up worth fixing."

A careless error with sound reasoning is resolved internally as a slip (never authored, only inferred); the learner sees Nearly there and it routes the same way — no penalty. On the course road, ideas read as solid, firming up, you're here or ahead.

The result card

Right after the check, the learner sees one card, in this order: the result's headline with a small mark in its colour; for a mix-up, the misconception's approved learner-facing label; the message written for that result; and "Your answer · kept for you" — their response, reasoning and confidence, read back exactly as submitted. The card is the only verdict; no raw result flashes on the check screen. Below it sit the options under "Where to next · you choose" (see Adaptivity & Routing).

Kept answer & a fresh check

The first answer is never overwritten. Under it, the learner's attempts so far are listed one quiet line each ("Attempt 2 · Nearly there"). When the routing offers it, a Try a fresh check button gives a different version of the same check — "fresh attempt 1 of 2".

• Two fresh tries per lesson, at most. The limit is enforced on the server, not only by hiding the button; a third is refused.
• The budget never refills and carries across result changes, so a learner can't earn new tries by bouncing between results.
• When the offer isn't there, nothing renders — never a disabled button.
• No "highest reading" is kept; each attempt is evidence the belief engine weighs in turn.

The variant pool

Each check has a pool of variants: the same idea, the same rubric, a different story and different attempts. Studio produces the pool with the lesson, and each variant is reviewed and approved before it can reach a learner. Fresh checks, Revision's "A fresh question" and the "Welcome back" look-back always draw a variant the learner hasn't seen, so a re-check is a genuine re-test, not a memory test. The worked example walks through one approved variant, which is then kept out of fresh questions for everyone. A lesson needs at least three approved variants before it can be published live.

One reading is not a verdict

The system acts on a single reading only as far as that reading deserves. If a fresh attempt reads worse than before, that one bad reading is treated as suspected, never as a confirmed downgrade. A mix-up is followed up later: a "Welcome back" look-back on a following day, and a "Look again at your answer" item on the Revision page, where the learner reads their own earlier answer and says what they would change. If the idea holds on return, the belief consolidates; if it doesn't, the item stays open. A single wrong result is always recoverable.

The learner model

The learner model is the spine — the running picture that turns separate lesson results into genuine adaptivity. It holds one record per learner, with a row per Knowledge Component (KC) encountered:

It holds	Meaning
Mastery belief	The probability the learner has this KC — climbs and dips with evidence, within bounds.
Belief state	Unconfirmed → consolidating → secure. "Secure" needs a survived re-encounter; one clean result never jumps straight there. A belief that loses its footing is marked revised.
Misconception flags	Any named mix-up currently open for this learner on this KC.
Confidence calibration	How well the learner's confidence tracks their correctness, accumulated over time.
History	Encounters, last seen, and whether the belief is holding or fading.

It carries real duties: privacy, the learner's right to see their own picture, and treating a low belief as a transient state to support, never a fixed label.

The misconception registry

Open-ended diagnosis ("invent the explanation for this error") is something LLMs do poorly. So Maestro never asks for it. Instead, misconceptions are a governed, finite list, and a check declares which of its wrong options embodies which one.

• The Lead Designer proposes tempting errors while designing a lesson. They join a shared misconception registry and, once approved, are bound to the lesson its check scores against.
• When writing a check, the writer declares the tempting option for each bound misconception — or states why none fits. Code verifies every declaration: an id that isn't an approved binding is faulted, and so is an undeclared wrong option that strongly matches one. Code never binds by itself.
• Each bound misconception carries an approved learner-facing label and its own mix-up message, produced in Studio and published with the lesson.

Formative vs summative assessment

Two categories of assessment run on different logic, and they are built as separate systems, not one merged "assessment engine".

	Formative — the lesson's check	Summative — milestone assessments
Where	At the end of every lesson.	At each milestone on the course road, entered through its Milestone Guide.
What it is	Response, reasoning, confidence; produces one of five results.	A substantial piece of work in a chosen context, then a spoken defense.
Judged by	Deterministic rules — mastery evidence, never a score.	A weighted report against the course's approved rubric and weights.
Reachable	Continuously through the journey.	Once the milestone opens and its required Revision items have had one honest try.

How a grade relates to mastery

The framing runs one way: grades do not translate into mastery decisions. Lesson results build the learner model and the Revision list; the milestone produces the grade.

The grade reflects the quality of the milestone work and its defense, not a re-derivation of whether each KC was mastered. A lesson check is never graded, and a milestone never retroactively re-decides a lesson's result. What each result does next is the subject of Adaptivity & Routing.