Learning Signals

What Journey captures when a learner answers a lesson's check, how those signals become a result and a belief about mastery, and who sees what: plain words for the learner, the internal detail for staff.

The 30-second version

Every check captures four signals: response, reasoning, confidence and process. Fixed rules turn them into one of the internal bands, which the learner only ever sees as five words: Solid · Nearly there · Not yet · Not sure · A mix-up. Each result updates the learner model, a belief per concept held in Maestro's own database and changed only through an append-only ledger by a pure, versioned engine. Staff views in Pulse and the learner's profile may use the internal names; learner pages never do. Signals stay inside Maestro; only summarised grades go back to the LMS.

The four check signals

Signal	What is captured
Response	The option the learner chose, or "Not sure". The choice is classed against the check's design: correct, a tempting error bound to a known misconception, or another wrong answer.
Reasoning	The learner's own explanation, kept verbatim. A reasoning judge (an AI call through the gateway, using an approved prompt version) rates it sound, partial or unsound. The prompt version is recorded with the evidence.
Confidence	How sure the learner felt. Choosing "Not sure" skips the confidence step: honest uncertainty is recorded as such, not as low confidence.
Process	How the answer was produced: time per step and in total, revisions, paste events and paste length, loss of focus, views of the material, and time spent on feedback.

Each answer is stored as an evidence record, together with the learner's choices about where to go next. A record is never edited; a retry or a fresh question is a new record.

Bands and learner words

The band comes from a fixed mapping of response, reasoning quality and confidence, authored per check and copied into the released course. The AI rates the reasoning; it does not choose the band. Internally the bands are:

Internal band	Learner sees	Notes
secure	Solid
fragile	Nearly there	Also used for a slip: a confident miss against a strong, well-evidenced belief. A slip is never authored; the engine resolves it so one bad moment does not undo earned progress.
knowledge_gap	Not yet
not_sure	Not sure	The first "Not sure" on a concept is honoured: the belief is left unchanged and a gentle revisit is scheduled. A second "Not sure" on the same concept counts as a knowledge gap.
misconception	A mix-up	Tied to a specific misconception from the shared registry. Never softened to a slip.

Learner words only

Band names and engine terms never reach a learner page. A shared label map supplies the five words and their headlines, and a plain-language check fails the build if an internal name appears in learner copy. What the learner sees for each result, and the options offered, are in Adaptivity & Routing.

Process signals and integrity

Process signals are read against the learner's own baseline, never a population norm: "fast" only means fast for this learner, and nothing is called unusual until there are enough past answers to compare. A large paste, an unusually fast answer, or repeated loss of focus lowers the weight of that piece of evidence; it never removes it entirely and never accuses anyone. Strong patterns add to a staff-only authenticity indicator. Nothing about this is shown to the learner. The wider approach is in Academic Integrity.

The learner model

The learner model holds one belief per knowledge concept per learner, outside any single course, so a concept met again in another course builds on what is known. Each belief carries:

a mastery probability, moved by each result (a Bayesian odds update, scaled by the evidence weight; a spaced re-encounter counts for more);
a state: unconfirmed, consolidating, secure (only after surviving a spaced re-encounter) or revised (a secure belief that lost its footing). A strong surprise moves the number but holds the state at unconfirmed until the concept is seen again;
depth of understanding on the SOLO scale, which climbs only on a secure result with sound reasoning and drops one level on a misconception;
calibration (how well confidence tracks correctness) and coherence (whether confidence, reasoning and result agree);
the number of pieces of evidence and the date of the next spaced re-check, sooner for a wobbling belief and further out for a secure one.

Misconceptions run their own episode per learner: suspected, confirmed, remediating, rechecked, cleared, with every transition logged.

For engineers — one write path

A belief changes only by appending an event to the belief ledger; the stored belief is the fold of that ledger, so every value can be replayed and explained. The maths lives in a pure, deterministic engine package with no I/O, and every belief, ledger event and routing decision is stamped with the engine version that produced it. The per-subtopic states on the course road are only a display summary of these beliefs.

Who sees what

Audience	What they see
The learner	The five result words, their own answers ("kept for you"), the options they chose, course-road states (solid, firming up, you're here, ahead) and their milestone results. Never bands, probabilities or process signals.
Course staff (Pulse and learner profiles)	Internal names are allowed: mastery per concept and belief states, live activity, misconceptions shared by several learners, suggested interventions with their evidence, and the staff-only authenticity indicator. Access follows each person's role and course.
The LMS	Milestone grades per gradebook column, and nothing else. Raw signals are never exported (see Integration — Moodle & LTI).

Staff routes never return a learner's Atlas conversation.