The Build

Interactives & the Evidence System

What the interactive system does, why it is designed this way, and where the dev team's work fits. The whole adaptive engine runs on one fuel — trustworthy evidence of what each learner actually knows — and everything here exists to keep that fuel clean.

The three kinds of assessment

It is critical not to confuse these — they have different purposes, different rules, and different security requirements.

Kind	Where	What it is & its rules
Practice interactives	Inside the node, during learning	Formative activities — sorting, matching, ordering, classifying. Their job is to help the learner learn: immediate, per-item, reasoning-based feedback; retryable. They are not how the system decides mastery, and they carry no grade.
The Evidence Check	At the exit of every node	A short diagnostic capturing three signals — the learner's response, their reasoning in their own words, and their confidence. The outcome converts to a diagnostic band (secure · fragile · knowledge gap · misconception) that drives routing and updates the learner model. Not a quiz, never graded directly — a diagnostic conversation whose value depends entirely on the answer being the learner's own.
Milestone Assessments	Formal, graded, controlled conditions	The longer exams that carry the marks. Evidence Checks prepare learners for milestones. The design law: mastery produces the grade; the grade never produces mastery.

The problem we design against

Any on-screen task whose complete answer is visible on the screen can be outsourced to an AI tool in seconds: screenshot the item, paste it into a chatbot, type back the answer. Drag-and-drop sorting, matching, and ordering are solved this way trivially. This matters twice over:

• Grade integrity — platform progress contributes to the course grade, so faked evidence corrupts marks.
• Adaptive integrity (the deeper problem) — a faked Evidence Check writes a false "secure" belief into the learner model. The system then routes the learner past the support they needed, and they arrive at the graded milestone unprepared. A cheated Evidence Check doesn't just cheat the institution — it sabotages the learner's own preparation and poisons the engine.

The Defense Stack — seven layers, and who builds each

The defense is not one mechanism; it is seven layers that reinforce each other. Layers 1–4 are authoring-side (built into content before any learner sees it); layers 5–7 are runtime-side (executed while the learner works). Which side a layer lives on tells you who builds it.

Layer	What it does	Studio (authoring)	Runtime (dev team + engine)
1 · Incentive design	Make honesty the smartest strategy: Evidence Checks are never graded directly; no correctness feedback in-flow (feedback and routing come after submission — protecting the diagnosis and blinding a cheater); a penalty-free "I'm not sure" option with its own diagnostic band; the platform tells learners openly how this works.	Policy fields, not-sure path, transparency text	Enforce feedback timing; render not-sure
2 · Task design	Require what an external AI cannot supply: the learner's own reasoning (required field), their own confidence judgment, or their own platform history ("Earlier you argued X — does that survive this new criterion?"). Maestro has the learner's record; an outside tool doesn't.	Required reasoning/confidence/context fields in schemas	Inject learner history into context-bound slots
3 · Per-learner variation	Answers don't travel. Every EC item is built from a template whose fields split into radicals (what the item tests — locked, SME-approved, never varies) and incidentals (surface details — scenario dressing, names, ordering). Studio pre-generates a variant pool (10–20 versions) at publish; every variant SME-reviewable. Shared screenshots match nobody else's item; no answer bank can form; a re-check is a genuine re-test, not a memory test. Scope note: this defends against answer circulation between people over time — it does not stop real-time AI use on one's own item; that is Layers 2 and 4.	Radical/incidental split; pool generation + SME review	Deterministic variant selection per learner (e.g. learner-id hash)
4 · Template integrity classification	Unfit vehicles cannot carry Evidence Checks — see the classification table below. Enforcement is mechanical at three points: the classification lives on the template profile; the generation pipeline may only assign an official EC to an eligible template; the validator hard-fails any EC riding an ineligible template. A rule the system physically cannot break.	ai_resistance_profile; generator filter; validator hard check	Nothing — enforced before publish
5 · Behavioural signals	Quiet context, never accusation: response latency vs. the learner's own baseline, focus-loss, paste-vs-typed, revision count. Three strict rules: never shown as "cheating flags"; only adjust the weight evidence carries; anomalies trigger support, never penalties. The same signal that suggests outsourcing also describes an anxious learner — the response to both is the same supportive check-in.	Signal-capture contract per instance	Actual capture in renderers
6 · Longitudinal triangulation	Mastery is believed, not recorded — the engine. A single result is one witness, never proof. Per-KC mastery belief consolidates only when independent witnesses agree: engagement predicts the expected outcome (an out-of-character result is held unconfirmed); a clean check moves belief up gradually, never jumps it; KCs resurface as spaced re-checks (different variant, same radicals) — real mastery survives, outsourced mastery collapses and retroactively discredits the earlier result; and the controlled-conditions milestone anchors everything. To defeat this a learner must fake coherently for weeks — at which point the only reliable way to fake mastery is to have it.	Diagnostic bands; recheck eligibility; evidence map	Belief-engine mathematics
7 · Dialogic verification	The conversation that confirms. When evidence doesn't cohere — or routinely at random, or on high-stakes KCs — the Companion opens a short conversation: "Walk me through how you approached this." For an honest learner, genuine reflection; for outsourced answers, where the absence of understanding becomes visible. Text-based by default; never announced as an integrity check. The Companion never improvises assessment content — probes come from the authored, SME-approved verification_probe_pack on each node.	Authored probe packs per node	Companion trigger + live dialogue

The template-first architecture

Interactives are never built bespoke per node:

Object	What it is	Built by
Interactive Template	A reusable interaction pattern plus an editable form schema (e.g. select_and_justify). Built once, QA-tested, accessibility-validated, approved into the library.	Dev team
Interactive Instance	One node's use of a template: the pipeline fills the form with that node's approved content — KC, misconception traps, evidence criteria, feedback logic, routing. Structured data (JSON), not code.	Studio pipeline (SME-gated)
Renderer	Turns any valid instance into the learner-facing UI and emits the declared signals. One per template.	Dev team

Why template-first: bespoke interactions are slow, untestable, impossible to keep accessible, and ungovernable at scale. With templates, the dev team builds and hardens ~11 renderers once; content scales through instances without new code. The template's form schema is also where the defense stack physically lives — required reasoning/confidence fields, the radical/incidental split, the diagnostic bands, and the signal contract are all schema, so a non-compliant instance cannot exist.

The V1 Template Library — 11 templates, 4 tiers

Tier	Templates	Classification	Evidence role
P — Presentation	accordion_reveal · tabbed_panels · process_steps	presentation_only	None; completion only.
B — Practice	classify_categorize · match_pairs · sequence_order	screenshot_solvable	Formative only; retryable; immediate feedback; never an official EC.
C — Evidence	select_and_justify · compare_and_decide · misconception_confrontation	reasoning_required	EC-eligible; required reasoning + confidence + not-sure path; diagnostic bands; variant pools.
D — Context-bound	prior_answer_probe · scenario_decision_pathway	context_bound	Strongest EC class — see the special contracts below.

Variant pools apply to Tiers C and D only. Tier B practice is retryable with immediate feedback anyway — there is no secret to protect, and pools there would multiply SME review load for no gain.

The two binding contracts

Contract	What it binds
The rendering contract (per template)	Interaction behaviour, states, events emitted, signals captured, and accessibility requirements — keyboard operable, screen-reader friendly, mobile responsive, text equivalent wherever the interaction encodes content.
The compatibility manifest (per published course package)	The package declares every template_id + version and every signal-capture capability it requires. If the renderer registry cannot satisfy the manifest, the package does not publish. No silent degradation.

Design laws — do not violate these

• Mastery → grade, never grade → mastery. The grade attaches to triangulated mastery belief, never to raw Evidence Check outcomes.
• No correctness feedback in-flow during an Evidence Check. The first attempt is stored; feedback comes after reasoning/submission.
• An Evidence Check is never a simple right/wrong quiz. Response + reasoning + confidence, always; "I'm not sure" always available and penalty-free.
• Screenshot-solvable templates never carry an official Evidence Check. Enforced by the validator; not negotiable in content design.
• Nothing reaches a learner unreviewed. Variant pools are pre-generated and SME-reviewable; the Companion only uses authored probe packs.
• Behavioural signals support, never accuse. They weight evidence and trigger support; never surfaced as cheating flags or used as standalone proof.
• Similarity is not teaching. An interactive must create the intended cognitive work, not merely resemble an activity type.
• Accessibility is first-class. Every template: keyboard operable, screen-reader supported, mobile responsive, with a text equivalent wherever the interaction encodes content.