Generative AI changes the economics of producing computing resources. It does not change the standard by which those resources should be judged.
The relevant question for a head of computing or curriculum lead is not whether a model can produce a plausible worksheet, code trace or examination response. It is whether a defined generation workflow preserves the conceptual boundary of the lesson, supports a valid inference about pupil understanding and costs less to verify than an established departmental process costs to construct.
That is a resource-engineering problem. Prompt fluency is secondary.
Distinguish three materially different workflows
“AI-generated” is too broad for a departmental decision record. It can describe three workflows with different assurance burdens:
- Direct generation: the system produces the pupil resource from a brief. The teacher supplies constraints but no authoritative resource text.
- AI-assisted adaptation: the system transforms an existing, reviewed resource—for example by changing context, reading demand or programming language—while the original remains available for comparison.
- Source-grounded generation: the system produces a new artefact from teacher-selected curriculum, specification or technical sources supplied for that task.
These categories should not be pooled when evaluating performance. Adaptation offers a comparator but can introduce semantic drift. Source grounding can improve traceability but does not guarantee that the model has represented the source correctly. Direct generation has the weakest provenance and therefore requires the strongest independent reconstruction of correctness.
Record the workflow category, model or product version, material supplied, generation date and teacher intervention. Otherwise a later reviewer cannot tell which process produced the observed defect rate.
Generation should begin from the intended inference
A teaching resource is an instrument for eliciting evidence. Its design should therefore begin with the inference a teacher intends to draw from pupil action.
Consider a trace table for nested iteration. A correct final state may result from a valid mental model, pattern matching, execution with a debugger or reconstruction from an AI-generated explanation. Those routes are not pedagogically equivalent. If the intended inference concerns pupils’ ability to coordinate loop invariants and state change, the task must discriminate between them.
This places stronger requirements on an AI-generated draft than topic alignment. The specification needs to define:
- the knowledge components under examination;
- the observable behaviour that would warrant the intended inference;
- plausible alternative explanations for success;
- the support conditions under which the evidence remains interpretable; and
- the transfer item that tests whether the model survives a change of surface features.
The specification is not a longer prompt for its own sake. It is the teacher’s account of validity. Generation is useful only after that account exists.
Apply four acceptance gates separately
A resource passes only when it clears four distinct gates. Do not allow strength in one category to compensate for failure in another.
Gate 1: technical correctness
Computing resources permit unusually strong forms of verification. Code can be executed. Queries can be run against controlled fixtures. Boolean expressions can be enumerated. Algorithms can be traced against adversarial cases. Protocol explanations can be checked against an explicit abstraction layer.
Use those affordances systematically and establish an executable oracle where the subject permits it.
For generated programs, define expected behaviour before inspecting the implementation. Include equivalence classes, boundary values, invalid states and repeated invalid inputs where relevant. Execute in the same language version and constrained environment pupils will use.
Passing tests does not settle the pedagogical question. A solution may be extensionally correct while being instructionally unsuitable: it may depend on library behaviour outside the curriculum model, conceal state mutation, introduce exception handling prematurely or reward a technique that will not generalise.
The review therefore needs both an output oracle and a representation standard. The first asks whether the artefact works. The second asks whether it makes the intended computing idea available for inspection. Any incorrect answer, non-terminating valid path, unsafe instruction or material dependency mismatch is a failure at this gate.
Gate 2: curricular validity
Map every claimed objective, explanation, task demand and success criterion to the department’s enacted curriculum. For qualification classes, check the current specification, command words, permitted notation and assessment conditions against the awarding body’s own documents. Generic references to “GCSE standard” or “the national curriculum” are not sufficient.
The national curriculum for computing names broad disciplinary expectations, including abstraction, algorithms, programming, Boolean logic, systems and data representation. It does not establish the sequencing, depth or terminology of a particular scheme of work. The generated artefact must therefore be checked against both statutory intent and the department’s chosen progression model.
For assessed work, separate teaching-resource review from candidate-use policy. Current JCQ guidance requires centres to protect the authenticity of candidates’ work and review their approach to AI use. A technically accurate revision resource can still be unsuitable if it rehearses material under conditions that conflict with the intended assessment or the centre’s rules.
Gate 3: construct validity
Every school-level model omits detail. The professional judgement lies in deciding whether an omission is productive or distorting.
An unplugged network simulation, for example, may support reasoning about routing decisions while deliberately suppressing timing and lower-layer behaviour. It becomes misleading when physical movement is allowed to stand in for packet switching without clarifying addressing, independent routing, loss, reordering or the distinction between the model and an implementation.
The same test applies elsewhere. A memory diagram should not collapse volatile main memory into persistent storage. A cyber-security case should not treat threat, vulnerability, exploit and impact as interchangeable. A database task should not make a coincidental row order appear to be a property of an unordered relation.
Ask which false generalisation the simplification makes available. Then ask whether success on the task warrants the inference the teacher intends to draw. If either answer is unclear, the resource fails this gate.
Gate 4: classroom usability
Review the artefact in its actual delivery form, not as model output. Check code legibility, keyboard operation, magnification, print behaviour, timing, teacher orchestration, recovery from predictable errors and the availability of a non-AI route through the task.
Usability includes opportunity cost. A resource that requires continuous teacher repair, produces avoidable clarification questions or displaces a stronger established task has failed even when its content is technically correct.
Distractors need a theory of error
Generated questions often contain plausible-looking distractors. Plausibility is not enough. Each distractor should correspond to a defensible model of pupil reasoning and should discriminate that model from the target understanding.
In a Parsons problem, multiple valid line orders, accidental syntax faults and visually ambiguous indentation damage the inference. A pupil may fail because the puzzle is under-specified, or succeed by exploiting surface cues. Neither result diagnoses understanding of sequence, selection, iteration or scope.
For every distractor, identify:
- the misconception or incomplete strategy it represents;
- why a pupil holding that model would select it;
- what follow-up question would distinguish misconception from misreading; and
- whether the distractor introduces knowledge outside the construct being assessed.
If a distractor has no account of error, remove it. Generated variety has no value when it increases measurement noise.
Worked example: validating a binary-search diagnostic
Suppose a Key Stage 4 resource asks pupils to diagnose four implementations of binary search. The intended inference is that pupils can reason about the algorithm’s precondition, interval invariant and termination—not merely recognise familiar code.
An AI system generates the programs, test data, explanations and mark scheme. The expert review is not proofreading. It reconstructs the task’s validity.
First, specify the oracle independently. Test an empty collection, one item, absent targets below and above the range, targets at both boundaries, odd and even lengths, duplicate values and unsorted input. Decide explicitly whether the algorithm returns membership, an index or an insertion point. Those contracts are not interchangeable.
Second, trace the interval represented by the lower and upper bounds. A mixed inclusive–exclusive implementation can pass ordinary examples while failing at a boundary or failing to terminate. Require each update to preserve the chosen invariant and make progress. An implementation that sorts its input internally may appear robust while concealing binary search’s sorted-input precondition; that makes it unsuitable for this diagnostic.
Third, classify each defect against the intended construct. An off-by-one update can expose reasoning about interval reduction. A language-specific integer-division surprise may instead measure syntax knowledge. A duplicate-value test is relevant only if the return contract makes the selected occurrence significant. Remove defects that cannot support the intended inference.
Fourth, audit the mark scheme. Credit should follow a valid trace and explanation of the invariant, not resemblance to one canonical implementation. If two repairs satisfy the declared contract, the scheme must recognise both.
Finally, use an unassisted transfer item in which the representation changes—for example, ask pupils to justify termination or construct a failing case rather than repair code. Agreement between the diagnostic and transfer evidence is more persuasive than success on additional near-identical items.
The model contributes candidate cases. The teacher establishes the oracle, the construct and the warranted inference.
Accessibility belongs inside validity
Accessibility is not a final presentation check. If colour carries program state, indentation disappears under magnification, a code-ordering interface is not keyboard operable or the reading demand obscures the computational relationship, the task is measuring access to the representation alongside the intended construct.
Experienced teachers already adapt representations. The additional risk with generated material is silent inconsistency: terminology changes across items, diagrams encode relationships differently, or examples introduce cultural and linguistic demands unrelated to the computing concept.
Review the complete resource as a system. Preserve necessary disciplinary complexity, but remove accidental variation that weakens interpretation of pupil responses.
Data provenance is part of technical quality
Requests for contemporary datasets create a particular verification burden. A fluent model may invent records, combine incompatible dates or reproduce personal information without making provenance visible.
Use authoritative, permitted data only when authenticity contributes to the learning goal. If the objective is relational modelling or query semantics, a deliberately constructed synthetic dataset usually offers stronger control: edge cases can be guaranteed, expected results can be calculated in advance and personal data can be excluded by design.
Do not submit identifiable pupil work, safeguarding information, private assessment records, credentials or unreleased examination material to a general-purpose AI service. Tool approval, retention, contractual terms and school policy remain prerequisites; de-identification alone does not settle the governance question.
Set risk tiers before evaluating
The assurance burden should follow the consequence of failure, not the polish of the output.
- Tier A—bounded practice: Retrieval, worked examples and low-consequence practice. Require author review against all four gates and executable checks where possible. Abandon the candidate if a major defect remains unresolved or the pre-declared time ceiling is exceeded.
- Tier B—diagnostic or sequence-defining: Diagnostic questions, hinge tasks and explanations that establish a later model. Require full four-gate review, a transfer item and independent sampling by another specialist. Abandon the candidate for any unresolved major defect, loss of diagnostic power or breach of the time ceiling.
- Tier C—high consequence: Assessed work, examination preparation, cyber-security activity or material involving sensitive data. Require full review by the author and an independent specialist, plus current awarding-body, JCQ and school-policy checks where applicable. Abandon the candidate for any critical defect, unresolved major defect, disputed assessment validity or breach of the time ceiling.
A critical defect makes the central answer technically false, creates a safety or data risk, or compromises assessment integrity. A major defect changes the intended inference, curriculum representation or accessible route. A minor defect is local and presentational, without changing meaning or access. Record all three; never average a critical failure into an otherwise favourable score.
Departments should set the actual time ceilings before generating resources. A universal number would be false precision: the defensible limit depends on the resource class, local expertise and the cost of the established route.
Measure total verification cost
Generation time is an unhelpful workload measure. The relevant quantity is total cost to classroom confidence:
- specifying the intended inference;
- generating and selecting candidates;
- executing technical checks;
- analysing distractors and accessibility;
- repairing the resource;
- preparing independent evidence; and
- reviewing what happened in use.
A department should compare this total with its established design route. Record defect classes as well as minutes. Repeated conceptual errors, unstable output between revisions or resources that need complete reconstruction are evidence against that use case even when the first draft arrives quickly.
Define an abandonment threshold in advance. If verification exceeds it, discard the candidate. The ability to continue refining a conversation is not evidence that refinement remains economical.
Run a paired, counterbalanced departmental evaluation
Treat AI generation as an alternative source of candidate artefacts, not as a method that deserves adoption by default. Compare it with the department’s established development process using matched briefs from each resource class under consideration.
For each brief, produce one resource through the established route and one through a defined AI workflow. Counterbalance which resource is produced first and which specialist reviews it first. Where practical, conceal provenance during the first quality review. Use the same four gates, pupil group, teaching time and evidence plan. Do not pool direct generation, adaptation and source-grounded generation into one result.
The evaluation record should capture:
- elapsed time from specification to classroom-ready decision;
- specialist attention, recorded as minutes and role;
- number of corrections by gate and severity;
- unresolved defects and reasons for abandonment;
- independent reviewer agreement;
- pupil responses on the intended diagnostic and transfer evidence; and
- post-use corrections before the resource could be reused.
The comparison is small and contextual, so report distributions and individual failures rather than presenting a fragile average as a general effect. A useful minimum is the median total preparation time for each workflow, the correction count by severity and the evidence outcome for every matched pair.
Make adoption a gated decision
Authorise a workflow for a named resource class only when every candidate has cleared its required review and the matched comparison shows:
- no unresolved critical or major defects;
- no material loss of diagnostic or transfer evidence against the established route;
- no regression in accessibility or classroom usability;
- lower median total preparation time, including all specialist review and repair; and
- agreement from the independent specialist for every Tier C resource.
Otherwise restrict the workflow to the classes for which it passed, or reject it. Do not use success with low-consequence retrieval practice to authorise AI-generated examination preparation.
Use a compact decision record for each resource and retain the evidence behind it:
- Resource class and risk tier: What was produced, for whom and with what consequence of failure?
- Workflow provenance: Direct generation, adaptation or source-grounded generation; system version, date, sources and teacher interventions.
- Intended inference: What conclusion should pupil action warrant, and what alternative explanations must be excluded?
- Gate outcomes: Technical correctness, curricular validity, construct validity and classroom usability—each recorded as pass, fail or unresolved.
- Defect burden: Critical, major and minor corrections; specialist minutes; elapsed time.
- Comparative evidence: Matched baseline, diagnostic and transfer result, reviewer agreement and post-use amendments.
- Decision: Authorised, restricted or rejected, with scope, owner and review date.
This makes the adoption claim falsifiable. It also gives a future curriculum lead enough information to distinguish a dependable workflow from a resource that survived through undocumented expert rescue.
Source and evidence boundary
This note was prompted by Simon Johnson’s AI Computing Prompts for Teachers, a practical collection covering lesson preparation, scaffolds, activities and assessment. The collection demonstrates the range of material a teacher might ask an AI system to draft. It is not presented here as evidence that those materials improve learning or reduce workload.
The national curriculum for computing supplies broad disciplinary expectations, while awarding-body specifications and the department’s enacted curriculum must supply the finer progression and assessment constraints. The Department for Education’s position on generative AI in education retains professional responsibility for how outputs are used and notes that evidence about learner impact remains limited. JCQ guidance on AI use in assessments provides the relevant assessment-integrity boundary.
The paired evaluation borrows general assurance principles from the NIST AI Risk Management Framework’s measurement guidance: context-matched testing, documented metrics, repeatable evaluation and independent review. NIST does not validate this educational protocol; the application to departmental resource decisions is this article’s proposal.
The verification protocol and examples in this note are original proposals. No documented classroom observation was supplied, so the recommendation is a controlled comparison rather than departmental adoption.