A well-formed paragraph tells you the sentences hold together. It does not tell you the writer holds a model of the thing. Those are two different signals, and if we conflate them we mis-grade the room.
The gap in one sentence
Fluent output is the surface. Understanding is an internal generative model: a compressed way the learner can predict what should happen next, notice when a prediction misses, and update. Active inference frames this directly, the mind as a prediction-and-update loop over a generative model of the world (Class E, Parr, Pezzulo, Friston, 2022). A paragraph that reads well can be produced without that loop running underneath. That is the whole problem.
We see this in two familiar shapes. A student paraphrases a source cleanly but cannot answer a "what would change if..." question about it. A student uses a chatbot to smooth their prose, and the smoothing hides that the underlying claim is a guess. The output looks strong. The model inside is thin.
Why this matters right now
Generative writing tools are inside almost every homework session (Class C, from what teachers report in our workshop intake). If our assessment reads only the surface, we reward polish and miss whether the learner can actually predict, check, and revise. Over a semester, a class trained on polish gets fluent and gets fragile. The AI-authorship fence, which we ask every artifact in this program to carry, is one guardrail against that drift. It is not the whole answer. The other half is how we probe.
Four classroom probes teachers can use in under five minutes
These are lightweight. Pick one, use it live, and listen for the shape of the answer, not the vocabulary.
-
Predict-then-check. Before the next paragraph, before the next step of the experiment, before the next line of the proof, ask the learner to say out loud what they expect to happen and why. Then run it. A learner with a working model has a specific prediction and a specific reason. A learner running on fluent surface has a shrug dressed as a sentence.
-
Perturb one variable. Take the artifact they just produced and ask: "What in your answer would change if we swapped X for Y?" A real model responds locally and precisely. A surface model responds by rewriting the whole paragraph in the same tone.
-
Reverse the direction. If they explained cause to effect, ask them to explain effect to cause. If they described the algorithm forward, ask them to walk it backward from the output. Generative models are approximately symmetric in that they can be queried from either end. A memorized sequence usually cannot.
-
Ask for the falsifier. "What would make you drop this answer?" A learner who cannot name a falsifier is not yet holding the claim as a claim. They are holding it as text. Naming a falsifier is the smallest, cheapest sign the model is real.
What to do with what you hear
Grade the loop, not the paragraph. If the probe surfaces that the model is thin, that is not a failure to punish, that is the point in the lesson where teaching happens. Reopen the prediction. Ask for the reason under the reason. Let them revise the artifact with the model now visible. That is a gate in the sense we use across this program, a small, honest place where the learner has to show the thinking before moving on.
Two guardrails to keep. First, no shaming for using tools. The fence is about disclosure, not purity. Second, the probes are not gotchas. They are the same probes we use on ourselves in our own build work, because the same failure mode is universal: fluent output outrunning the model underneath.
Where to go next
- What goes missing when we outsource thinking
- Designing gates for middle school
- Curriculum gates that teach thinking
- Come to the SolutionWright workshop to practice these probes with a live class artifact you bring.