A student pastes the prompt, copies the reply, hands it in. The paper looks fine. Something quiet happened, though: a loop that would have changed the student did not run.
This piece is not a scolding. It is a mechanical description of what a chatbot substitutes for, and what that substitution costs, in the language we teach here.
The loop that gets skipped
In an active-inference frame, thinking is a cycle: predict, act (or write), notice the error between what you expected and what came back, update the internal model, predict again (Class E, Parr, Pezzulo, Friston 2022). Learning is that update. It is the thing that carries over to the next problem.
When a chatbot writes the paragraph, that cycle does not run inside the student. It runs inside the model. The student sees the finished output but never held the wrong version long enough to feel it be wrong. The prediction error, which is the teaching signal, is emitted somewhere the student cannot receive it.
That is the whole mechanism. Everything below is a consequence.
What actually gets outsourced
Three specific things go missing when the chatbot writes:
- The bad draft. The bad draft is not waste. It is the substrate the correction acts on. Delete it and the correction has nothing to bite.
- The moment of surprise. "I thought this sentence would land and it did not" is the update signal. Reading a competent finished paragraph does not produce it.
- The felt sense of a working model. Students who write, revise, and reread build an internal sense of what their own thinking sounds like when it is running well. Copied output does not add to that library.
None of this requires any claim about what the chatbot is or is not. The point is about what happens, or does not happen, on the student's side of the exchange.
Why "just check it after" is not the same
A common patch is: let the chatbot draft, then have the student edit. This helps, but it is not equivalent to writing. Editing a fluent draft is a shallow prediction-error cycle: the student compares the text to a vague sense of "does this sound right," and the model updates in small ways. Composing from scratch is a deep cycle: the student generates the prediction, watches it fail on the page, and revises the underlying model that produced it (Class E).
Both cycles are real. They are not the same cycle. A course that always runs the shallow one and never runs the deep one will produce students who can polish, and cannot generate.
A classroom implication, not a rule
We are not telling teachers to ban chatbots. We are saying: know which loop you are asking the student to run today, and design the assignment so that loop actually runs.
If today's learning goal is "generate a hypothesis you did not have this morning," the chatbot writing the hypothesis defeats the assignment at the mechanism level, not at the ethics level. If today's goal is "polish a draft you already wrote," a chatbot as an editor is aligned with the loop you want.
The gate is not moral. It is mechanical. Match the tool to the loop.
Themesis, field context
Themesis has been arguing that the ground under "what counts as intelligence" is shifting, and that the field's framing choices now matter more than they did a year ago. We read that as field-level context for why the GNI framing, natural not artificial, matters right now, and why the education conversation cannot lag the science conversation. See The AGI Landscape Just Changed for her post; the one-line summary here is ours.
The classroom question is downstream of the field question. If intelligence is the loop, then outsourcing the loop is the thing to notice.
Where to go next
- Learner Agency in a World of Generative Models sets up the frame this piece assumes.
- Prediction Then Check in Daily Lessons shows the loop as a concrete classroom rhythm.
- When a Student Uses a Chatbot, What to Do is the practical companion, decisions, not rules.
- The workshop is where teachers practice designing assignments around the loop rather than around the tool.