Before a tool enters your room, you get to ask it four questions. What does it see, what does it store, what does it claim, and where are its gates.
This is a checklist. It works for any classroom software, whether it calls itself AI or not, whether it was built by a startup or a district vendor. You do not need a technical background to use it. You do need patience and a willingness to write down answers.
1. What does the tool see
Sit with the tool and list every input surface. Microphone, camera, keystrokes, screen recording, uploaded work, roster fields, learning management system pulls, third-party single sign-on tokens (Class C).
Then ask the vendor to confirm the list in writing. If the answer is longer than the list you drew, something is watching that you did not name. If it is shorter, the vendor may not know their own product. Both cases are a stop sign.
A useful frame: the tool's sensory boundary is a Markov blanket around your classroom. Everything the tool infers about a student is downstream of what crosses that boundary. If you cannot draw the boundary, you cannot govern it (Class E, following Parr, Pezzulo, and Friston, 2022, on the role of sensory states in active inference).
2. What does the tool store
Storage is a separate question from sensing. A tool can see a microphone stream and store nothing, or see a single click and store it for seven years.
Ask for: the storage duration for each data type, the physical or cloud location of storage, the retention policy after a student leaves the school, the deletion process a parent can request, and the audit log the school can pull (Class C).
If the vendor cannot show you the audit log during the sales call, they do not have one you can rely on.
3. What does the tool claim
This is where classroom tools go wrong most often. A tool that promises "personalized learning" is making a claim about what it can infer. A tool that promises "adaptive practice" is making a claim about how it updates. A tool that promises to "understand" a student is making a very large claim indeed.
For each marketing claim, ask: what is the evidence, and what class of evidence is it. Peer citation, published study, internal case study, or vendor assertion. Then ask what would falsify the claim. If the vendor cannot name a failure condition, the claim is not testable, and a non-testable claim is not a promise you can hold them to.
For the language we use about our own work, see honesty-in-classrooms-the-ai-authorship-fence. We treat every public statement about UNI as a working hypothesis with growing, evidence-classed evidence, science in the open. We ask vendors to hold themselves to the same posture.
4. Where are the gates
A gate is a moment where a human decides whether the tool's output goes forward. Grade suggestions, flagged behavior alerts, generated feedback, recommended next steps. Each of these should route through a person before it reaches a student or a family.
Ask the vendor to show you every automated action the tool can take without a teacher's click. If the list includes anything that touches a student directly, you have a gate to install or a feature to disable.
Themesis writes about the same idea from a governance angle, using different vocabulary. Their post on human-in-the-loop review pairs well with this checklist. We link it here because it lands in the same place: automation without a human gate is a promise the vendor cannot keep.
The one-page version
Write these four columns on a piece of paper:
- Sees: every input surface, confirmed in writing.
- Stores: every data type, with duration, location, and deletion path.
- Claims: every marketing claim, with evidence class and falsifier.
- Gates: every automated action, with the human who approves it.
If any cell is blank after your call with the vendor, the answer is not yet. If any cell has the phrase "trust us," the answer is no.
Next
- /transparency: our own posture, held to the same checklist we just gave you.
- a-school-leaders-checklist-before-any-ai-vendor: the leadership version of this document.
- /workshop: where we walk teams through the four questions with a live tool on the table.