A pilot is a small, bounded question with a written answer key before the term starts. If the measurement plan is vague, the report at the end will be vague, and everyone loses the year.
What a pilot is for
A single-classroom pilot is not a marketing exercise. It is a shared attempt by a teacher, a department chair, and a partner to answer a small, honest question about student thinking under a specific practice. The practice we bring is active inference in the classroom (Class E, per Parr, Pezzulo, and Friston, 2022): students hold a working model, meet an observation, and update on purpose. The question the pilot answers is whether that habit shows up in student work over a term, and whether it travels between subjects.
We write the question, the measures, and the refusals down before day one of the term. We publish them to the school and to the family. That written page is the contract for the pilot.
What we count
We count things a teacher can point at without a tool.
- Frequency of student-attached evidence tags on written claims. We use the six-letter set (A, B, C, E, F, U), described in evidence-classes-as-classroom-vocabulary. A pilot tracks how often students attach a tag at all, and how the mix shifts over the term.
- Frequency of stated falsifiers. When a student writes a claim, do they also write the observation that would make it wrong? We count the F tags and the sentences behind them.
- Number of on-record model updates. When new evidence lands, do students revise the earlier claim in writing rather than deleting it? We keep the revision, not just the final answer.
- Teacher time cost. How many minutes per week does the practice add or remove from planning and grading? A pilot that saves the teacher no time is a pilot the school should not scale.
Every count comes from artifacts the class already produces: boards, notebooks, lab reports, essays. We do not add a separate testing instrument on top of the term.
What we refuse to count
We name the refusals in advance so nobody, us included, drifts.
- We do not count "engagement" as a proxy for learning. Engagement is easy to stage and hard to defend.
- We do not count standardized test deltas as evidence for or against the practice inside a single-term pilot. The signal-to-noise ratio is wrong at that scale, and we will not claim otherwise.
- We do not count teacher enthusiasm as an outcome. It is a working condition we care about, not a finding.
- We do not run any measurement that requires a student to log into an outside service to be counted. The pilot has to work on paper.
Which classes of evidence apply
The findings from a pilot fall into a narrow band of evidence classes, and we will label them that way in the write-up (Class C, our configuration and reporting rule).
- Counts of tags, falsifiers, and revisions are Class A: empirical in-session, produced by the classroom during the term.
- The rubric, the tag definitions, and the reporting template are Class C: configuration. They are documented and re-runnable at another school.
- Any claim we make about active inference as a frame is Class E: expert citation, and it stays tied to the published literature (Parr, Pezzulo, Friston, 2022; see also the working posture we hold in /transparency).
- Any claim we cannot yet ground gets a Class U tag and stays there until we can ground it. U is not a failure. U is honesty.
What we refuse to claim from a pilot
A term of data from one classroom cannot say the practice raises achievement, closes gaps, or generalizes to other schools. It also cannot say the practice fails to do those things. We will not write either sentence into the report. What a good pilot can say is: the habit was present or absent, the teacher time cost was this, the artifacts look like this, and here is what a reader would need to see next to trust it further. That is enough to decide whether to run a second, larger pilot with fresh consent.
If a school wants to see the pilot template before we start, we send it. If a school wants to end the pilot mid-term, the artifacts stay with the school and the write-up is dropped.
Where this fits
- Read a-single-classroom-pilot-what-to-expect for the week-by-week shape of a pilot.
- See evidence-classes-as-classroom-vocabulary for the tag set students use.
- Our full posture on claims, gates, and refusals lives at /transparency.