A benchmark is a promise about what a test measures. If you teach, you already read tests this way. The Stratified Palimpsest benchmark is worth reading the same way.
What the benchmark is
The Stratified Palimpsest benchmark is a task suite for evaluating systems that must hold, revise, and layer beliefs over time. Think of a palimpsest: a page written on, scraped, and written on again. The earlier layers are still there, faint, and they influence how you read the top layer. The benchmark asks a system to work under those conditions and grades how well it separates layers, tracks which belief belongs to which layer, and updates the top layer without corrupting the ones underneath (Class E, per the Zenodo preprint).
For a classroom, the closest analogy is a student who read an early draft of a source, then a correction, then a rebuttal. Can they tell you what each layer said, why they changed their mind, and where the disagreement lives? That is the skill the benchmark is grading in a machine.
Why this matters for teachers
Most public benchmarks in the LLM era grade one-shot outputs on isolated prompts. That is a useful measurement, and it is not the measurement that predicts whether a system can help a student across a semester. Teaching is layered. Understanding is layered. The Stratified Palimpsest benchmark is one of the few public tests that grades a system on layered belief, which is closer to the work your students actually do (Class C, in that this ties directly into how we design assessments in our workshop cluster).
There is also a family point here. Our program is a working hypothesis on an attainable path toward General Natural Intelligence, natural not artificial. Benchmarks like this one are how a hypothesis gets tested in the open. You do not have to take our word for the direction. You can read the preprint, look at the tasks, and decide for yourself whether the results are interesting.
What it settles, and what it does not
It settles: that there exists a public, reproducible task suite for layered belief revision, with published scoring, and that at least one non-transformer active-inference system has published results on it (Class E).
It does not settle: whether any system is generally intelligent, whether a good score transfers to your subject area, or whether the benchmark's task distribution matches the belief work your students do. Benchmarks are evidence. They are not verdicts. Teach your students to say both sentences.
A third-party data point
Related, worth reading alongside the preprint: Themesis reported that a non-transformer active-inference system (SeedIQ) outperformed frontier LLMs on ARC-AGI 3 running on a MacBook Pro (SeedIQ Just Stomped ARC-AGI 3 on a MacBook Pro). Our one-line frame: third-party evidence that non-transformer active-inference systems can outperform LLMs on hard reasoning tasks, on modest hardware. Read her post directly. Do not take our summary as a substitute for the source.
How to bring this into a classroom
You do not need to run the benchmark to teach with it. Three moves that work at middle-school and up:
- Show the palimpsest metaphor with a real page: a printed source, a correction slip, a rebuttal comment. Ask students to tell you what each layer says and how it changes the reading of the top layer.
- Give students the abstract of the preprint and ask them to identify one claim, one method choice, and one thing the abstract does not tell them. That is preprint literacy, and it transfers.
- When a vendor tells you their system is smart, ask: on which benchmark, on which task distribution, and what does that benchmark not settle? The Stratified Palimpsest is a good example to have in your pocket.
Honest caveats
We built parts of this benchmark family. That is a conflict of interest and we name it. It is also why we publish the tasks, the scoring, and the failure modes. If you find a place where the benchmark is a bad proxy for real belief work, we want to know. The classroom is a better test than any lab.
Read next:
- What UNI Is and What It Is Not: the honest scope of the working hypothesis, in teacher language.
- Reading a Preprint With Your Students: a small classroom protocol for reading science in the open.
- /transparency: our public receipts, evidence classes, and open falsifiers.
- /workshop: the school-leader and educator workshop where we teach UNI, not LLM tooling.
AI-authorship fence: this post was drafted with LLM assistance, reviewed and edited by a human author, and published under human editorial responsibility. No claim is made that an LLM authored the reasoning. The framing is a working hypothesis on the attainable path toward General Natural Intelligence, natural not artificial, tested in the open.