There is a question I used to put on a quiz that a student can now answer, correctly, in about four seconds, without knowing anything. They type it into a phone. So can most of the questions I have written over twenty-two years of teaching. When the answer is that cheap to get, the answer stops being evidence of much, and the thing I write on the worksheet has to change.
For years the quiet assumption behind most assessment was that a right answer signals understanding. It never fully did, and now it barely does at all. A correct answer can mean the student understood, or memorized, or looked it up, or asked a chatbot. The answer has stopped being reliable evidence. The question is now the instrument, and writing a good one is the actual skill.
This post is about writing assessment questions, and the rubric criteria that go with them, so that what a student hands back tells you something true about what they understand. It is not about grading faster. It is about asking better.
A question is a measuring instrument
Treat every question you assign as an instrument you are pointing at a student's understanding. Like any instrument, it measures one thing well and other things badly, and if you point it wrong you get a clean reading of nothing.
"Define osmosis" is a real instrument. It measures whether a student can reproduce a definition. That is a legitimate thing to measure sometimes. But teachers routinely reach for the definition question when what they actually want to know is whether the student understands osmosis, and those are different measurements. A student can define osmosis perfectly and have no idea why a salted slug shrivels. Another can be fuzzy on the textbook wording and explain the slug immediately. If you only ever ask for the definition, you cannot tell those two students apart, and one of them is the student you most need to find.
Recall tells you who studied. Understanding tells you who gets it.
The most useful distinction I know for writing questions is between recall and understanding, because they fail in opposite directions.
A recall question rewards the student who studied last night. That is not nothing. But it is fragile: it decays within a week, it rarely transfers on its own, and it is exactly the kind of thing a language model hands over for free. If your assessment is mostly recall, an AI can now score full marks on it, which is a sign the assessment was never measuring much to begin with.
An understanding question rewards the student who can do something with the idea in a place they have not seen before. It asks for a why, a how, a what would happen if. It is harder to write, harder to answer, and far harder to fake, because the student has to reason in front of you rather than retrieve. When people worry about making an assignment AI-proof, this is the real move, though I would not frame it as defense. A question that reveals understanding is a better question whether or not a chatbot exists. The chatbot just made the weak questions visible.
Three ways to write a question that reveals understanding
Three moves do most of the work.
Ask for the reasoning, not the result.Instead of "What is the theme of the poem," ask "The poem could support two opposite readings of its ending. Argue for one, and explain what in the text forces your choice." The first question has an answer to retrieve. The second has no answer to retrieve, only reasoning to show.
Move it to unfamiliar ground. Understanding is the ability to carry an idea to a situation the lesson never covered. If you taught supply and demand with gas prices, assess it with concert tickets. If you taught force diagrams with a box on a ramp, assess it with a climber leaning back on a rope while rappelling down a rock face. A student who only memorized the gas-price example is now stuck; a student who understood the idea is fine. The gap between them is exactly what you were trying to see.
Make the answer explainable, not lookup-able.If the honest answer to your question is a sentence a student could copy from a source or a chatbot, the question is measuring retrieval. Add the step that cannot be copied: "and explain why the other obvious answer is wrong," or "and justify it using our lab data from Tuesday." The justification is where understanding lives. A chatbot will draft one if a student feeds it the lab data, but the student still has to decide which evidence matters and why, and that decision is the thing you are trying to see.
The rubric can quietly undo a good question
Here is the trap I fell into for years. I would write a genuinely good question, then attach a rubric that rewarded the wrong thing, and the rubric won.
Rubrics that award points for the presence of vocabulary, the number of examples, or the length of the response are recall rubrics wearing an understanding question's clothes. A student works out within one assignment that the way to score is to name-drop the key terms and pad the paragraph, and now your carefully written reasoning question is being answered with a keyword checklist. The question asked for understanding; the rubric paid for memorization; the students, correctly, optimized for the rubric.
The fix is to write rubric criteria that describe the quality of the reasoning, not the presence of ingredients. "Uses three vocabulary terms" rewards retrieval. "Explains why the mechanism produces the result, not just that it does" rewards understanding. The second is harder to mark quickly, which is the honest reason the first is so common. It is also the criterion that makes the whole question worth asking.
One good question, read across the whole class
There is a payoff to writing understanding questions that only shows up when you read the whole class at once.
When every student answers a recall question, a wrong answer mostly tells you they did not study, and there is not much to do with that beyond "study more." But when every student reasons through the same why question, their wrong answers are not random. They cluster. Twenty students who miss the same understanding question tend to miss it in two or three specific ways, and each way is a different misconception you can name and reteach. The pattern is the diagnosis.
Reading that pattern by hand across a full class set is the work that used to make this too expensive to do every week. It is what a class-wide read of every response is built to surface: not just who was right, but the shape of the misunderstanding across the room. I wrote separately about what a knowledge gap report actually is, but the short version is that a well-designed question is what makes the report worth reading. Ask twenty students to define a term and the report tells you who studied. Ask them to explain a mechanism and the report tells you what your class does not yet understand, which is the only thing that changes what you teach next.
A worked rewrite
Take a flat question and walk it up.
Version one:"List the causes of World War I." This measures whether a student memorized the list their teacher gave them. A phone answers it instantly. Every student who studied scores full marks, and you learn nothing about who understands.
Version two:"Which single cause of World War I do you think was most necessary, in the sense that without it the war does not happen? Defend your choice against the strongest objection to it." Now there is no stored answer to retrieve. The student has to weigh causes, commit, and anticipate a counterargument. A chatbot can draft a generic version, but the student still has to own the judgment, and your rubric can reward the quality of the defense over the choice itself.
Version two is more work to write and more work to mark. It is also the only one of the two that can tell you which students understand how historical causation works, which was presumably the point of the unit. The extra effort in the question is effort you save later, because now you are teaching from evidence about what your class actually thinks.
Common questions about writing better rubric questions
What makes a good rubric question?
A good rubric question asks for reasoning rather than recall. It has no answer a student can simply retrieve or copy, only a judgment or explanation the student has to build, and it works best when it moves the idea to a situation the lesson did not cover. That is what separates measuring understanding from measuring who studied last night.
How do you write questions that AI cannot just answer for students?
Ask for the justification, not the result. A language model will draft an answer to anything, including your lab data if a student feeds it in. What it cannot do is be the student: the question still forces the student to decide which evidence matters and defend it against the objection you raised in class, and that decision is what the answer reveals. The goal is not to defeat AI; it is to ask a question whose answer shows whether the student understands.
How is a knowledge gap report different from a grade?
A grade tells you how each student scored. A knowledge gap report reads the whole class at once and shows the pattern in how students went wrong, which usually clusters into two or three specific misconceptions you can reteach. A well-designed understanding question is what makes that pattern meaningful, because wrong reasoning is diagnostic in a way a wrong recall answer is not.
The answer is cheap. The question is the job.
AI has the answers. It had most of them before, in the back of the textbook and the first search result; that is just undeniable now. What it does not have is your question, aimed at your students, about the thing you are trying to teach them this week. Writing that question well is not a workaround for a world full of cheap answers. It is the part of teaching that was always the point, finally made unavoidable.
If this is the shift you are chasing, the companion piece on why questions are what we teach for makes the wider case. Then ask the question worth answering, read what the whole class hands back, and let it tell you what to teach next.