I stood in front of the whole of Key Stage 3 a few weeks ago and ran a live AI query on the screen. I asked it a factual question about a topic the students knew well enough to care about. The answer came back fluent, confident, and detailed.
It was also wrong.
I proved it, step by step, in front of several hundred students. The room went quiet. Not because they were impressed. Because they were embarrassed. Most of them use this tool every day. Many of them use it for schoolwork. And it had not occurred to them to do what I had just done: check.
Some of them changed how they talked about AI after that assembly. A few started asking better questions. But here is the thing that stayed with me: why did it take a live demo to make them sceptical of something they already know is fallible?
They laugh when AI produces five-legged animals. They know AI makes things up. They have seen it fail. And yet, when it generates text, fluent, structured, confident text, they accept it. Not because they have evaluated it and decided to trust. Because questioning it does not occur to them.
They are not directing their use of AI. They are being carried along by it.
What aviation's worst disaster teaches us about AI in schools
On 27 March 1977, two Boeing 747s collided on a fog-covered runway at Tenerife airport in the Canary Islands. 583 people died. It remains the deadliest accident in aviation history.
The cause was not mechanical failure. It was human. The captain of the KLM aircraft, one of the most experienced pilots in the airline, the man who literally appeared in their advertisements, began his takeoff roll without clearance. His co-pilot and flight engineer both had doubts. The flight engineer even questioned the captain aloud. But the challenge was tentative, indirect, and ultimately ineffective. The captain's authority was so absolute that the people best positioned to prevent the disaster could not bring themselves to override him.
The investigation coined a term for this: the authority gradient. When one person is perceived as far more competent or powerful than another, the junior party stops pushing back. Not because they cannot see the problem. Because the social cost of challenge feels higher than the risk of staying silent.
The pattern repeated across the industry. In crash after crash, the technical skills were present. The procedures existed. The checklists were followed. But the culture in the cockpit made it nearly impossible for a junior officer to challenge a senior captain's judgement, even when lives depended on it.
The checklists did not fail. The culture overrode them.
How Crew Resource Management rebuilt cockpit culture
Aviation's response was not another checklist. It was a complete redesign of cockpit culture.
Crew Resource Management, or CRM, was developed in the early 1980s. Its core insight was that technical competence alone was not enough. Crews needed shared communication protocols, structured challenge procedures, and explicit permission to question authority when something felt wrong.
- Shared mental models. Every crew member should have the same understanding of the current situation. If one person sees something differently, that difference must be surfaced, not suppressed.
- Graded assertiveness. Junior officers were taught a structured escalation: observe, state your concern, propose an alternative, and if necessary, take action. Not a vague encouragement to "speak up." A specific, practised sequence.
- Normalised challenge. The culture shifted from "the captain is always right" to "the captain is responsible, but anyone can challenge." Challenge became an expected part of the workflow, not a violation of hierarchy.
- Briefing and debriefing. Before every flight, the crew discusses the plan, the risks, and the roles. After every flight, they review what happened. The loop is closed. Learning is built into the routine.
CRM is widely credited as one of the most significant safety interventions in aviation history. The principles have since been adopted in medicine, nuclear power, and maritime operations. Education has not adopted them.
Why students defer to AI the same way co-pilots deferred to captains
The structural dynamics are the same. In a cockpit, the authority gradient runs between captain and first officer. In a classroom, it runs between AI and student. The AI produces output that is fluent, confident, and immediate. The student produces output that is hesitant, unfinished, and slow. The gradient is steep. And just like in a cockpit, the junior party defaults to deference.
My students defer to AI-generated text the way a nervous co-pilot defers to a captain. They assume fluent text equals correct information. They do not actively doubt it. The output carries the weight of authority: it sounds like a teacher, a textbook, an expert. And challenging authority is not something most teenagers do naturally, especially not in front of peers.
I have never seen a student openly challenge an AI output in front of their classmates. Not once. They will laugh at a five-legged animal because the error is visible and the cost of pointing it out is zero. But challenge a paragraph of confident, well-structured text? That requires a different kind of courage. It requires the student to say: I think this is wrong, even though it sounds more articulate than I do.
Aviation discovered that this silence is not a character flaw. It is a system design problem. The co-pilots at Tenerife were not cowards. They were operating in a system that had not given them the language, the permission, or the practice to challenge effectively. The same is true of your students.
Why school AI policies fail without a culture of challenge
My school has rules for AI use. Check your sources. Do not copy and paste. Verify claims. Most schools have something similar.
Students know the rules. They know they should check. They do not check, because checking requires them to doubt something that sounds more authoritative than they feel. The rule says "verify." The culture says "this sounds right, and questioning it would mean admitting I am not sure."
And so the student stops directing. Not because they chose to, but because the path of least resistance led them there. Every unchallenged AI output is a small abdication. Enough of them, and the student forgets they were ever supposed to be in charge.
You cannot fix an authority gradient with a rule. You fix it with a culture.
Classroom Resource Management: four protocols that actually work
Here is what CRM looks like translated for schools. Four elements, drawn from the aviation principles but shaped for the reality of a classroom where AI is present.
1. The callout
Give students a shared phrase they use before accepting any AI output. Aviation uses "Cross-check" and "Verified." Your version might be as simple as "Checked or accepted?" spoken aloud or written at the top of the work.
The point is not the specific words. The point is that the moment of acceptance becomes visible rather than invisible. Right now, students accept AI output silently. The callout makes the decision conscious.
2. The challenge pair
Pair students so that one is the "pilot" (using AI) and the other is the "co-pilot" (challenging the output). The co-pilot's job is not to check whether the AI is right. It is to ask: does this actually answer the question we were asked? Is there anything here that does not make sense? What would we change?
Rotate roles. The skill is not being the critic. The skill is practising challenge as a normal part of the workflow.
3. The visible routine
Put the challenge protocol on the wall. Not as a poster that gets ignored. As a living routine that gets used.
- Before you accept AI output: What did you ask it?
- During: Does this match what you already know?
- After: What would you change and why?
Make it visible enough that it becomes habit.
4. The debrief
CRM's most powerful tool is the post-flight debrief. In the classroom, this takes five minutes at the end of a task. Not "what did the AI get right?" but "where did we challenge it and where did we just accept it?"
Over time, students start noticing their own patterns of deference. They begin to see the authority gradient not as an invisible force but as something they can name and resist.
Attitude is cultural, not individual
The "a" in Noller's creativity equation is not just individual. It is cultural. A student can have perfect self-awareness, strong critical thinking, and genuine scepticism, and still defer to AI if the culture around them treats challenge as disruption.
The authority gradient is not inside the student. It is between the student and the system. And it is maintained by every classroom that has rules for AI but no culture of doubt.
Aviation did not fix authority gradients by telling co-pilots to be braver. It fixed them by redesigning the system so that challenge was expected, practised, and rewarded. Schools need to do the same thing.
Two things you can do this term
1. Run one challenge pair session this week. Pick a task where students are using AI. Pair them. One uses the tool, the other challenges the output. Rotate. Debrief for five minutes at the end. See what surfaces.
2. Introduce the callout. Choose a phrase and make it the standard question before any AI-assisted work is submitted. Written at the top of the page or spoken aloud. The goal is to make the moment of acceptance visible.
Your students don't need more rules. They need a culture where questioning AI is as normal as using it. That is what it means to direct intelligence, rather than be directed by it.
References
- Bainbridge, L. (1983). Ironies of Automation. Automatica, 19(6), pp.775-779.
- Helmreich, R.L., Merritt, A.C. and Wilhelm, J.A. (1999). The Evolution of Crew Resource Management Training in Commercial Aviation. International Journal of Aviation Psychology, 9(1), pp.19-32.
- Parasuraman, R. and Riley, V. (1997). Humans and Automation: Use, Misuse, Disuse, Abuse. Human Factors, 39(2), pp.230-253.
- Endsley, M.R. (1995). Toward a Theory of Situation Awareness in Dynamic Systems. Human Factors, 37(1), pp.32-64.
- Noller, R.B. (1977). Scratching the Surface of Creative Problem Solving: A Bird's Eye View of CPS. Buffalo, NY: DOK Publishers.
Matthew Wemyss is an AIGP-certified AI in Education consultant and practising school leader. Book a discovery call to discuss building an AI culture in your school.