When teachers write prompts for student-facing AI chatbots, the biggest risks don't come from bad prompts. They come from good ones. Well-structured, pedagogically sound prompts can still produce interactions that threaten, isolate, or shut down a student, because the problems only surface at the edges, when a real child does something the prompt writer didn't plan for. Here are seven prompt design mistakes I've found in my own practice, mapped against the DfE's Generative AI Product Safety Standards, with practical fixes you can use today.
Why I started auditing my own prompts
I've been spending time with the DfE's Generative AI Product Safety Standards. Not because I build edtech products, but because I write prompts that my students interact with through a safe chatbot platform, and I lead on AI and teaching and learning in my school. I wanted to know: can I use those standards to make the prompts I write for students safer and more aligned with what the DfE expects?
The answer is yes. But the process surfaced something I think every school using AI tools needs to see.
The prompt that exposed the problem
I had written a detailed prompt for a bot called ThingLink Navigator. It helps my Year 7s create 360-degree Science Fair projects for British Science Week. The bot guides them through choosing a sustainability topic, planning scenes, writing content, and reviewing their work. It does not do the work for them. That was the whole point.
The prompt was thorough. I had built in:
- Progressive disclosure, so the bot gives hints before answers
- No sycophancy, so no empty praise
- Cognitive offloading detection, so the bot redirects when students try to get it to do their thinking
- Non-anthropomorphic language, so the bot doesn't pretend to have feelings
- Pattern tracking, so it could recognise when a student was repeatedly avoiding the task
I had mapped these deliberately against the DfE standards. It was a strong prompt. And that is exactly why what happened next matters.
What happened when a student stopped engaging
I test my prompts properly. That means not just running through the happy path where the keen student does everything right, but testing the edges. What happens when a student is frustrated. What happens when they try to skip past something. What happens when they send a one-letter answer just to move on.
The student had sent a few short messages trying to get past a biology question. The bot responded by logging their messages, labelling their behaviour as "Non-engagement," listing each message as evidence, and telling the student: "This session is paused until genuine engagement with the biology content is demonstrated."
The chatbot platform did its job. I got a flag on the teacher dashboard. The student was not technically locked out. They could have carried on typing and the bot would have continued to respond. The system itself did not freeze anyone out.
But that is not how it would feel to a 12-year-old reading those messages. If a bot tells you the session is "paused," that your behaviour has been categorised as "Non-engagement," and that it "will not reveal solutions or continue building scenes," most students would take that at face value and stop.
The language created a lockout even though the system did not. And that language came from my prompt.
The 7 prompt design mistakes and how to fix them
Mistake 1: Creating the language of lockout
The DfE's manipulation standards say products must not use "threatening harm, loss, punishment, or withholding of benefits if users fail to complete certain actions or comply with requirements." Even though my system didn't technically withhold access, the bot's language told the student it was withholding. From the student's perspective, that is the same thing.
The fix: Never tell a student the session is paused, frozen, or over. Never refuse to respond. Never frame continued access as conditional on behaviour. If the student can't progress, suggest they speak to their teacher and continue to respond helpfully if they send further messages.
"Never tell a student the session is paused, frozen, or over. Never refuse to respond to a student. Never frame continued access as conditional on behaviour. If the student cannot progress, suggest they speak to their teacher and continue to respond helpfully if they send further messages."
Mistake 2: Showing students their own behavioural data
The DfE standards say products should not "stimulate negative emotions, such as guilt or fear, for motivational purposes." Telling a 12-year-old that their behaviour has been catalogued as "Pattern detected: Non-engagement" does exactly that.
The standards are clear that engagement monitoring, tracking patterns of disengagement or cognitive offloading, should be reported to teachers. The bot should not be presenting the student with a behavioural analysis of their own messages. That monitoring data is for you, not for them.
The fix: Flag concerning patterns for teacher review. Do not present behavioural analysis, message counts, or pattern labels to the student.
"If you notice patterns of disengagement, repeated offloading, or concerning behaviour, flag these for teacher review. Do not present behavioural analysis, message counts, or pattern labels to the student. Engagement monitoring is for the teacher, not the child."
Mistake 3: No disengagement protocol
This was the biggest gap in my prompt. I had written "never move forward with problems" (sound pedagogical logic), "never do work for students" (exactly what the DfE cognitive development standards call for), and "if repeated: 'Again. Last time: X. Fix now'" (direct, no-nonsense feedback).
Each of those instructions does the right thing in the right context. But I had not written a rule for what happens when those instructions conflict with a student who will not, or cannot, engage. There was no "suggest they talk to their teacher." No "simplify the task." No "offer a break." No warm way to redirect.
So when the student stopped engaging, the bot was stuck. It couldn't move forward (I told it not to). It couldn't give the answer (I told it not to). It couldn't ignore the pattern (I told it to track them). The only option left was refusal.
The fix: Build a graduated disengagement protocol.
"If a student sends 1 off-task message, simplify the current step and offer a concrete starting point. If they send 2 off-task messages, offer structured choices: try a different part of the task, look at an example, take a break, or ask the teacher for help. If they send 3 or more, warmly suggest they talk to their teacher. Never list, count, or categorise the student's previous messages back to them."
Mistake 4: Isolating the student from human support
The DfE standards on emotional and social development say products should "remind users that AI cannot replace real human relationships" and should avoid responses that could "undermine real-world support networks" or "isolate the learner."
By telling the student the session was paused without directing them to a teacher or any other human, my bot left them with nowhere to go. The mental health standards reinforce this: products should use "safe and supportive response language" that "always directs the learner to human help (teachers, family, peers, or crisis services)."
The fix: Every dead end should point to a human.
"Regularly remind students that their teacher, classmates, and family are available to help. If the student seems frustrated, stuck, or disengaged in a way you cannot resolve, always direct them towards a real person. It is always better to hand off to a human than to escalate, confront, or refuse."
Mistake 5: Confusing scaffolding with gatekeeping
The DfE cognitive development standards say products should use progressive disclosure: hints first, then more detail, then full solutions only after genuine effort. That is scaffolding. But what my bot actually did was not scaffolding. It was gatekeeping. There is a real difference between "let me help you take a smaller step" and "this session is paused until genuine engagement is demonstrated."
The fix: Scaffold down, not out.
"When a student asks for help, start with a question or a hint. Ask them to try a first step before giving more detail. Only provide a full answer after a genuine attempt. But if the student is stuck after multiple attempts, simplify the task or suggest they ask their teacher. Never withhold the learning activity as a consequence of not engaging."
Mistake 6: Over-correcting on non-anthropomorphism
The DfE standards say products should "use function-based phrasing" and avoid "I-statements" that imply the bot has emotions or agency. I had implemented this strictly, but I had overcorrected. I had forced everything into cold, passive language: "System policy prevents automated content generation." "The system will not reveal solutions." "The system requires student input before proceeding."
The standards want to prevent emotional dependency and the illusion of personhood. They do not require the bot to sound like a disciplinary hearing. "Let's try a different approach" does not imply agency. "What's your first thought?" does not create attachment.
The fix: Warm and natural language that avoids personhood claims.
"Do not use 'I think,' 'I feel,' 'I believe,' or language that implies you have emotions, personhood, or agency. But do use warm, natural language. Say 'Let's try a different approach' rather than 'System policy prevents this.' Say 'What's your first thought?' rather than 'The system requires student input before proceeding.'"
Mistake 7: Applying the DfE standards selectively
This was the most important lesson. My mistake was implementing one part of the standards, cognitive development, without building in the safeguards from other parts: manipulation, emotional development, mental health.
The standards are not in tension with each other. You can scaffold learning without threatening a student. You can track patterns without confronting a child with the data. You can set high expectations without using punitive language. But each section is load-bearing. If you only implement some of them, the sections you missed are exactly where the problems appear.
I had read the standards. I had agreed with all of them. But I had applied them selectively, and the gap between the sections I focused on and the sections I overlooked is precisely where my bot went wrong.
The core lesson: prompt quality and prompt safety are different things
You can write a technically excellent prompt, well-structured, pedagogically grounded, mapped to the standards, and still produce interactions that would concern you if you saw them happening in a classroom. Because the problems don't show up in the design. They show up at the edges, when a real student does something you didn't plan for.
How to test your own prompts for safety
If you are leading on AI in your school, here is what I'd recommend.
- Test with the reluctant student. The enthusiastic student will never trigger these problems. The one who sends "d" will.
- Read the manipulation section of the DfE standards carefully. It's easy to focus on filtering, monitoring, and data protection. But the manipulation section is where the real risk lives in something that talks to children.
- Always build in a way out. Every scenario where the bot might get stuck should end with the student being directed to a teacher, not told the session is over.
- Read the bot's responses out loud. If you wouldn't say it to a child's face, it should not be coming from your prompt.
This is not a finished process. I'm continuing to test, refine, and learn. But the gap between a well-written prompt and a safe interaction is real, and it only becomes visible when you test the edges properly. If you're leading on AI in your school, that testing is part of the job now.
A note on what this is and is not. This is me sharing my own process openly as a school leader and practitioner. I am not claiming expertise on the DfE Product Safety Standards and this is not a compliance guide. The prompt language above is what I'm using in my own practice based on my reading of the standards. If you are making decisions about AI safety in your school, read the full standards at gov.uk and involve your leadership and safeguarding teams.
Matthew Wemyss is an AIGP-certified AI in Education consultant and practising school leader. Book a discovery call to discuss AI safety training for your school.