A teacher wants to write feedback on thirty reports. The chatbot can do that, provided it gets the reports. So the text goes in, with the student's name at the top and sometimes a note from the support file after it, because that makes the feedback better. Nobody means any harm. Still, the school has now handed student data to a vendor it has no agreement with.
The Dutch Data Protection Authority (the Autoriteit Persoonsgegevens, AP) knows this pattern from other sectors. It received reports of data breaches in which employees had shared personal data of patients or customers with an AI chatbot. Often the employee did so on their own initiative, against the agreements in place. Its advice to organisations: make clear agreements, and spell out which data staff may and may not enter.
Why an agreement isn't enough
An agreement works for as long as someone remembers it. In the week of the grading meeting nobody does, because the teacher's goal is to get the work done. A rule in a handbook doesn't change that.
Two layers of TRACES meet here.
TRACES are the six layers on which you assess an AI system: Trail, Retrieval, Approval, Constraints, Expertise, Storage. Where a layer is missing, there is a risk.
Constraints. What a system technically can't do is stronger than what it's not allowed to do. "Teachers are instructed not to enter names" is a request. An environment where teachers only sign in with a school account, on a tool covered by a data processing agreement, is a boundary. Kennisnet, the Dutch public knowledge centre for ICT in education, advises managing accounts centrally and preventing staff or students from creating accounts of their own outside the school's view.
Storage. Where does a piece of data end up once it has been entered? Kennisnet warns that data entered into an AI service can end up in the training of the model, and advises setting tools up as privacy-friendly as possible, for instance so that the vendor may not use prompts as training data. Whether that holds for your tool is in the data processing agreement, and under Article 28 of the GDPR that is where it should be recorded.
Constraints catches the mistake up front. Storage decides how big the damage is if something gets through anyway.
A tool: three buckets
Not everything has to be shut. Much of a teacher's work isn't about people at all: building a lesson, a test in four difficulty levels, an explanation in plainer language. It helps to separate that explicitly, so a teacher can see in ten seconds which bucket a task falls into.
Bucket 1 · Anywhere. Nothing in it is about a person. A lesson outline, sample exercises, an explanation of a concept, an email to an anonymous audience.
Bucket 2 · Only in the school environment. Work that involves student data, in a tool the school has a data processing agreement for and has run a DPIA on. What that takes is covered in the article on what a school board asks before teachers get an AI tool. If no such environment exists, bucket 2 doesn't exist.
Bucket 3 · Never. Files, support notes, advice about a student, and anything the school treats as confidential.
Many tasks can move from bucket 2 to bucket 1 with a single step: replace the personal data.
Replace the data before the prompt
Kennisnet advises avoiding entering personal data and confidential information, and using anonymous or fictional data where possible. That fits Article 5 of the GDPR, which asks for data limited to what is necessary for the purpose. For feedback on a report, the student's name isn't necessary. The text is.
A made-up example.
- Not this: "Give feedback on the report by Sanne de Vries, 3B, who fell behind last year after her illness."
- This: "Give feedback on the report below by a student in year 3. Look at how the conclusion is built."
The second prompt gets the same feedback. What's missing didn't matter for the answer.
One caveat. A text without a name isn't automatically anonymous. If the report itself contains details that point to one student, such as an unusual event or a personal situation, the text can still be traced back, and belongs in bucket 2. A teacher who's unsure leaves it in bucket 2 or 3. That doesn't need a lawyer.
When it goes wrong anyway
It happens, even at a school with good agreements. Make sure a teacher doesn't have to wonder whether reporting is allowed. Agree beforehand who it goes to, and that a report never leads to a telling-off. According to the AP, reporting such a breach to the authority and to the people affected is mandatory in many cases, and a school that only hears about it weeks later can't do that in time. A teacher who can simply say "this morning I put a file in the wrong tool" is worth more to the school than one who stays quiet.
What a school can set up
- One list of approved tools, and one place where teachers can find it. If someone needs a tool that isn't on it, they can submit a request. Without that route, teachers go looking themselves.
- School accounts, not private ones. Then the data processing agreement already covers them.
- Settings in advance. That prompts aren't used for training is something an administrator sets, not the teacher.
- A sorting card on one A4. The three buckets, with examples from the school's own work, at every workstation.
- Explanation for the team. That fits the AI literacy Kennisnet mentions: everyone who uses AI should understand both the benefits and the risks.
What you can do this week
- Ask the team which AI tools teachers use now, and for what. Ask without judging.
- Take ten recent tasks from everyday work and sort them into the three buckets.
- For each task in bucket 2 and 3, write down which data can come out, and what's left.
- For the most-used tool, check whether a data processing agreement exists and whether teachers sign in with a school account.
- Hang up the sorting card, and discuss it once in a team meeting.
The Ketencheck on this site shows where in the chain behind a piece of teaching material the origin of a claim gets lost. For student data the same question applies, only at the entrance.
Sources
- Kennisnet, Privacy en AI: hier moeten scholen op letten (updated 22 September 2025). kennisnet.nl
- Autoriteit Persoonsgegevens, Caution: use of AI chatbot may lead to data breaches. autoriteitpersoonsgegevens.nl
- Regulation (EU) 2016/679 (GDPR), Article 5(1)(c) and Article 28. eur-lex.europa.eu