The internal AI assistant has been live for four weeks. Since then, a log has been sitting on your server: draft quotations, help with wording, questions about complaint handling. In between, a search on severance pay in termination agreements, the same grammar question three times from the same person, and a cover letter addressed to a company that is not yours. Four weeks ago, all of that would have ended up with a cloud provider. Now it sits with you.
Genau das war der Plan. Wer Sprachmodelle im eigenen Haus betreibt, muss keinem Anbieter vertrauen, keinem Auftragsverarbeiter hinterherprüfen und keine Drittlandübermittlung begründen. Das Argument stimmt. Wir vertreten es selbst, ausführlich und mit Quellen, im Artikel Warum On-Premise-KI fast immer sicherer ist. Es hat nur eine zweite Hälfte, die im Prospekt fehlt: Der Datenschutz verschwindet mit der Cloud nicht. Er zieht bei Ihnen ein.
This article is about that second half.
Whoever holds the data is the party it needs protecting from.
Data protection moves in with you
With operation in your own house, you change roles. You are now the controller in the GDPR's sense [1], and for a kind of data your company has never held before. Prompts are not usage data like login timestamps. People write things to a machine they would never write to a colleague: the mistake they just made and want to understand before reporting it. The question of whether a symptom is enough for a sick note. The conflict with their manager, wrapped as a request for a diplomatic reply to this one e-mail. Read over months, a prompt log is a behavioral profile, and a more intimate one than e-mail, because the inhibition threshold toward a machine is lower.
Legally, this lands in employee data protection, section 26 of the German Federal Data Protection Act [2]. The supervisory authorities' guidance on AI and data protection makes clear that they expect purposes, legal bases and access concepts to exist before production use starts [3]. The Mittelstand reality alongside that: there is no data protection department. There is an IT lead doing this on the side, and maybe an external data protection officer with four hours a month. The question of who may look into the log still has to be answered before the first request lands in it. Afterwards it answers itself, and wrongly.
The harmless question
Six weeks after go-live it arrives, usually in a management meeting: which department actually uses the thing the most? The question is legitimate. The machine cost money, somebody wants to know whether it earns its keep. One table, six rows. Done.
Except a usage statistic about employees is not an operating figure like power consumption. It says who works how much with a tool, which makes it objectively suitable for monitoring performance and behavior. The German Works Constitution Act has a provision for exactly this: section 87(1) no. 6 makes technical systems suitable for monitoring subject to co-determination, and intent does not matter [4]. We only wanted to see utilization is the sentence such evaluations start with.
That does not mean the statistic has to be dropped. It means it has to be built so that both sides can trust it: management, because the numbers add up, and the works council, because no individual can be read out of them. That sounds like a matter of negotiation. First of all, though, it is plain arithmetic.
Anonymization is arithmetic
Let us run it through a model company whose numbers come from our own test environment: 45 employees, six departments. Production 24, quality management 7, sales 5, purchasing 4, management 3, IT 2. In the first month they submit 1,240 requests between them: production 180, QM 310, sales 440, purchasing 150, management 90, IT 70.
The textbook approach is k-anonymity [5]: a row is only shown if at least k people stand behind it, with k = 5 as the usual choice. Sounds solid. In this company, the rule deletes three of the six rows, because purchasing, management and IT sit below the threshold. The remaining rows sum to 930. Anyone who knows the total does the math and asks who the missing 310 requests belong to. Suppression that stops the totals from adding up points straight at the thing it is meant to hide. The workable answer is a residual row: small departments merge into one line, 'other departments', and the arithmetic holds again.
The second trap sits in the question of what k actually counts. The department's headcount? Then here is what happens in a pilot: 24 people work in production, and exactly one of them works with the assistant, the foreman doing work preparation. The 180 requests in the 'production' row are his 180 requests, and everyone in the building knows it. A threshold on headcount waves the row through, 24 being greater than 5. Nobody is protected. What the threshold has to count is the people who were actually active in the evaluation window, because that is the set the number can have come from. With a single active user, that means the production row disappears into the residual line, however large the department is on paper.
A threshold on request volume would be even more crooked. Sales has five people and 440 requests. By volume the row looks entirely unremarkable, yet it is the small team with the heaviest use, which is exactly the case where activity is easiest to pin on individuals. And then the time range: a monthly statistic over 45 people looks harmless. But if the viewer can pick the window freely, they can query single days and take differences, Tuesday against Wednesday, the week before someone's vacation against the week after. Anyone who knows who was in when reads a name out of the difference between two harmless numbers. So the threshold has to apply within the queried window, not the calendar month. A single-day window then has too few active people almost everywhere, and simply returns the residual line.
None of this is AI-specific, statistical offices have wrestled with the same effects for decades. But AI assistants create, in many companies for the first time, a fine-grained, central record of knowledge work. And with that, this arithmetic migrates out of the academic literature and into the works agreement.
Not knowing as a design principle
In the GDPR, the relevant principle sits in Article 5: data minimization [1]. It is usually read as a frugality rule, collect little, delete early. Taken seriously, it is a construction rule: the system gets no path to certain data at all.
An example from our own engineering. The evaluation layer of our platform, the one that produces exactly the department statistic above, would naturally have been given a connection to the user directory, the LDAP that manages logins and roles. Then it knows headcounts and department lists, the whole organization. We cut that connection. The component that counts what was asked cannot enumerate the workforce. Of a person, it knows what their login brings along: an identifier and a department attribute. Someone who never logged in does not exist for it.
This was not generosity, and technically the decision even paid off: the anonymity threshold counts active people in the window anyway, and those are present in the request data itself. The directory connection would have been a second subsystem, a second copy of the bind credentials, one more place that goes down when the directory does. The renunciation removed a subsystem instead of adding one.
The real value lies elsewhere. What is not connected cannot be queried, by anyone: not by the curious report somebody retrofits in three years, not by a compromised admin account, and not by us as the vendor. Even management in a weak moment finds no lever. An access ban is a rule. Rules have to be enforced and can be circumvented. A missing connection is topology. It can even be verified from the outside: whether a component never opens a connection shows in its configuration and its network traffic. Whether it never abuses a permitted connection is something you have to believe.
The bill
So much for the theory, now the bill. Renunciation is paid for in questions that no longer have an answer.
Who has never used the tool? No answer. The system only sees who acts. It cannot compute the difference against the workforce, because it does not know the workforce. For the management report, that means there is no adoption rate across all employees. There is the number of active people per month, and that is where it ends.
The follow-up question has even had a legal hook since February 2025. Article 4 of the AI Act requires operators to ensure sufficient AI literacy among their staff [6]. The obvious metric would be a rate, 31 of 45 trained. A system without a staff list cannot compute that rate. It can say which of the actually active users can show a training record and which cannot, and nothing more. The rate across the whole workforce has to be kept somewhere else, in HR administration, where it belongs. The reflex to put everything into the new system because it happens to be there runs into a wall here, and day to day that feels like a division of labor or like a defect, depending on the day.
Smaller hardships remain, too. A failed login attempt names no person: a name that could not log in is a claim in an input field, not an identity. If you want to know whether somebody is systematically trying other people's accounts, you get to see the pattern, but no name out of the statistic. And the residual line from above deliberately counts imprecisely in one spot: a person who changes departments within the evaluation window is weighed twice in it. The error is known, and it sits, frankly, on the permissive side: in doubt, a residual line keeps standing that strict counting would have merged. The alternative would be to pull exactly that person-to-department mapping into the evaluation which is never supposed to arrive there. So the error stands named in the documentation instead of living silently in the code.
And then there is the moment in the sales conversation, our own. Can we see who submits the most requests? No. Can you unlock that? No. It is not switched off, it is not built, and building it would not be a configuration checkbox but a different architecture. There are prospects for whom the conversation gets shorter at this point. That is the price, and it would be dishonest to talk it small.
Against that stands a gain that is hard to demonstrate, because it consists of nothing happening. The report nobody can build shows up in no conflict with the works council, and the behavioral profile that never comes into being appears in neither an access request nor a data breach. Security gains of this kind have no demo. You recognize them by certain incidents not occurring, and non-occurrence is not a slide in a pitch.
Whose decision it is
The line between 'do we want to know' and 'do we want to be unable to know' is drawn by management and the works council together, and it belongs in the works agreement covering the AI rollout, before the system runs. No vendor can take that decision off your hands, us included. What a vendor can do is make sure the decision, once taken, ends up in the architecture rather than in a policy nobody enforces. The supervisory authorities' guidance says essentially the same: fix the purposes before data flows [3].
For the conversation with a vendor, us included, two test questions suffice. First: which component of your system can enumerate our staff list, and through what? Second: which evaluations does the system refuse by construction, and where is that documented? The first question has a correct short answer. If all you hear on the second is what is possible, you have your answer too.
Warum die Daten überhaupt ins eigene Haus gehören, steht im Artikel Warum On-Premise-KI fast immer sicherer ist. Wie eine Appliance aussieht, bei der solche Entscheidungen in der Architektur liegen, unter Produkt.