KNOWLEDGE · DATA SOVEREIGNTY

Why on-premise AI is almost always safer

Free models, exposed credentials, and anonymization that does not hold up: why local AI is the only consistently coherent answer to the security question.

On the large platforms for AI model access, a model whose price is zero euros regularly sits near the top. Hundreds of billions of parameters, a generous context window, ‘free’ right there in the name. The honest question is not whether someone pays for it — but who.

“If you are not paying for the product, you are not the customer — you are the product being sold.” Andrew Lewis, 2010 [1]

Whoever hands you inference on a frontier model for free is not concerned with whether your data is monetized, but how. Every prompt to a free service must be treated as potentially readable — for training, analytics, or plain resale. And this goes well beyond free models. The real question is: who do you trust with your data while it is being processed?

When credentials end up on the internet

Credentials ending up on the internet is not a theoretical worry, but measurable. Public exposure dashboards continuously document hundreds of thousands of publicly reachable AI instances on the internet — many with leaked credentials, hosted across a who's-who of global cloud providers. One firewall rule, one forgotten port, one default password, and the entire AI setup including prompts, documents, and keys is out in the open. At that scale it is not an edge case, but a pattern.

With a properly isolated on-premise system, this simply cannot happen. There is no public endpoint to scan. No IP on a watchboard. No credentials sitting out on the internet.

Anyone who thinks this only hits hobby setups misses the reality: even the internal AI tools of large, professional firms were rendered attackable in 2026 through unauthenticated endpoints and classic injection flaws — the fault was never ‘the AI’ as such, but ordinary application security around an AI-adjacent system [2].

Europe: regulation as a tailwind

Europe's legal landscape confirms what the technology already suggests: data belongs where you can control it. The NIS2 Directive creates an EU-wide cybersecurity framework for critical sectors — with systematic risk management, supply-chain control, and reporting duties. Crucially, it explicitly emphasizes management accountability; executives can be held personally liable [3].

The EU AI Act entered into force on 1 August 2024 and becomes broadly applicable from 2 August 2026; governance duties for general-purpose AI models have applied since August 2025 [4]. Anyone operating AI carries transparency and documentation duties — far easier to meet when you know exactly where your data sits and who can access it. Germany's BSI, too, explicitly warns about data exfiltration through model outputs and tool connectors [5].

And the catch with cloud: even where a provider promises ‘German data residency’, sub-processors, telemetry pipelines, CDN services, and support access from third countries can still create cross-border transfers. A US parent is moreover subject to the US CLOUD Act — regardless of whether the servers sit in Frankfurt [6]. On-premise removes that complexity: your data stays physically and legally within your domain.

Why anonymization is not enough

Proxy services and browser extensions promise to ‘anonymize’ prompts before they are sent. The problem is fundamental: you cannot anonymize semantics. In its Opinion 28/2024, the European Data Protection Board made clear that AI models trained on personal data cannot automatically be treated as anonymous in every case [7].

Picture these sentences in an AI chat:

No name is mentioned — and yet anyone who knows the context connects the dots. Pseudonymization protects against easy attribution, not against semantic reconstruction. OWASP ranks ‘Sensitive Information Disclosure’ as the second most important LLM risk (LLM02:2025) [8]. At a large electronics group in 2023, confidential source code and meeting notes were fed into a cloud AI within days — with no way to take them back [9]. If the prompt never leaves the building, it never needs anonymizing in the first place.

Zero trust: the trusted computing base

The decisive question in AI security is: who do you trust with the execution environment? In security architecture, the total set of components that must be trusted is the Trusted Computing Base (TCB). The smaller the TCB, the safer the system.

Dimension Cloud On-premise
Trusted Computing Base Large: your admins + provider layers + hypervisor + multi-tenancy Small: your admins + your hypervisor + dedicated hardware
Multi-tenancy Structural; isolation depends on hypervisor and control plane None; dedicated hardware removes the risk
Privileged access Tenant admins + provider operations = larger insider surface Only your admins, with JIT access and MFA
Incident forensics Dependent on provider logs and contract boundaries Full chain of custody under your control

Neither cloud nor on-premise can fully protect ‘data in use’ cryptographically today — during execution, data sits in plaintext in memory. The strategic question is therefore not ‘which encryption?’ but ‘who has access to the execution environment?’ In the cloud the answer is: you, your admins, the provider, its admins, its sub-processors, and every jurisdiction that can compel access. On-premise: you and your admins. Full stop.

In an air-gap, there is no cloud

In an isolated environment there are no cloud providers. No sub-processors. No cross-border transfers. No leaked credentials on watchboards. No free models paid for with your prompts. No jurisdiction questions. No ‘shared responsibility’ model where, in the worst moment, no one is actually responsible.

There is only your hardware, your network, your models, your data. Zero trust in its purest form: no data leakage, no trust assumptions, no third parties. That is exactly what we build — an appliance that sits in your data center and belongs to you. How that looks technically, read under Product.

References

  1. Andrew Lewis, MetaFilter (2010); core idea tracing back to Serra/Schoolman, ‘Television Delivers People’ (1973).
  2. OWASP — Top 10 for LLM Applications 2025 (Anwendungssicherheit rund um KI-Systeme).
  3. NIS2-Richtlinie (EU) 2022/2555 — EUR-Lex.
  4. EU AI Act — Zeitplan & Anwendbarkeit, Europäische Kommission.
  5. BSI — Generative KI-Modelle: Chancen und Risiken.
  6. US CLOUD Act (H.R. 4943, 2018).
  7. EDPB — Opinion 28/2024 (KI-Modelle und Anonymität).
  8. OWASP — LLM02:2025 Sensitive Information Disclosure.
  9. Samsung-ChatGPT-Datenleck (2023), The Register.
  10. DSGVO — Volltext, EUR-Lex.
  11. Lenovo Press LP2368 — TCO On-Premise vs. Cloud LLM.
  12. NVIDIA Developer Blog — MLPerf Inference Records.

Sovereignty is an architecture decision.

Let us spend thirty minutes on what the appliance looks like in your data center.

Book intro call →
Book an intro call →