Every serious evaluation of on-premise AI lands on the same handful of questions. They're good questions, the kind an IT lead should ask before putting a new box on the network. Here are the ones we hear most, answered the way we answer them in the room.
“Do we need a GPU cluster and a DevOps team?”
No. This is the assumption that stops most teams from even evaluating on-prem, and it's a few years out of date. A modern open-weights model, quantized and served well, delivers a genuinely useful assistant on a single appliance-class machine. Kaldryn installs on a fresh Ubuntu server in under ten minutes, no Kubernetes, no platform team. If you can rack a NAS, you can run your own AI.
“Are open models actually good enough?”
For the everyday work an internal assistant does, drafting, summarising, answering questions over your own documents, yes, and the gap to frontier cloud models has narrowed every quarter. The trick is matching the model to the job: a fast small model for chat and retrieval, a bigger one for heavy drafting. Kaldryn ships 200+ curated models with signed weights, so choosing isn't a research project.
The question that matters isn't “is this the best model in the world?”, it's “is this model, running over our documents, more useful than a frontier model that isn't allowed to see them?” For regulated teams, the second one wins.
“Who maintains it?”
The same person who maintains the rest of your infrastructure, which is to say, not much of anyone, most weeks. The platform monitors itself, the admin console shows what your users are doing (and what they're trying to do, the DLP log is illuminating), and updates are explicit: you decide when, they're signed, and you can defer them indefinitely on air-gapped deployments.
“What happens when the internet goes down?”
Nothing. That's the whole design. Retrieval, chat, the API, everything runs on-box. An internet outage that takes the office offline takes your cloud AI subscriptions with it; the appliance in the corner doesn't notice.
“What does it cost, really?”
Owned hardware is a fixed cost; per-seat cloud AI is a linear one. The crossover point depends on your headcount and usage, but the shape of the curve is the part to internalise: an appliance costs the same whether ten people use it or two hundred. Past a modest team size, “unlimited users, just electricity” stops sounding like marketing and starts being the spreadsheet.
“How do we prove to compliance that nothing leaves?”
This is where on-prem flips from being the cautious choice to the easy one. The answer isn't a vendor questionnaire and a sub-processor list, it's a firewall rule and a packet capture. Kaldryn seals its network perimeter on boot and ships the evidence tooling (immutable audit log, SIEM export, isolation checks) to demonstrate it. Your DPO gets an architecture where the sensitive question, where does the data go?, has a one-word answer: nowhere.
Evaluating a deployment for your own team? The reference architecture covers the technical detail, and our EU regulation briefings cover the compliance side.