When you download an open-weights model, you are adding a multi-gigabyte binary of opaque floating-point numbers to your most sensitive systems. Treated casually, that is one of the largest unreviewed dependencies in the enterprise. Model provenance is the discipline of knowing where those weights came from, that they have not changed, and whether they are fit to run, and it is becoming a procurement requirement for defence, public-sector and regulated buyers.

Models are a supply-chain dependency

We have spent a decade learning to scrutinise software supply chains, signing artefacts, pinning versions, generating SBOMs, scanning for tampering. Model weights have largely escaped that scrutiny despite carrying comparable risk: a poisoned or swapped checkpoint can subtly degrade outputs, leak training data, or behave differently from the artefact you evaluated. The fix is to apply the same supply-chain hygiene to models that we already apply to code.

Hash on first load, then pin

The foundational control is simple: compute a content hash of the weights the first time a model is loaded, record it, and pin to it. On every subsequent load the hash is checked. If a file changes, through tampering, corruption or an unnoticed substitution, it no longer matches and the system refuses to run it. This is the model equivalent of a checksummed, signed release, and it gives you a hard guarantee that the weights you reviewed are the weights serving your users.

“Provenance is not about distrusting open models. It is about being able to prove, at any moment, that the thing running is exactly the thing you approved.”

A trust score, not a vibe

Beyond integrity, administrators need to decide which models are appropriate to deploy at all. A structured provenance grade makes that decision auditable by scoring each model across dimensions such as:

  • Source and license, is the origin reputable and the license compatible with your use?
  • Stability, is the model mature and widely exercised, or experimental?
  • Integrity, does it pass hash verification and load cleanly?
  • Behaviour, does it pass a canary evaluation on install before anyone relies on it?

Rolling these into a single 0-100 grade, backed by a short canary eval at install time, turns 'which model should we trust?' from a matter of reputation into a documented, repeatable decision a security reviewer can sign off.

Why it matters more on-premise

Paradoxically, running your own models raises the provenance bar rather than lowering it. In a cloud service the provider implicitly owns model integrity; when you self-host, that responsibility moves to you. The upside is that you also gain the control to do it properly, pinning the exact weights, keeping them running regardless of upstream deprecation, and proving the chain of custody to an auditor. Provenance is the discipline that lets sovereignty and trust coexist.

Provenance is one plane of a larger sovereign-AI architecture. For the full picture, see our reference architecture for air-gapped LLM deployment and our primer on sovereign AI.