Most security programs have spent the last several years getting comfortable with the software supply chain. Package registries are scanned. Container images are signed. Build provenance is recorded. The same teams have not extended any of that to the model weights flowing into their AI systems, and the gap is widening at a pace that should be uncomfortable. A model is a binary artifact pulled from a registry, loaded into memory, and given access to the same data and the same tools as any other production component. The fact that no one writes the file by hand does not make it safer than code. In practice, it is often less safe, because the validation steps that catch a poisoned npm package were never wired up to catch a poisoned checkpoint.
The typical pattern is consistent. A data science team picks a model from a public registry. The team downloads the weights to a workstation, evaluates the model against an internal benchmark, and ships it to a serving environment. Somewhere in that path a checksum is checked or not. Somewhere a license is reviewed or not. The model loader deserializes whatever it is given, and the security team learns about the artifact for the first time during an incident review six months later. The path from registry to production rarely passes through any control that would have caught a malicious upload, and that path is the actual attack surface.
What Has Already Been Demonstrated
The community catalog of supply chain incidents involving model weights is no longer short. Pickle based weight formats have been used to execute arbitrary code at load time, because pickle deserialization runs Python by design and the people loading the file rarely treat that as a code execution boundary. Registry typosquatting has placed look alike repositories alongside the legitimate ones, and the misclick is harder to spot when the model card is convincingly copied. Maintainer account compromises on public registries have allowed attackers to publish backdoored variants of widely used models under names the community already trusted. Hidden behavior triggered by specific inputs has been demonstrated in research and observed in the wild, and the detection problem for that class of backdoor is not solved by reading the model card.
None of these are theoretical. They have all happened to real artifacts that real organizations pulled into production. The defense is not to assume the registries are policing themselves, because the registries have been candid that they cannot. The defense is to build the controls that should have been there from the start, in the same way the software supply chain conversation reached that conclusion several incident cycles ago.
The Boundaries a Working Program Has to Draw
The first boundary is the registry itself. The public model hubs are convenient, and they are the right answer for evaluation work, but they are not the right answer for production loading. An organization that lets engineers pull weights straight from a public host into a serving environment has accepted whatever changes that host allows the upstream maintainer to make. The teams we have seen handle this well operate an internal mirror that fronts the public registries. Models are pulled into the mirror once, scanned, signed, and tagged with the version of the artifact at the moment it entered. Production loaders only accept artifacts from the mirror, and the mirror keeps a record of what came from where.
The second boundary is the file format. Pickle and the formats built on top of it should not be loaded inside a trust boundary that matters. Safetensors and other restricted formats exist for exactly this reason, and the migration cost is low enough that there is no defensible argument for accepting pickle files into production at this point. The conversion has to happen before the artifact crosses into a serving environment, and the conversion has to fail closed when it cannot be performed. A model that only ships in pickle form is a signal about how the maintainer thinks about safety, and that signal should weigh on the decision to use the model at all.
The third boundary is integrity. Every artifact entering the mirror should be checksummed against a value the team controls, not against a value the registry publishes alongside the file. A hash that lives next to the artifact is not a defense against an attacker who can write to the artifact. The hash that matters is the one your team recorded when the artifact was accepted, stored somewhere the registry cannot reach, and verified at load time. This is the same logic that container image signing arrived at after a similar set of incidents in the container ecosystem. The conclusion was not novel and it does not need to be rediscovered for model weights.
The fourth boundary is provenance. Where did the weights come from. Who fine tuned them. What dataset did the tuning use. What was the license at the time of acceptance. None of those questions are answered by the file itself. They have to be captured at intake, attached to the artifact in the mirror, and carried through to the production manifest. A model that loses its provenance between download and deployment is a model that cannot be evaluated when a question is asked later, and the questions are increasingly being asked by customers and regulators rather than by internal teams.
The fifth boundary is behavior. Static scanning catches some classes of risk and misses the most interesting ones. Backdoored weights pass static review because the file itself is structurally fine. The check that catches them is behavioral, run against a suite of evaluation inputs the team controls, and the failures are flagged when outputs deviate from the baseline established for the same model at acceptance. This is the part of the program most teams do not have, and it is the part that does the most work when the upstream artifact has been quietly modified.
Where Programs Tend to Stall
The most common stall is the assumption that the model registry is somebody else's problem. The data science team treats it as an artifact source. The security team treats it as outside its remit. The platform team treats it as a network destination to allow. No one owns the question of which registries are trusted, which artifacts are allowed to enter, and what evidence has to follow them in. The fix is not to assign every team partial responsibility. The fix is to put model intake under a single owner, the same way package registries eventually ended up under a single owner in mature programs.
The second stall is the speed problem. Engineering teams move fast on model evaluation, and a heavy intake process pushes them back toward unsanctioned channels. A program that requires a week of paperwork to admit a model from Hugging Face will produce shadow pulls into developer laptops within a quarter. The cadence has to match what the engineering work actually requires, and the intake controls have to be automated enough that the friction is in the policy decisions rather than the mechanical steps. Mirroring, scanning, and signing should happen in minutes. The human review should be reserved for the cases where the automation flags something worth a person's time.
The third stall is the rollback story. A model that turned out to be compromised has to be removed from production with confidence, and the confidence has to extend to anywhere the artifact reached during its time in service. The teams that have handled this well keep a model deployment ledger that lists every environment a given artifact ran in, every dataset it processed, and every downstream system it produced output for. The ledger is the input to an incident response process that would otherwise have no anchor. The teams that have not built one tend to discover at incident time that they cannot answer the simplest version of the question, which is where the model has been.
"The model weights moving into production are arriving from places the security team has never reviewed, in formats that execute code at load time, with hashes the registry vouches for and the receiver does not verify. That is not a supply chain. It is a delivery mechanism waiting for someone to use it."
What Belongs on the Roadmap
The starting point for most organizations is a model registry policy, narrow enough to be enforceable. The policy names the registries the organization will pull from, the formats that are accepted into production, and the path an artifact has to follow to be considered eligible for loading. A policy that lists ten registries and accepts every file format is not a policy. A policy that lists two registries, accepts safetensors only, and requires intake through the internal mirror gives the platform team something to enforce in code.
The next layer is the intake pipeline. The mirror has to be in place. The scanning has to run on every artifact. The signing has to happen at acceptance, and the verification has to happen at load. None of those steps is technically difficult. The work is in wiring them into the path engineering already uses, so that the production loader is the only entry point and the entry point only accepts artifacts the program has cleared.
The third layer is the behavioral evaluation. Every model that enters the mirror should be run against a fixed suite of inputs and the outputs recorded as a baseline. Every refresh of the model from upstream should be compared against that baseline before it is promoted. Deviations should fail the promotion until a human decides whether the change is acceptable. This is the control that catches backdoors that survive every other check, and it is the control that the registries cannot run for you because they do not know what your baseline is.
The fourth layer is the ledger and the rollback process. Every artifact in production should be traceable to a registry source, an intake record, and the environments it has been loaded into. The ledger does not need to be elaborate. It needs to be complete, and it needs to be queryable at incident time without rebuilding it from logs. The organizations that have a working ledger are the ones that survive a supply chain incident with their AI services intact. The ones that do not have to pause every AI workload until they can answer where the bad weights went, and that pause is the part of the incident that gets reported to customers.
The Posture That Holds Up
The posture that holds up under review is one where model weights are treated the same way package binaries are treated in a mature software supply chain program. The artifacts come in through a controlled path. The path produces a signed, scanned, version pinned object that the production runtime is the only thing allowed to load. The provenance follows the artifact through its life in the environment. The behavioral baseline catches the changes the static checks cannot. The ledger answers the questions that incident response will ask before the questions are asked. The posture is not exotic. It is the posture the rest of the supply chain conversation arrived at after a long line of incidents, and the AI version of the same conversation is now far enough along that organizations can adopt the conclusion without waiting for their own incident to force the work.
The organizations that are doing this well treat the model as a build output rather than a download. The mirror is the artifact store. The intake is the build step. The signature is the proof. The production loader is the only consumer. Once that shape is in place, the question of whether a given weight file is safe to load stops being a judgment call and starts being a control answerable by the system itself. That is the standard the rest of the supply chain reached after years of breaches, and it is the standard model weights have to reach before the next round of registry incidents finds the organizations that have not built it yet.