nullifAI: malicious PyTorch models evaded pickle scanning on Hugging Face

What happened

Researchers identified two Hugging Face-hosted PyTorch models containing malicious pickle payloads linked to web shells at a hard-coded IP address. The malformed files exploited a scanner weakness: Python can deserialize pickle opcodes until an error later in the stream, while the scanner at the time stopped after validation failure. The reported remediation changed Picklescan to continue collecting imports as far as possible in broken pickle files. Reporting described the models as closer to proof-of-concept artifacts than evidence of an active attack; the models were removed and Picklescan was updated.

Why an agent changes the risk

Model supply chains are part of the agent threat surface. An unattended workflow that selects, downloads, or loads a community model can turn a malicious artifact into code execution on a notebook, CI runner, tool worker, or agent host with credentials and network access.

How to test for this

• Test model-intake pipelines only in sandboxes. Reject pickle-based weights by policy where possible and prefer non-executable formats such as safetensors.
• Pin model revisions to verified commits, check publisher identity and hashes, and test downgrade, typosquatting, and malicious-update paths before automation trusts a new artifact.
• Run model conversion, loading, and agent workers with least privilege, network restrictions, short-lived scoped credentials, and alerts for outbound connections or unexpected process execution.