What Socair does not detect
This attestation does not certify the absence of unknown backdoors.
What a report cannot see, whatever its rows say. Every report carries the same list under Out of scope, and a test in the product holds the two equal.
- Backdoors, trojans, or poisoning in the model weights. Tier 1 reads the weights' layout, never their behavior.
- Triggered, sleeper, or polymorphic behavior. Tier 1 runs no inference.
- Behavior that appears only after quantization, or only on particular hardware or serving stacks, unless Tier 2 ran on the production node class.
- Behavioral safety: jailbreak susceptibility, harmful capability, and bias.
- Malicious behavior that only emerges at runtime under real traffic.
- Artifact formats we do not parse.
- Pickle code execution through a reviewed class's state, loader bugs, parse differentials, a tampered environment, or code carried as data for a later stage to run.
- Chat-template instructions written as ordinary guidance (no override or concealment phrase, URL, hidden or obfuscated text, or condition on message content other than a standard content-part type such as text or image): the text a rendered template adds, and the text a template adds for a content-part type, is listed in the report, not judged, unless the template matches a reviewed template.
- Legal review of a license, and compliance with its usage policies. The license is identified, and checked only against a policy you configure.
How to read this
- The ceiling is the detection boundary of the current check set. It is not a confidence statement about the artifact.
- Every surface outside the checks that ran is named in the attestation: a row the scan could not examine is NOT_TESTED with its reason, a check this level does not run is listed as not run, and everything else is on this list.
- A clean attestation is a bounded statement about a named hash under the checks that ran, never a guarantee.
- A LEAD is a suspicious signal that needs a person's review, not a finding of malice and not a gap an acceptance can clear.
Two things the list assumes
- Tier 2 runs the model on your production node class. Its foundation is in the release, opt-in, but no real Tier 2 check ships yet, so the lines that mention it stay out of scope; its checks are on the roadmap.
- Socair’s detection benchmark runs defanged reproductions of published attacks and pins every outcome. With every input supplied it detects 67 of 73 and misses three. The other three are withheld as gaps, NOT_TESTED rather than passed: a Keras layer class from a package outside Keras, which only the caller’s code supplies, and two OpenVINO IR models whose .bin holds bytes outside the ranges the graph reads. Of the misses, one is the chat-template line above: a default system prompt that steers answers as ordinary guidance. One is a protocol 0 pickle under a name that is not a pickle’s, which a directory scan finds only by its extension; something must still load it by that name, and the benchmark says why it is not on this list. The third is repository Python that no config names and no loader imports by its path, which the Remote code row lists and does not judge, because Socair cannot tell whether anything runs it. With Socair’s defaults it detects 63 of 73: a remapped tokenizer passes, since the Tokenizer row compares tokens only against a reference table you configure; so does a reviewed chat template with an added trigger branch, since the Chat template row compares a render only with reviewed template texts you supply; so do edited weights, since the Tensor reference row compares tensors only with a reviewed model’s manifest you supply; and a file on a known-bad list is not matched, since that check runs only against a list you supply.
Ceiling v5, a generated copy of docs/detection-ceiling.json in the product repository at v0.1.0-rc.6, the release that made the reports published here.