Why Scanning Architecture Matters: Comparing Skill Registry Security

The architecture of when and where security checks run determines what attacks they catch — not whether checks exist at all. Multiple registries in the AI skill ecosystem run security checks today. The question worth asking is what guarantees each approach actually provides and what it leaves open.

This post is a technical comparison of three models. It is an architecture discussion, not a competitive review.

The Three Models

Model 1: Reactive Community Moderation

How it works: Users flag suspicious skills. Above a report threshold, the skill is auto-hidden or sent for manual review. Some registries run periodic scans after flagging.

Examples in practice: ClawHub used this model before ClawHavoc. Three community reports triggered auto-hide.

What it catches:

• Skills that have already been flagged by at least one user
• Behaviors so obviously malicious that users recognize them quickly

What it misses:

• Everything before the first user notices and reports
• Sophisticated attacks designed to look entirely legitimate
• Novel techniques with no existing reports
• The window between upload and first install — which can be minutes for targeted attacks

The structural problem: The system is entirely reactive. A malicious skill has already been distributed and potentially installed before any protection activates. The attacker’s winning condition is simply: move faster than the community notices. In ClawHavoc, 354 packages were uploaded in under 6 hours. The community reporting system was not designed for that rate.

Model 2: Install-Time-Only Scanning

How it works: When a user installs a skill, the registry or install toolchain scans the downloaded files against detection rules or known-malware databases. If the scan flags something, the install is blocked or warned.

Examples in practice: Skills.sh integrated Snyk, Socket, and generative AI analysis for install-time scanning as of February 2026.

What it catches:

• Known malware signatures present in the downloaded files
• Declared dependency vulnerabilities
• Behavioral patterns matching known malicious code

What it misses:

The tamper window. Install-time scanning checks the artifact as received. It cannot detect whether that artifact matches what the publisher originally submitted. If the registry is compromised, the CDN is tampered with, or the distribution pipeline is attacked between publish and install, the scanner will analyze the tampered version — and may pass it as clean, because the modified artifact hasn’t been seen before.

Novel variants. Snyk and Socket maintain databases of known-bad packages. AMOS-based malware (the payload in ClawHavoc) is actively developed to evade signature detection. The specific AMOS variant used in ClawHavoc was not in VirusTotal’s database at the time of initial distribution. Novel variants pass signature checks for days or weeks before detection databases catch up.

Deferred payload delivery. Some attacks don’t embed malware in the skill archive at all. The skill contains instructions that cause the AI agent to download and execute an external payload during normal use. Install-time scanning sees a clean archive. The malicious payload is fetched later, entirely outside the scan window.

The fundamental problem: Install-time scanning verifies one artifact at one moment against one set of signatures. It cannot establish that the artifact is identical to what the publisher submitted, and it cannot catch attacks that defer payload delivery past the scan.

Model 3: Dual-Side Verification with Cryptographic Tamper Detection

How it works: Security checks run on both sides of the distribution pipeline.

1. Publisher side: Before sharing, the publisher runs a local security scan. The scan report and a SHA-256 cryptographic tree hash of the archive are stored on the registry. Skills with critical findings cannot be shared — the gate is enforced before the skill is published to the index.

2. Registry: The tree hash is stored as an immutable record at publish time.

3. Consumer side: When installing, the consumer independently re-scans the downloaded files. The server compares the consumer’s scan report against the publisher’s original report, and the consumer’s computed hash against the stored hash.

Only if both reports agree (or diverge within acceptable bounds) and the hashes match does the install receive a verified verdict. A hash mismatch produces an automatic critical verdict and blocks installation — no human review required, no database update needed.

What it catches:

Registry-level tampering. If the stored archive is modified after publish — by a compromised admin, a breach, or an infrastructure attack — the hash comparison fails immediately. The consumer’s computed hash will not match the stored hash. Installation is blocked before any human reviewer notices anything is wrong.

CDN and transit tampering. If the distribution network delivers a modified file, the hash comparison catches it regardless of what the consumer’s scan result is. The cryptographic check is independent of detection databases.

Publisher-side attack staging. Because the publisher-side scan runs before the skill can be shared, skills containing instructions to download external payloads, access credential files, or execute obfuscated code are blocked at the source. The attack cannot reach the registry index at all.

What it doesn’t guarantee:

• A publisher who crafts a skill specifically to evade the scanner’s detection rules — scanner evasion is possible in any static analysis system
• Composition-level attacks that use entirely normal-looking operations in combination to achieve malicious outcomes

Attack Vectors: Side-by-Side

Attack Vector	Reactive	Install-time	Dual-side
Known malware	❌ (depends on reports)	✅ (signature match)	✅ (publisher-side blocked)
Novel malware variant	❌	❌ (not in DB yet)	✅ (behavioral rules catch staging)
Registry infrastructure breach	❌	❌ (scans tampered file)	✅ (hash mismatch)
CDN/transit tampering	❌	❌ (scans delivered file)	✅ (hash mismatch)
Fake prerequisite / external payload fetch	❌	⚠️ (depends on analysis depth)	✅ (publisher scan catches staging)
Credential file access (~/.ssh, .env)	❌	⚠️ (may catch)	✅ (publisher scan + behavioral rules)
Prompt injection in .md instructions	❌	❌ (not a code scanner target)	✅ (dedicated prompt injection rules)
Agent persona/memory persistence (SOUL.md)	❌	❌	✅ (publisher scan)
Social engineering (ClickFix-style UX)	❌	❌	⚠️ (partially detectable)
Typosquatting	❌	❌	✅ (verified namespace enforcement)

The Cryptographic Guarantee

The core distinction between install-time scanning and dual-side verification comes down to what each approach proves about the artifact.

Install-time scanning proves: “This artifact, as I received it right now, does not match any known-bad signatures in the current database.”

Dual-side verification with tree hashing proves: “This artifact is byte-for-byte identical to the artifact the publisher submitted. The publisher’s pre-share scan passed. The consumer’s post-download scan independently agrees. Any deviation — in the archive, in transit, in the registry — would have broken the chain and blocked installation.”

These are categorically different security properties. The first is a probabilistic check against an external database that may or may not include the specific threat. The second is a cryptographic binding between the submitted and delivered artifact, with independent verification at both ends that does not depend on any external database being current.

This distinction is most consequential in three scenarios that have all occurred in adjacent ecosystems:

1. Registry compromise — npm and PyPI have both experienced infrastructure breaches. When a registry’s storage layer is compromised, install-time scanning of the compromised artifact provides no protection. Hash comparison catches it immediately.

2. CDN tampering — TLS interception, BGP hijacking, and compromised edge nodes are realistic threats for high-value software distribution. An attacker who can modify bytes in transit defeats install-time scanning; they cannot defeat a hash comparison the consumer computes locally.

3. Sophisticated novel malware — As AI agents become high-value targets, attackers will invest in evasion. A purpose-built skill designed to avoid known detection signatures is exactly the scenario where behavioral publisher-side scanning — which targets what the skill does, not what it looks like — provides durable protection.

The practical formulation: install-time scanning tells you the artifact looks clean today. Dual-side verification tells you the artifact is what the publisher submitted and both parties agree on its contents. For a security-sensitive distribution channel, the second property is the one that matters.

Why Install-Time Scanning Is Still Worth Having

This is not an argument that install-time scanning is worthless. It provides genuine value:

• Defense in depth: Multiple independent checks catch more than any single check alone
• Known-malware database coverage: For previously documented malware families, signature matching is fast and reliable with well-maintained databases
• Dependency vulnerability detection: Tools like Snyk excel at identifying vulnerable declared dependencies — a different threat class from malicious skill behavior

The argument is that install-time scanning alone leaves structural gaps that grow more dangerous as the ecosystem matures and AI agents become more autonomous and more credentialed.

What a Fourth Model Could Look Like

The three models above operate at the artifact level: they analyze what a skill contains before and after distribution. A fourth model — currently at the frontier of registry security research — would operate at the runtime level: enforcing what a skill is actually permitted to do during execution.

This approach would look something like a declared permission manifest:

# skill.permissions.yaml (hypothetical)
network:
  outbound: false
filesystem:
  read: ["./"]        # project directory only
  write: ["./output/"]
shell:
  allowed: false
agent:
  persona_modification: false
  memory_write: false

At install time, the skill’s declared permissions are cryptographically bound to the archive. At runtime, the AI agent enforces the manifest — refusing to execute any operation the skill did not declare. An attempt to read ~/.ssh/ by a skill that declared filesystem.read: ["./"] would be blocked at the agent level, not the scan level.

This model shifts the guarantee from “we scanned this and it looked clean” to “this skill is structurally incapable of performing undeclared operations.” It’s analogous to the evolution from antivirus scanning to OS-level sandboxing in mobile app ecosystems — a transition that took years but materially changed the threat model.

Several practical challenges remain: defining a permission grammar expressive enough to cover legitimate skill behaviors without being so broad it provides no constraint; getting AI agent runtimes to enforce the manifest consistently; handling skills that legitimately need broad access (e.g., a deployment helper that needs SSH and network access). These are solvable engineering problems, not fundamental blockers.

Runtime enforcement via declared permissions would not replace dual-side verification — it would complement it. Publisher-side scanning and cryptographic tamper detection remain necessary because they operate before the skill ever runs. Runtime enforcement adds a third layer: even if a malicious skill somehow reaches the consumer’s machine, it cannot exceed its declared permissions.

This is Phase 3 thinking. The industry isn’t there yet. But the trajectory from reactive moderation → dual-side verification → runtime permission enforcement is the logical progression as AI agents become more capable and the consequences of a compromised skill grow more severe.

The Practical Implication for Developers

When you install a skill from a registry that uses install-time-only scanning, you are trusting:

• That the registry has not been compromised since the skill was published
• That the delivery infrastructure has not been tampered with
• That the current detection database covers the specific attack technique in the skill
• That the attacker did not defer payload delivery past the scan window

When you install a skill from a registry using dual-side verification, you additionally verify:

• That the archive you received is byte-for-byte identical to what the publisher submitted
• That the publisher’s pre-share scan passed before the skill was made available
• That any tampering in transit would be caught by hash mismatch before installation completes

The extra verification takes seconds. The guarantee it provides is structural rather than probabilistic — and structural guarantees hold regardless of whether the attacker anticipated your defenses.

Conclusion

Security architecture is not a product feature. It is a set of commitments about what an adversary would have to do to defeat your defenses — and whether those commitments are enforced by code or depend on human reaction speed.

Reactive moderation requires attackers to be slow. Install-time scanning requires attackers to be unsophisticated or their malware to be old. Dual-side verification with cryptographic tamper detection requires attackers to simultaneously compromise the publisher, evade static analysis, and defeat a hash comparison that the consumer computes locally against an immutably stored reference. The latter is a meaningfully harder problem.

The ClawHavoc campaign demonstrated what happens when a high-velocity attack meets a system that depends on human reaction speed. As AI agents accumulate more credentials, more file access, and more autonomous capability, the cost of getting registry security architecture wrong increases proportionally.

The minimum viable security model for a skill registry today includes publisher-side scanning, cryptographic tamper detection, and independent consumer verification. Install-time scanning is a useful addition to that baseline, not a substitute for it. Runtime permission enforcement is the next evolution — and the ecosystem should be building toward it now, before the attack surface matures faster than the defenses.

SkillSafe’s scanner ruleset is publicly documented at skillsafe.ai/security. We publish this comparison to make the architectural tradeoffs concrete — not to position our implementation as the final word on registry security.