Agent Upstart

Cool things people are building with agents — and practical help getting the most out of them.

Playbooks · Coding tasks

Vet a skill before you install it

Written October 29, 2026

For a solo developer × vetting one skill before it goes into Claude Code × Claude Code, ending with a written decision — install, install with limits, or do not install — made before the skill runs anything.

Before you start

A skill is a folder of instructions: a SKILL.md file that tells the agent what to do, plus any scripts, hooks, and reference files it ships with. Installing one hands the agent new instructions and, depending on the skill, new tools and commands it may run. The description in the skill's frontmatter states what it is for; it does not prove what it does. This page vets one skill in one pass: a scanner run, a read of the skill as an attacker would read it, a provenance check, and a written decision.

Scanners pattern-match text. They do not run the skill, and they cannot tell a documented attack example from a live instruction. On one security-teaching skill — vbsec, the vbs-scan-security skill, 76 files — all three scanners below returned their maximum severity. vetskill 0.1.0 returned BLOCK at 100/100, with 16 high, 10 medium, and 7 low findings. skillvet 0.2.0 returned RED at 100/100. SkillSpector 2.12.0, in static-only mode, scored 100, CRITICAL, with a DO NOT INSTALL recommendation and 141 issues, at 81.6% coverage in a scan run on October 29, 2026. The findings were pattern matches on documented examples inside the skill's own rule files — exploit code, example hosts such as 169.254.169.254 and evil.com, and calls such as eval( shown as attacks to detect — not instructions the skill runs. A scanner verdict alone would block a teaching skill. The human read below is what separates examples from executable behaviour, and it is not optional.

What "done" looks like

At the end of one pass you have a short decision note for the skill: its name, the commit you looked at, the scanner verdict, what the attacker-read found, and one of three decisions — install, install with limits, or do not install. The skill is not installed until the note exists.

Steps

  1. Copy the skill to a throwaway folder; run nothing yet. Clone the repo or copy the skill folder somewhere outside your project, and note the exact commit. Do not run its install script, and do not open it in the agent yet. Every later step works on this copy, so a bad skill never touches a live project.
  2. Run one scanner over the copy. Any one of these three runs the same kind of static pass; pick one and run it from outside the skill folder:
    • vetskill scan <skill folder> — install with pip install git+https://github.com/bayuh85/vetskill. As of November 14, 2026, that repository address returns a 404, so the install does not currently work; run skillvet or SkillSpector below instead. The reveal command reports hidden characters; the rules command lists the 21 rules it checks.
    • npx -y skillvet <skill folder> — no install step; it prints a points table and a verdict.
    • skillspector scan <skill folder> --no-llm — static-only mode, no model key needed. Install it first with uv tool install git+https://github.com/NVIDIA/skillspector.git (SkillSpector); without that step the scan command is not found. It checks 71 vulnerability patterns in 17 categories and scores 0–100.
    Record the verdict and the finding count. Expect teaching material to score badly: on the vbsec skill above, all three scanners gave their maximum severity for documented examples. A maximum verdict is a reason to read, not a verdict by itself.
  3. Read SKILL.md as an attacker. Read the whole file, not just the description. Check four things. First, the frontmatter: a skill with no allowed-tools field and no hooks field declares no tool scope, so the body text is the only statement of what it may run — on the vbsec skill, the frontmatter carried no allowed-tools or hooks at all. Second, hidden text: run the scanner's reveal or hidden-character check, and look for instructions inside HTML comments. Third, every host, command, and file the skill names: a host the skill actually calls is different from a host named in an example, and a command the skill runs is different from a command it quotes to detect. Fourth, the install and helper scripts: read them without executing them, and note anything that downloads code, reads credentials, or phones home.
  4. Check provenance before trusting the read. Confirm the repo is the one the skill claims to come from, that it carries a licence file, and that the commit you noted matches what you scanned. A skill copied into a chat message or a gist has no provenance to check: treat it as untrusted text, not as the author's release. Note the install path the README gives, and whether it runs a script you have now read.
  5. Write the decision note, then act on it. One entry: skill name, commit, scanner name and verdict, the attacker-read findings in one line each, and the decision. Install means the scanner findings are explained by the skill's own purpose and the read found nothing executable beyond it. Install with limits means the skill runs only on a copy of the project, or with the network-dependent and destructive options left off — on the vbsec skill, the live OSV.dev queries and the opt-in patch-applying mode are the options that change what the skill does. Do not install means any finding survives the read: a host the skill actually calls that it does not document, a script that downloads and runs code, or instructions hidden where a reader would not look.
  6. Re-scan after any update or delisting. When the skill changes, or when a scanner or registry removes it from a listing, repeat the pass on the new commit before reinstalling: run the scanner again, read the diff, and write a new decision note. A skill cleared once is cleared for that commit only.

Check the agent's work

What can go wrong

Limits

Related playbooks

Sources

Sources: vbsec — skill scanned, vetskill, SkillSpector.

Back to: Coding playbooks hub · Also: Open an untrusted repo without running its code