Agent Upstart

Cool things people are building with agents — and practical help getting the most out of them.

Playbooks · Coding tasks

Make sure your Claude Code skill actually loads when it should: test it in each agent before you rely on it

Written October 30, 2026

For skill authors shipping the same skill to more than one agent: test whether it actually loads when it should, in each agent separately, with one prompt it should answer and one it should leave alone. A skill is a reusable pack of instructions that teaches the agent how to do one job. Without a test in each agent, a skill can sit unused in one agent while a pass in another is mistaken for a pass in all of them.

Before you start

When a skill loads — when it activates — the agent reads the skill's instructions and follows them for the task at hand. A skill that loads in one agent may sit unused in another. Each agent decides for itself whether a skill is relevant, usually from the skill's name and description, and only then loads the full instructions. A skill that was never loaded has not failed at its job; it was never given the job. The test below keeps those two outcomes apart, and it keeps a pass in one agent from standing in for a pass in another.

The discipline is borrowed from an ordinary audit: a rack audit traces each cable against the plan instead of assuming the rack matches the drawing. This test traces each agent separately against the same plan, prompt by prompt, and records what was actually seen.

You need the skill installed in each agent you test, one test prompt the skill should answer, and one negative-control prompt it should leave alone. The test prompt must not name the skill: naming it tests manual invocation, not automatic activation. If you cannot install the skill in an agent yet, that agent is untested, not failed. Write it down as untested and move on.

What "done" looks like

At the end of one pass you have a table with one row per agent per prompt: the agent, the prompt, whether the skill was loaded, and the evidence you saw. No cell says the skill activates in an agent unless that agent's row shows it happening.

Steps

  1. Write the test prompt in plain words. Phrase it the way a user would, without the skill's name and without its trigger words copied from its description. If the prompt only works when it repeats the description, it tests the description, not the skill. Keep the prompt identical across agents: same words, same attached files, same starting state.
  2. Write the negative control. Pick a prompt on a nearby topic that the skill should not touch. A skill that fires on everything is as broken as one that never fires: it will pollute unrelated work. The negative control is what catches that. One published single-agent method runs twenty requests, half that should load the skill and half near misses that should not, before and after a description rewrite; you do not need twenty, but you need at least one near miss per agent.
  3. Start a fresh session in the first agent. Skills load when a session starts, so a skill added or edited mid-session may be invisible until the session restarts. Open a fresh session, send the test prompt, and watch for the agent's own signal that the skill loaded: a skill indicator, a tool call naming the skill, or the agent naming the skill in its reply. Save the reply and the signal, not just your impression of it.
  4. Send the negative control in a fresh session. Same agent, new session, unrelated prompt. If the skill loads here, record a false positive, not a pass. A skill that fires on the control needs its description narrowed before it ships, whatever it did on the test prompt.
  5. Repeat both prompts, unchanged, in each other agent. Claude Code, Codex, and Cursor each get the same test prompt and the same negative control, each in a fresh session. Do not rephrase the prompt for an agent that missed, and do not skip the control for an agent that passed. Rephrasing after a miss tests a different prompt, and the table no longer compares like with like.
  6. Record three kinds of activation separately. Discovery means the agent shows it found the skill, for example by naming it or showing a skill indicator, even if it then does something else. Explicit activation means the agent invokes the skill by name or through its skill tool. Implicit activation means the reply follows the skill's distinctive steps or wording without naming it. These are different evidence levels, so they get different rows. An implicit match on a generic task is weak evidence: if any competent agent would answer that way without the skill, do not count it as activation.
  7. Run the manual check before blaming activation. If a skill never fires automatically in an agent, invoke it by name in that agent. If manual invocation works, the problem is discovery: the description does not match how users phrase the task. If manual invocation fails too, the problem is installation or loading, and no description rewrite will fix it. One published troubleshooting guide makes exactly this split its first diagnostic, and it is the cheapest split in this test.
  8. Fix the description, then rerun the whole table. The usual fix is a concrete description written in the user's words, stating what the skill does and when to use it, sometimes with a direct instruction about what not to do instead. After any change, rerun every agent and both prompts from fresh sessions. A fix proven in one agent is unproven in the others until their rows say so.

How to record it

One row per agent per prompt, with these columns: agent, prompt (test or control), fresh session (yes or no), skill signal seen (indicator, tool call, named in reply, distinctive steps, or none), verdict (discovery, explicit, implicit, no activation, false positive, or untested). A verdict of "activates in Claude Code, Codex, and Cursor" needs a passing row in all three. A verdict of "activates in Claude Code" needs only Claude Code's row, and says nothing about the others.

Keep the prompts and the replies with the table. A table without the prompt text cannot be rerun, and a rerun with different words is a different test.

What can go wrong

Limits

Related playbooks

Sources

Sources: Dev Encyclopedia skill trigger guide, Scott Spence sandboxed activation evals, 650-trial description study, Rack audit pattern.

Back to: Coding playbooks hub