Agent Upstart

Cool things people are building with agents — and practical help getting the most out of them.

Playbooks · Coding tasks

Catch bad agent behaviour chains with Quarry

Written November 4, 2026

A single agent action can be permitted and harmless, and the same action after two other permitted actions can be the incident. Quarry checks the sequence. It does not check your files, your configs, or your extensions: those checks already have their own pages, and this one is not another of them.

What Quarry checks

Project Quarry is Cisco's open-source detection language and engine for agent behaviour. A rule is written over messages, tool calls, tool results, and event order, and the engine compiles each rule to a symbolic finite-state automaton when it loads. A scan runs that compiled program over a trace: the step-by-step record of what an agent saw, said, and did. Cisco's announcement post gives the shape of the problem in one example: reading a configuration file may be legitimate, and sending a request to an external endpoint may be legitimate, but a rule can produce a finding when the trace shows the sensitive read followed within a defined window by the external send. The risk sits in the relationship between the two actions, so the finding preserves evidence from both.

When a rule matches, Quarry returns the evidence: the steps that caused the match, the field each one matched in, and a short excerpt, with common secret formats masked. A rule can draw on two kinds of evidence. Symbolic checks handle what can be stated exactly, such as an unsafe command, a string match, or the order of two tool calls. Neural signals — machine-learning classifiers and model judges — handle questions that need judgment, such as whether a tool result contains a prompt injection, and only when a model adapter is configured. The repository is in alpha, its DSL syntax may change before version 1.0 in ways that require updates to existing rules, it supports Python 3.11 through 3.13, and it is distributed under the Apache 2.0 licence. It ships a command-line scanner, a Python SDK, and plugins for Claude Code, Codex, and other MCP-compatible agents.

Quarry is a shared policy layer, not a replacement for every detector or enforcement control. It does not replace fine-grained authorization, which decides whether a specific action is permitted. Cisco's own framing is that rules complement the monitoring that other systems provide rather than replacing it, and that a human should review and approve each rule before it moves to enforcement, because a rule written by an agent can inherit gaps in that agent's understanding of the risk it is meant to capture.

Not a file or config check

Quarry must not be confused with the checks that audit a machine's state before an agent runs. Audit a repository's git config before any agent opens it reads one config file. Check where your coding agents store API keys on your own machine looks for key files in known places. Check your Codex and Claude Code config for relay tampering reads CLI configuration for tampering. Check a Cursor machine for the tampered Claude Code extension looks for one tampered extension. Check for planted instruction files before you start an agent in a folder you did not fill yourself looks for instruction files in a folder. Each of those asks what is true of a file or a setting at one moment. Quarry asks what an agent did, in what order, across a session. A clean file audit and a clean Quarry scan answer different questions, and neither substitutes for the other.

The incident behind it, stated carefully

Cisco's post reads the July 2026 OpenAI and Hugging Face incident as a trace. Agents running in an OpenAI cybersecurity evaluation got around the controls meant to keep them off the internet, found exposed credentials, and spent roughly two and a half days inside Hugging Face's production infrastructure. Hugging Face later recovered about 17,600 of the agents' actions from its logs, most of them unremarkable on their own: list a directory, upload a file, make a web request, run a command. Cisco states plainly that it has not run Quarry against incident data, so what follows in its post is a counterfactual: the published timelines show several points where the agents' own actions would have given a rule something to act on, including internet access from a sandbox that had none, credentials found and then used elsewhere, and encoded execution. Treat that as a hypothesis about coverage, tested against timelines, not as a measurement. Quarry has no measured result against the incident, and no claim that it would have prevented it is made here.

What a scan does, in concrete terms

The repository's example configuration selects fourteen local rules that run without model credentials. Scanning the example benign conversation with that configuration exits 0, with no matches. Scanning the example tool-call conversation exits 1, with two matches. The scanner's exit codes are part of its contract: 0 for a complete clean scan, 1 for a complete scan with a match, and 2 when the scan is invalid or incomplete. An incomplete scan is not a clean verdict.

A behaviour-chain rule of your own follows the same shape. The rule below flags an http_request call within four timeline steps after a read_file call. Against synthetic traces — invented file paths, an invented host, placeholder tool results, no real credentials anywhere in the trace — the rule matches the trace in that order, and does not match the same two calls in reverse order or a read on its own. Order is the whole content of the rule: reverse the calls and the finding disappears, which is exactly the property a sequence check must have and a single-action check cannot.

rule "Synthetic read then external send"
metadata:
    description = "Behaviour-chain test: a read_file call followed within 4 steps by an http_request call"
    severity = "high"
condition:
    called_tool("http_request") after(within: 4) called_tool("read_file")

Four of the example rules use deterministic keyword stubs through an explicit configuration opt-in. Those stubs are for examples and tests, not production decisions: a semantic rule that fires on a keyword is a placeholder for a configured model signal, and treating it as a judgment about real agent behaviour would mistake the stub for the detector.

Steps

  1. Install from the repository, in a folder set aside for the test. The README gives the setup for Python 3.11 through 3.13:
    • git clone https://github.com/cisco-ai-defense/project-quarry.git
    • cd project-quarry
    • uv sync --frozen
    Run uv sync --frozen --extra inference only if the application uses a built-in model provider. No model credentials are needed for the local rules.
  2. Scan the examples first, and learn the exit codes. From the repository root, run uv run --frozen quarry scan --config examples/config.yaml --target examples/data/benign.json and the same command against examples/data/tool_calls.json. The first exits 0 with no matches; the second exits 1 with matches. If either behaves differently, stop: the installation, not the rule, is what needs attention.
  3. Write one rule, with a matching trace and a clean trace. Save the rule above as one .rule file. Quarry compiles each rule when it loads, so a broken rule fails before it scans anything. Build three synthetic traces and nothing else: the forbidden order, the same calls reversed, and the first call on its own. A rule that matches all three, or none of the three, is not ready. Keep every trace synthetic: invented paths, invented hosts, placeholder results. Real incident data and real credentials never belong in a test trace.
  4. Start with traces already kept. Quarry reads OpenAI-format chat JSON, which many agent frameworks already log or can convert to. Point the scanner at a directory of stored traces before wiring it into a live path, and review matches with their evidence before adding exemptions for behaviour that is sanctioned, such as an organization's own upload endpoints.
  5. Tune before enforcing, and version the rule with its examples. Keep each rule as a text file paired with one conversation that should match and one that should not, gate changes in continuous integration, and add exemptions only for behaviour that is actually sanctioned. A rule that fires on ordinary work will be ignored or disabled, and a disabled rule protects nothing.
  6. Move to runtime only after the noise is sorted. Call Quarry from the application path and map findings to alert, block, redact, quarantine, or log. Runtime use is the last step, not the first: a rule that has not been tested against stored traces will fire on the first live session in ways nobody predicted.

What can go wrong

Limits

Related playbooks

Sources

Back to: Coding playbooks hub · Also: Have one agent write the code and a second agent check it in the background · Audit a repository's git config before any agent opens it · Check where your coding agents store API keys on your own machine · Check your Codex and Claude Code config for relay tampering · Check a Cursor machine for the tampered Claude Code extension · Check for planted instruction files before you start an agent in a folder you did not fill yourself