Experiment result
Stroq attack suite
Written November 8, 2026
We ran Stroq's own attack command to see whether its hooks stop the attacks it ships with.
What was tested
Stroq guards coding agents with local hooks that judge each tool call before it runs. Its attack command runs a set of known attacks against your policy in a throwaway home directory, to show what gets stopped and what gets through. We tested whether the command's result on version 0.30.0 matches what the README says it should be.
How
One command, on version 0.30.0 of the Stroq CLI:
npx -y @stroq/cli attack
The attacks run in a sandbox the command sets up itself. Nothing else was installed and no agent session was involved.
What happened
| Outcome | Scenarios |
|---|---|
| Blocked (denied) | 15 |
| Asked (stopped for a human decision) | 21 |
| Passed through | 0 |
| Total | 36 |
That matches the README's claim for 0.30.0 exactly: "on 0.30.0, 15 are denied, 21 are asked about, 0 pass through." The README also says the 36 attacks come from public reports and synthetic test cases.
What it means for the reader
Stroq's own self-test passes on its own terms: nothing in its attack set got through unchallenged on this version. But most of the set — 21 of 36 — was not blocked outright; it was stopped and handed to a person to decide. The protection there depends on the person saying no, so the result is a prompt to check your own habits, not a promise that nothing can slip past.
Limits
This is a self-test, not an independent evaluation. The 36 attacks are Stroq's own set, so the run shows Stroq stopping attacks Stroq chose, not attacks an outsider devised. No transcript audit was run: there were no real agent sessions on this machine, so the result says nothing about what Stroq would catch in a live session, or about attacks outside its set. A new version may change the counts.
Related experiments
- Aider /test passes arguments through
- Snorkel reward-code reproduction
- Stroq: audit what your coding agent touched
- Fix a bug with an agent without breaking the test suite
Sources
- Stroq — GitHub repository and README — the attack-suite description and the 0.30.0 claim we checked
- Stroq: audit what your coding agent touched — our playbook on using Stroq