Playbooks · Coding tasks
Measure agent friction with Gauge CLI
Written November 6, 2026
Gauge CLI measures agent friction and retests improvements. What it measures, and whether the measurement survives a retest.
The problem: iterating without a friction metric
Agent builders change prompts, skills, and tooling, then judge the result by feel. A task that used to work stops working, or a fix that looked good in one session fails in the next, and there is no number to point at. Gauge CLI is an open-source tool for exactly that gap: it runs real coding agents against your product, records where they get stuck, and lets you retest a fix against the same task to see whether the friction actually went away.
What Gauge measures
Gauge runs coding sessions and analyzes them to find the issues agents run into. The kinds of findings it names in its README are concrete: an interactive command that blocks a coding agent, an agent reaching for a deprecated command or an outdated API pattern, a skill going unused even when it should help, an agent confidently claiming a limitation that does not exist, and an agent unable to work out the intended workflow across several commands. The fix might be in error messages, skill content, help text, code, or documentation — Gauge's job is to show you the friction, not to pick the fix.
It works with Claude Code, Codex, Pi, and OpenCode, using your existing logins or API keys; agent usage is billed through your provider or subscription. It needs Node.js 22.13 or newer, npm, and Docker running Linux containers. The full loop runs locally with no Gauge account: npm install -g @withgauge/cli, gauge local runtime pull, then gauge init --local in your project. A hosted option exists for managed compute and shared measurement over time, and that one uses workspace credits.
How the improvement loop works
- Define realistic tasks. Save evals or preference prompts with stable identities, editable defaults, and observable outcomes. Shared Markdown cases in
gauge-cases/are for measurements meant for teammates or CI. - Test your current product. Gauge starts fresh Docker sessions and injects the working-tree version of your inputs, so each test agent sees the task and the configured inputs inside its sandbox.
- Find friction. Gauge normalizes events from the four supported agents into consistent transcripts. In local mode, your coding agent reads the evidence, checks the outcome against the criteria, and identifies what slowed or blocked the task.
- Change and retest. Edit the product or its guidance, then rerun the same eval. The new batch snapshots the current input files while the baseline stays intact, so the comparison is the same task, before and after.
- Compare.
gauge run comparelines up the baseline batch against the candidate batch and summarizes completion, turns and duration, and criterion judgments. Read the findings and the evidence to decide whether the fix worked.
The retest is the point. A single measurement tells you a task is hard for an agent; a before-and-after pair tells you whether your change made it easier. Gauge keeps the task, the criteria, and the model pin fixed between runs, which is what makes the comparison mean something.
What it does not do
- It does not judge for you in local mode. Your coding agent authors the analysis from the session evidence; in hosted mode the analysis happens automatically after each session.
- It does not test everything locally. Local mode supports skills, SDKs, CLIs, and files; docs, authenticated MCPs, and authenticated APIs need hosted compute.
- It does not measure discovery on the web. Supplying a skill measures choice after exposure to it, not whether the agent would have found it on its own.
- A measurement here is one task on one agent. A fix that survives one retest has survived one retest, not every task your users will try.
Limits
- No measurement run was done for this page. Docker was not available in the environment where it was written, so the loop above is described from the README, not from a run. The commands and the workflow are the project's own; whether they behave as described on your machine is something only a run settles.
- Gauge is a young project — created in October 2026, with no releases published at the time of writing — so expect the CLI and its docs to move. Pin the version you measure with.
Related playbooks
- Antigravity CLI 1.5.3: subagent models, sandbox fix
- Antigravity CLI 1.5.4: ACP Auto mode
- Antigravity CLI 1.5.5: swarms in preview
- Antigravity CLI 1.5.7: Auto becomes the default
Sources
- Gauge CLI — GitHub repository and README
Back to: Coding playbooks hub · Also: Run your own agent bake-off on one real task · Fingerprinting AI coding agents from 33,580 PRs · Run parallel subagents in Claude Code