Agent Upstart

Cool things people are building with agents — and practical help getting the most out of them.

Playbooks · Coding tasks

Cut coding-agent costs with TokenEconomizer

Written November 4, 2026

A plugin that learns how one repository works from finished sessions, then trims what later sessions are sent. The authors' benchmark results are real and narrow: treat them as a reason to try it on your own work, not as a saving you will see.

What it does

TokenEconomizer is a plugin for Claude Code, Codex, the GitHub Copilot CLI, and Pi, with experimental support for OpenCode, Cursor Agent, and Hermes Agent. It leaves the agent and its model alone and works through what the agent already exposes: the instruction prefix, the loaded tools, and the live event stream. After each finished session in a repository, it updates what the next session in that repository is given. What it learns stays on the machine, under ~/.tokeneconomizer, and the plugin needs no API key of its own: its own model calls run through the agent, on the same plan or login, and their cost is included in the reports it produces.

It has three mechanisms. Micro-skills learn short facts about the repository from finished sessions — a build step that must run first, the spelling of a command that works in this repository, an environment variable a command needs — and add a fact to a session only after repeated sessions confirm it. Tool pruning leaves out tools that recent sessions left unused, through the agent's own settings, while always keeping the tools for shell execution and for reading, writing, and editing files. Failure-loop intervention watches a turn for repeated failures and, when its judge is sure enough the turn is stuck, redirects the agent once with a summary of what it has tried; if the turn is still stuck after that, the plugin says so and tells you how to stop it. Stopping the turn is your call: the plugin never stops a turn itself. If any part of the plugin fails, the session carries on as if it were not installed.

The waste it targets is the waste that repeats. In the authors' profiling of 60 unmodified SWE-bench runs, described on the project page, 30% of tool calls repeated an earlier task's action once a repository had five or more tasks behind it, 16 of the 23 tool descriptions billed on every turn were never called, and the 17% of tasks that failed took 32% of the spend. A fixed saving applied to every repository cannot match that shape: on the same 60 tasks, each of the three fixed approaches the authors tried was cheaper in some repositories and dearer in others. TokenEconomizer learns per repository instead.

What the authors measured

The paper, by researchers at UC Berkeley, Microsoft Research, and Microsoft Security, ran five agent and model combinations on the same 277 SWE-bench tasks from six repositories, with each repository's tasks in order and starting with nothing learned. Costs include every model call the plugin made, charged at cache-aware prices. Across the five combinations, mean cost was 18.4–30.9% lower than the default run, cost fell in all 30 repository-by-combination cells, and Pass@1 rose in four of the five combinations. The one fall was 1.0 percentage point, on Claude Code with Opus 4.8.

Agent and modelMean cost changePass@1 change
Claude Code + Sonnet 5−23.6%+1.3 points
Claude Code + Opus 4.8−30.9%−1.0 points
Codex + GPT-5.6 Terra−24.7%+0.9 points
Codex + GPT-5.6 Sol−26.1%+1.6 points
GitHub Copilot CLI + Sonnet 5−18.4%+2.6 points

Lowering the reasoning effort instead of using the plugin cut cost by a similar amount in the paper's comparison and lowered success in every setting, which is the contrast the authors draw: the plugin's savings arrived with success roughly unchanged, while the cheaper-effort savings did not. The savings also carried beyond SWE-bench in the authors' runs, at 16–21% lower on Web-Bench and 6–20% lower on two security-auditing benchmarks, where every task sat in a new repository and micro-skills were off. Savings accumulated over a task stream rather than arriving at once: on Claude Code, every repository was cheaper by the sixth task.

What the results do not establish

The headline range is an SWE-bench result, from the authors' own harness, on six repositories and five combinations. It is not a measurement of your repository, your tasks, or your model, and it was not independently reproduced. SWE-bench tasks are self-contained coding tasks with tests; long, open-ended work, work in a repository the plugin has never seen, and work where the agent's tools differ from the tested setups can all behave differently. The plugin installs for Pi, but the experiments ran only on Claude Code, Codex, and the Copilot CLI, so the percentages say nothing about Pi, and the experimental OpenCode, Cursor, and Hermes support was not part of the benchmark at all.

Two costs sit inside the reported figures and deserve a plain reading. The plugin's own model calls — the fact proposer and the loop-guard judge — are charged to the task that triggered them, so a repository with few repeated tasks may pay the overhead without gaining the reuse. And tool pruning changes what the agent can reach: a tool left out because recent sessions did not call it is a tool the next session cannot use until it is given back. The plugin's own defaults reflect that trade-off. Tool pruning and the loop guard are on by default; micro-skills are off by default, because the proposer adds model calls and how much it saves depends on the repository and the work.

Steps

  1. Check the supported versions first. The README lists a supported version range for each agent, and the install commands were tested on the oldest and newest version in each range. If your agent version sits outside its range, do not assume the plugin will behave as described here.
  2. Install for Claude Code. In the folder where you want to keep the clone, run:
    • git clone https://github.com/microsoft/TokenEconomizer.git
    • claude plugin marketplace add "$PWD/TokenEconomizer"
    • claude plugin install tokeneconomizer@tokeneconomizer
    Keep the marketplace path absolute: the Copilot CLI reads a relative path as the name of a GitHub repository. Start the agent in one of your repositories, trust the folder, and run /tokeneconomizer:status. Codex and Copilot CLI installs follow the same shape with their own plugin commands, and Pi installs directly from the git repository; the README gives each form.
  3. Start with tool pruning in propose mode. Run /tokeneconomizer:config set toolprune=propose in the repository, work as usual for a few sessions, then read /tokeneconomizer:toolprune to see which tools it would leave out and on what evidence. Nothing changes until you run toolprune apply. A tool given back with toolprune restore is not removed again.
  4. Turn micro-skills on deliberately, or leave them off. With /tokeneconomizer:config set microskills=on, the proposer adds model calls of its own. Try it on a repository where the same facts keep being rediscovered, and compare the cost shown by /tokeneconomizer:status across sessions with it on and off, on the same kind of work. A fact the plugin injects is worth reading: /tokeneconomizer:microskills shows the facts in use and those still forming.
  5. Refresh the tool catalog after an agent update. Once per machine, and again after Claude Code or Copilot CLI updates, run /tokeneconomizer:toolprune refresh-catalog. It records every tool a session loads, without a model call, so pruning can consider all of them.
  6. Judge it on your own sessions, in order. Run at least six tasks of one kind in one repository before judging, since the plugin starts with nothing learned and the authors' savings accumulated over a stream. Compare cost per task with the plugin on against the same kind of work without it, and watch Pass@1 in your own terms: tests passing, not the plugin's say-so. If cost does not fall, turn a mechanism off with config set rather than assuming the next repository will behave differently.

What can go wrong

Limits

Related playbooks

Sources

Back to: Coding playbooks hub · Also: What an agent costs, in plain figures · Agent cost estimator · Baseline usage before the shared allowance · DeepSeek, Exa, Brave, Tavily: which web-search API should your agent actually pay for now that search is metered