Agent Upstart

Cool things people are building with agents — and practical help getting the most out of them.

About · About page

How we check things

What we run before a page is published, what docs-only means, what gets cut, and how corrections work.

Most pages on this site describe a tool or a technique. Before one is published, we try to run the thing it describes, and we only print claims we could check. This page explains what that means in practice, so you can judge for yourself how much weight to give a page.

What we run before publishing

Where a run is possible, we install the tool and run it in a sandbox — a throwaway copy of the setup, not a machine with real work on it. We run the commands the page describes and compare what actually happens with what the tool's own pages say should happen. Where the author published a notebook or a dataset, we reproduce the published result from it instead of taking the headline number on trust. The runs we publish as experiment pages carry the run transcript as text, so you can read exactly what happened instead of taking our word for it.

What "docs-only" means

Sometimes a run is not possible here — the tool needs an account, a paid plan, or hardware we do not have. Then the page is labelled docs-only: every claim on it comes from the vendor's own pages, and no run backs it. A docs-only page tells you what to check yourself before you rely on it, usually by trying the tool's own free tier or reading the linked source. Treat it as a starting point, not a result.

What gets cut

Any claim we cannot check against the vendor's own page or a run is cut, not hedged. If a vendor page says a feature exists and we cannot find it in the docs or in a run, the feature does not appear on this site. If a price or an allowance appears only in a third-party roundup, it does not appear either — prices and allowances change without warning, and only the vendor's own page counts. Cutting a claim leaves a gap in a page; we would rather print the gap than fill it with a guess.

What to do with a claim on this site

Open the linked source. Every load-bearing claim on this site links to the page it came from — usually the vendor's own docs, repo, or changelog. Read that page before you spend money or change a setup on the strength of ours. If a page is docs-only, treat it as a starting point and run the check it suggests before you trust it. If a claim has no link, treat it as our wording, not a fact.

How pages get corrected

Pages are checked again on a schedule, and anything that no longer matches its source gets fixed or cut. When a correction changes what a page says, the change is listed on the changelog so you can see what moved and when.

See the runs

Sources

Back to: About Agent Upstart · Also: What we believe · Changelog