Agent Upstart

Cool things people are building with agents — and practical help getting the most out of them.

Playbooks · Coding tasks

Google's AQuA watches for silent agent failures

Written November 5, 2026

AQuA is an ambient quality agent that reads production traces to catch failures agents never report. What it checks, and what it can't see.

What AQuA is

AQuA, the Ambient Quality Agent, was released by Google on October 8, 2026 as an open reference implementation — composable building blocks you can run in your own project, not a managed service. It runs unattended beside your agent inside your Google Cloud project, sweeping production trajectories from Cloud Trace, Cloud Logging, or BigQuery on a schedule, after each deployment, or on demand. Session transcripts, source snapshots, and BigQuery tables stay inside your project boundary. AQuA never sits in the request path and never writes back to your agent.

The problem it addresses is the failure that leaves no trace of itself as an error. An agent can return HTTP 200, stay inside its latency budget, and raise no tool error — and still record a seat as confirmed without checking whether it is free, or recommend a steakhouse to a traveller whose profile says vegan. Health checks stay green; the failure is silent and happens inside the conversation.

What it checks

Each run reads a sample of up to 1,000 recent sessions and puts it through a five-stage pipeline:

  1. Sample. Pulls a random sample of up to 1,000 recent sessions from your telemetry, capped to keep run cost predictable.
  2. Review. Grades each session against a nine-point checklist of common breakdowns — procedure, tool selection and arguments, grounding, task completion, and others — with the agent's instructions, tools, and an optional developer goal file (goal.md) in view. When a session fails, it writes a structured actual-versus-expected finding.
  3. Cluster. Groups findings that share the same failure mechanism into candidate issue clusters.
  4. Verify. A separate model checks each candidate cluster against up to three full transcripts and discards clusters the evidence does not support.
  5. Track. Matches surviving clusters against open insights in BigQuery as new, recurring, or automatically resolved after 14 days unseen.

When an insight is worth investigating, root-cause analysis runs from the dashboard chat or from agents-cli aqua run. AQuA reads the failing trajectories alongside the immutable source snapshot captured at deploy time, cites the file and line range where the defect sits, and proposes an edit anchored to those lines. It never applies an edit or opens a pull request on its own.

Google demonstrates the pipeline on a travel-concierge agent with subagents for inspiration, places, flights, seats, and booking. Across a 32-session sweep, 5 sessions passed cleanly and 27 produced 42 findings, which clustered into 9 candidates; the verifier rejected 3 as false positives, leaving 6 verified issues. The largest was a seat-availability bypass in 15 sessions: the planning agent saved seat numbers straight into session state without calling the seat-selection subagent. A second was dietary constraints dropped across a subagent boundary in 7 sessions. After two one-line prompt fixes and a redeploy, replaying the same 32 sessions showed the seat bypass down 87 percent, from 15 sessions to 2, the vegan omissions down from 7 to 0, and full-session passes more than doubled, from 5 of 32 to 13 of 32. Those figures come from Google's own demonstration, not an independent test.

What it can't see

What to borrow without AQuA

You do not need AQuA running to take the habits behind it. Write down the failure modes you care about in plain language, the way a goal.md file does, and grade a sample of real sessions against that list instead of only checking that calls succeeded. Write each failure as an actual-versus-expected pair, so the fix is visible in the finding. And treat a cluster of failures as a claim until a second read of the full transcripts supports it — Google's design rejects unverified clusters before they enter the queue, and on an 87-trace internal benchmark its verifier rejected 4 of 24 candidate clusters.

To try the dashboard with no cloud project, no credentials, and no model calls, clone the adk-recipes repository and run make demo in the ambient-quality-agent directory, which serves the dashboard locally on a synthetic month of runs and insights.

Limits

Related playbooks

Sources

Back to: Coding playbooks hub · Also: What AI monitoring missed, per the METR audit · Catch bad agent behaviour chains with Quarry