Playbooks · Teams tasks
Copilot Automations learn from real outcomes
Written November 7, 2026
Copilot Automations now propose prompt improvements from real outcomes. What it learns from, what it proposes, and what to check before accepting.
What changed
GitHub's October 28 changelog says Copilot Automations can now learn from their outcomes and propose improvements to themselves. An automation is a cloud workflow in a repository that runs on a trigger or a schedule — sorting issues, reviewing pull requests, opening optimization pull requests. Writing its prompt is the easy part; keeping the prompt honest as the work changes is the hard part, and that is the gap this change addresses. The proposal is the whole feature: nothing about the automation changes until a person accepts it.
What "real outcomes" means here
Automations leave a trail of signals on GitHub that show how well they are working. The changelog names three:
- Issue triage: whether people change the labels the automation applied.
- Code review: whether developers accept the comments the automation posted.
- Performance optimization: whether maintainers merge the pull requests the automation opens.
Those are the outcomes the improvement loop reads. A label change, an accepted comment, and a merged pull request are all things people do after the automation acts — not scores the automation gives itself.
How the loop works
When you ask Copilot to set up a self-improving automation, it can create a group of automations that work together. The changelog's example is a custom code reviewer: one automation runs when a pull request opens, reviews the changes, and posts comments; a second, weekly improvement automation analyses how developers responded to those comments and proposes refinements to the first automation. When the improvement automation proposes a change to the original automation's prompt, tools, or triggers, GitHub notifies you. You review the proposal and accept or reject it before anything is applied.
Copilot Automations is available with Copilot Pro, Pro+, Max, Business, and Enterprise plans.
What to check before accepting a proposal
A proposal that optimizes the wrong outcome is worse than a stale prompt. The signals are ambiguous in specific ways, and each one is worth checking against the actual work before accepting:
- A merged pull request is not proof the automation's output was right. A maintainer may have repaired the change before merging it. If the proposal treats the merge as a success, it learns the wrong lesson. Look at what changed between the automation's version and the merged version.
- An accepted review comment may have been accepted for the wrong reason. A developer can accept a comment to clear the thread, or because the fix was trivial, not because the comment caught something real. If the proposal steers the reviewer toward more comments of that kind, it is optimizing for acceptance, not for useful review.
- A relabelled issue may reflect a policy change, not a bad triage. If the team redefined what a label means, the automation's old labels will look wrong even though they were right under the old definition. A proposal that retrains the triage prompt on those relabels bakes in a definition the team has already moved past.
- A proposal can narrow the automation. An improvement that makes the prompt more specific to the cases it has seen can make it worse on the cases it has not. If the proposal adds conditions drawn from a handful of recent outcomes, ask whether those conditions hold generally or describe one week of work.
The check is the same in each case: read the proposal against the outcome it claims to come from, not against the automation's own description of that outcome. The changelog offers no results showing the loop raises quality over time, so "better results over time" is GitHub's claim for now, not a measured result.
Limits
- This page is a read of the changelog. It describes the feature; it reports no run of one.
- The changelog describes the signals and the approval step. It does not say how often proposals arrive, how large a change a proposal can make, or what happens to a proposal nobody answers.
- The failure modes above are checks to apply, not observed incidents. No example proposal is shown because none is published in the changelog.
Related playbooks
- Test Gemini 4 Carbon in Copilot before Dec 1
- Copilot reviews: how many get addressed
- Require Copilot customizations to come from plugins
- Share a Copilot Automation and cap its credits
Sources
- Copilot Automations now learn from outcomes and self-improve — GitHub Changelog
- Copilot automations — GitHub Docs
Back to: Small-team playbooks hub · Also: Copilot reviews: how many get addressed · Decide whether Copilot should approve PRs · Triage a backlog of issues with an agent