Playbooks · Coding
Copilot computer use: drive the GUI
Written November 10, 2026
GitHub Copilot can now drive desktop applications — read a window, click a button, type into a field, move between apps. It is a public preview, local sessions only, off by default, and GitHub's own warning is the first thing to read.
What it is
Computer use entered public preview on October 1, 2026, in GitHub Copilot CLI and the GitHub Copilot app, on macOS and Windows. GitHub's description: Copilot reads accessible application content and visual context through the operating system's accessibility tree or screenshots, then clicks controls, enters and edits text, presses keys, scrolls, drags, and navigates workflows across applications. It is aimed at legacy and GUI-only software that provides no API, command-line interface, or MCP integration — the programs where the screen, keyboard and mouse are still the only interface.
It is available for local sessions only. Linux is not supported for the desktop-control part, and neither is VS Code or the cloud agent. The feature is a bundled computer-use plugin running on its own MCP server inside the CLI.
How to turn it on
Computer use is disabled by default. In an interactive Copilot CLI session:
/computer on # enable desktop app control
/computer show # show plugin, MCP server and skill status
/computer off # disable it again
The CLI saves your preference and enables the bundled plugin. In the Copilot app, the same switch sits under Settings, then Computer Use. On macOS, turning it on guides you through granting Accessibility permission to interact with controls and Screen Recording permission to inspect windows. Copilot follows the active permission mode: depending on the mode, it asks for approval before controlling an application, and you can grant it for the current session or save an "Always allow" decision. To interrupt an operation in progress in the CLI, press Esc twice.
An organisation can disable computer use through managed settings, and a local preference cannot override that policy. If the CLI reports the feature as blocked by managed settings, that is the reason.
The warning, in full
GitHub warns that computer use interprets interfaces that can change between application versions, operating systems and window states. It can select the wrong control, enter text in the wrong location, or struggle with non-standard or dynamic controls and complex workflows. Ambiguous instructions or unexpected on-screen content may cause unintended actions that affect your device, data, or connected accounts — including access to personal, financial, or enterprise systems. GitHub's advice: use a direct tool — an API, MCP server, terminal command, filesystem tool, or browser tool — whenever one can complete the task, because those return more structured and predictable results than clicking through a screen.
Three habits follow from that. Start with non-critical applications until the feature has a track record in your hands. Review the target application, the requested permissions, and the result — particularly before allowing actions that modify data or affect other people. And do not choose "Always allow" for applications that hold sensitive information or support high-impact actions: a saved approval lets later computer-use actions control the app without asking again.
Limits
- Public preview, and GitHub says it is subject to change. No general-availability date, usage limits, or per-action pricing have been published for it.
- Local sessions on macOS and Windows only. It does not work in VS Code, in the cloud agent, or on Linux.
- Application windows may display sensitive information, including information about other people. Only use computer use with applications and tasks whose visible content you are comfortable providing as context to Copilot.
Related playbooks
- Tencent Hy4: vision and computer use, weights pending
- Copilot CLI 1.0.106: /code, /settings, GPT-6.1 Terra
- Copilot CLI 1.0.108: index, PDFs, Ctrl+C
- Copilot CLI 1.0.109: CIDR sandbox rules
Sources
- About computer use in GitHub Copilot — GitHub Docs
- Using GitHub Copilot CLI to interact with desktop applications — GitHub Docs
Back to: Coding playbooks hub · Also: Copilot CLI 1.0.108: index, PDFs, Ctrl+C · Review a pull request with a coding agent