Agent Upstart

Cool things people are building with agents — and practical help getting the most out of them.

Playbooks · Coding tasks

Foundry Local runs Qwen3.8-27B on your GPU

Written November 7, 2026

Microsoft's Foundry Local serves Qwen3.8-27B on-device and plugs it into Copilot via BYOK. What it costs in VRAM, what it scores, and what stays in the cloud.

What it is

Foundry Local brings agentic coding directly to developer hardware: it runs Qwen3.8-27B locally and connects it to GitHub Copilot, so code and context never leave the machine in front of you. The model is served on ONNX Runtime, and Copilot connects through Bring Your Own Key. The setup needs Foundry Local CLI 0.13.0 or later, and for NVIDIA GPUs the CUDA and cuDNN runtimes have to be installed separately — Foundry Local does not download them.

Microsoft picked Qwen3.8-27B as its main optimization target because it pairs strong coding capability with a size suited to local deployment. The pitch is threefold: predictable inference costs instead of per-token cloud billing, greater privacy for sensitive code, and the freedom to keep working offline.

What it costs in VRAM

With 4-bit quantization, the model is reduced to about 18 GB, which makes it runnable on consumer GPUs with 24 GB of VRAM, supporting context windows of up to 64K tokens. GPUs with 32 GB or more of VRAM can support longer contexts, up to the model's maximum of 262K tokens. 24 GB is the floor: the setup steps start by asking whether you have an NVIDIA GPU with at least 24 GB of VRAM.

The optimization stack behind those numbers combines DFlash2 speculative decoding, INT4 weight handling, KV-cache quantization, prefix caching, continuous batching, and CUDA graph fusions tuned for NVIDIA's SM121 GPU architecture.

What it scores

All four benchmark figures below are Microsoft-reported, from the model card Microsoft points to. They have not been independently reproduced.

BenchmarkQwen3.8-27B
Terminal-Bench 2.1 (agentic terminal coding)73.0
SWE-bench Pro (agentic coding)61.7
NL2Repo-Bench (repo-level code generation)42.3
DeepSWE 1.1 (agentic coding)42.2

Microsoft also compared Foundry Local with llama.cpp using the 4-bit model on two of its own systems — an NVIDIA GeForce RTX 5090 with 32 GB of VRAM and an NVIDIA DGX Spark with 128 GB of unified memory — on a dataset of real GitHub issues. Foundry Local delivered higher generation speed at every tested prompt length, with the advantage widening beyond 131K tokens. That is Microsoft's own test, on Microsoft's systems, with Microsoft's dataset: it is a vendor comparison, not an independent one.

Setting it up

Install the CLI with winget install Microsoft.FoundryLocal, then check the version reports 0.13.0 or higher. Load the model and check the server with foundry model load qwen3.8-27b and foundry server status. Then, in the GitHub Copilot app, open Settings, select Model providers, choose Add provider, select Foundry Local, enter the API endpoint from the server status in the Base URL field, and search for Qwen3.8-27B in Browse Catalog to add it. Switch the model provider to Foundry Local and select Qwen3.8-27B in the Copilot model picker. Use the endpoint your own machine reports, not the port shown in Microsoft's screenshots.

What stays in the cloud

Microsoft says it plainly: local models may have limited reasoning capability compared with advanced cloud models because of their smaller size. For complex, high-reasoning tasks, cloud models remain the strong choice, while local models handle everyday coding and targeted subtasks — a hybrid workflow, not a replacement. Support for Macs and AMD GPUs is on the roadmap, not shipped: the post says Microsoft plans to expand Qwen3.8-27B support to more devices. The model catalog and telemetry are Microsoft-controlled, and the local path still runs through Copilot's BYOK setup.

Limits

Related playbooks

Sources

Back to: Coding playbooks hub · Also: MAI-Code-1.1-Flash: local coding on RTX Spark · Copilot reviews: how many get addressed