Agent Upstart

Cool things people are building with agents — and practical help getting the most out of them.

Playbooks · Coding tasks

MAI-Code-1.1-Flash: local coding on RTX Spark

Written November 7, 2026

Microsoft squeezed a 137.9B coding model onto RTX Spark at 51 tokens a second with a 120K context. What the low-bit build keeps — and what it gives up.

What it is

MAI-Code-1.1-Flash is a 137.9-billion-parameter mixture-of-experts coding model, built to run on a developer's own machine instead of in the cloud. Microsoft's post frames the question plainly: how much useful coding intelligence fits inside one machine, on an RTX Spark PC with 128 GB of unified memory. The answer it offers is a low-bit build, not a smaller model. Mixed-precision quantization takes the weight footprint from about 276 GB at 16-bit precision down to a 53.0 GiB checkpoint at 3.31 bits per weight, with a more compact 46.1 GiB variant at 2.87 bits per weight also developed. The 53.0 GiB checkpoint is the released one, available through Microsoft Foundry and as the local model in GitHub Copilot.

The point of the squeeze is not the size itself. Lower precision frees memory and bandwidth that can be spent elsewhere: larger models, longer contexts, bigger key-value caches, and speculative decoding. But the post is clear that smaller is not automatically better. Quality, footprint, and responsiveness have to be optimized together, because the target model, the key-value cache, and the runtime all compete for the same device memory, and every token streams the active weights through memory.

What it does on the machine

On RTX Spark, with 128 GB of unified memory and a 64 GB GPU allocation, Microsoft measured the 53.0 GiB build with a DFlash2 speculative decoding draft model across input lengths from 1K to 120K tokens. Each request generated 8,192 tokens, with three repetitions per case, and the reported figures are medians. The headline number: a median of 51.12 tokens per second with a 120K-token input context, up from 25.81 tokens per second without the draft model. Across the tested input lengths, the draft raised median decode throughput by 1.52 to 1.98 times. At 120K input, median end-to-end latency fell 27 percent, from 527.25 to 382.41 seconds.

Those numbers describe one specific setup, and the post says so directly: the figures describe this Q2-XS and DFlash2 configuration and should not be generalized to different serving conditions. Peak GPU memory in the draft-enabled configuration rose only from 59.42 to 63.50 GB as input grew from 1K to 120K tokens, which is the number that matters if you are sizing a machine. The draft model's own key-value cache is kept in check with sliding-window attention in its first four layers, cutting reserved draft cache memory at 262K context from 5,120 MiB to 1,156 MiB, a 77.4 percent reduction, while draft acceptance and decode throughput differed by no more than 2 percent from the full-attention draft.

The demo in the post shows the model running locally through GitHub Copilot: Copilot audits a codebase for unused code, imports, dependencies, and obsolete feature flags, investigates findings across the repository, and produces a cleanup plan. That is Microsoft's demo, not an independent test. No benchmark scores for coding quality are quoted in the post's text beyond the observation that candidate quantization variants that look nearly equivalent on one benchmark can separate on another, which is why the team compared candidates across a broad set of coding and non-coding benchmarks rather than relying on a single metric.

The trade, in one paragraph

Low-bit inference buys room: room for a 137.9B model on a machine with 128 GB of memory, room for a 120K-token context, room for a draft model that roughly doubles decode speed at long inputs. What it costs is the work of preserving quality under quantization. Microsoft's approach assigns different precisions by tensor role, with expert gate and up projections at about 2 bits per weight and expert down projections at about 4, and calibrates sensitivity from representative coding trajectories, then selects the final build through end-to-end evaluation rather than from the sensitivity signals alone. The post's own lesson is that the operating point is an empirical choice: two builds with similar coding quality at different sizes sit at different points on the quality-size frontier, and neither is universally optimal.

What to do with it

Limits

Related playbooks

Sources

Back to: Coding playbooks hub · Also: Run LongCat-2.5 in Claude Code without the 200K cap · Model specs: context, tools, and cutoffs in one table · Foundry Local runs Qwen3.8-27B on your GPU