GitHub Copilot's Agentic Harness: Same Model, Fewer Tokens

GitHub published benchmark results comparing its Copilot agentic harness against model-native harnesses like Claude Code and Codex CLI. The headline: same model, same task, but fewer tokens burned—without sacrificing resolution.

The harness is the shared runtime behind Copilot CLI, the Copilot app, and Copilot code review. Improve it once, and every surface benefits. GitHub ran five benchmarks—SWE-bench Verified, SWE-bench Pro, SkillsBench, TerminalBench, and an internal Windows container benchmark—across Claude Sonnet 4.6, Claude Opus 4.7, GPT-5.4, and GPT-5.5.

Token efficiency is the standout. Across most configurations, GitHub's harness used fewer tokens than the model vendor's own harness while hitting equivalent task completion rates. The differences in resolution were within stochastic noise—effectively parity.

GitHub also published a cost-vs-resolution scatter plot for TerminalBench 2.0. The Copilot harness markers sit in the same or better quadrant (up and to the left) compared to same-model competitors. GPT models deliver the best value; Claude Opus hits the highest resolution at a premium. The harness lets you pick either, or let auto-model selection decide.

The multi-model story is the real differentiator. The harness supports 20+ models across GPT, Claude, Gemini, and MAI families, plus bring-your-own-key for open-source or local models. A model-vendor harness can't offer that. It also enables cross-model critique features like Rubber Duck, where one model reviews another's output.

This is a pragmatic approach: don't bet the farm on one model, don't waste tokens, and don't sacrifice results. Benchmarks are just one signal, but the direction is clear.