--- title: "How coding agents compete: the model, the bundle, the perimeter, not the agent" canonical: https://thebeat.dev/threads/how-coding-agents-compete/ updated: 2026-11-06 status: open momentum: rising opened: 2026-07-28 tags: - ai-coding-agents - positioning - distribution - agent-rails collection: threads site: https://thebeat.dev/ license: https://creativecommons.org/licenses/by/4.0/ --- # How coding agents compete: the model, the bundle, the perimeter, not the agent > Coding agents compete on which models a plan reaches and what a task costs. Two independent studies find the harness mostly moves the bill. A third, Endor Labs' security benchmark, finds it can move results by 12 to 23 points, but in a rival's favour: Cursor beat Codex on OpenAI's GPT-5.5 and Claude Code on Anthropic's Fable 5.0. The harness sellers outside the labs now sell routing between labs' models as the harness's job, and GitHub sells Copilot as a model subscription inside other agents. Six days before OpenAI's proposed November 12 cutoff, Cursor still lists no GPT-6 model. **The question**: Now that coding agents no longer compete on what they can do, does the market consolidate around whoever controls the model supply and the subscription - or does the harness stay a layer a buyer can swap without changing anything else? - **Status**: open · momentum rising - **Opened**: 2026-07-28 · last worked 2026-11-06 - **Guide**: https://thebeat.dev/guide/positioning/, https://thebeat.dev/guide/channels/, https://thebeat.dev/guide/agent-rails/ Coding agents compete on which models a plan lets a developer reach and what each task costs. The harness, the agent program that wraps a model, reliably changes the bill. It sometimes changes the result too, but no measurement so far shows a lab's own harness getting more out of its own model than a rival's does. The cost half is settled by two independent studies on different models. HarnessTax, from UC Berkeley and Arena researchers (Melissa Pan, Ion Stoica, Matei Zaharia and others, released September 16), ran seven models through Claude Code, Codex and the minimal Pi harness, three attempts at each of 30 sampled tasks. On SWE-bench Lite, of 14 within-model comparisons with a Holm correction, eight cost gaps survived and one success-rate gap did: Claude Opus 4.8 scored 6.7 points higher in Codex than in Pi. GPT-5.6 Sol cost $1.54 per attempt in Claude Code and $0.44 in Pi, with its success rate within a few points. On October 3 Yangze Liu and Zhongyi Han posted "What Does a Harness Buy? Tokens, Mostly" (arXiv 2610.04433). They ran five models, two of them served locally, through Claude Code, OpenCode and mini-SWE-agent on SWE-bench Verified. On the 45 hardest tasks, changing the harness flipped about as many outcomes (13%) as rerunning the same one did, and cost per task moved up to three times. The one gap that cleared the noise, OpenCode trailing by up to 9 points, came about half from runs cut off at the output limit, which Claude Code automatically continues (VentureBeat's reading of the paper). Both are small, and HarnessTax reprices token usage on one schedule rather than reporting anyone's bill. The result half is contested. Endor Labs, which sells code security and has no agent of its own, runs an Agent Security League: 200 vulnerability-fixing tasks from open-source Python projects, scored on whether the patch passes the project's tests and, separately, hidden security tests. It has run eight models through two harnesses each. On the functional score, GPT-5.5 reached 84.9% in Cursor and 62.0% in OpenAI's own Codex, both run on April 25. Claude Fable 5.0 reached 71.5% in Cursor and 59.2% in Anthropic's Claude Code, two days apart in June. On Claude Opus 4.7 and 4.8 the two harnesses were within a few points. Endor's reading is that "the agent harness matters as much as model capability". It does not say how many attempts each task got, the models are from the spring, and the tasks are security fixes rather than issue resolution, so it does not overturn the two studies. It does show that where the harness moved the result, the model's own lab lost. Fan and colleagues' study of harness design (arXiv 2609.20804, 176 settings, four models) offers one reconciliation: planning mostly buys accuracy for weaker models and mostly saves cost for stronger ones. The sellers price as if the cost half is what counts. Three of the biggest harness sellers outside the labs now sell routing between labs' models as the harness's job, and each pitches it on cost. Cursor Router (July 22) classifies each request and sends it to a frontier or a cheaper model, with "Grok 4.5 required as a price-efficient routing option", its owner's model. Cognition's Devin Fusion pairs "a frontier lead model with a cost-efficient sidekick model", the default sidekick being Cognition's own SWE-2, and bills each at its own rate. GitHub made Copilot Fusion generally available on every Copilot plan on October 28, billing each model it picks at its standard rate, with 30% off for a limited time. GitHub says Fusion's Max setting scored 60 on Artificial Analysis' Coding Agent Index, within two points of Claude Code running Opus 5.5, at about half the API cost per task. That is GitHub's reading of an index that has changed versions five times since May. GitHub has also started selling Copilot as a model subscription inside other people's agents. Since October 23 its usage API counts Copilot model traffic by the tool that sent it, naming Claude Code, Codex, OpenCode and Zed beside its own clients. On November 3 Copilot became an agent in JetBrains Air, with cloud tasks billed to the developer's Copilot plan rather than JetBrains' own AI credits, and on November 4 it arrived inside Xcode's built-in coding assistant on every plan, in public preview. The same week its usage API began separating the developer's own keys and local models from what Copilot serves. And on October 28 GitHub Enterprise began including Copilot for every licensed user buying through an account team, with no monthly AI credits, so every model call is metered from the first request. Developers do move, and they say they move for better work. Stack Overflow's 2026 Developer Survey asked whether respondents had switched from one AI tool to another for work in the past year, and 65.7% of 13,144 said yes. The most common primary reason was better output quality (25.1%), then an employer requiring or recommending a different tool (10.6%), then a better fit with their workflow or editor (7.4%). Lower cost came to 6.4%, and access to a preferred model or provider to 2.1%. The question covers AI tools at work rather than coding agents alone, and it takes one reason per person. Read beside the studies, it says switching for better output is mostly switching for a model. That reading is this thread's, not the survey's. A correction this thread made on October 20 stands: the September signal and this thread's first version gave HarnessTax's Sol figures as cost per resolved task and said no quality gap survived correction across 42 comparisons. The study's published data gives cost per attempt, and 20 within-model comparisons, one of which survives. ## What they compete on **The model supply.** SpaceX closed its $60 billion all-stock acquisition of Cursor's maker, Anysphere, on August 14 by Cursor's own post, putting it in a SpaceXAI division beside xAI. Cursor's own stated reason was access to "the largest fleet of GPUs in the world". Two weeks later OpenAI said it would wind down the contract letting Cursor serve OpenAI models. It proposed a November 12 shutoff, the longest notice the contract allows, on the stated ground that it cannot trust SpaceX to honour its terms. Cursor may end it sooner, and OpenAI will supply no new models in the meantime (Engadget's reading of the announcement). That second half is visible. GPT-6 Astra has been generally available in GitHub Copilot since September 4, and OpenAI has since published GPT-6.1 Terra, but on November 6 Cursor's model list still ended at GPT-5.6, with no GPT-6 model of any kind and no end-of-life note. Users keep other routes to the same models, their own API keys and OpenAI's own extensions, but a widely used agent is a model generation behind on one supplier because of who owns it. One thing still does not reconcile: the same Cursor page quotes a promotional price for GPT-5.6 Sol "through November 21, 2026", nine days past the proposed end. **What the plan includes.** On October 12 xAI launched The X Subscription: one plan, five tiers from Free to a $200 Ultra, spanning Grok, Grok Bot, Cursor and X Premium. "Every tier includes a pool of AI usage shared across Cursor and Grok", in Cursor's words. xAI pitches the $30 tier as SuperGrok, Cursor Pro and X Premium+ that cost "$90/month combined", xAI's own figure. Cursor's pricing page sells that ladder, "only sold directly through X Subscription Corp.", and says plainly what the bundle is for: on Plus, "Only Grok models are available on this plan", and the higher tiers "include generous usage of Grok models but do not include an allowance for third-party models", which are billed at API list price. Three weeks in, the bundle is under cost strain. Many Grok Bot subscribers reported running through their weekly limits four or more days early in late October, and on November 1 Elon Musk said serving Grok Bot was "unsustainably high relative to revenue" (Crypto Briefing, which does not say which tiers). The $30, $100 and $200 tiers include Cursor; no report so far says Cursor's coding usage was cut. Bundling a coding agent with things that are not coding is not xAI's invention: Google's AI Pro plan carries Antigravity beside 5 TB of storage and YouTube Premium Lite. Google is also winding down Jules, its asynchronous coding agent: read-only after November 11, shut down on December 31, with Google naming Antigravity as its agent-first platform, and the plan page no longer mentions it. What xAI did first is fold an independent agent, one that had its own price page, into a parent's subscription. **The menu as the product.** GitHub sells Copilot as an agent and runs it as a host for every lab's models: xAI's Grok 4.7 the week of September 21, Claude Sonnet 5.5 a week later, Gemini 4 Argon on the day it became generally available, October 13. Copilot Fusion grew out of HydraFusion, an orchestration project GitHub opened in VS Code on September 30, whose reviewer was drawn from "a different model family". GitHub reported that in offline tests its best-tuned HydraFusion configuration scored 4.9 points above Claude Opus 5 on TerminalBench 2.1 at 67% lower estimated cost, GitHub's own numbers on benchmark revisions it chose. Cognition makes the same pitch from the other side of the market, calling itself "an independent agent lab" that combines models "rather than tie customers to one provider". It raised more than $2 billion at a $48 billion valuation on September 8, on run-rate revenue it puts at almost $900 million, up from $492 million in May (company-reported). **The surface and the perimeter.** OpenAI repositioned Codex on August 21 as an open agent platform, a harness pitched for embedding in someone else's software. Cursor built the forge instead: Origin, a git host in early beta since August 17, with an app marketplace of more than 20 apps opened October 13; its pricing page includes Origin from the $30 tier up, not in the $8 Plus. GitHub took its agent to where the request is made, Copilot's cloud agent in Slack and Microsoft Teams in public preview from August 21. The SpaceX side answered with Supabase's partnership, in which Grok Bot hands a backend change to Cursor as scoped work (October 14). For the regulated buyer, Coder's Agent Relay keeps an agent's reasoning in the vendor's cloud and its execution inside the customer's own workspace: Cursor was the first partner (September 4) and Claude Code followed on September 16, in early access with design partners. The IDE incumbent has chosen to sit under the agents rather than compete with them. JetBrains sells "Java and Kotlin intelligence inside your coding agent", an extension for Cursor, VS Code and Google Antigravity that needs an IntelliJ IDEA Ultimate subscription after a 30-day trial. Its own agentic IDE, Thinkrail (signal, October 26), is built around the third-party Pi harness, with "no accounts of its own", and its Air app now runs Copilot on the developer's Copilot plan. These hold the buyer who cannot switch on a whim, but none of them changes which model does the work. ## The tension The swap layers keep arriving. Vercel's AI SDK put GitHub Copilot behind the same `HarnessAgent` interface as Claude Code and Codex on September 10, beside Cursor, OpenCode, Grok Build, Pi and others, so a team changes the agent without touching its own tooling. Spotify opened Xirp, a vendor-neutral environment running Claude Code, Codex and Gemini CLI side by side, after more than 36,000 internal sessions (its own count). DeepSeek shipped an open-source harness into the same category. Ramp skipped the market: its in-house agent, Inspect, authors 75% of merged pull requests by Ramp's figures, via a reported Pragmatic Engineer piece, not audited. The best evidence against all this is Endor's league and who leads. If the harness can move a result by 23 points, choosing one is not a cost-only decision, and Harness-Bench (arXiv 2605.27922, 13 authors, revised October 15) agrees from general workflow tasks: it reports "substantial variation" across model and harness pairings and argues capability should be reported per configuration rather than per model. In Stack Overflow's survey the coding agents with the most use are Claude Code (66%) and GitHub Copilot (59%), a lab's own harness first. Neither carries the day yet. Every large same-model gap found so far favoured a rival's harness over the model's own lab, which is the opposite of a moat. The lab that leads in agents also makes the leading model, so its rise fits "follow the model" as well as "the harness wins". And Harness-Bench's abstract gives no effect sizes. ## Where this started The reading in early August was that the coding agent had become a commodity. Three terminal agents launched inside forty-eight hours - Meta's Muse Code, Memcode, Clark Code - and none said "ours writes better code": Meta competed on a price structured as a data trade, Memcode on repository memory, Clark on an Apache-2.0 licence. A fourth arrival, Mirafold, sold a browser UI over Claude Code, Codex and Gemini CLI rather than an agent, and a policy layer, Kastra, had launched in late July covering Claude Code, Cursor and Codex at once. The entrants were right that the harness alone would not win. What the survey and the studies add is that the developer still chooses on output, and gets it mostly from the model, and the harness sellers' answer has been to choose the model for them. ## What would settle it The November 12 shutoff completes the first test of supply as a weapon. Cursor's model menu the day after, and whether a GPT-6 model ever reaches it, says whether a model owner can take its models out of a rival's agent or whether the agent kept enough leverage to negotiate. The second test is a current frontier model run through its own lab's harness and a rival's, several attempts each: if the lab's harness wins beyond noise, the harness is a moat after all. The third is a switching number for coding agents specifically, with the agent left and the agent chosen. This thread follows the head-to-head between the agents; the price unit is left to the pricing thread. ## Open loops - Does OpenAI's wind-down of Cursor's model access land on November 12 as proposed, and does Cursor's model menu still list GPT-5.6 the day after? On November 6 Cursor's model docs still listed every GPT-5.6 model, with Sol's promotional price running to November 21, past the proposed end, and no GPT-6 model at all. A shutoff on schedule completes the first case of a model owner using supply as a competitive lever against a rival harness; an extension or a quiet reversal says the harness still holds leverage over the lab. *(by November 12, 2026)* - A switching number for coding agents specifically - which agent developers left and for which, rather than Stack Overflow's all-AI-tools question with one reason each. JetBrains' Developer Ecosystem research and the next Stack Overflow survey are where it would appear; until then the swap is shown to be common but not shown to follow the model. *(by end of 2026)* - Does a current frontier model, Opus 5.5 or GPT-6 Astra, still score differently in a rival's harness than in its own lab's? Endor Labs' Agent Security League, where the 12 to 23 point gaps came from, has not run one model through two harnesses since June. A new pair on its league, or on Artificial Analysis' Coding Agent Index, says whether the gaps hold for this generation and which way they point. *(by January 31, 2027)* ## Sources - [Liu and Han - What Does a Harness Buy? Tokens, Mostly (arXiv 2610.04433)](https://arxiv.org/abs/2610.04433) - [VentureBeat - A coding agent's harness can triple its cost without making it more accurate, study finds](https://venturebeat.com/orchestration/a-coding-agents-harness-can-triple-its-cost-without-making-it-more-accurate-study-finds) - [HarnessTax - How Much Does the Harness Matter for Coding Agents?](https://harnesstax.github.io/) - [HarnessTax - the study's published data and text](https://github.com/HarnessTax/HarnessTax.github.io) - [Endor Labs - Agent Security League](https://www.endorlabs.com/research/ai-code-security-benchmark) - [Endor Labs - GPT-5.5 sets a new code security record with Cursor, not Codex](https://www.endorlabs.com/learn/gpt-5-5-sets-a-new-code-security-record-with-cursor-not-codex-in-agent-security-league) - [Endor Labs - Claude Fable 5, take two: same model, different harness](https://www.endorlabs.com/learn/claude-fable-5-take-two-same-model-different-harness-and-a-very-different-result) - [Fan et al. - An Empirical Study of Harness Design for Coding Agents (arXiv 2609.20804)](https://arxiv.org/abs/2609.20804) - [Harness-Bench: Evaluating Agent Harnesses across Models and Realistic Workflows (arXiv 2605.27922)](https://arxiv.org/abs/2605.27922) - [Stack Overflow Developer Survey 2026 - Switching AI tools](https://survey.stackoverflow.co/2026/ai/data/ai-tool-switch) - [Stack Overflow Developer Survey 2026 - AI](https://survey.stackoverflow.co/2026/ai) - [Cursor - Models](https://cursor.com/docs/models) - [GitHub changelog - GPT-6 Astra is generally available in GitHub Copilot](https://github.blog/changelog/2026-09-04-gpt-6-astra-is-generally-available-in-github-copilot) - [Cursor - Pricing](https://cursor.com/pricing) - [Cursor - Cursor is now a part of SpaceX](https://cursor.com/blog/joining-spacex) - [OpenAI - Our decision on Cursor following its acquisition by SpaceX](https://openai.com/index/our-decision-on-cursor-following-its-acquisition-by-spacex) - [Engadget - OpenAI will pull its models from Cursor due to SpaceXAI acquisition](https://www.engadget.com/2246969/openai-pull-its-models-from-cursor-due-to-spacexai-acquisition/) - [xAI - Introducing The X Subscription](https://x.ai/news/x-subscription) - [Crypto Briefing - Musk says Grok Bot usage limits were cut because costs ran too high](https://cryptobriefing.com/musk-grok-bot-usage-limits-high-costs/) - [Google One - Google AI plans](https://one.google.com/about/google-ai-plans/) - [Jules - Changelog](https://jules.google/docs/changelog/) - [Cursor changelog - Cursor Router](https://cursor.com/changelog/router) - [Devin docs - Fusion](https://docs.devin.ai/cli/fusion.md) - [GitHub changelog - Route and orchestrate models with GitHub Copilot Fusion](https://github.blog/changelog/2026-10-28-route-and-orchestrate-models-with-github-copilot-fusion) - [GitHub - Project HydraFusion: Frontier quality via multi-model orchestration](https://github.blog/ai-and-ml/github-copilot/project-hydrafusion-frontier-quality-via-multi-model-orchestration/) - [GitHub changelog - HydraFusion in VS Code and the GitHub Copilot app](https://github.blog/changelog/2026-09-30-hydrafusion-in-vs-code-and-the-github-copilot-app) - [GitHub changelog - GitHub Enterprise now includes access to GitHub Copilot](https://github.blog/changelog/2026-10-28-github-enterprise-now-includes-access-to-github-copilot) - [GitHub changelog - Usage metrics API adds usage from tools like OpenCode](https://github.blog/changelog/2026-10-23-usage-metrics-api-adds-usage-from-tools-like-opencode) - [GitHub changelog - GitHub Copilot is now available in JetBrains Air](https://github.blog/changelog/2026-11-03-github-copilot-is-now-available-in-jetbrains-air) - [GitHub changelog - GitHub Copilot is now available in Xcode's coding assistant](https://github.blog/changelog/2026-11-04-github-copilot-is-now-available-in-xcodes-coding-assistant) - [GitHub changelog - BYOK and enterprise custom models in the Copilot usage API](https://github.blog/changelog/2026-11-03-byok-and-enterprise-custom-models-in-the-copilot-usage-api) - [JetBrains - Java and Kotlin by IntelliJ IDEA](https://www.jetbrains.com/idea/extension/) - [JetBrains - Thinkrail on GitHub](https://github.com/JetBrains/thinkrail) - [OpenAI - Codex as a platform: build on the open agent harness](https://developers.openai.com/blog/codex-as-a-platform) - [Vercel changelog - GitHub Copilot is now available in the AI SDK harness layer](https://vercel.com/changelog/github-copilot-ai-sdk-harness-adapter) - [Cognition - Do it all with Devin: announcing our Series E](https://cognition.com/blog/series-e) ## Related - [Guide - Positioning for developers](https://thebeat.dev/guide/positioning/) - [Guide - Channels & distribution](https://thebeat.dev/guide/channels/) - [Guide - Agent rails](https://thebeat.dev/guide/agent-rails/) --- Part of [The Beat - developer marketing, on the record](https://thebeat.dev/). Content license: CC BY 4.0 - quote it, link the canonical page.