Coding agents and harnesses, compared

Start from the framing that makes this category legible: agent = model + harness. The model supplies raw capability. The harness supplies everything that turns capability into finished work — the tool loop, the context strategy, the permission gate, the retry-on-failing-test behaviour, the thing that stops it wandering off after twenty minutes. Hand the same frontier model to two harnesses and you get wildly different completion rates on the same repo. That is why a benchmark number only earns a place on this page when the harness, the model and the benchmark version are all named — and why the Terminal-Bench column below is currently empty rather than populated with the model scores everyone else reprints. A harness score with no harness attached is a marketing artefact, not a measurement.

The field that actually separates these tools is where does it stop? We score it 1–5 on a fixed ladder: suggests → edits files → runs tests → opens PR → merges and deploys. Almost every comparison stops at feature checklists and never asks how far down that ladder the tool goes without a human. The answer is sobering, and it is narrower than the ladder suggests: every scored harness here sits at rung 3 or rung 4, so in practice the column separates "stops at your working tree" from "reaches a pull request" and nothing finer. Claude Code, Codex CLI, opencode, Cursor, Devin and GitHub Copilot reach rung 4 — a branch pushed and a pull request opened through a path the vendor ships. Nothing on this page ships at rung 5. Copilot's coding agent cannot approve its own PR by design; Devin's ceiling is a review request. Anyone selling "autonomous deployment" is describing a CI pipeline you wired yourself, sitting downstream of a rung-4 agent.

There are two camps and they are drifting apart. The vendor-coupled harnesses — Claude Code, Codex CLI, Gemini CLI, Devin, Grok Build — are built against one model family's quirks, so they ship agentic features first: subagents, hooks, sandboxing, cloud runners. Amp belongs beside them for the same practical reason with a different mechanism: it is not coupled to one vendor so much as opaque about which it uses, because there is no model picker and multiple vendors sit behind it. The provider-agnostic ones — opencode, Cline, Goose, Aider, Zed, Qwen Code, Kilo Code — trail on features by a quarter or two but survive vendor pricing changes, run local models, and never leave you renegotiating your tooling because someone re-tiered a subscription. If your team is one procurement decision away from switching model vendors, take the agnostic one and accept the lag — opencode in particular now reaches a pull request through its own GitHub App, so the feature gap is narrower than it was.

The pricing hides more than the features do. Three models are in play: BYOK tokens (you pay the provider, the harness is free — opencode, Aider, Goose, Zed, Cline, Qwen Code), a flat subscription with soft limits (Claude Code on Pro or Max, Codex on ChatGPT plans), and credits (Windsurf, Devin, Amp). Credits are the one to watch. A credit is an abstraction over tokens whose conversion rate the vendor controls and can change, which makes month-to-month cost genuinely unforecastable for an agent that might burn ten times more context on a bad day. Copilot's premium requests are the same problem in different clothing. If you need a number you can put in a budget, a subscription or your own API key are the only two honest answers.

Expect this page to be wrong faster than any other on the site. Our intended cadence here is a full re-verification every fourteen days, and we are not currently meeting it: every dated cell below was last checked on 15 January 2026, so the table is roughly six months old and you should treat pricing and feature cells as leads to confirm rather than as current fact. We would rather show you the dates and the gap than quietly restamp them. In the eighteen months to that pass, Windsurf changed hands once — after an OpenAI acquisition collapsed and Google licensed the technology and hired the leadership, Cognition bought what remained — Aider's release cadence slowed to a trickle, GitHub rebuilt Copilot around a multi-agent control plane, and Grok Build appeared in July 2026 with essentially nothing verifiable published. pi.dev arrived after that pass and is not on the roster below; a row of dashes reads as a bad result rather than as absent evidence, so it stays in prose as one to watch until someone here has actually shipped code with it. Pick tools whose exit cost is low: a harness that reads AGENTS.md, takes your API key and leaves your repo unchanged costs you a weekend to replace. One that owns your credits, your index and your PR flow does not.

Rung 1 is suggests: it proposes a diff or a snippet and you apply it. Rung 2 is edits files: it writes directly to your working tree. Rung 3 is runs commands and tests: it executes the build and test loop and iterates on the failures it sees. Rung 4 is opens a pull request: it commits, pushes a branch and raises a PR through a first-class path. Rung 5 is merges and deploys: it lands work on the default branch or triggers a release with no human in the loop.

The interesting gap is between 3 and 4, and it is organisational rather than technical. Any harness with shell access can run gh pr create. What rung 4 actually requires is that the vendor has thought about identity — whose token pushes the branch, what CI runs, what happens on a failed check, whether the agent can respond to review comments. GitHub Copilot's coding agent is the clearest example: it runs in Actions, pushes under a bot identity, and is structurally barred from approving its own work. Devin and Cursor's background agents solve the same problem with a hosted VM and their own identity. Claude Code and opencode solve it with published GitHub Apps that answer a mention on an issue, which is why an MIT-licensed terminal harness and a proprietary one land on the same rung — the licence has nothing to do with it.

Note also how little of the ladder is actually in play. Nothing on this page scores 1 or 2: shipping a coding agent in 2026 means writing to the working tree, and the tools that only suggest diffs are autocomplete products in a different category. Nothing scores 5 either. So the killer field is really a two-value sort, and rung 3 hides a wide band inside it — Aider driving a test command and fixing what it breaks is doing more unattended work than an IDE agent that pauses for approval on every command, and this column will not tell them apart. Read it with the permissions and headless columns rather than alone.

The gap between 4 and 5 is nobody's to close unilaterally. Merging is a policy decision that lives in branch protection rules, not in a harness. Treat any tool claiming rung 5 as claiming to have been given credentials, and audit it accordingly.

The flat subscription is the most honest deal on this page, but read the limits. Every subscription-based harness enforces a rolling usage window, and the ceiling is expressed in vendor-specific units that do not map cleanly onto tokens. Heavy agentic use — long sessions, big repos, subagents — burns through those windows far faster than chat does, and the answer when you hit one is either wait or move to metered billing.

Credits are worse for planning. Windsurf, Devin and Amp all price in units the vendor defines. The unit economics can be perfectly fair and still be unforecastable, because the vendor can re-price the conversion between a credit and a token when the underlying model price changes, and you find out afterwards. Copilot's premium-request multipliers have the same shape. If someone in finance needs a number for next quarter, a per-seat subscription or your own provider key are the only two things you can defend.

BYOK looks cheapest and often is, but it moves the variance onto your API bill rather than removing it. A harness that re-reads a large file on every turn costs multiples of one that caches aggressively, and the difference shows up in your provider invoice rather than the tool's price page. If you are running BYOK at team scale, meter it per repo for a fortnight before you extrapolate, and check whether the harness supports prompt caching against your provider at all.

Three decision paths cover most teams. If the work is well-specified tickets on a repo with good tests, buy autonomy: Claude Code or Codex CLI in headless mode behind a PR, or Copilot's coding agent if you already live in GitHub and want the identity problem solved for you. Optimise for rung 4, background execution and a sandbox you trust.

If the work is exploratory or architectural, buy ergonomics instead. Cursor and Windsurf exist because reviewing a large agentic diff inside an editor with real navigation beats reading it in a terminal, and both maintain a persistent embeddings index that grep-based harnesses cannot fake on a million-line monorepo. They are not alone in that: Kilo Code indexes too, and GitHub Copilot and Devin read an index alongside live search, which is a richer substrate again. What Cursor has that the others do not is that index sitting inside the editor you are reviewing in. Zed is the interesting third option: a fast editor that hosts other agents over the Agent Client Protocol, so you keep your harness choice open while getting an editor's review surface.

If you are constrained by procurement, data residency or a fear of lock-in, take a provider-agnostic open-source harness. opencode if you want a modern TUI and the broadest provider coverage, Goose if MCP is already central to your stack, Cline if your team is VS Code-native, Aider if you want the smallest auditable thing that works. You will be a quarter behind on features. You will also be able to point every one of them at a different model provider on a Tuesday afternoon.

Whatever you pick, write AGENTS.md, pin a test command, and keep the sandbox on. Those three decisions outlast the tool.

Every cell here comes from vendor documentation, the tool's own repository, or hands-on use, and each carries its provenance. Where we are reading a pricing page we mark it vendor-claimed with the URL. Where we are inferring from adjacent behaviour, or making a judgement call like a score, we mark it inferred. Where the fact is widely reported by users but not documented, it is community. Where we do not know — and in a category moving this fast that is often — the cell is null and renders as a dash rather than a guess. Closed-source harnesses have null GitHub stars because no comparable repository exists, not because nobody stars them.

The verify cadence for this page is fourteen days, and the dated cells below currently sit well outside it: the last full pass was 15 January 2026. We publish the dates rather than refresh them, because a date is a claim about work someone did, and moving one without redoing the check is the cheapest lie a comparison site can tell. Read every date as the last time a human actually looked.

The killer field, where does it stop?, is scored on a fixed five-rung ladder and always at the highest rung the tool reaches out of the box: rung 4 requires a first-class path to a pull request — a built-in command, a vendor-published GitHub App with its own identity, or a hosted runner — not merely the ability to invoke gh because you asked it to run a shell command. Any harness with shell access can be scripted up a rung; that is a property of your scripts, not the product. A vendor-published Action you wire into your own workflows, with no App identity behind it, is graded partial rather than a full rung, and that rule is applied to first-party and third-party harnesses alike.

The Terminal-Bench column ships empty, and we would rather that than the alternative. The published figures we could trace either name a model and a benchmark version without naming a harness, or were produced by the benchmark's own reference agent — which makes them evidence about the model, not about anybody's harness, and importing them into a harness column is exactly the laundering this page exists to argue against. Terminal-Bench 1.0 results are also superseded by 2.0 and not comparable to it, and the gaps between the top entries were smaller than one task on an eighty-task benchmark, which is not a ranking. An empty column is the honest state of this evidence until we run the benchmark ourselves against pinned harness versions.