DIY harness

DIY harness — Your own loop over a model API with an agent SDK — the build-it-yourself reference point. The baseline: a few hundred lines over whichever agent SDK your provider ships — every vendor on this page has one, and this row deliberately endorses none of them — with the tools your codebase actually needs and none of the ones it does not. This is genuinely the right answer for narrow, repeated tasks — a migration bot, a test-backfill job — where a general harness spends most of its context rediscovering your conventions. It is the wrong answer for open-ended engineering, because you will spend a quarter rebuilding permissions, checkpointing and context management badly. Open source. We carry it as the do-it-yourself baseline, not as a vendor.

We track it in 1 comparison — Coding agents — so every claim below is a cell in a table you can open and check rather than an impression. Across those rosters it sits against 16 other tools, and what follows is where it visibly separates from them.

Where it wins.

• Hooks — Yes, which only 3 of 10 tools here manage.

Each of those is ranked against the whole roster on its category page, not against a hand-picked subset, so a first place here means first of everything we list.

Where it gives ground.

• Permissions — applies everything, the weakest of the 16 here. Claude Code records allowlist + OS sandbox.
• Plan mode — No. Claude Code records Yes.
• Checkpoint & undo — No. Claude Code records Yes.
• Skills / commands — No. Claude Code records Yes.

None of these disqualify it on their own. They are the fields to check against your own requirements before you commit, because they are the ones where a competitor genuinely does better.

Provenance. 20 of 26 tracked fields carry a value for DIY harness, and 0 of those cite a document you can open. Nothing here carries a verification date yet, which is itself a reason to check before committing. Every figure keeps its own provenance — measured by us, claimed by the vendor, inferred, or community-reported — and we would rather print a dash than a guess.

Its nearest neighbour in our data is Aider. DIY harness is ahead on Hooks (Yes against No) and Opens PRs (Partial against No). Aider takes Checkpoint & undo (Yes against No) and Plan mode (Partial against No). That pattern repeats across the rest of the roster — see DIY harness alternatives for the other rivals, each compared the same way.

Licence

Open source

Fields we track

20 of 26

Right for narrow, repeated jobs — a codemod runner, a test-backfill bot — where a general harness wastes most of its context rediscovering conventions you could have hardcoded. Wrong for open-ended engineering, where you will spend a quarter rebuilding permissions, checkpointing and context management worse than the incumbents.

Permissions

applies everything16th of 16

Inferred

Plan mode

○No16th of 16

Inferred

Checkpoint & undo

○No14th of 16

Inferred

Skills / commands

○No16th of 16

Inferred

Repo indexing

manual context only16th of 16

Inferred

Hooks

●Yes1st of 10

Inferred

BYOK

●Yes1st of 16

Inferred

Headless / CI

●Yes1st of 16

Inferred

DIY harness is the right call in these situations, each one drawn from a field we actually record:

• One repeated task with a known shape and a tight tool surface.
• Environments where no third-party harness can be installed.
• Understanding what the commercial harnesses are actually doing for you.
• Hooks is your binding constraint. DIY harness records Yes, which only 3 of the 10 tools in the Coding agents roster do. We define that field as user-defined commands fired on lifecycle events — before a tool call, after an edit, on session end — that can block or rewrite the action.

Reach for something else when any of the following is a requirement rather than a nice-to-have:

• Permissions. DIY harness records applies everything, the weakest of the 16 tools in the Coding agents roster. Claude Code records allowlist + OS sandbox on the same field. If that is a hard requirement rather than a preference, start elsewhere.
• Plan mode. DIY harness records No on the Coding agents table. Claude Code records Yes on the same field. If that is a hard requirement rather than a preference, start elsewhere.
• Checkpoint & undo. DIY harness records No on the Coding agents table. Claude Code records Yes on the same field. If that is a hard requirement rather than a preference, start elsewhere.
• Skills / commands. DIY harness records No on the Coding agents table. Claude Code records Yes on the same field. If that is a hard requirement rather than a preference, start elsewhere.
• Repo indexing. DIY harness records manual context only, the weakest of the 16 tools in the Coding agents roster. Devin records hybrid index + agentic on the same field. If that is a hard requirement rather than a preference, start elsewhere.

We publish this block because a comparison that only lists what a tool is good at is marketing. Every figure above sits on the same page as its source, and the field definitions are on the category tables if you want to check how we measured them.

What we don't know. 6 of the 26 fields we track for DIY harness are still blank: Where it stops, Licence, GitHub stars, Terminal-Bench, Bench model and First release. Those render as dashes rather than as zeroes or assumptions, because an empty cell and a bad cell are not the same thing and only one of them is honest. If you know any of these figures and can point at a document, tell us.