Findings
Measured on 2026-11-18 on macOS with jj 0.47.1, GitButler 0.23.4, Mergiraf 0.20.0 and Python 3.12. Agents were Claude Haiku subagents. Each setup ran once on a scenario built to collide, so these results show failure modes, not frequencies.
The cart scenario
cart.py has total(items) and item_count(items). Three agents start from the same base, none told about the others:
| Agent | Task | Collides with |
|---|---|---|
| A | rename total to subtotal | B (same function), C (C calls it) |
| B | add discount_pct=0 to total | A (same function); its formula makes the result a float |
| C | add receipt() calling total; its test expects "3 items, total 25" | A (name), B (the float prints 25.0) |
A correct combined result passes 4 tests. Two of the bugs are invisible to text merging: the rename merges cleanly while the caller still calls total, and the float causes no textual conflict at all.
Results
| Setup | Where the conflict surfaced | Combined tests |
|---|---|---|
plain git worktree | merge stops at B; C cannot merge until a human resolves it | same code as the Mergiraf row |
| git + Mergiraf | stops at B; Mergiraf resolves the rename and body change | 2/4 |
| jj workspaces | stored as a 3-sided conflict in the merge commit, nothing blocks | B+C alone fails on 25.0 |
| jj + Mergiraf | jj merge tools take 2 sides, so merges go pairwise | A+B already 1/3 failing |
| GitButler, one shared directory | hunks of different agents fuse while editing; 0/3 agents committed | 2/4, B's code never landed |
GitButler + but worktree new | commits marked conflicted; refuses to land a conflicted commit | 2/4, landed B with failing tests |
| jj + claims and notes (v1) | B stacked on A, C stacked on A | 3/4: rename caught, float missed |
| jj + concord board + gate | A, B, C serialized on total; C resolved its own merge | 4/4 on the first gate run |
What each layer is worth
- A combined-test gate before landing is the only thing that detected every semantic bug. Every setup without it landed broken code.
- Coordination prevents semantic bugs instead of only detecting them. The first version missed the float because C finished before B announced it. A
planbarrier (all intents known before work) and a blockingfinishthat returns exit 10 fixed it: C switched tosubtotaland formatted the float. - Structural merging with Mergiraf resolves conflicts inside one function. It does not resolve two top-level functions appended at the same spot, in git or jj.
- jj keeps merges and rebases from blocking and makes undo cheap. The difference between jj and git was merge flow, not correctness.
Known gaps
- B's note said "returns int|float … a number" instead of "now always float". C caught the float by reading code, not from the note. Free-text notes are too weak; structured contracts are an open decision.
- The claim key is the original name: B's
done subtotalwas rejected. Keys must follow renames. - The board serializes agents that touch the same function, so this scenario shows correctness, not speedup.
Dogfood: concord developing concord
Three Claude Sonnet subagents, one workspace each, briefed only with the concord skill, their task and their scope, built three features of concord itself. Two of them planned to modify the same function: the second waited, claimed after the first one's done, and got its change merged in before editing. The gate passed on the first run with clippy clean and 25 tests passing.
The run showed two gaps in the skill, both now stated in it: an agent first planned modifies=- for a function whose body it changed, and edited before claiming. Docs are not functions: all three would have edited the design doc in parallel, so the orchestrator kept docs to itself and edited them after the gate.
On an existing git repository
Before landing was designed in, a git user saw a detached HEAD, a branch that did not move, an untracked file committed into the agents' base, and an index out of step with HEAD. The fixes: agents start from the last commit, the main workspace tracks files like git, and the gate tests in a scratch child and lands like a pull or restores everything on FAIL. experiments/git-landing/run.sh checks both paths.
Public benchmarks: merge tool A/B
ConflictBench: 159 real Java merge scenarios. A pass means the merged files equal the developer's resolution, text for text.
| Gate merge tools | Pass | Conflict | Wrong |
|---|---|---|---|
plain git merge-file (2 cases error) | 0 | 157 | 0 |
| Mergiraf | 27 | 106 | 26 |
| Mergiraf, then Weave | 37 | 85 | 37 |
Weave turned 21 conflicts into merges: 10 equal the developer's resolution and 11 differ. Of those 11, 3 were equivalent, 3 kept both sides harmlessly, and 5 were broken, including code that does not compile and a dropped parameter. The rule set before the run was to keep Weave only if passes rose without wrong merges rising. It failed, so the gate uses Mergiraf alone. "Wrong" means text inequality, so some wrong merges are fine code; the column compares tools, it does not count bugs.