Lesson 16 / 29

Parallel Agents With Git Worktrees

Run several agents at once without them overwriting each other.

One working copy per agent

Two agents editing the same folder will overwrite each other's files and run each other's half-finished code. The standard fix is isolation: each agent gets its own working directory and branch. Git worktrees do this cheaply: git worktree add -b agent-a ../agent-a creates a second checkout of the same repository on a new branch, sharing history but with separate files. Cloud agents achieve the same with separate sandboxes. Run agents in parallel on independent tasks (different modules, separate bug fixes), review each branch separately, and merge them one at a time, re-running tests after each merge, since independent branches can still conflict in meaning. The limit is your review capacity, not the number of agents you can start.

Two agents, two worktrees, run

I ran this with plain Python 3 (standard library only), using a throwaway project created in a temporary folder. In a throwaway repository, two branches are created with separate working directories. Agent A adds a line, agent B rewrites the file, and main is untouched. git diff --shortstat shows each branch's change size. This uses real git in a temporary folder, never your own repository.

import subprocess, tempfile, os

def git(cwd, *args):
    return subprocess.run(["git", *args], cwd=cwd, capture_output=True, text=True,
                          env={**os.environ, "GIT_AUTHOR_NAME": "t", "GIT_AUTHOR_EMAIL": "t@example.com",
                               "GIT_COMMITTER_NAME": "t", "GIT_COMMITTER_EMAIL": "t@example.com"})

with tempfile.TemporaryDirectory() as base:                      # a throwaway repository, never your real one
    main = os.path.join(base, "repo"); os.makedirs(main)
    git(main, "init", "-q", "-b", "main")
    open(os.path.join(main, "app.py"), "w").write("print('v1')\n")
    git(main, "add", "."); git(main, "commit", "-q", "-m", "initial")

    for name in ("agent-a", "agent-b"):                           # one isolated worktree + branch per agent
        git(main, "worktree", "add", "-q", "-b", name, os.path.join(base, name))
    open(os.path.join(base, "agent-a", "app.py"), "w").write("print('v1')\nprint('feature A')\n")
    open(os.path.join(base, "agent-b", "app.py"), "w").write("print('fixed B')\n")

    print("main still has:", open(os.path.join(main, "app.py")).read().strip())
    for name in ("agent-a", "agent-b"):
        git(os.path.join(base, name), "add", "."); git(os.path.join(base, name), "commit", "-q", "-m", name)
        stat = git(main, "diff", "--shortstat", "main", name).stdout.strip()
        print(f"{name}: {stat}")
    print("branches:", sorted(b.strip("* ").strip() for b in git(main, "branch", "--format=%(refname:short)").stdout.split()))

Output:

main still has: print('v1')
agent-a: 1 file changed, 1 insertion(+)
agent-b: 1 file changed, 1 insertion(+), 1 deletion(-)
branches: ['agent-a', 'agent-b', 'main']

Merge one branch at a time

Re-run the tests after each merge; independent branches can still conflict in behaviour.

Quick check: What is the limiting factor for running many agents in parallel?

  • The colour of the terminal
  • The number of available branches
  • Your capacity to review their changes
  • Nothing at all
Answer

Your capacity to review their changes — Unreviewed changes are risk, however fast they were produced.