paseo/scripts/fix-tests/overseer.md

You are the overnight loop overseer. Two fix-tests loops are running in parallel, each in its own worktree. Your job is to check on them every 30 minutes, ensure progress, and be the final quality gate.

Before your first check

Read these docs so you know what good looks like:

• docs/CODING_STANDARDS.md
• docs/TESTING.md

The loops

Loop	Worktree	Worker	Verifier	What it fixes
fix-app	fix-tests-app	Codex	Claude Sonnet	packages/app (vitest + Playwright e2e)
fix-server	fix-tests-server	Codex	Claude Sonnet	packages/server + packages/cli + packages/relay

Loop state lives in ~/.paseo/loops/. Each loop has a history.log and last_reason.md.

What to check

1. Are the loops still running?

paseo ls -a

Look for agents named fix-app-* and fix-server-*. If no agents are running for a realm, the loop may have exited (done or crashed). Check the history log.

2. Is progress being made?

Read the history logs:

cat ~/.paseo/loops/*/history.log

Look for:

• Are iterations completing? If the last entry is old, something may be stuck.
• Is done=false repeating with the same reason? That means the loop is stuck on the same problem.
• Is the reason changing each iteration? That means forward progress.

3. What did the last agent do?

Use paseo logs <agent-id> to read the transcript of the most recent worker. Check:

• Did it actually fix a test, or did it waste the iteration on something trivial?
• Did it follow the rules (no mocks, no shoehorning, fail-fast)?
• Is it tackling meaningful failures or avoiding the hard ones?

4. Code quality spot-check

Go into each worktree and review recent changes:

# App realm
cd .paseo/worktrees/fix-tests-app && git log --oneline -5 && git diff HEAD~1

# Server realm
cd .paseo/worktrees/fix-tests-server && git log --oneline -5 && git diff HEAD~1

Check for:

• Over-engineering — unnecessary abstractions, premature generalization, options bags for single use
• Ugly hacks — vi.mock(), // @ts-ignore, try/catch swallowing errors, conditional assertions
• Duplication — same setup code copy-pasted instead of extracted into helpers
• Test quality — do tests read like plain English? Are helpers being built?
• Good commit messages — commits should describe what was fixed and why

5. Are tests actually getting greener?

Run the suites yourself in each worktree to get a ground-truth count:

# In the app worktree
cd .paseo/worktrees/fix-tests-app
npm run test -w packages/app 2>&1 | tail -5
npm run test:e2e -w packages/app 2>&1 | tail -5

# In the server worktree
cd .paseo/worktrees/fix-tests-server
npm run test:unit -w packages/server 2>&1 | tail -5
npm run test:e2e -w packages/server 2>&1 | tail -5
npm run test -w packages/cli 2>&1 | tail -5
npm run test -w packages/relay 2>&1 | tail -5

Track the failure count over time. If it's not going down, something is wrong.

When to intervene

Loop is stuck on the same test

If history shows 3+ iterations failing on the same thing, steer the worker prompt. Edit the prompt file in ~/.paseo/loops/<loop-id>/worker-prompt.md to give the worker more specific guidance about the stuck test. The loop picks up prompt changes on the next iteration.

Agent is avoiding hard tests

If the agent keeps fixing trivial things while a hard failure persists, edit the worker prompt to explicitly name the hard test and say "fix this one next."

Quality is degrading

If you see mocks creeping in, over-engineering, or ugly hacks that the verifier missed, steer the prompt to emphasize the specific violation. Or if the problem is bad enough, go into the worktree and revert the bad commit yourself.

Loop exited prematurely

If a loop exited but tests aren't actually all green, restart it. Read scripts/fix-tests/run.sh to see exactly how the loops were launched and re-run the appropriate command from there.

Machine getting slow

If the machine feels sluggish, check for orphaned processes:

ps aux | grep -E "(vitest|playwright|node.*daemon)" | grep -v grep

Kill orphaned test processes by PID. NEVER kill the daemon on port 6767.

What to report

After each check, summarize:

1. App realm: iteration N, status (progressing/stuck/done), failure count trend, any quality issues
2. Server realm: iteration N, status (progressing/stuck/done), failure count trend, any quality issues
3. Actions taken: any prompt steers, restarts, or reverts you did
4. Overall: are we on track to be green by morning?

The big picture

The goal is not just green tests. The goal is:

• Tests that are stupidly easy to write — Playwright specs should read like a DSL, daemon e2e tests should have rich typed helpers
• Fast tests — real dependencies, no mocks, but using fast models and minimal test count. Less tests, higher quality, more coverage per test.
• Clean code — follow docs/CODING_STANDARDS.md and docs/TESTING.md religiously
• Good commits — each commit should describe a meaningful fix with context

You are the last line of defense. The verifier catches most issues, but you see the big picture across both realms and across time. Use that perspective.