paseo/plan.md

# Plan

## Context

Build a new Codex MCP provider side‑by‑side with the existing Codex SDK provider. The new provider lives in `packages/server/src/server/agent/providers/codex-mcp-agent.ts` and is selected via a new provider id (e.g. `codex-mcp`). All testing is **E2E only** (no mocks/fakes). Use `/Users/moboudra/dev/voice-dev/.tmp/happy-cli/src/codex/` as reference for MCP + elicitation.

## CRITICAL RULES - READ BEFORE EVERY TASK

1. **NO VAGUE REPORTS**: Never say "test hung", "was interrupted", "failed locally" without:

   - The EXACT error message or stack trace
   - The SPECIFIC line of code causing the issue
   - A concrete hypothesis for the root cause

2. **NO SKIPPING/DISABLING TESTS**: Skipping tests, adding `.skip`, or "opt-in gating" is **NOT ACCEPTABLE**. Fix the actual problem. If a test hangs, find out WHY and fix the code, not the test.

3. **NO WORKAROUNDS**: Adding timeouts, fallbacks, or "defensive" code that hides bugs is forbidden. The code must work correctly, not appear to work.

4. **INVESTIGATE DEEPLY**: When something fails:

   - Read the actual source code
   - Add debug logging if needed
   - Trace the exact execution path
   - Find the ROOT CAUSE, not symptoms

5. **BE SPECIFIC**: Every "Done" entry must include:
   - What the actual problem was (specific)
   - What code was changed (file:line)
   - How you verified it works

## Completed Work (Compacted)

### Codex MCP Provider (2025-12-24)

- ✅ Created `codex-mcp-agent.ts` with MCP stdio client, event mapping, permissions, persistence, abort handling
- ✅ Fixed model availability (removed hardcoded gpt-4.1), permission elicitation, exit code handling
- ✅ Fixed thread/item event mapping for file_change, mcp_tool_call, web_search, todo_list
- ✅ Fixed persistence to include conversationId metadata
- ✅ Resolved Codex MCP vs Codex SDK elicitation parity (CRITICAL FINDING: `codex exec` ignores approval events)
- ✅ All 14 Codex MCP unit tests pass; Codex is now the only provider (deprecated SDK provider removed)
- Reports: `CODEX_MCP_MISMATCH_REPORT.md`, `REPORT-codex-mcp-audit.md`

### UI/UX Fixes (2025-12-25)

- ✅ Removed duplicate "Codex MCP" option - now shows only "Codex"
- ✅ Fixed duplicate user/assistant messages (provider was emitting, but agent-manager already dispatches)
- ✅ Fixed Codex agent-control MCP parity with Claude (added MCP servers to Codex config)
- ✅ Fixed agent timestamp not updating on click without interaction

### DaemonClient Implementation (2025-12-25)

- ✅ Created `packages/server/src/server/test-utils/daemon-client.ts` (~550 lines)
- ✅ Created `packages/server/src/server/test-utils/daemon-test-context.ts`
- ✅ Created `packages/server/src/server/daemon.e2e.test.ts` (25 passing tests)
- Reports: `REPORT-daemon-client-design.md`, `REPORT-daemon-e2e-audit.md`, `REPORT-claude-permission-tests.md`

**DaemonClient API:**

- Connection: `connect()`, `close()`
- Agent lifecycle: `createAgent()`, `deleteAgent()`, `listAgents()`, `listPersistedAgents()`, `resumeAgent()`
- Agent interaction: `sendMessage()`, `cancelAgent()`, `setAgentMode()`, `initializeAgent()`, `clearAgentAttention()`
- Permissions: `respondToPermission()`, `waitForPermission()`
- Git: `getGitDiff()`, `getGitRepoInfo()`
- Files: `exploreFileSystem()`
- Models: `listProviderModels()`
- Waiting: `waitForAgentIdle()`
- Events: `on()`, `getMessageQueue()`, `clearMessageQueue()`

**E2E Test Coverage:**

- Basic flow (Codex + Claude): create agent, send message, verify response
- Permissions (Codex + Claude): approve/deny, permission_requested/resolved cycle
- Persistence: delete agent, resume from handle, verify conversation context
- Multi-agent: parent creates child via agent-control MCP
- Agent management: cancelAgent, setAgentMode, listAgents
- Timestamp: verify clicking agent doesn't update timestamp
- Git: diff (staged/unstaged/modified), repo info (branch, dirty state)
- Files: list directory, read file content
- Models: list Codex and Claude provider models
- Images: single and multiple image attachments to Claude

## Tasks

- [ ] **BUG (CRITICAL)**: Claude agent race condition in `forwardPromptEvents` causes garbled text.

  **ROOT CAUSE**: Race condition in `claude-agent.ts` when `stream()` calls overlap.

  **The Bug** (3 factors):
  1. **Fire-and-forget async** (`claude-agent.ts:480`): `this.forwardPromptEvents(...).catch(...)` NOT awaited
  2. **Shared mutable state** (`claude-agent.ts:368-369`): `streamedAssistantTextThisTurn` and `streamedReasoningThisTurn` are instance variables
  3. **Flag corruption**: Turn 2 resets flags (line 769-770) while Turn 1 is still reading them (line 824-825)

  **Why E2E Tests Pass**: Sequential messages, each awaited. App/agent-control sends rapid overlapping messages.

  **TDD REQUIREMENTS**:
  1. **TEST FIRST**: Write E2E test in `daemon.e2e.test.ts`:
     - Create Claude agent
     - Send message 1 (long prompt like "Write a 500 word essay")
     - IMMEDIATELY send message 2 WITHOUT waiting for message 1 (this should interrupt)
     - Capture `assistant_message` chunks from message 2
     - Assert: chunks are coherent, no garbled/missing text
     - This test MUST FAIL before the fix
  2. **FIX**: Move flags from instance vars to local vars in `forwardPromptEvents()`:
     - Delete lines 368-369 (instance vars)
     - Add local vars at start of `forwardPromptEvents()` (line ~769)
     - Pass flags through to suppression logic (line 824-825)
  3. **VERIFY**: Test passes, typecheck passes, manual verification in app

  **Files**: `packages/server/src/server/agent/providers/claude-agent.ts:368-369, 480, 769-770, 824-825, 1123, 1138, 1149`

- [x] **BUG (MCP)**: `send_agent_prompt` errors when agent already running.
  - **Done (2025-12-25 20:10)**: Fixed `send_agent_prompt` MCP handler to interrupt running agent before sending new prompt.

  **WHAT**:
  - Modified `packages/server/src/server/agent/mcp-server.ts:418-463`
  - Added check for `snapshot.lifecycle === "running" || snapshot.pendingRun` at start of `send_agent_prompt` handler
  - If running: calls `agentManager.cancelAgentRun(agentId)` to interrupt
  - Added polling wait (max 5s, 50ms interval) for agent to become idle after cancellation
  - Matches behavior of `session.ts:interruptAgentIfRunning()`

  **WHY**:
  - The error `"Agent {id} already has an active run"` came from `agent-manager.ts:454` in `streamAgent()`
  - The MCP handler was calling `startAgentRun` without checking/cancelling existing runs
  - `cancelAgentRun` only initiates cancellation (fires and forgets), doesn't wait for `pendingRun` to clear
  - Polling wait ensures generator fully terminates before starting new run

  **TEST**:
  - Added E2E test `packages/server/src/server/agent/agent-mcp.e2e.test.ts`: "send_agent_prompt interrupts running agent and processes new message"
  - Test creates agent, sends prompt in background mode, then sends second prompt while first is running
  - Verifies no "already has an active run" error is returned

  **VERIFICATION**:
  - Test passes: `npx vitest run packages/server/src/server/agent/agent-mcp.e2e.test.ts --testNamePattern "send_agent_prompt interrupts"`
  - Server typecheck passes: `npm run typecheck` (server package)
  - Unit tests pass: `npx vitest run src/server/agent/mcp-server.test.ts`

- [x] **BUG (Server)**: Claude streaming sends incomplete chunks to long-running agents.
  - **Done (2025-12-25 22:15)**: Investigated with E2E test for long-running agents. **Server-side streaming is NOT the cause.** All chunks from Claude SDK are correctly forwarded.

  **INVESTIGATION**:
  1. Created E2E test `daemon.e2e.test.ts:1883-2028` "streaming chunks remain coherent after multiple back-and-forth messages"
  2. Test creates Claude agent, sends 3 messages, verifies streaming chunks on 3rd message
  3. Added debug logging to `mapBlocksToTimeline` in `claude-agent.ts:1139-1140` to trace chunk flow
  4. All `text_delta` events from Claude SDK are received and forwarded
  5. Final `text` message is correctly suppressed to avoid duplicates

  **TEST OUTPUT**:
  - 12 chunks received, all coherent: "The elephant at the zoo celebrated its 42nd birthday..."
  - No missing chunks between server receipt and WebSocket send
  - Chunks like " celebrate" + "d its" are normal token boundaries, not corruption

  **CONCLUSION**: The server correctly forwards all streaming chunks from Claude SDK. The original bug observed at the client (`REPORT-garbled-text-bug.md`) must have a different cause:
  - Possible React Native WebSocket implementation differences
  - Possible transient network issue during original observation
  - Bug may have been fixed by subsequent changes

  **FILES**:
  - `packages/server/src/server/daemon.e2e.test.ts:1883-2028` - New E2E test added
  - `packages/server/src/server/agent/providers/claude-agent.ts:1134-1145` - Verified chunk handling

  **VERIFICATION**: `npx vitest run src/server/daemon.e2e.test.ts --testNamePattern "streaming chunks remain coherent"`

- [x] **BUG (App-side)**: Claude assistant text garbled in React Native app rendering.
  - **Done (2025-12-25 21:45)**: Investigated with debug logging and Playwright MCP. **App-side code is NOT the cause.** See `REPORT-garbled-text-bug.md` for full analysis.

  **INVESTIGATION SUMMARY**:
  1. Added debug logging to `appendAssistantMessage` - state transitions are correct
  2. Reproduced bug via Playwright MCP on `localhost:8081`
  3. Console logs show chunks received by client are already incomplete (e.g., " a" + "check error" instead of " a" + "type" + "check error")
  4. Client-side Zustand state updates work correctly - no race condition
  5. FlatList renders `item.text` directly - no manipulation

  **ROOT CAUSE**: The server is sending incomplete text chunks to the client. The E2E test passes because it tests NEW agent creation; the bug appears in LONG-RUNNING agents during streaming.

  **NEXT STEPS**: Investigate server-side Claude agent streaming (NOT app-side):
  - `packages/server/src/server/agent/providers/` - Claude agent streaming implementation
  - Add server-side logging to compare Claude API output vs what's sent to clients

- [x] **BUG**: Claude agent assistant text is garbled/corrupted during streaming.
  - **Done (2025-12-25 20:15)**: Added E2E test `daemon.e2e.test.ts:1760-1881` that verifies server-side streaming text integrity. **Test passes** - server sends clean, non-corrupted text chunks. The bug is NOT on the server side.

  **INVESTIGATION RESULT**:
  - Server-side data is clean (verified via E2E test)
  - Each `assistant_message` timeline event contains correct text delta
  - Text chunks are properly accumulated in order
  - Bug must be in React Native app rendering layer (`packages/app`)

  **App-side code reviewed (no obvious bugs found)**:
  - `packages/app/src/types/stream.ts:216-244`: `appendAssistantMessage` correctly appends to last assistant message
  - `packages/app/src/contexts/session-context.tsx:698-718`: Zustand functional updates for state management
  - `packages/app/src/components/agent-stream-view.tsx`: FlatList rendering of stream items

  **ALSO FIXED**: Removed duplicate `DropdownField` import in `create-agent-modal.tsx:66` that was causing build errors.

  **NEXT STEPS** (for follow-up task):
  1. Test in actual React Native app to observe garbled text reproduction
  2. Check for React concurrent rendering issues with rapid state updates
  3. Investigate FlatList virtualization edge cases
  4. Consider adding debug logging to `appendAssistantMessage` in production to capture actual state transitions

- [x] **REFACTOR**: Remove module-level `SESSION_HISTORY` Map from `codex-mcp-agent.ts`.
  - **Done (2025-12-25 19:28)**: Removed `SESSION_HISTORY` global Map and refactored to use instance-level `persistedHistory` field.

  **WHAT**:
  - Removed `SESSION_HISTORY` Map declaration (was at `codex-mcp-agent.ts:118`)
  - Constructor: now sets `historyPending = true` for resume instead of looking up `SESSION_HISTORY.get()` (`codex-mcp-agent.ts:2564-2565`)
  - `connect()`: simplified condition to always load from disk when resuming (`codex-mcp-agent.ts:2606-2608`)
  - `loadPersistedHistoryFromDisk()`: removed `SESSION_HISTORY.set()` line (`codex-mcp-agent.ts:2629-2639`)
  - `recordHistory()`: appends to `this.persistedHistory` instead of `SESSION_HISTORY` (`codex-mcp-agent.ts:3147-3152`)
  - `flushPendingHistory()`: appends to `this.persistedHistory` instead of `SESSION_HISTORY` (`codex-mcp-agent.ts:3155-3160`)

  **RESULT**: Typecheck passes. Persistence E2E test "persists session metadata and resumes with history" passes (8.3s). The test now exercises disk loading since there's no global Map to provide in-memory history. Net change: -12 lines.

  **NOTE**: Two flaky tests ("maps thread/item events..." and "captures tool call inputs/outputs...") also failed before this change - they depend on LLM choosing to call MCP tools/web search which is non-deterministic.

- [x] **REVIEW (App)**: New agent page (`/agent/new`) is missing features from old modal.
  - **Done (2025-12-25 23:45)**: Completed comprehensive review. See `REPORT-new-agent-page-review.md` for full analysis.

  **WHAT REVIEWED**:
  - `packages/app/src/app/agent/new.tsx` (560 lines) - New agent creation page
  - `packages/app/src/components/create-agent-modal.tsx` (~2700 lines) - Old modal with full features
  - `packages/app/src/components/home-footer.tsx` - Entry points for agent creation
  - `packages/app/src/components/agent-input-area.tsx` - Already has dictation + image support

  **KEY FINDINGS**:

  1. **CRITICAL BUG** (`new.tsx:216-218`): Image attachments silently fail - early return without error:
     ```typescript
     if (images && images.length > 0) { return; }  // BROKEN!
     ```
     Fix: Remove early return, include images in `createAgent()` call.

  2. **CRITICAL BUG** (`new.tsx:269-271`): Creation failures silently ignored:
     ```typescript
     if (payload.status === "agent_create_failed") {
       pendingRequestIdRef.current = null;
       return;  // No error shown to user!
     }
     ```
     Fix: Add `setErrorMessage(payload.error)`.

  3. **Missing error state**: No `errorMessage` state, no UI to display errors.

  4. **Missing loading state**: No `isLoading` state, button doesn't disable during creation.

  5. **Missing daemon offline handling**: No error shown when daemon is unavailable.

  6. **Missing Git Options Section** (~200 lines in old modal):
     - Base branch selection
     - Create new branch toggle + input
     - Create worktree toggle + input
     - Git validation errors
     - Dirty directory warnings

  7. **Dictation/Images already work**: `AgentInputArea` component has full `useDictation` integration and image picker - just needs image support wired in.

  8. **Dead code in home-footer.tsx**:
     - Line 25: `showCreateModal` state never set to `true`
     - Lines 206-209: `CreateAgentModal` rendered but never shown

  **IMPLEMENTATION PRIORITY**:
  | Priority | Item | Effort |
  |----------|------|--------|
  | P0 | Fix image attachments | 5 lines |
  | P0 | Fix creation failure display | 2 lines |
  | P0 | Add error message state | 20 lines |
  | P1 | Add loading state | 15 lines |
  | P1 | Add daemon offline handling | 20 lines |
  | P2 | Git Options Section | ~200 lines |

  **FOLLOW-UP TASKS NEEDED**:
  - [ ] **FIX (App)**: New agent page - fix image attachments (remove early return, wire images to createAgent)
  - [ ] **FIX (App)**: New agent page - add error/loading states and failure display
  - [ ] **FEATURE (App)**: New agent page - add Git Options Section
  - [ ] **CLEANUP (App)**: Remove dead CreateAgentModal code from home-footer.tsx