706ae3d9bb
* refactor(claude): derive subagents from the SDK task protocol Claude Code announces subagent identity and lifecycle on the SDK stream (task_started, task_updated, task_notification). Paseo consumed only task_notification and reconstructed the rest from sidechain frames, in two independent derivations that had drifted apart. Both paths now translate into one small observation vocabulary and fold it with one function, so identity and status are derived once. The fold is stateless: the task protocol is edge-triggered and the store's sticky merge already accumulates. - Replay links a subagent through the agent-<id>.meta.json sidecar, which carries the same Task tool_use id the live stream keys on. The agentId text scrape stays as a fallback for sessions recorded without it. - The sidechain tracker keeps mapping frames to timeline items but no longer owns the descriptor when the task protocol is active, detected from the session rather than from a version string. - Adds replay coverage, which did not exist. Subagent ids are unchanged, so no protocol, store, or client change. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(claude): keep non-subagent tasks out of the track, open the child timeline with its prompt Backgrounded shell commands announce as task_started with a tool_use_id but task_type local_bash and no subagent_type, so filtering on the id alone put `sleep 20` in the subagents track. Discriminate on task_type, falling back to subagent_type for releases that predate it. task_started also carries the prompt. Emitting it as the child's first timeline item means the pane no longer opens mid-conversation, and matches replay, where the sidechain transcript already begins with the prompt. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(claude): drop status for tasks that were never declared as subagents task_notification also fires for backgrounded shell commands and carries a tool_use_id. Routing status off that id readmitted exactly what the declaration filter rejected, as a descriptor with a status and no identity — rendered in the track as a nameless "Loading..." row. Status now routes only through the task_id map populated at declaration. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(app): name a subagent row after its task, not its type The descriptor carries the subagent type in title and the task in description. The client collapsed them and rendered only the type, so every child in a fan-out read "general-purpose". The task is what tells siblings apart, so it names the row and the type moves to the muted secondary line, which the track computed but never drew. Providers that report no task keep the previous behavior. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(claude): label the parent's Task card from the subagent declaration A backgrounded Task subagent emits no frames carrying parent_tool_use_id, so the sidechain tracker never runs for one and never supplies the sub_agent detail. Its card in the parent transcript rendered as a bare "Task". The declaration already carries the type and the task, so the card is built from it. For a foreground subagent the tracker still emits the same card enriched with the child's action log; the two collapse on the shared tool-call id, so that case is unchanged. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(subagents): surface a subagent's model, effort, and cost Adds model, effort and usage to the provider subagent descriptor, populated from observations the child itself reports. - model: read per assistant frame, live and on replay. Never snapshotted, since a provider can swap models mid-flight on overload or refusal fallback, and never inherited from the parent. - effort: replay only. The SDK's assistant message does not carry it and it can be silently downgraded for the chosen model, so a live value would be a guess. - usage: task_progress reports tokens and tool count once per tool use, and task_notification reports the terminal totals. This matters most for backgrounded subagents, which emit no sidechain frames and would otherwise show no sign of activity at all. Status becomes optional on the upsert event: a report that only carries a model or a cost says nothing about whether the child is still running, and must not revert a finished one. Usage merges rather than replaces, because duration only arrives at the end. Fields are optional on the wire, so an older daemon simply omits them and the client renders nothing rather than substituting the parent's settings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(claude): read subagent effort from hooks Effort appears nowhere on the SDK message stream — verified by scanning every message type at depth — and the level Paseo requests is not necessarily the one that runs, since a model that does not support it is silently downgraded. Hooks are the source. Every hook input carries agent_id and effort.level, the active level for that turn after any downgrade, and agent_id is the same id task_started calls task_id, so it routes through the table already built at declaration. PreToolUse, PostToolUse and SubagentStop are registered purely to observe; they record what they see and always return empty, so they cannot affect tool execution or turn control. Paseo's hooks merge with any the user configured rather than replacing them: extraClaudeOptions is spread after the base options, so assigning would have silently stopped effort being observed for anyone with their own hooks. Verified end to end through ClaudeAgentClient against a real session: a subagent descriptor now carries model, effort and usage together. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(subagents): show a subagent's runtime facts in its pane, and its cost on replay The track row carried model, effort and cost; the pane carried none of it, so opening a subagent showed only its task. The pane gets a header strip rather than a longer tab subtitle: the subtitle feeds the width-constrained tab strip, whose job is to name the task. Duration appears in the pane, which has room, and stays out of the dense row. Replay had no usage source, so a reopened agent lost its subagents' token and tool counts permanently — descriptors are in-memory and cleared when an agent closes. Usage is now derived from the child's own transcript, matching the live producer exactly: total tokens are the four counters summed on the last assistant turn (a context reading, not a cumulative sum, which would multiply the cached prefix by the turn count), and tool uses are tool_use blocks across assistant entries. Duration is an approximation — the transcript's first entry is written after the task starts — and is the one field replay cannot match. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(claude): record the task protocol's undocumented wire facts The subagent sources depend on Claude Code internals the SDK types do not explain: which announced tasks are subagents, why a status for an undeclared task is dropped, why task ids outlive a turn, why effort is only reachable through hooks, and why a replayed session must skip its grandchildren. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Generalize provider subagent presentation --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: Mohamed Boudra <boudra.moha@gmail.com>
Voice Assistant
A voice-controlled terminal assistant that runs as a single local service.
Quick Start
# Install dependencies
npm install
# Copy environment variables
cp .env.example .env
# Edit .env and add your API keys (OpenAI, Deepgram)
# Run development servers
npm run dev
# Open browser to http://localhost:5173
Architecture
- Express Server (port 3000) - Serves API and built UI in production
- Vite Dev Server (port 5173) - Hot-reload React UI in development
- WebSocket (
/ws) - Real-time bidirectional communication - Agent - STT → LLM → TTS pipeline with terminal control
- Daemon - tmux-based terminal management (in-process)
Development
# Run both servers (recommended)
npm run dev
# Or run separately:
npm run dev:server # Express on port 3000
npm run dev:ui # Vite on port 5173
# Type checking
npm run typecheck
# Build for production
npm run build
# Start production server
npm start
Project Status
✅ Completed (Phases 1-2):
- Package setup and configuration
- Express server with WebSocket
- React UI with Vite
- WebSocket client with ping/pong testing
⏳ In Progress (Phase 3):
- Terminal control (tmux integration)
📋 Planned (Phases 4-9):
- LLM integration (OpenAI GPT-4)
- Agent orchestrator
- Speech-to-Text (Deepgram)
- Text-to-Speech (OpenAI)
- Audio streaming
- UI polish
See IMPLEMENTATION_PLAN.md for complete details.
Environment Variables
OPENAI_API_KEY=your-openai-key-here # GPT-4 and TTS
DEEPGRAM_API_KEY=your-deepgram-key-here # Streaming STT
STT_MODEL=whisper-1 # Optional: override to gpt-4o-transcribe, etc.
STT_CONFIDENCE_THRESHOLD=-3.0 # Optional: reject low-confidence clips
STT_DEBUG_AUDIO_DIR=.stt-debug # Optional: persist raw dictation audio for debugging
PASEO_HOME=~/.paseo # Runtime state directory (agents/, etc.)
PASEO_LISTEN=127.0.0.1:6767 # Listen address (host:port or /path/to/socket)
PASEO_HOME defaults to ~/.paseo and isolates runtime artifacts like agents/. PASEO_LISTEN controls the daemon listen address. For blue/green testing you can run a parallel server without touching production state:
PASEO_HOME=~/.paseo-blue PASEO_LISTEN=127.0.0.1:7777 npm run dev
Tech Stack
- Server: Express, TypeScript, ws (WebSocket)
- Client: React 18, Vite, TypeScript
- Terminal: tmux (via child_process)
- AI: OpenAI (LLM + TTS), Deepgram (STT)
Testing
Currently manual testing via:
- Start servers:
npm run dev - Open http://localhost:5173
- Test WebSocket connection (green status indicator)
- Click "Send Ping" button to test communication
More testing guidance as features are implemented.
License
MIT