jb/paseo
1
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

feat: add sqlite storage and terminal ui

Browse Source
This commit is contained in:
Mohamed Boudra
2026-03-29 23:20:18 +07:00
parent e5fd61f131
commit 7d02e24d84
189 changed files with 17868 additions and 4663 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/DEVELOPMENT.md
+19
View File
@@ -35,6 +35,25 @@ In worktrees or with `npm run dev`, ports may differ. Never assume defaults.
Check `$PASEO_HOME/daemon.log` for trace-level logs.
### Database queries
Run arbitrary SQL against the SQLite database:
```bash
# Show table row counts
npm run db:query
# Run any SQL
npm run db:query -- "SELECT agent_id, title, last_status FROM agent_snapshots"
npm run db:query -- "SELECT agent_id, seq, item_kind FROM agent_timeline_rows ORDER BY committed_at DESC LIMIT 10"
# Point at a specific DB directory
npm run db:query -- --db /path/to/db "SELECT ..."
```
Auto-detects the running dev daemon's database from `/tmp/paseo-dev.*`, `PASEO_HOME`, or `~/.paseo/db`.
Pass either a DB directory or a `paseo.sqlite` file to `--db`. The script opens the database directly in read-only mode.
## Build sync gotchas
### Relay → Daemon
docs/STORAGE_REVAMP_PLAN.md
+217
View File
@@ -0,0 +1,217 @@
# Storage Revamp Plan
Status: active rollout, phases 1 and 2 complete
This document now tracks the storage revamp as it exists today, not as a speculative design exercise.
The DB foundation and the project/workspace identity cutover have landed. What remains is the explicit
creation/archive surface cleanup, timeline durability cutover, and final removal of legacy paths.
## Goals
- make structured records durable in Drizzle + SQLite
- make projects and workspaces explicit first-class records
- stop deriving project/workspace identity from agent `cwd`
- keep agent snapshot persistence behind clear ownership
- move committed timeline history to storage-owned rows
- remove legacy JSON and in-memory authority once the DB path is proven
## Out of scope
- moving config, keypairs, push tokens, or server identity into the DB
- persisting raw provider deltas or transport-only chunk streams
- designing a hosted/remote database story beyond keeping the schema portable
- durable reasoning history unless product explicitly asks for it later
## Current state
The storage revamp is no longer hypothetical.
Completed:
- Drizzle + SQLite database bootstrap is in place
- `projects`, `workspaces`, and `agent_snapshots` use integer primary keys
- `workspaces.project_id` and `agent_snapshots.workspace_id` cascade on delete
- `agent_snapshots.workspace_id` is `NOT NULL`
- legacy JSON import feeds the DB-backed structured records
- project/workspace records use explicit `directory` fields instead of path-as-identity
- session read paths now use persisted workspace/project rows instead of cwd/git derivation
- `workspace-reconciliation-service.ts` is deleted
- `workspace-registry-bootstrap.ts` is deleted
- `workspace-registry-model.ts` is reduced to `normalizeWorkspaceId`
Still pending:
- explicit `create_project` / `create_workspace` API cleanup
- final archive cascade behavior for descendants and live agents
- committed timeline storage cutover
- removal of remaining legacy JSON and in-memory committed-history authority
## Converged decisions
### Structured record authority
Projects, workspaces, and agent snapshots are DB-backed structured records.
The server should not recreate project/workspace identity from:
- git remotes
- worktree main-repo roots
- normalized cwd strings
Temporary exception:
- agent creation may still find-or-create a workspace by directory if the UI has not yet provided
`workspaceId` explicitly
That fallback is transitional and should be deleted once the client always sends the workspace id.
### Storage seams
The useful seams remain concrete and domain-shaped:
- `ProjectRegistry`
- `WorkspaceRegistry`
- `AgentSnapshotStore`
- `AgentTimelineStore`
There is no reason to reintroduce a reconciliation service layer for project/workspace identity.
### Timeline contract
The long-term timeline contract remains:
- committed rows are durable, canonical history
- provisional live updates are transient subscription state
- committed history is fetched by seq
- provider history replay is not the durability mechanism
The structured-record cutover is complete before the timeline cutover so timeline rows can rely on
stable DB-backed agent and workspace identity.
## Remaining phases
### Phase 3: Explicit creation and archive cleanup
Goal:
Remove the last transitional write paths that still infer state from directories.
Required work:
- add explicit `create_project` handling
- add explicit `create_workspace` handling
- make agent creation require `workspaceId` once the UI is ready
- finish archive semantics for workspaces/projects and any descendant agent state
- remove the temporary find-or-create-by-directory fallback from agent creation
Exit gate:
- project/workspace creation is explicit end to end
- no normal creation path infers identity from cwd or git metadata
- archive flows behave consistently for structured records and live runtime state
### Phase 4: Timeline storage cutover
Goal:
Make committed history durable and storage-owned.
Required work:
- make `AgentTimelineStore` authoritative for committed history
- write one committed row per finalized logical item
- support tail, before-seq, and after-seq queries from storage
- stop treating provider history hydration as the normal refresh/load path
- keep provisional live updates in memory only
Exit gate:
- committed history survives daemon restart
- reconnect uses committed catch-up plus future live events without gaps or duplicates
- unloaded agents can serve committed history from storage alone
### Phase 5: Legacy cleanup
Goal:
Remove compatibility paths after the DB-backed model is fully authoritative.
Required work:
- remove legacy JSON authority for structured records
- remove in-memory committed-history ownership
- remove provider-history rehydrate compatibility paths
- trim dead protocol and reducer logic from the pre-storage model
- update architecture docs to match the final model
Exit gate:
- there is one durable storage path for structured records
- there is one durable storage path for committed timeline history
- the runtime no longer depends on the removed JSON/in-memory model
## Data model summary
### Projects
- integer primary key
- `directory` is unique
- `display_name`
- `kind`: `git | directory`
- optional `git_remote`
- timestamps and archive state
### Workspaces
- integer primary key
- belongs to a project by `project_id`
- `directory` is unique
- `display_name`
- `kind`: `checkout | worktree`
- timestamps and archive state
### Agent snapshots
- `agent_id` remains the primary key
- belongs to a workspace by integer `workspace_id`
- `workspace_id` is required
- timestamps, lifecycle state, persistence metadata, attention metadata, archive state
### Timeline rows
Target shape once Phase 4 lands:
- `agent_id`
- committed `seq`
- committed timestamp
- canonical finalized item payload
Not part of durable history:
- raw streaming chunks
- provisional assistant text
- provisional reasoning text
## Verification requirements
Every remaining phase should keep the same bar:
- `npm run typecheck`
- targeted tests for the touched storage/session/runtime paths
- migration/import coverage when storage authority changes
- reconnect and catch-up scenario coverage when timeline behavior changes
At minimum, timeline cutover must explicitly prove:
- `fetch-after-seq`
- `fetch-before-seq`
- restart durability
- no-gap/no-duplicate reconnect behavior
## Main risks
- timeline work reintroduces provider-history replay as hidden authority
- archive behavior diverges between stored records and live in-memory agents
- explicit creation work leaves the transitional cwd fallback in place too long
- cleanup stalls after compatibility paths stop being exercised
## Rule of thumb
If a new change needs to ask "what can we infer from this cwd?" for project or workspace identity,
it is probably moving in the wrong direction.
docs/TERMINAL-MODE.md
+506
View File
@@ -0,0 +1,506 @@
# Terminal Mode — Implementation Plan
## Concept
Terminal mode wraps an agent TUI (Claude Code, Codex, OpenCode, Gemini, etc.) in a Paseo agent entity. The agent is tracked in sessions, has a provider/icon/title, and can be archived — but instead of rendering a structured chat view, it renders a terminal running the agent's CLI.
**Key principle:** `agent.terminal` is a boolean flag on the agent entity. If `true`, the panel renders a terminal. If `false` (default), it renders the current structured AgentStreamView.
## What Changes
### Phase 1: Server — Data Model & Provider Interface
#### 1.1 Add `terminal` flag to `ManagedAgentBase`
**File:** `packages/server/src/server/agent/agent-manager.ts`
```typescript
type ManagedAgentBase = {
// ...existing fields...
terminal: boolean; // NEW — if true, this agent renders as a terminal TUI
};
```
This flag is set at creation time and never changes. A terminal agent is always a terminal agent.
#### 1.2 Add `terminal` to `AgentSessionConfig`
**File:** `packages/server/src/server/agent/agent-sdk-types.ts`
```typescript
export type AgentSessionConfig = {
// ...existing fields...
terminal?: boolean; // NEW — create as terminal agent
};
```
#### 1.3 Add `terminal` to the Zod schema
**File:** `packages/server/src/shared/messages.ts`
Add to `AgentSessionConfigSchema`:
```typescript
terminal: z.boolean().optional(),
```
Add to the `AgentStateSchema` (the wire format sent to clients):
```typescript
terminal: z.boolean().optional(),
```
#### 1.4 Add terminal command builders to `AgentClient`
**File:** `packages/server/src/server/agent/agent-sdk-types.ts`
```typescript
export type TerminalCommand = {
command: string;
args: string[];
env?: Record<string, string>;
};
export interface AgentClient {
// ...existing methods...
/**
* Build the shell command to launch this agent's TUI for a new session.
* Only available if capabilities.supportsTerminalMode is true.
*/
buildTerminalCreateCommand?(config: AgentSessionConfig): TerminalCommand;
/**
* Build the shell command to resume an existing session in the agent's TUI.
* Only available if capabilities.supportsTerminalMode is true.
*/
buildTerminalResumeCommand?(handle: AgentPersistenceHandle): TerminalCommand;
}
```
#### 1.5 Add `supportsTerminalMode` capability
**File:** `packages/server/src/server/agent/agent-sdk-types.ts`
```typescript
export type AgentCapabilityFlags = {
// ...existing flags...
supportsTerminalMode: boolean; // NEW
};
```
Also add to the Zod schema in `messages.ts`:
```typescript
supportsTerminalMode: z.boolean(),
```
#### 1.6 Implement terminal command builders in providers
**Claude** (`packages/server/src/server/agent/providers/claude-agent.ts`):
```typescript
buildTerminalCreateCommand(config: AgentSessionConfig): TerminalCommand {
const args: string[] = [];
if (config.modeId === "bypassPermissions") {
args.push("--dangerously-skip-permissions");
}
if (config.model) args.push("--model", config.model);
// mode mapping: default → nothing, plan → --plan, etc.
return { command: "claude", args, env: {} };
}
buildTerminalResumeCommand(handle: AgentPersistenceHandle): TerminalCommand {
return {
command: "claude",
args: ["--resume", handle.sessionId],
env: {},
};
}
```
**Codex** (`packages/server/src/server/agent/providers/codex-app-server-agent.ts`):
```typescript
buildTerminalCreateCommand(config: AgentSessionConfig): TerminalCommand {
const args: string[] = [];
if (config.model) args.push("--model", config.model);
if (config.modeId) args.push("--approval-mode", config.modeId);
return { command: "codex", args, env: {} };
}
buildTerminalResumeCommand(handle: AgentPersistenceHandle): TerminalCommand {
return {
command: "codex",
args: ["--resume", handle.nativeHandle ?? handle.sessionId],
env: {},
};
}
```
**OpenCode** (`packages/server/src/server/agent/providers/opencode-agent.ts`):
```typescript
buildTerminalCreateCommand(config: AgentSessionConfig): TerminalCommand {
return { command: "opencode", args: [], env: {} };
}
// No resume support for OpenCode initially
```
Capabilities for each provider:
- Claude: `supportsTerminalMode: true`
- Codex: `supportsTerminalMode: true`
- OpenCode: `supportsTerminalMode: true`
#### 1.7 Handle terminal agent creation in `AgentManager.createAgent()`
**File:** `packages/server/src/server/agent/agent-manager.ts`
When `config.terminal === true`:
1. Do NOT call `client.createSession()` — there is no managed session
2. Call `client.buildTerminalCreateCommand(config)` to get the command
3. Create a `TerminalSession` via `terminalManager.createTerminal()` with the command
4. Register the agent with `terminal: true`, `lifecycle: "idle"`, `session: null`
5. Store the terminal ID in the agent's metadata or a new field
6. The agent's persistence handle can be populated later (the CLI will create its own session file)
```typescript
async createAgent(config: AgentSessionConfig, agentId?: string, options?: { labels?: Record<string, string> }): Promise<ManagedAgent> {
const resolvedAgentId = validateAgentId(agentId ?? this.idFactory(), "createAgent");
const normalizedConfig = await this.normalizeConfig(config);
const client = this.requireClient(normalizedConfig.provider);
if (normalizedConfig.terminal) {
// Terminal mode — no managed session, just build the command
const buildCmd = client.buildTerminalCreateCommand;
if (!buildCmd) {
throw new Error(`Provider '${normalizedConfig.provider}' does not support terminal mode`);
}
const cmd = buildCmd.call(client, normalizedConfig);
return this.registerTerminalAgent(resolvedAgentId, normalizedConfig, cmd, {
labels: options?.labels,
});
}
// ...existing managed agent flow...
}
```
New method `registerTerminalAgent()`:
- Creates a ManagedAgent with `terminal: true`
- Stores the `TerminalCommand` in agent metadata for later use (resume, reconnect)
- Sets lifecycle to `"idle"` (the terminal itself manages the agent's internal state)
- Does NOT have an `AgentSession` — the `session` field is `null` (like closed agents)
- Broadcasts `agent_state` event so clients know about it
#### 1.8 New message: create terminal for agent
The client needs a way to request a terminal for a terminal agent. Options:
**Option A:** Extend `createTerminal` to accept an agent ID. When provided, the server looks up the agent, gets the command, and creates a terminal pre-configured with that command.
**Option B:** New message type `create_terminal_agent_request` that combines agent creation + terminal creation in one step.
**Recommendation: Option A.** Add optional `agentId` to `CreateTerminalRequestMessage`. If provided:
- Look up the agent (must be a terminal agent)
- Use the agent's stored command to create the terminal
- Associate the terminal with the agent
**File:** `packages/server/src/shared/messages.ts`
```typescript
const CreateTerminalRequestMessageSchema = z.object({
type: z.literal("create_terminal_request"),
cwd: z.string(),
name: z.string().optional(),
agentId: z.string().optional(), // NEW — if provided, create terminal for this terminal agent
requestId: z.string(),
});
```
#### 1.9 Terminal → Agent lifecycle binding
When a terminal associated with a terminal agent exits:
- Set agent lifecycle to `"closed"`
- Attempt to detect the agent's session file for persistence handle
- Broadcast state update
When a terminal agent is opened from the sessions page:
- Server calls `buildTerminalResumeCommand(handle)` if persistence handle exists
- Otherwise calls `buildTerminalCreateCommand(config)`
- Creates a new terminal with that command
#### 1.10 Extend `createTerminal()` to support command + args
**File:** `packages/server/src/terminal/terminal.ts`
```typescript
export interface CreateTerminalOptions {
cwd: string;
shell?: string;
env?: Record<string, string>;
rows?: number;
cols?: number;
name?: string;
command?: string; // NEW — if provided, run this instead of shell
args?: string[]; // NEW — arguments for command
}
```
In `createTerminal()`:
```typescript
const spawnCommand = options.command ?? shell;
const spawnArgs = options.command ? (options.args ?? []) : [];
const ptyProcess = pty.spawn(spawnCommand, spawnArgs, {
name: "xterm-256color",
cols, rows, cwd,
env: { ...process.env, ...env, TERM: "xterm-256color" },
});
```
---
### Phase 2: App — Draft UI & Terminal Toggle
#### 2.1 Add terminal toggle to draft tab
**File:** `packages/app/src/screens/workspace/workspace-draft-agent-tab.tsx`
Add a toggle switch in the draft UI: **"Chat" / "Terminal"**
State:
```typescript
const [isTerminalMode, setIsTerminalMode] = useState(false);
```
The toggle should be persistent per draft (stored in the draft store or as a preference).
When terminal mode is selected:
- The provider/model pickers still work (same UI)
- The mode picker still works
- The "send" button label changes to "Launch" or "Start"
- The initial prompt input may be hidden or optional (terminal agents don't need an initial prompt — the user types directly into the TUI)
#### 2.2 Modify agent creation to pass `terminal: true`
When the user submits a draft in terminal mode:
```typescript
const config: AgentSessionConfig = {
provider: selectedProvider,
cwd: workspaceId,
model: selectedModel,
modeId: selectedMode,
terminal: true, // NEW
};
```
The `CreateAgentRequestMessage` already carries `config`, so no new wire message needed.
#### 2.3 Terminal mode in `AgentStatusBar`
**File:** `packages/app/src/components/agent-status-bar.tsx`
When rendering a draft's status bar, filter the capability:
- If `supportsTerminalMode` is false for a provider, disable the terminal toggle when that provider is selected
- The terminal toggle can live next to the provider selector or as a segmented control above the input area
---
### Phase 3: App — Agent Panel Rendering
#### 3.1 Branch rendering in `AgentPanel`
**File:** `packages/app/src/panels/agent-panel.tsx`
```typescript
function AgentPanelContent({ agentId, ... }) {
const agent = useAgentState(agentId);
if (agent?.terminal) {
return <TerminalAgentPanel agentId={agentId} agent={agent} />;
}
return <AgentPanelBody agent={agent} ... />;
}
```
#### 3.2 New component: `TerminalAgentPanel`
**File:** `packages/app/src/panels/terminal-agent-panel.tsx` (new file)
This component:
1. Gets the terminal ID associated with the agent (from agent metadata or a new field)
2. Renders a `TerminalPane` connected to that terminal session
3. If no terminal exists yet (agent from sessions page), requests terminal creation via `createTerminal({ agentId })`
4. Handles terminal exit → agent close lifecycle
Essentially: it's the existing `TerminalPane` component, but associated with an agent entity instead of a standalone terminal.
#### 3.3 Tab descriptor for terminal agents
**File:** `packages/app/src/panels/agent-panel.tsx` → `useAgentPanelDescriptor`
The tab descriptor (icon, label) already comes from the agent's provider. Terminal agents get the same icon/label as managed agents — that's the whole point. No changes needed here unless we want a "terminal" badge.
Optional: add a small terminal icon badge to distinguish terminal agents from managed agents in the tab bar.
---
### Phase 4: Sessions Page
#### 4.1 Terminal agents appear in sessions list
No changes needed for listing — terminal agents are real agents, they already show up via `AgentManager.getAgents()`.
#### 4.2 Opening a terminal agent from sessions
**File:** `packages/app/src/screens/sessions/` (sessions screen)
When the user clicks a closed terminal agent:
1. Server calls `buildTerminalResumeCommand(handle)` if persistence exists
2. Creates a new terminal with that command
3. Opens agent tab in workspace
If no persistence handle (session was ephemeral), show "Start new session" which calls `buildTerminalCreateCommand(config)`.
---
### Phase 5: CLI Gating
#### 5.1 `paseo send` — error for terminal agents
**File:** `packages/cli/src/commands/send.ts`
```typescript
if (agent.terminal) {
throw new Error("Cannot send messages to terminal agents. Open the terminal in the UI instead.");
}
```
#### 5.2 `paseo run` — could support `--terminal` flag (future)
Not in v1. For now, `paseo run` always creates managed agents. Terminal mode is UI-only.
#### 5.3 `paseo ls` — show terminal flag
Add a `terminal` column or badge to `paseo ls` output so users can distinguish terminal agents.
---
## Wire Format Changes Summary
### AgentSessionConfig (create request)
```diff
{
provider: string;
cwd: string;
model?: string;
modeId?: string;
+ terminal?: boolean;
...
}
```
### AgentState (server → client)
```diff
{
id: string;
provider: string;
lifecycle: string;
+ terminal?: boolean;
...
}
```
### AgentCapabilityFlags
```diff
{
supportsStreaming: boolean;
supportsSessionPersistence: boolean;
+ supportsTerminalMode: boolean;
...
}
```
### CreateTerminalRequest
```diff
{
type: "create_terminal_request";
cwd: string;
name?: string;
+ agentId?: string;
requestId: string;
}
```
### TerminalCommand (new type)
```typescript
{
command: string;
args: string[];
env?: Record<string, string>;
}
```
---
## Implementation Phases & Agent Assignments
### Phase 1: Server data model (1 agent)
- Add `terminal` to types, schemas, and agent manager
- Add `TerminalCommand` type and `buildTerminalCreateCommand`/`buildTerminalResumeCommand` to `AgentClient`
- Add `supportsTerminalMode` capability flag
- Extend `createTerminal()` to support command+args
- Implement terminal agent creation flow in `AgentManager`
- Wire terminal exit → agent close lifecycle
- Implement command builders in Claude, Codex, OpenCode providers
- Typecheck must pass
### Phase 2: App draft UI + terminal toggle (1 agent)
- Add terminal mode toggle to `workspace-draft-agent-tab.tsx`
- Pass `terminal: true` in config when toggle is on
- Filter toggle based on `supportsTerminalMode` capability
- Persist toggle preference
- Typecheck must pass
### Phase 3: App panel rendering (1 agent)
- Branch `AgentPanelContent` on `agent.terminal`
- Create `TerminalAgentPanel` component
- Handle terminal creation for agent on open
- Handle terminal exit lifecycle
- Typecheck must pass
### Phase 4: Sessions page + CLI gating (1 agent)
- Terminal agents show in sessions with badge
- Opening from sessions resumes or creates terminal
- `paseo send` errors for terminal agents
- `paseo ls` shows terminal badge
- Typecheck must pass
---
## Feature Interaction Guards
Terminal agents are explicitly excluded from automated dispatch paths:
- **LoopService**: `buildWorkerConfig` and `buildVerifierConfig` set `terminal: false`
- **ScheduleService**: `executeSchedule` rejects terminal agents with a clear error for agent-targeted schedules; new-agent schedules set `terminal: false`
- **Voice mode / `handleSendAgentMessage`**: Guarded by `getStructuredSendRejection()` before send
- **CLI `paseo send`**: Returns error for terminal agents
- **MCP agent creation**: Programmatic paths don't pass `terminal: true`
All session-specific operations (`runAgent`, `streamAgent`, `setMode`, `cancelAgentRun`, etc.) are guarded by the centralized `requireSessionAgent()` which rejects terminal agents.
## What This Does NOT Change
- The existing managed agent flow is untouched
- Terminal sessions (non-agent) still work as before
- The `AgentSession` interface is unchanged
- Mobile experience is unchanged (terminal mode is web/desktop only for now)
- No new providers are added (existing providers gain terminal command builders)
- No hooks, no env injection, no process tree detection (v1 keeps it simple)
## Future Work (Not In This Plan)
- Auto-detect agent type from PTY process tree (for standalone terminals)
- "Convert to chat" / "Convert to terminal" actions
- Terminal title/icon from OSC sequences
- `paseo run --terminal` CLI support
- Mobile terminal mode (if xterm.js works well enough on mobile web)
- Gemini / Aider / Goose provider definitions (terminal-only providers)