Added nono profiles & orca integration - Added some extensions to start working on
This commit is contained in:
@@ -0,0 +1,568 @@
|
||||
# subagents — Design Plan
|
||||
|
||||
A pi extension that fires off background subagents from a parent pi session, where each
|
||||
subagent can be powered by one of three backends — **pi** (in-process SDK session),
|
||||
**Claude Code** (`@anthropic-ai/claude-agent-sdk`), or **Codex** (`codex app-server`) —
|
||||
unified behind a single Effect v4 service interface.
|
||||
|
||||
> **Status:** this document describes the original v1 plan (stubbed backends). All
|
||||
> three backends are now REAL implementations — see `src/backends/{pi,claude,codex}.ts`.
|
||||
> The stub machinery survives in `src/backends/stub.ts` for the manager test registry.
|
||||
|
||||
**Scope of the first version:** interface design + stubbed backend internals + the v1 UI
|
||||
carried over. No real Claude/Codex process integration yet; the pi backend may also stay
|
||||
stubbed initially so the manager/UI/tool loop can be exercised end to end with zero
|
||||
external dependencies.
|
||||
|
||||
**Location:** `/Users/davis/.pi/agent/extensions/subagents/` — fully self-contained
|
||||
(no imports from `../shared` or `../subagents`; the handful of shared helpers v1 uses are
|
||||
copied in).
|
||||
|
||||
---
|
||||
|
||||
## 1. V1 inventory (what must be preserved)
|
||||
|
||||
Source: `/Users/davis/.pi/agent/extensions/subagents/` (`index.ts`, `manager.ts`,
|
||||
`prompt.ts`, `result-delivery.ts`, `takeover.ts`) plus `../shared/` helpers.
|
||||
|
||||
### 1.1 Tools exposed to the parent LLM
|
||||
|
||||
| Tool | Parameters | Behavior |
|
||||
|---|---|---|
|
||||
| `subagent_spawn` | `prompt`, `title`, `working_dir?`, `model?`, `provider?`, `reasoning_effort?` | Fire-and-forget spawn. Returns immediately with an id (`sa-N`). Enforces `MAX_RUNNING = 4` with a synchronous reservation so parallel tool calls can't race past the cap. Validates `working_dir`, resolves model against the registry (inherit parent model/thinking level by default), truncates title to 160 chars. |
|
||||
| `subagent_wait` | `ids[]` (max 64) | Blocks until all listed subagents settle; respects the tool `AbortSignal`; streams `Waiting for ...` via `onUpdate`. Marks the awaited results "consumed" so they are not also auto-delivered. Output budgets: 48KB total, 16KB per agent, with per-section fallbacks (`[omitted: ...]`). Errors on unknown ids (lists known ids). |
|
||||
| `subagent_cancel` | `ids[]` | Aborts running subagents (marks consumed first to avoid duplicate delivery), waits for settlement, reports per-id `Cancelled ...` / `was already <status>`. Partial transcripts remain on disk. |
|
||||
| `subagent_check` | `id` | Non-blocking peek: status line, turn count, error text, up to 2KB/20 lines of latest output (includes the live streaming assistant message). Does not consume the result. |
|
||||
| `subagent_list` | — | One `describeSubagent()` line per agent: `id [status] "title" (provider/model, ctx%, elapsed, cwd)`. |
|
||||
|
||||
Prompt metadata (all strings live in `prompt.ts`): `subagent_spawn` has a
|
||||
`promptSnippet` and two `promptGuidelines` (delegate self-contained tasks; don't block on
|
||||
`subagent_wait` unless necessary). Tool descriptions explain fire-and-forget semantics,
|
||||
the concurrency cap, and that children can't orchestrate/see the parent conversation.
|
||||
|
||||
### 1.2 State tracking (v1 `SubagentManager`)
|
||||
|
||||
- Plain class with `Map<string, Subagent>`; each `Subagent` = `{ id, title, prompt, cwd,
|
||||
session: AgentSession, status: "running" | "done" | "error", createdAt, settledAt?,
|
||||
errorText?, unsubscribeLifecycle }`.
|
||||
- Children are **in-process pi `AgentSession`s** created via the SDK
|
||||
(`createAgentSession` + `SessionManager.create(cwd)` → real session files visible in
|
||||
`/resume`), with child resources loaded per-cwd (`DefaultResourceLoader`, trust-gated
|
||||
project resources) and a tool denylist (`excludeTools`: the subagent_* tools,
|
||||
`workflow`, `ask_user`).
|
||||
- Settlement is driven by session lifecycle events (`agent_start` re-marks running;
|
||||
`agent_settled` settles). Failure detection: thrown prompt error, last assistant
|
||||
`stopReason === "error" | "aborted"`, error text bounded to 4096 chars.
|
||||
- Change notification: `addChangeListener()` + `nextChange(signal)` promise — used by
|
||||
`waitFor`, the footer status, and the dashboard.
|
||||
- `waitFor(ids, signal, onPending)` keeps a `waitInterest` refcount per id so settles
|
||||
during an active wait are marked consumed.
|
||||
- `send(sub, text)`: steer via `session.steer()` while streaming, else start a fresh
|
||||
`prompt()` run (used by takeover).
|
||||
- Caps and cleanup: `MAX_RUNNING = 4`, `MAX_TRACKED = 64` with LRU pruning of settled
|
||||
agents, `STOP_TIMEOUT_MS = 5s` bounded aborts, force-dispose fallback, idempotent
|
||||
`disposeAll()` on `session_shutdown`.
|
||||
|
||||
### 1.3 Result delivery back to the parent
|
||||
|
||||
- When a child settles **unconsumed**, `onSettled` defers it into a tiny
|
||||
`createDeferredResultDelivery` buffer (defer/consume/drain/clear keyed by id).
|
||||
- Flush happens when the parent goes idle: immediately if `sessionContext.isIdle()`,
|
||||
otherwise on the parent's `agent_settled` event. A later `subagent_wait` can still
|
||||
consume a deferred result before flush (that's why it is a buffer, not an immediate
|
||||
send).
|
||||
- Delivery = `pi.sendMessage({ customType: "subagent-result", content, display: true,
|
||||
details: { id, title, status } }, { deliverAs: "followUp", triggerTurn: true })`.
|
||||
Content is built by `buildSubagentResultMessage` (`Subagent sa-N "title"
|
||||
finished/failed.` + optional `Error:` line + output truncated to 24KB/600 lines with a
|
||||
pointer to the child session file for the full transcript).
|
||||
|
||||
### 1.4 UI (carried over into v2 essentially as-is)
|
||||
|
||||
1. **Footer status** (`ctx.ui.setStatus("subagents", ...)`): `subagents: ■ 2 running ·
|
||||
■ 1 done · ■ 1 failed · /subagents to view` (warning/success/error colored squares;
|
||||
cleared when no subagents). Driven by manager change listener.
|
||||
2. **`subagent-result` message renderer**: status icon (`■`/`x`) + bold accent header
|
||||
`subagent sa-N · title · finished/failed`; collapsed = first 8 body lines +
|
||||
`... (ctrl+o to expand)`; expanded = header + `Markdown` component render of the body.
|
||||
3. **`/subagents` command** → `openSubagentPicker` loop (TUI mode only; notifies and
|
||||
bails in non-TUI or when there are no subagents):
|
||||
- **SubagentDashboard** — fullscreen overlay (`anchor: "center", width: "100%",
|
||||
maxHeight: "100%"`), bordered list panel titled `agents · settled/total`. Each row:
|
||||
selection marker `❯`, status glyph, title, dim id on the left; model id · context
|
||||
utilization (`%/capacity`) · elapsed · status word on the right. Scroll window
|
||||
centered on the selection with `... N more` markers. 1Hz ticker re-render for
|
||||
elapsed/token columns + manager change subscription. Keys: `tui.select.up/down`
|
||||
**and** `j`/`k` to move, `tui.select.confirm` to take over, `x` to abort the
|
||||
selected running agent, `tui.select.cancel` to close. Hint line shows the
|
||||
*configured* keys via `keybindings.getKeys()`.
|
||||
- **TakeoverView** — fullscreen overlay for one subagent: header line (status glyph,
|
||||
`id · title · status · elapsed · provider/model · ctx%`), fixed-height transcript
|
||||
viewport (error line and scroll indicator consume viewport rows so height never
|
||||
jumps), an `Input` line, and a hint row. Keys: `tui.input.submit` send (steer if
|
||||
streaming, new run if idle), `app.interrupt`/`tui.select.cancel` back to dashboard,
|
||||
`app.clear` abort run, `tui.editor.cursorUp/Down` scroll ±6 lines,
|
||||
`tui.editor.pageUp/Down` page. Renders are throttled to 50ms because streaming can
|
||||
emit per-token events.
|
||||
- **Transcript rendering** (`buildTranscriptLines`): sanitizes ANSI/tabs/control
|
||||
chars; user messages as `> ` accent-prefixed wrapped lines; assistant text wrapped
|
||||
plain; thinking as dim italic `~ ` lines; tool calls as `→ toolname {args}`; tool
|
||||
results as one dim `output:`/red `error:` first line. Includes the **live streaming
|
||||
assistant message**, **live tool executions** (running/done/error marker + first
|
||||
output line preview, tracked from `tool_execution_*` events until the final tool
|
||||
result message lands), and **queued steering/follow-up messages** (`> [queued
|
||||
steer] ...`) so Enter visibly acknowledges input.
|
||||
|
||||
**V2 requirement:** all of the above renders from a *normalized* per-subagent view
|
||||
instead of poking at `sub.session` directly — that is the main UI refactor, everything
|
||||
else ports over mostly verbatim.
|
||||
|
||||
---
|
||||
|
||||
## 2. Backend integration facts
|
||||
|
||||
These shape the interface even though v1-of-v2 stubs the internals. (Traced from the
|
||||
"T3 Code" codebase which integrates Codex and Claude Code.)
|
||||
|
||||
| | Interactive sessions | One-shot tasks | Event shape | Interrupt | Steering |
|
||||
|---|---|---|---|---|---|
|
||||
| **pi** | In-process `createAgentSession()` (pi SDK); real session files; `session.subscribe()` | `session.prompt()` and read final assistant message (or `pi -p` subprocess, not needed) | `AgentSessionEvent` (message_start/update/end, tool_execution_*, agent_start/settled, queue_update, ...) | `session.abort()` | `session.steer()` / `followUp()` |
|
||||
| **Claude Code** | `@anthropic-ai/claude-agent-sdk` `query()` — SDK launches the `claude` executable and streams JSON messages (assistant/user/result/system, streaming partials, tool_use blocks) | `claude -p --output-format json` | SDK message stream (async iterable) | `query.interrupt()` / abort controller | streaming-input mode: push more user messages into the input iterable |
|
||||
| **Codex** | spawn `codex app-server` child process, JSON-RPC over stdin/stdout (`newConversation` / `sendUserTurn`, notifications: `agentMessageDelta`, `execCommandBegin/End`, `taskComplete`, `tokenCount`, ...) | `codex exec` (prints result to stdout, `--json` for events) | JSON-RPC notifications | `interruptConversation` request | send another `sendUserTurn` on the same conversation |
|
||||
|
||||
Common denominator all three can supply:
|
||||
|
||||
- an async event stream with: lifecycle (started/turn/settled), assistant text
|
||||
(deltas and/or completed messages), reasoning text, tool execution begin/update/end,
|
||||
token usage, errors;
|
||||
- a way to send a follow-up/steering user message into a live session;
|
||||
- an interrupt operation;
|
||||
- a final result text per run;
|
||||
- metadata: backend name, model identifier, session/log file path (pi session file,
|
||||
Claude session id + projects dir JSONL, Codex rollout path), working dir.
|
||||
|
||||
That is exactly what the normalized event model below encodes.
|
||||
|
||||
---
|
||||
|
||||
## 3. Architecture
|
||||
|
||||
### 3.1 Effect v4 conventions used
|
||||
|
||||
- Single `effect` package (v4 beta). Services defined with
|
||||
`ServiceMap.Service<T>()("id", ...)` / `ServiceMap.Key`; wiring via `Layer`;
|
||||
`ManagedRuntime.make(layer)` at the extension edge with `runtime.runPromise(effect,
|
||||
{ signal })` inside `async execute()` tool handlers and `await runtime.dispose()` on
|
||||
`session_shutdown`.
|
||||
- `Effect.gen` generators throughout the internals. `async`/`Promise` appears **only**
|
||||
in: tool `execute()` bodies, `pi.on(...)` handlers, the `/subagents` command handler,
|
||||
and the imperative TUI component classes (which are callback-driven, not effectful).
|
||||
- Streams: `Stream<SubagentEvent>` per subagent, produced by backends
|
||||
(`Stream.callback` for push-based sources like JSON-RPC notifications / SDK
|
||||
iterables), consumed by a manager fiber per subagent.
|
||||
- Errors: tagged error classes (`Data.TaggedError`) — `SpawnError`, `BackendUnavailable`,
|
||||
`SubagentNotFound`, `ConcurrencyLimitError`, `SendError`, `InterruptTimeout`. Tool
|
||||
handlers map these to thrown `Error`s with the same user-facing messages v1 uses.
|
||||
|
||||
### 3.2 Domain model (`src/domain.ts`)
|
||||
|
||||
```ts
|
||||
type BackendName = "pi" | "claude" | "codex";
|
||||
type SubagentStatus = "running" | "done" | "error"; // unchanged from v1
|
||||
|
||||
interface SpawnTask {
|
||||
prompt: string;
|
||||
title: string;
|
||||
cwd: string;
|
||||
// Generic model hint; each backend interprets/validates it its own way.
|
||||
model?: string; // pi: "provider/model-id"; claude: model alias; codex: model slug
|
||||
reasoningEffort?: string; // pi thinking level; codex reasoning effort; claude: ignored/mapped
|
||||
parentContext: { // resolved by the tool layer, passed opaquely
|
||||
parentCwd: string;
|
||||
projectTrusted: boolean;
|
||||
inheritedModelRef?: { provider: string; id: string }; // pi only
|
||||
inheritedThinkingLevel?: string;
|
||||
};
|
||||
}
|
||||
|
||||
interface SubagentMeta {
|
||||
backend: BackendName;
|
||||
modelLabel?: string; // "anthropic/claude-opus-4-5", "gpt-5-codex", ...
|
||||
contextWindow?: number; // for utilization %, when known
|
||||
sessionFilePath?: string; // pi session file / claude JSONL / codex rollout path
|
||||
nativeSessionId?: string; // claude session id, codex conversation id
|
||||
}
|
||||
```
|
||||
|
||||
### 3.3 Normalized event model (`src/domain.ts`)
|
||||
|
||||
One discriminated union covers everything the v1 UI and manager need. Backends translate
|
||||
their native streams into this; nothing downstream knows which backend produced it.
|
||||
|
||||
```ts
|
||||
type SubagentEvent =
|
||||
// lifecycle
|
||||
| { _tag: "RunStarted" } // pi agent_start / claude init / codex turn start
|
||||
| { _tag: "RunSettled"; outcome: RunOutcome } // terminal per run (a session can run again via send())
|
||||
// transcript building blocks
|
||||
| { _tag: "UserMessage"; text: string } // initial prompt + takeover sends, echoed by backend
|
||||
| { _tag: "AssistantDelta"; kind: "text" | "thinking"; delta: string }
|
||||
| { _tag: "AssistantMessage"; parts: TranscriptPart[] } // finalized message (replaces live buffer)
|
||||
| { _tag: "ToolStart"; toolId: string; name: string; argsPreview?: string }
|
||||
| { _tag: "ToolUpdate"; toolId: string; outputPreview?: string }
|
||||
| { _tag: "ToolEnd"; toolId: string; isError: boolean; outputPreview?: string }
|
||||
// bookkeeping
|
||||
| { _tag: "QueueChanged"; queued: ReadonlyArray<{ text: string; kind: "steer" | "follow-up" }> }
|
||||
| { _tag: "UsageChanged"; tokens?: number; contextWindow?: number }
|
||||
| { _tag: "MetaChanged"; meta: Partial<SubagentMeta> } // model switched, session file known, ...
|
||||
| { _tag: "BackendError"; message: string }; // non-fatal diagnostics (fatal → RunSettled outcome)
|
||||
|
||||
type RunOutcome =
|
||||
| { _tag: "Completed"; finalText: string }
|
||||
| { _tag: "Failed"; errorText: string; partialText?: string }
|
||||
| { _tag: "Interrupted"; partialText?: string };
|
||||
|
||||
type TranscriptPart =
|
||||
| { type: "text"; text: string }
|
||||
| { type: "thinking"; text: string; redacted?: boolean }
|
||||
| { type: "toolCall"; toolId: string; name: string; argsPreview?: string };
|
||||
```
|
||||
|
||||
Mapping sanity check against what the v1 UI renders:
|
||||
|
||||
| v1 render source | v2 event source |
|
||||
|---|---|
|
||||
| `sub.session.messages` (user/assistant/toolResult) | fold of `UserMessage` / `AssistantMessage` / `ToolEnd` into the snapshot transcript |
|
||||
| streaming message (`agent.state.streamingMessage`) | live buffer fed by `AssistantDelta` (cleared on `AssistantMessage`/`RunSettled`) |
|
||||
| live tool map from `tool_execution_*` | `ToolStart/Update/End` (entries drop when the finalized assistant/tool item lands, same as v1) |
|
||||
| queued steer/follow-up messages | `QueueChanged` |
|
||||
| status / errorText / settledAt | `RunStarted` / `RunSettled` |
|
||||
| model + context utilization columns | `MetaChanged` + `UsageChanged` |
|
||||
| `finalOutput` / `latestOutput` for check/wait/result delivery | `RunOutcome.finalText` (+ live buffer for `latestOutput`) |
|
||||
|
||||
Previews (`argsPreview`, `outputPreview`) are pre-flattened single-line strings because
|
||||
the UI only ever shows one sanitized line — this avoids leaking three different native
|
||||
"tool result" shapes through the interface.
|
||||
|
||||
### 3.4 The `SubagentBackend` service (`src/backend.ts`)
|
||||
|
||||
One interface; three implementations; a registry keyed by `BackendName`.
|
||||
|
||||
```ts
|
||||
interface SubagentBackend {
|
||||
readonly name: BackendName;
|
||||
readonly capabilities: {
|
||||
steering: boolean; // can send() into a live run (all three eventually; stubs: true)
|
||||
modelSelection: boolean;
|
||||
reasoningEffort: boolean;
|
||||
};
|
||||
/** Probe availability (binary on PATH, SDK importable, API key). Cheap + cached. */
|
||||
readonly available: Effect.Effect<boolean>;
|
||||
/**
|
||||
* Spawn a session. Scoped: releasing the scope interrupts/kills the underlying
|
||||
* session/process. Returns a live handle immediately (fire-and-forget semantics
|
||||
* live in the manager, not here).
|
||||
*/
|
||||
spawn(task: SpawnTask): Effect.Effect<SubagentSession, SpawnError, Scope.Scope>;
|
||||
}
|
||||
|
||||
interface SubagentSession {
|
||||
readonly meta: Effect.Effect<SubagentMeta>; // snapshot; also updated via MetaChanged
|
||||
/** All activity. Single consumer (the manager). Ends after the final RunSettled on close. */
|
||||
readonly events: Stream.Stream<SubagentEvent, never>;
|
||||
/** Steer while running, or start a fresh run when idle (v1 `manager.send` semantics). */
|
||||
send(text: string): Effect.Effect<void, SendError>;
|
||||
/** Interrupt the active run; resolves when the backend acknowledges. Bounded by the caller. */
|
||||
readonly interrupt: Effect.Effect<void>;
|
||||
/** Resolves with the outcome of the most recent run (mirrors the last RunSettled). */
|
||||
readonly awaitSettled: Effect.Effect<RunOutcome>;
|
||||
}
|
||||
|
||||
// Registry: a plain ServiceMap.Key holding a ReadonlyMap<BackendName, SubagentBackend>,
|
||||
// built by a Layer that collects the three backend layers. Adding a 4th backend = one
|
||||
// new file + one line in the registry layer.
|
||||
class BackendRegistry extends ServiceMap.Key<BackendRegistry,
|
||||
ReadonlyMap<BackendName, SubagentBackend>>()("subagents/BackendRegistry") {}
|
||||
```
|
||||
|
||||
Design choices worth calling out:
|
||||
|
||||
- **`spawn` is scoped, not paired with an explicit `dispose`.** The manager opens one
|
||||
`Scope` per subagent and closes it on cancel/prune/disposeAll — this replaces v1's
|
||||
`shutdownAndDisposeChildSession` + WeakMap-idempotence machinery with Effect's own
|
||||
guaranteed-once finalization. Timeout-bounded teardown (`5s`, then force) is a
|
||||
finalizer concern inside each backend.
|
||||
- **`send` unifies steer/new-run.** v1's `manager.send` already had these semantics; the
|
||||
interface keeps the decision inside the backend because "is a run active" is
|
||||
backend-native state.
|
||||
- **Events, not message arrays, are the contract.** The manager folds events into
|
||||
snapshots; backends never expose native message types. The pi backend has the richest
|
||||
native data and simply down-converts.
|
||||
- **No `exec()` one-shot operation in v1 of this extension** — see Open Questions; the
|
||||
interface deliberately leaves room to add `exec(task): Effect<RunOutcome>` later
|
||||
without touching the manager.
|
||||
|
||||
### 3.5 `SubagentManager` service (`src/manager.ts`)
|
||||
|
||||
Owns the registry of running/finished subagents. Effect service; internally a `Ref` (or
|
||||
plain mutable map guarded by the single-threaded JS model) of entries plus a
|
||||
**synchronous read model** for the TUI.
|
||||
|
||||
```ts
|
||||
interface SubagentEntry {
|
||||
id: string; // "sa-N", same scheme as v1
|
||||
backend: BackendName;
|
||||
title: string; prompt: string; cwd: string;
|
||||
scope: Scope.Closeable; // owns the SubagentSession
|
||||
session: SubagentSession;
|
||||
eventPump: Fiber.Fiber<void>; // folds events → snapshot, fires settle hooks
|
||||
}
|
||||
|
||||
interface SubagentSnapshot { // what the UI and tools read; plain immutable data
|
||||
id: string; backend: BackendName; title: string; cwd: string;
|
||||
status: SubagentStatus;
|
||||
createdAt: number; settledAt?: number;
|
||||
errorText?: string;
|
||||
meta: SubagentMeta;
|
||||
usage: { tokens?: number; contextWindow?: number };
|
||||
transcript: ReadonlyArray<TranscriptItem>; // finalized items
|
||||
liveAssistant?: { text: string; thinking: string };
|
||||
liveTools: ReadonlyArray<LiveToolState>; // v1's LiveToolEvent, verbatim
|
||||
queued: ReadonlyArray<{ text: string; kind: "steer" | "follow-up" }>;
|
||||
finalText: string; // last Completed finalText (v1 finalOutput)
|
||||
latestText: string; // finalText or live buffer (v1 latestOutput)
|
||||
turns: number; // count of AssistantMessage events (for subagent_check)
|
||||
}
|
||||
|
||||
class SubagentManager extends ServiceMap.Key<SubagentManager, {
|
||||
spawn(backend: BackendName, task: SpawnTask):
|
||||
Effect.Effect<SubagentSnapshot, SpawnError | ConcurrencyLimitError | BackendUnavailable>;
|
||||
waitFor(ids: string[], onPending?: (pending: string[]) => void):
|
||||
Effect.Effect<void, SubagentNotFound>; // interruption = tool signal abort
|
||||
cancel(ids: string[]): Effect.Effect<CancelReport, SubagentNotFound>;
|
||||
send(id: string, text: string): Effect.Effect<void, SubagentNotFound | SendError>;
|
||||
get(id: string): Effect.Effect<SubagentSnapshot | undefined>;
|
||||
list: Effect.Effect<ReadonlyArray<SubagentSnapshot>>;
|
||||
disposeAll: Effect.Effect<void>;
|
||||
/** Synchronous read model for the TUI (see 3.6). */
|
||||
readonly view: SubagentReadModel;
|
||||
}>()("subagents/SubagentManager") {}
|
||||
```
|
||||
|
||||
Behavior preserved from v1, expressed in Effect terms:
|
||||
|
||||
- **Concurrency cap**: `MAX_RUNNING = 4` enforced with a synchronous
|
||||
reserve-before-first-yield counter (same race-avoidance rationale as v1); the cap
|
||||
counts *running* agents across all backends (see Open Questions for per-backend caps).
|
||||
- **Settlement**: the per-subagent event pump fiber updates the snapshot on every event;
|
||||
on `RunSettled` it computes `status`/`errorText` (bounded to 4096 chars) and invokes
|
||||
the settle hook with `consumed = waitInterest > 0`. `waitFor` keeps the same
|
||||
wait-interest refcounts (a `Ref<Map<string, number>>`) and wakes on snapshot changes
|
||||
(a `Latch`/`PubSub`-based "next change" primitive replacing v1's resolver array).
|
||||
- **Cancel**: mark consumed → `session.interrupt` with 5s bound → close scope on
|
||||
timeout → wait for settle. Same "already \<status\>" reporting.
|
||||
- **Pruning**: `MAX_TRACKED = 64`, oldest settled non-wait-interested entries pruned by
|
||||
closing their scopes; cleanup tracked so `disposeAll` can await it.
|
||||
- **Settle → result delivery hook**: the manager exposes `onSettled` wiring identical in
|
||||
spirit to v1: the extension layer registers a callback that defers into the
|
||||
`result-delivery` buffer and flushes on parent idle / `agent_settled`. The
|
||||
`createDeferredResultDelivery` module is copied over unchanged (it is pure and already
|
||||
has a test).
|
||||
|
||||
### 3.6 Synchronous read model for the TUI (`src/read-model.ts`)
|
||||
|
||||
The TUI components (`Component` classes with `render(width)`/`handleInput(data)`) are
|
||||
imperative and render synchronously — they cannot `yield*` effects. Bridge:
|
||||
|
||||
```ts
|
||||
interface SubagentReadModel {
|
||||
list(): ReadonlyArray<SubagentSnapshot>; // sync snapshot reads
|
||||
get(id: string): SubagentSnapshot | undefined;
|
||||
subscribe(listener: () => void): () => void; // any-change notification (dashboard, footer)
|
||||
subscribeTo(id: string, l: () => void): () => void; // per-agent (takeover view)
|
||||
// sync fire-and-forget commands, executed via the ManagedRuntime under the hood:
|
||||
requestSend(id: string, text: string): void; // TakeoverView input submit
|
||||
requestAbort(id: string): void; // dashboard `x`, takeover app.clear
|
||||
}
|
||||
```
|
||||
|
||||
The manager's event pump writes each new snapshot into this store (plain mutable map +
|
||||
listener set) as its last step, so the UI is always at most one microtask behind the
|
||||
Effect world and v1's dashboard/takeover code ports with only these substitutions:
|
||||
|
||||
| v1 | v2 |
|
||||
|---|---|
|
||||
| `manager.list()` / `manager.get(id)` | `view.list()` / `view.get(id)` |
|
||||
| `manager.addChangeListener` | `view.subscribe` |
|
||||
| `sub.session.subscribe(handleSessionEvent)` + local live-tool map | `view.subscribeTo(id, ...)` + read `snapshot.liveTools` / `liveAssistant` (the fold moved into the manager) |
|
||||
| `manager.send(sub, text)` / `manager.abort(sub)` | `view.requestSend(id, text)` / `view.requestAbort(id)` |
|
||||
| `buildTranscriptLines(sub, ...)` reading `session.messages` | `buildTranscriptLines(snapshot, ...)` reading `snapshot.transcript` + live state + `snapshot.queued` |
|
||||
|
||||
Keybindings, layout math, scroll behavior, 1Hz ticker, 50ms render throttle, sanitize
|
||||
logic: copied as-is.
|
||||
|
||||
### 3.7 Extension edge (`index.ts` + `src/runtime.ts`)
|
||||
|
||||
```
|
||||
Layer graph:
|
||||
PiBackendStub.layer ─┐
|
||||
ClaudeBackendStub.layer ─┼→ BackendRegistry.layer ─→ SubagentManager.layer ─→ AppLayer
|
||||
CodexBackendStub.layer ─┘
|
||||
```
|
||||
|
||||
- `const runtime = ManagedRuntime.make(AppLayer)` — created lazily on first use (per the
|
||||
extension-docs guidance to not start background resources in the factory;
|
||||
`ManagedRuntime` builds its layer on first run, which satisfies this, but we still
|
||||
gate creation behind `session_start`). `session_shutdown`: `runtime.runPromise(
|
||||
manager.disposeAll)` then `await runtime.dispose()`, then recreate on the next
|
||||
`session_start` (handles `/new`, `/resume`, `/reload`).
|
||||
- **Tool handlers are the async boundary.** Each `execute(toolCallId, params, signal,
|
||||
onUpdate, ctx)` builds one `Effect.gen` program and runs it with
|
||||
`runtime.runPromise(program, { signal })`; tool-visible errors are converted from
|
||||
tagged errors to `Error` messages matching v1 wording. `onUpdate` and `ctx`
|
||||
(model registry, cwd, trust) are captured into the program as plain values/callbacks.
|
||||
- **Tool schema change:** `subagent_spawn` gains
|
||||
`agent: StringEnum(["pi", "claude", "codex"])` (optional, default `"pi"`), and
|
||||
`model`/`provider`/`reasoning_effort` keep their v1 shapes but are documented as
|
||||
backend-interpreted (pi validates against the registry; claude/codex validate against
|
||||
their own known-model rules — stubs accept anything). `describeSubagent` lines and the
|
||||
dashboard gain the backend name (e.g. `sa-3 [running] "title" (codex, gpt-5-codex,
|
||||
41%/272k, 1m32s, /repo)`).
|
||||
- `pi.registerMessageRenderer("subagent-result", ...)`, `pi.registerCommand(
|
||||
"subagents", ...)`, footer status updates, and the result-delivery flush hooks
|
||||
(`agent_settled`, idle-check on settle) are wired exactly like v1 — these all live
|
||||
outside the runtime and call into it only via `runPromise`/the read model.
|
||||
|
||||
### 3.8 What the stubs do (v1 of this extension)
|
||||
|
||||
All three backends share a `createStubSession(profile)` helper (`src/backends/stub.ts`)
|
||||
that fakes a plausible session so the manager, tools, result delivery, and both TUI
|
||||
views are exercised end to end:
|
||||
|
||||
- **spawn**: emits `MetaChanged` (backend-flavored model label + fake session file path
|
||||
under `os.tmpdir()`, e.g. `.../subagents-stub/sa-1.jsonl`, actually written with the
|
||||
transcript so "full transcript in session file" pointers resolve), then `RunStarted`,
|
||||
then a scripted turn: 2–3 `AssistantDelta` batches on a timer (~200ms cadence so
|
||||
streaming is visible), one fake `ToolStart/Update/End` cycle (`bash` with an args
|
||||
preview), `UsageChanged` ramping tokens, a final `AssistantMessage`, and `RunSettled`
|
||||
with `Completed` — final text echoes the task: `"[stub:claude] completed: <first 200
|
||||
chars of prompt>"`. Total runtime ~3–6s (configurable per profile) so `subagent_wait`,
|
||||
the footer counters, and the dashboard's running→done transition are observable.
|
||||
- **send**: emits `UserMessage` + `QueueChanged` (briefly, to exercise the queued-line
|
||||
rendering) and runs another scripted turn — so takeover steering works.
|
||||
- **interrupt**: stops the script timer and settles with `Interrupted` (→ status
|
||||
`error`, errorText `"Run was aborted"`, matching v1) — so `subagent_cancel` and the
|
||||
`x`/`app.clear` keybindings work.
|
||||
- **failure path**: a magic prompt prefix (e.g. `FAIL:`) makes the run settle with
|
||||
`Failed` — so error rendering, `errorText` rows, and failed result delivery are
|
||||
testable without real backends.
|
||||
- **Backend differentiation**: per-backend profiles vary the model label
|
||||
(`anthropic/claude-opus-4-5` vs `claude-sonnet-4-5` vs `gpt-5-codex`), fake context
|
||||
window, tool names, and delta cadence — enough to verify the UI treats backends
|
||||
uniformly. `available` returns `true` for stubs (real impls will probe binaries/SDK).
|
||||
- The **pi stub** can later be swapped for the real in-process SDK implementation by
|
||||
porting v1's `manager.ts` session code behind the same `SubagentSession` shape; that
|
||||
port is the first post-stub milestone.
|
||||
|
||||
---
|
||||
|
||||
## 4. File/module layout
|
||||
|
||||
```
|
||||
/Users/davis/.pi/agent/extensions/subagents/
|
||||
├── package.json # name, "effect": "^4.0.0-beta.x"; pi extension entry via pi.extensions
|
||||
├── package-lock.json / node_modules/ (after npm install)
|
||||
├── docs/
|
||||
│ └── design-plan.md # this document
|
||||
├── index.ts # extension factory: runtime lifecycle, 5 tools, /subagents
|
||||
│ # command, message renderer, footer status, result flush hooks
|
||||
└── src/
|
||||
├── domain.ts # BackendName, SubagentStatus, SpawnTask, SubagentEvent,
|
||||
│ # RunOutcome, TranscriptItem/Part, SubagentSnapshot, tagged errors
|
||||
├── backend.ts # SubagentBackend + SubagentSession interfaces, BackendRegistry
|
||||
│ # key + registry layer
|
||||
├── backends/
|
||||
│ ├── stub.ts # shared scripted fake-session machinery
|
||||
│ ├── pi.ts # PiBackend layer (v1: stub profile; later: real pi SDK sessions)
|
||||
│ ├── claude.ts # ClaudeBackend layer (v1: stub; later: @anthropic-ai/claude-agent-sdk)
|
||||
│ └── codex.ts # CodexBackend layer (v1: stub; later: codex app-server JSON-RPC)
|
||||
├── manager.ts # SubagentManager service + layer: registry, cap, waitFor,
|
||||
│ # cancel, prune, settle hook, event-fold into snapshots
|
||||
├── read-model.ts # sync SubagentReadModel bridge for the TUI
|
||||
├── runtime.ts # AppLayer composition + ManagedRuntime create/dispose helpers
|
||||
├── result-delivery.ts # deferred delivery buffer (copied from v1, unchanged)
|
||||
├── result-delivery.test.ts
|
||||
├── prompt.ts # all model-facing strings (v1 copy + `agent` param description)
|
||||
├── format.ts # elapsed/context-utilization/activity-status formatting
|
||||
│ # (merged copies of ../shared/{context-utilization,activity-status}.ts)
|
||||
└── ui/
|
||||
├── transcript.ts # sanitize + buildTranscriptLines over SubagentSnapshot
|
||||
└── takeover.ts # SubagentDashboard + TakeoverView + openSubagentPicker (ported)
|
||||
```
|
||||
|
||||
Notes:
|
||||
- `package.json` is needed because `effect` is an npm dependency (extension-with-deps
|
||||
style from the extension docs). Everything else avoids new dependencies.
|
||||
- v1's `child-session.ts` trust/tool-policy helpers are **not** copied in v1 of v2 (the
|
||||
stubs don't need them); the real pi backend will bring the needed subset into
|
||||
`backends/pi.ts` when implemented. The `resolveStandaloneChildProjectTrust` logic *is*
|
||||
still referenced by the design (SpawnTask.parentContext.projectTrusted) so the tool
|
||||
layer computes trust the same way v1 does.
|
||||
- Suggested project scripts (per house rules, to be added): `check` (`tsc --noEmit`),
|
||||
`test` (`node --test` or vitest for `result-delivery` + manager fold tests against
|
||||
stub backends).
|
||||
|
||||
---
|
||||
|
||||
## 5. Migration/coexistence note
|
||||
|
||||
v1 and v2 register the same tool names (`subagent_spawn`, ...) and the same
|
||||
`/subagents` command. While both live in `~/.pi/agent/extensions/`, pi will suffix
|
||||
duplicate commands (`/subagents:1`, `/subagents:2`) and both tool sets would be
|
||||
registered. During development, either (a) v2 uses temporary names
|
||||
(`subagent2_spawn`, `/subagents2`), or (b) v1 is moved out of the auto-discovery dir.
|
||||
Recommendation: (a) during development, rename to final names when v2 replaces v1.
|
||||
|
||||
---
|
||||
|
||||
## 6. Open questions (need user input)
|
||||
|
||||
1. **Per-backend spawn options.** v1's `model`/`provider`/`reasoning_effort` are
|
||||
pi-shaped. Options: (a) keep one generic `model` string + `reasoning_effort` that
|
||||
each backend interprets (proposed above — simplest for the LLM); (b) add a
|
||||
`backend_options` free-form object; (c) per-backend defaults in a config file with no
|
||||
per-spawn override. Which surface do you want the parent LLM to have?
|
||||
2. **One-shot exec mode.** Should the interface expose a separate cheap
|
||||
`exec(task): Effect<RunOutcome>` (mapping to `codex exec` / `claude -p
|
||||
--output-format json` / a fresh in-memory pi session), or is one interactive-session
|
||||
path enough? Exec would forfeit takeover/steering for that subagent — is a
|
||||
`mode: "session" | "exec"` spawn parameter desirable, or backend-internal
|
||||
optimization only?
|
||||
3. **Permissions/sandboxing for Claude/Codex children.** Subagents are headless, so
|
||||
interactive permission prompts are impossible. Do we run Claude with
|
||||
`bypassPermissions`/`--dangerously-skip-permissions` and Codex with
|
||||
`--full-auto`-style sandbox + never-ask approval policy? Should this be a global
|
||||
extension setting, per-spawn, or hardcoded? (Pi children inherit v1's trust-store
|
||||
logic — keep that as-is?)
|
||||
4. **Concurrency cap scope.** Keep one global `MAX_RUNNING = 4`, or per-backend caps
|
||||
(e.g. 4 pi + 2 claude + 2 codex)? Global is proposed as default.
|
||||
5. **Steering support parity in real backends.** Codex steering means
|
||||
interrupt-then-new-turn or queued `sendUserTurn`; Claude requires streaming-input
|
||||
mode from the start. OK to declare `capabilities.steering` and have the TakeoverView
|
||||
input show "(steering not supported)" if a backend can't, or is steering a hard
|
||||
requirement for all three?
|
||||
6. **Model/thinking inheritance across backends.** When `agent: "claude"` and no model
|
||||
given, what's the default (e.g. always `opus`/`sonnet`)? Inheriting the parent pi
|
||||
model is meaningless cross-backend. Proposal: per-backend default model in a small
|
||||
config block; confirm.
|
||||
7. **Binary/SDK discovery + failure UX.** When `codex`/`claude` isn't installed or has
|
||||
no credentials, should `subagent_spawn` fail fast with a clear tool error (proposed),
|
||||
or should the backends be hidden from the `agent` enum dynamically?
|
||||
8. **Result truncation budgets.** Keep v1's numbers (24KB result message, 48KB wait
|
||||
total, 16KB per agent, 2KB check preview) unchanged?
|
||||
9. **Effect version pinning.** Effect v4 is beta — pin an exact `4.0.0-beta.x` and
|
||||
accept manual bumps, or track the beta dist-tag?
|
||||
10. **Persistence across reloads.** v1 loses all subagents on `session_shutdown`
|
||||
(disposeAll). Codex/Claude children are external processes that *could* outlive a
|
||||
pi reload — should v2 keep v1's kill-everything behavior (proposed for v1 of v2) or
|
||||
plan for reattach later?
|
||||
@@ -0,0 +1,354 @@
|
||||
# Building a pi extension on the Effect v4 setup
|
||||
|
||||
> Practical, migration-oriented companion to [`effect-v4-notes.md`](./effect-v4-notes.md).
|
||||
> That file is the Effect API cheat-sheet (v3→v4 renames, child processes, concurrency);
|
||||
> **this** file is how to stand up _one pi extension_ on the same toolchain and where to
|
||||
> draw the line between "wrap in Effect" and "leave as plain TS".
|
||||
>
|
||||
> **Verified against** `effect@4.0.0-beta.98`, `@effect/platform-node@4.0.0-beta.98`,
|
||||
> `@effect/tsgo@0.19.0`, `typescript@7.0.2` (checked 2026-07-13, in `extensions/subagents`).
|
||||
> `npm run check` there passes clean — use it as the reference implementation.
|
||||
>
|
||||
> Audience: the agents migrating `firecrawl-search`, `ask-user`, `model-info`,
|
||||
> `git-info`, `ui-customization`, and `copy-all`.
|
||||
|
||||
---
|
||||
|
||||
## 0. The one rule: Effect is for the async core, not the whole file
|
||||
|
||||
A pi extension's public surface is **plain callbacks** the host calls:
|
||||
`export default function (pi: ExtensionAPI)`, `pi.registerTool({ …, async execute() })`,
|
||||
`pi.registerCommand`, `pi.on(...)`, renderers. None of that changes. Effect lives _inside_
|
||||
those callbacks — you run an effect at the top of `execute` and return its value.
|
||||
|
||||
Reach for Effect only where you actually get something from it:
|
||||
|
||||
- **Yes:** async work that needs typed errors, cancellation via the tool `AbortSignal`,
|
||||
timeouts, retries/polling, or a resource whose lifetime must outlive one call
|
||||
(child process, subscription) → child processes (`git-info`, `copy-all`), the
|
||||
Firecrawl SDK calls (`firecrawl-search`), git/gh polling (`git-info`).
|
||||
- **No / barely:** pure TUI popups and rendering (`ask-user`, `ui-customization`),
|
||||
cross-extension channel plumbing, cost/token bookkeeping (`model-info`). These are
|
||||
synchronous or already-Promise UI code; wrapping them in Effect adds ceremony and no
|
||||
safety. Migrate them by keeping the logic and only touching whatever genuinely async
|
||||
part benefits (usually nothing).
|
||||
|
||||
If an extension has no async core worth typing, the "migration" may be just adopting the
|
||||
toolchain (§1) and leaving the body plain. Don't invent an Effect layer to have one.
|
||||
|
||||
---
|
||||
|
||||
## 1. Per-extension toolchain (copy this exactly)
|
||||
|
||||
Each extension is its own npm package with its own `node_modules`. Replicate the pinned
|
||||
setup — do **not** float the versions.
|
||||
|
||||
`package.json`:
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"name": "<ext>",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
"check": "tsc --noEmit -p .",
|
||||
"prepare": "effect-tsgo patch", // patches the Effect LS into the tsgo binary
|
||||
},
|
||||
"dependencies": {
|
||||
"effect": "4.0.0-beta.98", // EXACT pin, no ^
|
||||
"@effect/platform-node": "4.0.0-beta.98", // only if you touch fs / child processes
|
||||
},
|
||||
"devDependencies": {
|
||||
"@effect/tsgo": "^0.19.0",
|
||||
"typescript": "^7.0.2",
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
`tsconfig.json` (extends the repo root, adds the Effect language-service plugin):
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"extends": "../../tsconfig.json",
|
||||
"compilerOptions": {
|
||||
"plugins": [{ "name": "@effect/language-service" }],
|
||||
},
|
||||
"include": ["index.ts", "src/**/*.ts", "*.test.ts"],
|
||||
}
|
||||
```
|
||||
|
||||
The root `tsconfig.json` already sets `strict`, `module`/`moduleResolution: NodeNext`,
|
||||
`verbatimModuleSyntax`, `allowImportingTsExtensions`, `target: ES2022`, `types: ["node"]`.
|
||||
Keep local `.ts` imports written **with the `.ts` extension** (`./src/manager.ts`), matching
|
||||
the house style.
|
||||
|
||||
### The LSP / language-service, precisely
|
||||
|
||||
- `typescript@7` is the **native (tsgo) TypeScript** — its `tsc` is a Go binary, and it's
|
||||
what `npm run check` runs.
|
||||
- `@effect/language-service` is **not an installed npm package** here (you won't find it in
|
||||
`node_modules`). It's delivered by `effect-tsgo patch`, run automatically by the `prepare`
|
||||
lifecycle script on `npm install`. The patch injects the Effect Language Service into the
|
||||
tsgo binary; the tsconfig `plugins` entry then turns on Effect-aware editor diagnostics and
|
||||
quickfixes (e.g. "yield missing services", floating effects).
|
||||
- Practical sequence for a fresh/edited extension: `npm install` (runs the patch) →
|
||||
`npm run check`. If editor diagnostics look stale, re-run `npx effect-tsgo patch`;
|
||||
`npx effect-tsgo get-exe-path` prints the patched binary it resolved.
|
||||
- The plugin drives the _editor_; it does not change `tsc` exit codes. `npm run check` is
|
||||
still your ground-truth green/red signal.
|
||||
|
||||
---
|
||||
|
||||
## 2. The async boundary: one `ManagedRuntime` + a `runTool` helper
|
||||
|
||||
Every extension that uses Effect needs exactly this shape. Build **one** runtime lazily,
|
||||
dispose it on shutdown, and funnel every tool handler through a single helper that turns
|
||||
Effect exits into what pi expects (a value, or a thrown `Error`; interruption → a friendly
|
||||
message). This is lifted verbatim from `src/runtime.ts` — reuse it.
|
||||
|
||||
```ts
|
||||
// src/runtime.ts
|
||||
import { Cause, Exit, Layer, ManagedRuntime, type Effect } from "effect";
|
||||
import { NodeServices } from "@effect/platform-node"; // only if you need fs / processes
|
||||
|
||||
// Compose your services here (see §3). NodeServices.layer =
|
||||
// ChildProcessSpawner | FileSystem | Path | Crypto | Stdio | Terminal.
|
||||
const AppLayer = Layer.mergeAll(NodeServices.layer /*, MyServiceLive */);
|
||||
|
||||
export function createRuntime() {
|
||||
return ManagedRuntime.make(AppLayer);
|
||||
}
|
||||
export type ExtRuntime = ReturnType<typeof createRuntime>;
|
||||
|
||||
/** Run an effect from an async handler: value on success, thrown Error otherwise. */
|
||||
export async function runTool<A, E>(
|
||||
runtime: ExtRuntime,
|
||||
effect: Effect.Effect<A, E>,
|
||||
options: { signal?: AbortSignal; interruptMessage?: string } = {},
|
||||
) {
|
||||
const exit = await runtime.runPromiseExit(
|
||||
effect,
|
||||
options.signal ? { signal: options.signal } : undefined,
|
||||
);
|
||||
if (Exit.isSuccess(exit)) return exit.value;
|
||||
if (Cause.hasInterruptsOnly(exit.cause)) {
|
||||
throw new Error(options.interruptMessage ?? "Operation was aborted.");
|
||||
}
|
||||
const [first] = Cause.prettyErrors(exit.cause);
|
||||
throw new Error(first?.message ?? Cause.pretty(exit.cause));
|
||||
}
|
||||
```
|
||||
|
||||
Wire it into the extension lifecycle (lazy build, dispose on shutdown):
|
||||
|
||||
```ts
|
||||
// index.ts
|
||||
export default function (pi: ExtensionAPI) {
|
||||
let runtime: ExtRuntime | undefined;
|
||||
const getRuntime = () => (runtime ??= createRuntime());
|
||||
|
||||
pi.registerTool({
|
||||
name: "my_tool",
|
||||
parameters: Type.Object({/* typebox */}),
|
||||
async execute(_id, params, signal) {
|
||||
return await runTool(getRuntime(), myEffect(params), {
|
||||
signal,
|
||||
interruptMessage: "Cancelled.",
|
||||
});
|
||||
},
|
||||
});
|
||||
|
||||
pi.on("session_shutdown", async () => {
|
||||
const closing = runtime;
|
||||
runtime = undefined;
|
||||
await closing?.dispose(); // runs all finalizers: kills scoped child processes, etc.
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
Key facts:
|
||||
|
||||
- `runPromiseExit(effect, { signal })` — passing the tool's `AbortSignal` makes cancelling
|
||||
the tool interrupt the fiber; scoped resources (child processes) are torn down.
|
||||
- `runtime.runFork(effect)` for fire-and-forget background work (polling loops, §5).
|
||||
- `dispose()` is the single teardown hook — put resource cleanup in Effect finalizers, not
|
||||
in ad-hoc `session_shutdown` code.
|
||||
- If an extension has **no** services (pure `Effect.tryPromise` calls, no fs/process), you
|
||||
can skip `NodeServices.layer` and even skip `ManagedRuntime` — `Effect.runPromiseExit(effect, { signal })`
|
||||
works standalone. Add the runtime only when you have a `Layer` to share.
|
||||
|
||||
---
|
||||
|
||||
## 3. Services, layers, errors (house style)
|
||||
|
||||
Mirror the subagents code. Class-style `Context.Service`, `Layer.effect`/`Layer.sync`,
|
||||
`Data.TaggedError` for domain errors.
|
||||
|
||||
```ts
|
||||
import { Context, Data, Effect, Layer } from "effect";
|
||||
|
||||
// Domain error — yieldable, tag-narrowable with Effect.catchTag:
|
||||
export class GitError extends Data.TaggedError("GitError")<{
|
||||
readonly message: string;
|
||||
}> {}
|
||||
|
||||
// Service: the class value is the key AND the type.
|
||||
export interface GitShape {
|
||||
readonly status: Effect.Effect<string, GitError>;
|
||||
}
|
||||
export class Git extends Context.Service<Git, GitShape>()("gitinfo/Git") {}
|
||||
|
||||
// Layer building it (Layer.effect can use scoped resources; Layer.sync for pure):
|
||||
export const GitLive: Layer.Layer<Git, never, ChildProcessSpawner> =
|
||||
Layer.effect(
|
||||
Git,
|
||||
Effect.gen(function* () {
|
||||
const spawner = yield* ChildProcessSpawner; // dependency, provided by NodeServices.layer
|
||||
return Git.of({
|
||||
status: spawner
|
||||
.string(ChildProcess.make("git", ["status", "--porcelain"]))
|
||||
.pipe(
|
||||
Effect.mapError(
|
||||
(cause) => new GitError({ message: String(cause) }),
|
||||
),
|
||||
),
|
||||
});
|
||||
}),
|
||||
);
|
||||
```
|
||||
|
||||
Provide dependencies with `Layer.provide`, compose with `Layer.mergeAll` (see §2 `AppLayer`).
|
||||
Full rename table (`Effect.fork`→`forkChild`, `catchAll`→`catch`, `Either`→`Result`, …) and
|
||||
the child-process / concurrency APIs live in `effect-v4-notes.md` — don't re-derive them.
|
||||
|
||||
---
|
||||
|
||||
## 4. Recipe: wrapping a Promise SDK (firecrawl-search)
|
||||
|
||||
The Firecrawl client is Promise-based. Wrap each call in `Effect.tryPromise` with a typed
|
||||
error; the callback receives an `AbortSignal` tied to fiber interruption — forward it to any
|
||||
SDK that accepts one so tool cancellation propagates.
|
||||
|
||||
```ts
|
||||
import { Data, Effect } from "effect";
|
||||
|
||||
class FirecrawlError extends Data.TaggedError("FirecrawlError")<{
|
||||
readonly cause: unknown;
|
||||
}> {}
|
||||
|
||||
const search = (client: Firecrawl, query: string) =>
|
||||
Effect.tryPromise({
|
||||
try: (signal) => client.search(query, {/* …, signal? if supported */}),
|
||||
catch: (cause) => new FirecrawlError({ cause }),
|
||||
});
|
||||
```
|
||||
|
||||
There's no `Layer` here unless you want the client as a service. For a single tool this is
|
||||
fine to call directly through `runTool` (or even plain `Effect.runPromiseExit`, §2). Keep the
|
||||
existing `.env`/API-key reading and result truncation as plain TS — no reason to Effect-ify
|
||||
string trimming.
|
||||
|
||||
---
|
||||
|
||||
## 5. Recipe: child processes + timeout + polling (git-info, copy-all)
|
||||
|
||||
`git-info` shells out to `git`/`gh` with per-command timeouts and polls on an interval;
|
||||
`copy-all` pipes text into `pbcopy`. Two viable levels — pick the lightest that fits.
|
||||
|
||||
**Simple, one-shot, small:** if all you do is "run a command, capture stdout, with a
|
||||
timeout," the Effect win is `Effect.timeout` + interruption killing the child. Use the
|
||||
spawner service:
|
||||
|
||||
```ts
|
||||
import { Effect } from "effect";
|
||||
import { ChildProcess } from "effect/unstable/process";
|
||||
import { ChildProcessSpawner } from "effect/unstable/process/ChildProcessSpawner";
|
||||
|
||||
const gitStatus = Effect.gen(function* () {
|
||||
const spawner = yield* ChildProcessSpawner;
|
||||
return yield* spawner
|
||||
.string(ChildProcess.make("git", ["status", "--porcelain"], { cwd }))
|
||||
.pipe(Effect.timeout("3 seconds")); // fails with Cause.TimeoutError (tag "TimeoutError")
|
||||
});
|
||||
```
|
||||
|
||||
Note the import gotcha: `ChildProcessSpawner` the **class** comes from the
|
||||
`.../ChildProcessSpawner` submodule; the `effect/unstable/process` index gives you the
|
||||
namespace. Provide `NodeServices.layer` (or `ChildProcessSpawner.layer`). Full command
|
||||
builder / streaming / kill semantics are in `effect-v4-notes.md §6`.
|
||||
|
||||
**Polling on an interval:** replace `setInterval` with a forked, scheduled effect so it
|
||||
cancels cleanly on dispose:
|
||||
|
||||
```ts
|
||||
import { Effect, Schedule } from "effect";
|
||||
|
||||
const pollLoop = refresh.pipe(
|
||||
Effect.catchCause(() => Effect.void), // don't let one failure kill the loop
|
||||
Effect.repeat(Schedule.spaced("3 seconds")),
|
||||
);
|
||||
const fiber = runtime.runFork(pollLoop); // interrupted by runtime.dispose()
|
||||
```
|
||||
|
||||
**When to stay plain:** `copy-all` spawning `pbcopy` is a trivial one-shot with no
|
||||
cancellation need — the existing `node:child_process` + Promise wrapper is honestly fine.
|
||||
Migrate it only for consistency; if you do, `Effect.callback` around `child.once("exit", …)`
|
||||
(see notes §4) is the minimal wrapper. Don't add a service/layer for a clipboard write.
|
||||
|
||||
---
|
||||
|
||||
## 6. Recipe: UI popups & rendering (ask-user, ui-customization, model-info)
|
||||
|
||||
These are the "leave it mostly plain" cases.
|
||||
|
||||
- `ask-user` is a TUI popup that resolves a Promise when the user picks. That Promise already
|
||||
models the one async thing. Effect adds nothing; if you want uniformity, wrap the final
|
||||
await in `Effect.tryPromise` at the boundary and stop there. Do **not** build a service.
|
||||
- `ui-customization` and `model-info` are renderers / event bookkeepers driven by
|
||||
`pi.on(...)` and cross-extension channels (`shared/dashboard-state.ts`). Channels are a
|
||||
pi-native mechanism — keep them. State counting and formatting stay synchronous TS.
|
||||
- If `model-info` has a periodic "live update" tick, the §5 polling pattern applies; but a
|
||||
plain timer here is also acceptable since there's no resource to tear down.
|
||||
|
||||
The migration bar for these: adopt the toolchain (§1) so they typecheck under TS7 + the
|
||||
Effect LS, and only touch runtime code that has a real async/resource concern.
|
||||
|
||||
---
|
||||
|
||||
## 7. Verify — scoped to the one extension
|
||||
|
||||
Run everything from inside the extension directory so you never trigger a root/global build
|
||||
or format:
|
||||
|
||||
```bash
|
||||
cd extensions/<ext>
|
||||
npm install # first time / after dep or script edits — runs `prepare` (effect-tsgo patch)
|
||||
npm run check # tsc --noEmit -p . ← ground-truth green/red
|
||||
npm run test # only if the extension defines tests; keep them minimal
|
||||
```
|
||||
|
||||
`extensions/subagents` is the known-green reference: `npm run check` there exits 0 against
|
||||
the pinned versions. If a migrated extension fails `check` with `Effect.fork`/`ServiceMap`/
|
||||
`Either` errors, it's using stale v3/early-beta APIs — consult the rename table in
|
||||
`effect-v4-notes.md`.
|
||||
|
||||
---
|
||||
|
||||
## 8. Don'ts (keep it lean)
|
||||
|
||||
1. **Don't float versions.** Pin `effect` and `@effect/platform-node` to the exact same
|
||||
`4.0.0-beta.98`; `unstable/*` can break between betas.
|
||||
2. **Don't Effect-ify pure/UI code.** No service or layer for a clipboard write, a string
|
||||
truncation, or a popup. §0 is the test: is there a typed-error / cancellation / resource /
|
||||
retry concern? If not, leave it.
|
||||
3. **Don't guess APIs.** If it's not in this guide or `effect-v4-notes.md` and you haven't
|
||||
grepped it in `node_modules/effect/dist/*.d.ts`, don't write it. Watch for hallucinated
|
||||
`ServiceMap` (reverted to `Context`), `Effect.fork` (→ `forkChild`), `Effect.either`
|
||||
(→ `Effect.result`).
|
||||
4. **Don't hand-roll teardown.** Put cleanup in Effect finalizers and let `runtime.dispose()`
|
||||
in `session_shutdown` run them.
|
||||
5. **Don't over-test.** A `check` that passes plus one focused runtime test (where behavior
|
||||
is non-obvious) beats a wall of defensive unit tests.
|
||||
6. **Don't run root scripts.** No repo-root `tsc`, `prettier`, or `npm run format`; stay
|
||||
inside `extensions/<ext>`.
|
||||
</content>
|
||||
@@ -0,0 +1,571 @@
|
||||
# Effect v4 — Practical Notes for the Subagents Extension
|
||||
|
||||
> **Building a new pi extension on this stack?** Start with the migration-oriented
|
||||
> [`effect-v4-extension-guide.md`](./effect-v4-extension-guide.md) (toolchain setup, the
|
||||
> `ManagedRuntime` + `runTool` boundary, when *not* to use Effect, per-extension recipes).
|
||||
> This file is the deeper Effect API reference it points back to.
|
||||
|
||||
> **Verified against:** `effect@4.0.0-beta.98` and `@effect/platform-node@4.0.0-beta.98`
|
||||
> (npm dist-tag `beta`, checked 2026-07-13). Every snippet in this doc was type-checked
|
||||
> with `tsc --strict` against these packages, and the process-spawning / runtime snippets
|
||||
> were executed with Node 24. v4 source lives in the **`Effect-TS/effect-smol`** repo
|
||||
> (not `Effect-TS/effect`); the official migration guide is
|
||||
> [`effect-smol/MIGRATION.md`](https://github.com/Effect-TS/effect-smol/blob/main/MIGRATION.md)
|
||||
> plus the `migration/*.md` files next to it (including a machine-readable
|
||||
> `migration/v3-to-v4.md` rename map).
|
||||
|
||||
## Install
|
||||
|
||||
```bash
|
||||
npm install effect@beta @effect/platform-node@beta
|
||||
```
|
||||
|
||||
Big structural facts:
|
||||
|
||||
- **One version number for everything.** All ecosystem packages release together at
|
||||
`4.0.0-beta.N`. `@effect/platform-node` must match `effect` exactly.
|
||||
- **`@effect/platform` is gone — merged into core `effect`.** `FileSystem`, `Path`,
|
||||
`PlatformError`, `Terminal`, `Stdio` are now top-level `effect` modules.
|
||||
`@effect/platform-node` remains as the Node *implementation* package.
|
||||
- **`effect/unstable/*` namespace.** Modules that may break in minor releases:
|
||||
`effect/unstable/process` (child processes — the one we need), `http`, `rpc`, `cli`,
|
||||
`ai`, `workers`, etc. Everything outside `unstable/` follows strict semver.
|
||||
- **`"type": "module"`**, ESM-first. Works with `moduleResolution: NodeNext` or `Bundler`.
|
||||
- Runtime keep-alive is built in now: a fiber suspended on `Deferred.await` etc. keeps
|
||||
the Node process alive without `NodeRuntime.runMain` (v3 needed runMain for that).
|
||||
|
||||
## Cheat sheet: v3 → v4 renames you will actually hit
|
||||
|
||||
| v3 | v4 |
|
||||
| --- | --- |
|
||||
| `Context.Tag` / `Effect.Service` | `Context.Service` (`Effect.Service` is **gone**) |
|
||||
| `Effect.fork` | `Effect.forkChild` |
|
||||
| `Effect.forkDaemon` | `Effect.forkDetach` |
|
||||
| `Effect.async` | `Effect.callback` |
|
||||
| `Effect.catchAll` | `Effect.catch` |
|
||||
| `Effect.catchAllCause` / `catchAllDefect` | `Effect.catchCause` / `catchDefect` |
|
||||
| `Effect.zipRight` / `zipLeft` | `Effect.andThen` / `Effect.tap` |
|
||||
| `Effect.either` | `Effect.result` |
|
||||
| `Either` module | `Result` (`Result.succeed` / `Result.fail`) |
|
||||
| `Layer.scoped` | `Layer.effect` (all layers are scoped-capable now) |
|
||||
| `Mailbox` | `Queue` (Queue absorbed Mailbox: done/fail signalling built in) |
|
||||
| `FiberRef` | `Context.Reference` (module `effect/References`) |
|
||||
| `Scope.extend` | `Scope.provide` |
|
||||
| `Stream.async*` (all 4 variants) | `Stream.callback` |
|
||||
| `Stream.fromChunk(s)` | `Stream.fromArray` / `Stream.fromArrays` (Chunk de-emphasized; arrays used) |
|
||||
| `Schema.TaggedError` | `Schema.TaggedErrorClass` |
|
||||
| `@effect/platform/Command` | `effect/unstable/process` → `ChildProcess` |
|
||||
| `@effect/platform/CommandExecutor` | `effect/unstable/process` → `ChildProcessSpawner` |
|
||||
| `@effect/platform/FileSystem` | `effect/FileSystem` |
|
||||
| `Runtime.runPromise(runtime)(...)` | gone — use `ManagedRuntime` methods directly |
|
||||
| `UnknownException` (tryPromise default) | `Cause.UnknownError` |
|
||||
|
||||
Removed with no replacement: `Effect.forkAll`, `Effect.forkWithErrorHandler`.
|
||||
Early v4 betas renamed `Context` → `ServiceMap`; **that was reverted** — beta.98 uses
|
||||
`Context` again (ignore older blog posts/AI answers mentioning `ServiceMap`).
|
||||
|
||||
---
|
||||
|
||||
## 1. Basics: `Effect.gen`, running, the Promise boundary
|
||||
|
||||
Unchanged in spirit from v3. `Effect<A, E, R>` = success `A`, typed error `E`,
|
||||
required services `R`.
|
||||
|
||||
```ts
|
||||
import { Effect, Exit } from "effect"
|
||||
|
||||
const double = (n: number) => Effect.succeed(n * 2)
|
||||
|
||||
const program = Effect.gen(function* () {
|
||||
const a = yield* Effect.succeed(1)
|
||||
const b = yield* double(a)
|
||||
yield* Effect.log(`result: ${b}`)
|
||||
return b
|
||||
})
|
||||
|
||||
// Entry-point conversion — only at the outermost layer (tool handlers):
|
||||
const p: Promise<number> = Effect.runPromise(program) // rejects on failure/defect
|
||||
const pe: Promise<Exit.Exit<number, never>> = Effect.runPromiseExit(program) // never rejects
|
||||
const s: number = Effect.runSync(program) // throws if async
|
||||
const fiber = Effect.runFork(program) // fire-and-forget fiber
|
||||
```
|
||||
|
||||
`runPromise` only accepts `Effect<A, E, never>` — all services must be provided first
|
||||
(or use a `ManagedRuntime`, §8). `runPromiseExit` is the right choice inside tool
|
||||
handlers when you want to convert failures to structured results instead of throws.
|
||||
|
||||
`Effect.fn` gives you named, traced effect functions (nice stack traces):
|
||||
|
||||
```ts
|
||||
const spawnJob = Effect.fn("spawnJob")(function* (name: string) {
|
||||
yield* Effect.log(`spawning ${name}`)
|
||||
return name
|
||||
})
|
||||
// spawnJob("x") : Effect<string>
|
||||
```
|
||||
|
||||
## 2. Services & Layers
|
||||
|
||||
`Context.Tag` and the v3 `Effect.Service` helper class are **both gone**. The single
|
||||
API is `Context.Service`, in two forms:
|
||||
|
||||
```ts
|
||||
import { Context, Effect, Layer } from "effect"
|
||||
|
||||
// Class-style (recommended — class value doubles as the key):
|
||||
class Clock extends Context.Service<Clock, {
|
||||
readonly now: Effect.Effect<number>
|
||||
}>()("app/Clock") {}
|
||||
|
||||
// Function-style key:
|
||||
const Random = Context.Service<{ next: Effect.Effect<number> }>("app/Random")
|
||||
|
||||
// Yielding the key retrieves the service (keys are Effects):
|
||||
const use = Effect.gen(function* () {
|
||||
const clock = yield* Clock
|
||||
return yield* clock.now
|
||||
})
|
||||
```
|
||||
|
||||
Layers work like v3 (`Layer.scoped` merged into `Layer.effect` — every `Layer.effect`
|
||||
build can use scoped resources):
|
||||
|
||||
```ts
|
||||
const ClockLive = Layer.succeed(Clock, { now: Effect.sync(() => Date.now()) })
|
||||
|
||||
const RandomLive = Layer.effect(
|
||||
Random,
|
||||
Effect.gen(function* () {
|
||||
yield* Effect.log("building Random")
|
||||
return { next: Effect.sync(() => Math.random()) }
|
||||
})
|
||||
)
|
||||
|
||||
// Dependencies between layers:
|
||||
class Ids extends Context.Service<Ids, { readonly nextId: Effect.Effect<string> }>()("app/Ids") {}
|
||||
|
||||
const IdsLive = Layer.effect(
|
||||
Ids,
|
||||
Effect.gen(function* () {
|
||||
const random = yield* Random
|
||||
return Ids.of({ nextId: Effect.map(random.next, (n) => `id-${n}`) })
|
||||
})
|
||||
)
|
||||
|
||||
const AppLayer = Layer.mergeAll(ClockLive, IdsLive.pipe(Layer.provide(RandomLive)))
|
||||
|
||||
const runnable = use.pipe(Effect.provide(AppLayer)) // R = never
|
||||
|
||||
// One-off service injection without a layer:
|
||||
const withTestClock = use.pipe(Effect.provideService(Clock, { now: Effect.succeed(0) }))
|
||||
```
|
||||
|
||||
Also useful:
|
||||
|
||||
- `Context.Reference("key", { defaultValue: () => ... })` — a service **with a default**
|
||||
(this replaces `FiberRef`); no layer needed, override with `Effect.provideService`.
|
||||
- **Layer memoization changed:** layers are memoized *globally across separate
|
||||
`Effect.provide` calls* by default in v4 (per-runtime MemoMap), so providing the same
|
||||
layer to two effects builds it once.
|
||||
|
||||
## 3. Error handling
|
||||
|
||||
`Data.TaggedError` survives unchanged; `Schema.TaggedError` is now
|
||||
`Schema.TaggedErrorClass`. Errors are yieldable directly.
|
||||
|
||||
```ts
|
||||
import { Data, Effect, Schema } from "effect"
|
||||
|
||||
class SpawnError extends Data.TaggedError("SpawnError")<{
|
||||
readonly command: string
|
||||
readonly cause: unknown
|
||||
}> {}
|
||||
|
||||
class TimeoutError extends Data.TaggedError("TimeoutError")<{ readonly ms: number }> {}
|
||||
|
||||
// Schema-validated variant:
|
||||
class NotFound extends Schema.TaggedErrorClass<NotFound>()("NotFound", {
|
||||
id: Schema.Number
|
||||
}) {}
|
||||
|
||||
const risky: Effect.Effect<string, SpawnError | TimeoutError> = Effect.gen(function* () {
|
||||
if (somethingBad) return yield* new SpawnError({ command: "codex", cause: "enoent" })
|
||||
return "ok"
|
||||
})
|
||||
|
||||
// catchTag narrows the error union:
|
||||
const handled: Effect.Effect<string, TimeoutError> = risky.pipe(
|
||||
Effect.catchTag("SpawnError", (e) => Effect.succeed(`spawn failed: ${e.command}`))
|
||||
)
|
||||
|
||||
const handledAll: Effect.Effect<string> = risky.pipe(
|
||||
Effect.catchTags({
|
||||
SpawnError: (e) => Effect.succeed(`spawn: ${e.command}`),
|
||||
TimeoutError: (e) => Effect.succeed(`timeout ${e.ms}ms`)
|
||||
})
|
||||
)
|
||||
```
|
||||
|
||||
Renames: `catchAll` → `Effect.catch`, `catchAllCause` → `catchCause`,
|
||||
`catchAllDefect` → `catchDefect`, `tapErrorCause` → `tapCause`, `Effect.either` →
|
||||
`Effect.result` (returns `Result`, the renamed `Either`). `Effect.match`,
|
||||
`Effect.exit`, `Effect.orDie`, `Effect.mapError`, `Effect.tapError` all still exist.
|
||||
The `Cause` data structure was **flattened** (no nested `Sequential`/`Parallel` trees;
|
||||
a cause is now a flat list of failures — simpler to render in job status output).
|
||||
|
||||
## 4. Promise / async / callback interop
|
||||
|
||||
```ts
|
||||
import { Cause, Data, Effect } from "effect"
|
||||
|
||||
// Promise that "can't" reject — rejection becomes a defect:
|
||||
const a = Effect.promise(() => Promise.resolve(42))
|
||||
|
||||
// Promise that may reject — default error type is Cause.UnknownError (v3: UnknownException):
|
||||
const b = Effect.tryPromise((signal) => fetch("https://x", { signal }))
|
||||
|
||||
// Typed error:
|
||||
class HttpError extends Data.TaggedError("HttpError")<{ cause: unknown }> {}
|
||||
const c = Effect.tryPromise({
|
||||
try: (signal) => fetch("https://x", { signal }),
|
||||
catch: (cause) => new HttpError({ cause })
|
||||
})
|
||||
```
|
||||
|
||||
Both forms receive an `AbortSignal` that fires on fiber interruption — pass it to SDKs
|
||||
(claude SDK, fetch, etc.) so interrupting a subagent fiber cancels the underlying call.
|
||||
|
||||
Callback APIs — `Effect.async` is now **`Effect.callback`**:
|
||||
|
||||
```ts
|
||||
// signature: Effect.callback<A, E, R>((resume, signal) => void | cleanupEffect)
|
||||
const waitForExit = (child: import("node:child_process").ChildProcess) =>
|
||||
Effect.callback<number>((resume) => {
|
||||
child.once("exit", (code) => resume(Effect.succeed(code ?? -1)))
|
||||
})
|
||||
|
||||
// cleanup on interruption via the AbortSignal:
|
||||
const sleepy = Effect.callback<number>((resume, signal) => {
|
||||
const t = setTimeout(() => resume(Effect.succeed(1)), 1000)
|
||||
signal.addEventListener("abort", () => clearTimeout(t))
|
||||
})
|
||||
```
|
||||
|
||||
## 5. Scopes & resource management
|
||||
|
||||
Same shape as v3. Relevant rename: `Scope.extend` → `Scope.provide`.
|
||||
|
||||
```ts
|
||||
import { Effect, Exit, Scope } from "effect"
|
||||
|
||||
const managedProc = Effect.acquireRelease(
|
||||
acquireEffect, // acquire (uninterruptible)
|
||||
(proc, exit) => Effect.sync(() => proc.kill()) // release, gets the Exit
|
||||
)
|
||||
|
||||
// Scoped region: release runs when the region ends
|
||||
const useIt = Effect.scoped(
|
||||
Effect.gen(function* () {
|
||||
const proc = yield* managedProc
|
||||
return proc.pid
|
||||
})
|
||||
)
|
||||
|
||||
// addFinalizer:
|
||||
Effect.scoped(Effect.gen(function* () {
|
||||
yield* Effect.addFinalizer((exit) => Effect.log(`closing, ok=${Exit.isSuccess(exit)}`))
|
||||
}))
|
||||
```
|
||||
|
||||
**Key pattern for background subagents** — a manually-controlled scope so the process
|
||||
outlives the spawning tool call, and killing the job = closing the scope:
|
||||
|
||||
```ts
|
||||
const startJob = Effect.gen(function* () {
|
||||
const scope = yield* Scope.make()
|
||||
const handle = yield* Scope.provide(managedProc, scope) // resource lives in `scope`
|
||||
// store `scope` in your job registry; later, from any fiber:
|
||||
// yield* Scope.close(scope, Exit.void) // runs finalizers -> kills process
|
||||
return { handle, scope }
|
||||
})
|
||||
```
|
||||
|
||||
`Effect.acquireUseRelease(acquire, use, release)` for one-shot bracketing.
|
||||
|
||||
## 6. Child processes & FileSystem (the important part)
|
||||
|
||||
The v3 `@effect/platform` `Command`/`CommandExecutor` modules were redesigned into
|
||||
**`effect/unstable/process`** with `ChildProcess` (command builder) and
|
||||
`ChildProcessSpawner` (the service). The Node implementation comes from
|
||||
`@effect/platform-node`.
|
||||
|
||||
Import gotcha: `import { ChildProcessSpawner } from "effect/unstable/process"` gives you
|
||||
the **module namespace**, not the service class. Import the class from the submodule:
|
||||
|
||||
```ts
|
||||
import { ChildProcess } from "effect/unstable/process"
|
||||
import { ChildProcessSpawner } from "effect/unstable/process/ChildProcessSpawner"
|
||||
import { NodeServices, NodeFileSystem } from "@effect/platform-node"
|
||||
```
|
||||
|
||||
`NodeServices.layer` provides everything at once:
|
||||
`ChildProcessSpawner | Crypto | FileSystem | Path | Stdio | Terminal`.
|
||||
|
||||
### Building commands
|
||||
|
||||
```ts
|
||||
// Template literal form:
|
||||
const cmd1 = ChildProcess.make`echo hello`
|
||||
// Options + template:
|
||||
const cmd2 = ChildProcess.make({ cwd: "/tmp" })`ls -la`
|
||||
// Array form (best for dynamic args — no shell parsing):
|
||||
const cmd = ChildProcess.make("codex", ["exec", "--json", prompt], {
|
||||
cwd: workDir,
|
||||
env: { CODEX_API_KEY: key }, // merged over process.env when extendEnv: true
|
||||
extendEnv: true,
|
||||
stdin: "pipe", // "pipe" | "inherit" | "ignore" | a Stream<Uint8Array>
|
||||
stdout: "pipe", // "pipe" | "inherit" | "ignore" | a Sink
|
||||
stderr: "pipe"
|
||||
})
|
||||
// pipelines: cmdA.pipe(ChildProcess.pipeTo(cmdB, { from: "stderr" }))
|
||||
// modifiers: ChildProcess.setCwd, ChildProcess.setEnv, ChildProcess.prefix
|
||||
```
|
||||
|
||||
Note `detached` defaults to **true on non-Windows** — the child gets its own process
|
||||
group (good: killing it kills the group).
|
||||
|
||||
### Running: two styles
|
||||
|
||||
**Style A — via the spawner service (simple, non-interactive):**
|
||||
|
||||
```ts
|
||||
const simple = Effect.gen(function* () {
|
||||
const spawner = yield* ChildProcessSpawner
|
||||
const text = yield* spawner.string(cmd) // full stdout as string
|
||||
const lines = yield* spawner.lines(cmd) // string[]
|
||||
const code = yield* spawner.exitCode(cmd) // ExitCode (branded number)
|
||||
const lineStream = spawner.streamLines(cmd, { includeStderr: true }) // Stream<string>
|
||||
})
|
||||
```
|
||||
|
||||
**Style B — the Command IS an Effect.** Yielding a command spawns it and returns a
|
||||
`ChildProcessHandle`, with the process lifetime tied to the ambient `Scope`
|
||||
(`Command extends Effect<ChildProcessHandle, PlatformError, ChildProcessSpawner | Scope>`):
|
||||
|
||||
```ts
|
||||
import { Effect, Stream } from "effect"
|
||||
|
||||
const streaming = Effect.scoped(
|
||||
Effect.gen(function* () {
|
||||
const handle = yield* cmd // <-- spawns; killed when scope closes
|
||||
handle.pid // ProcessId (branded number)
|
||||
|
||||
// stdout is Stream<Uint8Array, PlatformError>; decode + split lines:
|
||||
yield* handle.stdout.pipe(
|
||||
Stream.decodeText(),
|
||||
Stream.splitLines,
|
||||
Stream.runForEach((line) => Effect.log(`out: ${line}`))
|
||||
)
|
||||
|
||||
// stdin is a Sink<void, Uint8Array>; write by running a stream into it:
|
||||
yield* Stream.fromArray([new TextEncoder().encode("hello\n")]).pipe(
|
||||
Stream.run(handle.stdin)
|
||||
)
|
||||
|
||||
// handle.stderr, handle.all (stdout+stderr interleaved) also available
|
||||
const code = yield* handle.exitCode // waits for exit
|
||||
const running = yield* handle.isRunning
|
||||
yield* handle.kill({ killSignal: "SIGTERM", forceKillAfter: "5 seconds" })
|
||||
return code
|
||||
})
|
||||
)
|
||||
```
|
||||
|
||||
Verified at runtime: interrupting a fiber that spawned a process inside its scope
|
||||
(e.g. `Fiber.interrupt` on a fiber running `Effect.scoped(...)` around `sleep 30`)
|
||||
kills the child process. This is the backbone of subagent cancellation.
|
||||
|
||||
There's also `handle.unref` for letting the parent exit independently, and
|
||||
`additionalFds` for extra file descriptor channels (fd3+).
|
||||
|
||||
### FileSystem / Path (now core)
|
||||
|
||||
```ts
|
||||
import { Effect, FileSystem, Path } from "effect"
|
||||
|
||||
const files = Effect.gen(function* () {
|
||||
const fs = yield* FileSystem.FileSystem
|
||||
const path = yield* Path.Path
|
||||
const content = yield* fs.readFileString("/tmp/in.txt")
|
||||
yield* fs.writeFileString(path.join("/tmp", "out.txt"), content)
|
||||
})
|
||||
// provide NodeFileSystem.layer, or NodeServices.layer for everything
|
||||
```
|
||||
|
||||
## 7. Concurrency toolbox for a job manager
|
||||
|
||||
All verified compiling + running:
|
||||
|
||||
```ts
|
||||
import {
|
||||
Cause, Deferred, Effect, Fiber, FiberMap, PubSub, Queue, Ref, Stream, Exit
|
||||
} from "effect"
|
||||
|
||||
const forking = Effect.gen(function* () {
|
||||
const f1 = yield* Effect.forkChild(work) // v3 fork — dies with parent
|
||||
const f2 = yield* Effect.forkScoped(work) // tied to ambient Scope
|
||||
const f3 = yield* Effect.forkDetach(work) // v3 forkDaemon — outlives parent ✅ for background jobs
|
||||
// all fork variants accept { startImmediately?: boolean, uninterruptible?: boolean | "inherit" }
|
||||
|
||||
const r: number = yield* Fiber.join(f1) // propagates failure
|
||||
const exit: Exit.Exit<number> = yield* Fiber.await(f3) // failure as data
|
||||
yield* Fiber.interrupt(f2)
|
||||
// fiber.pollUnsafe() -> Exit | undefined for sync status checks
|
||||
})
|
||||
```
|
||||
|
||||
**Deferred** — one-shot completion signal (job finished):
|
||||
|
||||
```ts
|
||||
const done = yield* Deferred.make<Result, JobError>()
|
||||
yield* Effect.forkDetach(job.pipe(Effect.exit, Effect.flatMap((e) => Deferred.done(done, e))))
|
||||
const result = yield* Deferred.await(done) // keeps Node process alive in v4, no runMain needed
|
||||
```
|
||||
|
||||
**Ref** — shared job-table state: `Ref.make({})`, `Ref.update`, `Ref.get`, `Ref.set`.
|
||||
(`SynchronizedRef` for effectful updates.)
|
||||
|
||||
**Queue** — absorbed v3's `Mailbox`: it can be ended/failed, and converts to a Stream.
|
||||
To use `Queue.end`, the error type must include `Cause.Done`:
|
||||
|
||||
```ts
|
||||
const q = yield* Queue.make<string, Cause.Done>() // capacity/strategy via options
|
||||
yield* Queue.offer(q, "line")
|
||||
yield* Queue.offerAll(q, ["a", "b"])
|
||||
yield* Queue.end(q) // signal "no more items"
|
||||
const asStream = Stream.fromQueue(q) // ends when queue ends
|
||||
// Queue.take / takeAll / takeN / poll for direct consumption
|
||||
```
|
||||
|
||||
**PubSub** — broadcast (e.g. multiple watchers of one job's output):
|
||||
|
||||
```ts
|
||||
const ps = yield* PubSub.unbounded<string>()
|
||||
const sub = yield* PubSub.subscribe(ps) // scoped acquisition
|
||||
yield* PubSub.publish(ps, "event")
|
||||
const msg = yield* PubSub.take(sub) // function call, not sub.take
|
||||
// Stream.fromPubSub(ps) for stream consumption
|
||||
```
|
||||
|
||||
**FiberMap / FiberSet / FiberHandle** — keyed fiber registries, ideal for a subagent
|
||||
manager (auto-interrupts everything when its scope closes; adding under an existing key
|
||||
interrupts the old fiber):
|
||||
|
||||
```ts
|
||||
const jobsDemo = Effect.scoped(Effect.gen(function* () {
|
||||
const jobs = yield* FiberMap.make<string>() // FiberMap<string> (keys)
|
||||
yield* FiberMap.run(jobs, "job-1", work) // fork into the map
|
||||
const fiber = yield* FiberMap.get(jobs, "job-1")
|
||||
yield* FiberMap.remove(jobs, "job-1") // interrupts it
|
||||
}))
|
||||
```
|
||||
|
||||
**Stream** essentials for process output: `Stream.decodeText()`, `Stream.splitLines`,
|
||||
`Stream.runForEach`, `Stream.runCollect`, `Stream.run(sink)`, `Stream.fromQueue`,
|
||||
`Stream.fromPubSub`, `Stream.toAsyncIterable`, `Stream.callback` (v3 `Stream.async`),
|
||||
`Stream.fromArray` (v3 `fromChunk` — v4 uses plain arrays, Chunk is de-emphasized).
|
||||
Also `Stream.mkString` to collect into one string.
|
||||
|
||||
Combinators: `Effect.all([...], { concurrency: n })`, `Effect.race`,
|
||||
`Effect.timeout("5 seconds")` (fails with `Cause.TimeoutError`, tag `"TimeoutError"`),
|
||||
`Semaphore.make(n)` (moved out of `Effect.makeSemaphore`), `Latch.make()`.
|
||||
|
||||
## 8. ManagedRuntime — the entry-point pattern for the extension
|
||||
|
||||
v3's `Runtime.runPromise(runtime)(effect)` API is gone; `effect/Runtime` now only holds
|
||||
`runMain` plumbing. **`ManagedRuntime` is the way** to share layers across async entry
|
||||
points:
|
||||
|
||||
```ts
|
||||
import { Context, Effect, Layer, ManagedRuntime } from "effect"
|
||||
import { NodeServices } from "@effect/platform-node"
|
||||
|
||||
class JobStore extends Context.Service<JobStore, {
|
||||
readonly list: Effect.Effect<Array<string>>
|
||||
}>()("app/JobStore") {}
|
||||
|
||||
const JobStoreLive = Layer.sync(JobStore, () => ({ list: Effect.succeed([]) }))
|
||||
|
||||
const AppLayer = Layer.mergeAll(JobStoreLive, NodeServices.layer)
|
||||
|
||||
// Build ONCE at extension activation. Layers are built lazily on first run.
|
||||
const runtime = ManagedRuntime.make(AppLayer)
|
||||
|
||||
// pi tool handler — the only place we touch Promises:
|
||||
export async function handleToolCall() {
|
||||
return await runtime.runPromise(
|
||||
Effect.gen(function* () {
|
||||
const store = yield* JobStore
|
||||
return yield* store.list
|
||||
})
|
||||
)
|
||||
}
|
||||
|
||||
// fire-and-forget background work from a sync/async context:
|
||||
const fiber = runtime.runFork(backgroundEffect)
|
||||
|
||||
// extension deactivation — closes the runtime scope, runs ALL finalizers
|
||||
// (i.e. kills any still-scoped child processes):
|
||||
export async function shutdown() {
|
||||
await runtime.dispose()
|
||||
}
|
||||
```
|
||||
|
||||
`ManagedRuntime` also exposes `runSync`, `runSyncExit`, `runPromiseExit`, `runCallback`,
|
||||
a `memoMap` (share layer memoization between multiple runtimes), and `.scope`.
|
||||
Prefer `runPromiseExit` in tool handlers if you want to map typed failures to
|
||||
tool-result errors instead of catching thrown `Cause` wrappers.
|
||||
|
||||
---
|
||||
|
||||
## Architecture sketch for subagents
|
||||
|
||||
```
|
||||
ManagedRuntime.make(Layer.mergeAll(NodeServices.layer, SubagentManagerLive))
|
||||
│ built once at extension init; dispose() on shutdown
|
||||
▼
|
||||
SubagentManager service (Context.Service):
|
||||
- Ref<HashMap<JobId, JobEntry>> job table
|
||||
- start(cmd): Scope.make() → Scope.provide(ChildProcess handle, scope)
|
||||
→ Effect.forkDetach(pump stdout → Queue<string, Cause.Done>)
|
||||
→ Deferred<ExitCode> completed on exit
|
||||
- status(id): Deferred.isDone / handle.isRunning / Ref lookup
|
||||
- output(id): drain Queue (takeAll) or Stream.fromQueue for tailing
|
||||
- kill(id): Scope.close(scope, Exit.void) → finalizer kills the process
|
||||
Tool handlers: async fns calling runtime.runPromise(Effect.gen(...))
|
||||
```
|
||||
|
||||
## Surprises & gotchas (learned the hard way)
|
||||
|
||||
1. **`effect/unstable/process` exports module namespaces.** The
|
||||
`ChildProcessSpawner` class must be imported from
|
||||
`"effect/unstable/process/ChildProcessSpawner"` (or use
|
||||
`ChildProcessSpawner.ChildProcessSpawner` off the namespace).
|
||||
2. **`Effect.fork` does not exist** — code (or an LLM) writing v3-style `Effect.fork`
|
||||
fails to compile. Use `forkChild` / `forkScoped` / `forkDetach` / `forkIn`.
|
||||
3. **`Queue.end` needs `Cause.Done` in the queue's error type** —
|
||||
`Queue.make<A>()` defaults to `E = never` and won't accept `end`.
|
||||
4. **Early-beta content mentioning `ServiceMap` is stale** — it was renamed back to
|
||||
`Context` during the beta. Similarly, some AI-generated content mentions APIs that
|
||||
never shipped.
|
||||
5. **`Either` is `Result`**, `Effect.either` is `Effect.result`.
|
||||
6. **`Data.TaggedError` errors are yieldable** — `yield* new SpawnError({...})` fails
|
||||
the effect directly, no `Effect.fail` wrapper needed (works in v3 too, but idiomatic
|
||||
in v4 docs).
|
||||
7. **Chunk → Array**: stream element groups are plain arrays (`Stream.fromArray`,
|
||||
`runCollect` returns `Array<A>`), not `Chunk`.
|
||||
8. `tryPromise` default error is `Cause.UnknownError` (was `UnknownException`).
|
||||
9. `unstable/*` modules can break between v4 minors — pin exact versions
|
||||
(`4.0.0-beta.98`) and keep `effect` and `@effect/platform-node` in lockstep.
|
||||
10. Scratch workspace with all verified test files: `/tmp/effect-v4-scratch`
|
||||
(`test1-basics.ts` … `test8-runtime.ts`, `smoke.mts` executed successfully).
|
||||
Reference in New Issue
Block a user