Files
paseo/packages/app/src/contexts/session-context.tsx
T
Mohamed BoudraandGitHub 8021d2b6c9 Keep agent timelines stable from submission through resume (#2718)
* Reapply: Stop completed turns from appearing stuck (#2484)

* Reapply: Keep older chat history and image previews stable (#2490)

* Lock timeline layout stability with user-driven specs

Two reds reproduce the reported jank:
- new agent, first prompt: the row paints at top=100 then jumps to 144 through
  authoritative hydration, because the draft tab and the agent panel each render
  their own AgentStreamView for the same logical row.
- long loaded timeline, submit: scrollHeight collapses 10793 -> 9111 mid-turn,
  because splitOrderedTail reclassifies measured rows into the virtualized segment
  where the virtualizer falls back to estimateStreamItemHeight.

Also restores centered timeline rows: the pagination anchor wrapper introduced in
#2490 was a plain block div, which broke alignSelf: center.

* Measure the turn footer blink from when the footer appears

Bounding the window at the prompt row conflated two things: the footer arriving a
frame after the row, and the footer leaving the layout and coming back. Bounded from
the footer's first frame, the blink still reproduces, so the footer-ownership race is
real rather than a sampling artifact.

* Rename the shadowed prompt row binding in the frame recorder

* fix(desktop): allow concurrent worktree dev instances

Desktop terminals can inherit another instance's profile path and debugging flags. Scope the profile to the current dev root and select an available debugging port so instances do not share global runtime state.

* fix(desktop): expose development providers in dev builds

Desktop-managed daemons inherited no Paseo runtime mode, so dev builds constructed the production provider registry. Mark unpackaged internal launches as development while continuing to force packaged launches to production.

* fix(timeline): stabilize optimistic turn layout

Keep initial history loading dormant until user intent and preserve active-turn layout across delayed transport updates. Add painted-frame coverage for optimistic timeline transitions.

* fix(timeline): preserve optimistic prompts through hydration

Canonical replacement relied on a transport submission record that can settle before a provider echoes the user row. Keep presentation continuity owned by the unreconciled row while destructive history replacement remains governed by active transactions.

* fix(timeline): make submitted prompts canonical

Submission settlement previously depended on status updates that could arrive after the optimistic row. Persist every accepted prompt in the canonical timeline and keep the footer active until RPC and canonical acknowledgement converge.

* fix(timeline): stop older history pages repeating entries

Backward pagination counted source sequence rows, so page boundaries inside a merged projected item could return that item repeatedly. Count projected entries and advance from their display anchors instead, with server and browser regressions covering the contract.

* fix(timeline): resume agents from the latest history

Use a bounded tail probe on resume so long background runs appear atomically instead of replaying every missed page. Preserve unchanged views, reconcile local/live rows, and keep skipped history reachable through backward pagination.

* fix(timeline): reconcile turn state after missed updates

Turn completion events can be missed while an agent timeline is unsubscribed, leaving footer activity stale even after the directory reports idle. Reconcile the single-owned turn lifecycle from authoritative resume snapshots while preserving newer live evidence.

* test(timeline): cover running state across refresh

Protect the hydration boundary where directory state must restore consistent tab, composer, and footer activity. Expose the running indicator semantically so the regression checks user-visible state instead of spinner implementation details.

* fix(timeline): keep history loading geometry stable

Reserve a fixed history-start gutter regardless of pagination state so showing or hiding the loader cannot shift timeline content.

* fix(timeline): keep turn activity coherent

Derive footer, tab, and composer activity from one lifecycle and submission projection. Idle directory transitions now request an authoritative resume so missed terminal events converge without treating directory delivery order as turn authority.

* fix(tests): align checks with runtime boundaries

* fix(desktop): stop dev app surviving its terminal

Electron was detached from the desktop runner, allowing it to outlive a stopped workspace terminal with broken output pipes and retain the single-instance lock. Keep Electron owned by the runner and preserve explicit process-group cleanup for detached child trees.

* fix(timeline): preserve activity through turn handoff

Correlate lifecycle events and snapshots with daemon-owned turn identity, and keep optimistic submissions active until their canonical row arrives. This prevents stale updates and RPC timing from exposing a false idle state.

* refactor(timeline): remove submission timing state

* fix(timeline): restore prompt identity on resume

* fix(timeline): exclude attachments from canonical prompt text

Structured provider prompts can include hidden history attachments before the user's text. Persist only plain prompt blocks so authoritative user messages match what the composer displayed while provider input remains unchanged.

* refactor(agent): remove unused pending-run message identity

* fix(desktop): let Chromium allocate the debugging port

* refactor(agent): remove impossible finalized-turn tracking

* fix(timeline): load history after keyboard scrolling

Treat observed upward displacement as user intent so keyboard and scrollbar navigation can wake dormant pagination.

* test(server): preserve accepted turn stream order

The manager-owned accepted turn boundary intentionally precedes coalesced output. Assert that lifecycle event explicitly while retaining the timeline-before-terminal ordering contract.

* test(app): scope sidebar fixtures by identity

Shared E2E daemon state may contain unrelated projects. Keep stale-route coverage tied to its named missing project and drag only the two rows seeded by the reorder spec.

* fix(timeline): gate prompt revisions by client capability

* fix(timeline): finalize canonical prompt delivery

Treat provider identity as a single stored-row transition so legacy clients receive one deliverable canonical prompt. Release identity-pending rows before later foreground output or the current terminal while preserving stale-turn isolation.

* fix(server): preserve legacy prompt pagination

* fix(timeline): keep assistant block identity stable

Assign canonical block IDs during live promotion so terminal finalization cannot remount image-bearing rows.

* fix(timeline): apply covered revisions during resume append

Run the exact canonical revision pass before overlapping resume pages append newer rows, while leaving cursor advancement to the incremental path.

* fix(server): preserve queued agent transitions

Project each queued live snapshot before awaiting stored metadata so later lifecycle state cannot overwrite an earlier transition.

* fix(server): snapshot queued agent updates

Capture each live agent payload before it enters the per-agent queue so shared nested state cannot change while an earlier storage read is pending.

* docs(app): date agent turn compatibility cleanup

* refactor(timeline): mint submitted prompt identity at acceptance

Canonical prompt rows now keep one immutable wire identity while provider identity remains internal for rewind translation. This removes the unreleased same-sequence revision and legacy projection contract.

* refactor(app): remove prompt revision reconciliation

Submitted prompt rows are immutable on the wire, so resume and live reducers no longer need to apply already-covered same-sequence updates.

* test(app): cover submitted prompt rewind roundtrip

* docs(timeline): describe immutable submitted prompt identity

Provider echoes now enrich daemon-internal identity only; wire rows keep their acceptance-time identity and rewind performs the provider translation.

* test(server): validate mock rewind targets

Reject unknown provider message IDs so app rewind coverage exercises wire-to-provider identity translation.

* fix(server): validate rewind before interrupting

* fix(server): omit rewind identity for daemon prompts

* test(app): guard rewind eligibility and rejection

* fix(server): preserve prompt identity across hydration

* refactor(timeline): remove stale revision remnants

* Revert "fix(server): preserve prompt identity across hydration"

This reverts commit 6adcf7e5b0bc8601761af59b461a746cb5e395fa.

* fix(timeline): settle missed submission acknowledgements
2026-08-01 14:13:47 +02:00

1283 lines
42 KiB
TypeScript

import { useRef, ReactNode, useCallback, useEffect } from "react";
import { Buffer } from "buffer";
import { AppState } from "react-native";
import { useQueryClient } from "@tanstack/react-query";
import { useTranslation } from "react-i18next";
import { useClientActivity } from "@/hooks/use-client-activity";
import { useAppVisible } from "@/hooks/use-app-visible";
import { usePushTokenRegistration } from "@/hooks/use-push-token-registration";
import {
createSetAgentInitializing,
refreshAgentInitializationTimeout,
} from "@/hooks/use-agent-initialization";
import { prefetchProvidersSnapshot } from "@/hooks/use-providers-snapshot";
import { generateMessageId, type StreamItem } from "@/types/stream";
import {
createSessionAgentStreamReducerQueue,
deriveAgentStreamTurnLiveness,
processTimelineResponse,
type ProcessTimelineResponseOutput,
type TimelineReducerSideEffect,
} from "@/timeline/session-stream-reducers";
import { useCreateFlowStore } from "@/stores/create-flow-store";
import { isTimelineResumeSnapshotAuthoritative } from "@/timeline/timeline-sync-plan";
import {
createViewedTimelineSync,
type TimelineDeliveryMode,
type ViewedTimelineSync,
} from "@/timeline/viewed-timeline-sync";
import type { AgentAttachment, SessionOutboundMessage } from "@getpaseo/protocol/messages";
import { parseServerInfoStatusPayload } from "@getpaseo/protocol/messages";
import {
buildAgentAttentionNotificationPayload,
type AgentAttentionReason,
type AgentAttentionNotificationPayload,
type NotificationPermissionRequest,
} from "@getpaseo/protocol/agent-attention-notification";
import type { DaemonClient } from "@getpaseo/client/internal/daemon-client";
import type { AgentSessionConfig } from "@getpaseo/protocol/agent-types";
import type { GitSetupOptions } from "@getpaseo/protocol/messages";
import type { AgentPermissionResponse } from "@getpaseo/protocol/agent-types";
import { getHostRuntimeStore, useHostRuntimeIsConnected } from "@/runtime/host-runtime";
import { useVoiceAudioEngineOptional, useVoiceRuntimeOptional } from "@/contexts/voice-context";
import type { AudioPlaybackSource } from "@/voice/audio-engine-types";
import {
selectAgentTimelineState,
useSessionStore,
type MessageEntry,
type SessionState,
} from "@/stores/session-store";
import { useWorkspaceSetupStore } from "@/stores/workspace-setup-store";
import { sendOsNotification } from "@/utils/os-notifications";
import { getIsAppActivelyVisible, getIsAppVisible } from "@/utils/app-visibility";
import {
getInitKey,
getInitDeferred,
createInitDeferred,
resolveInitDeferred,
rejectInitDeferred,
} from "@/utils/agent-initialization";
import { encodeImages } from "@/utils/encode-images";
import { derivePendingPermissionKey } from "@/utils/agent-snapshots";
import { getSendingClientMessageIds } from "@/composer/submission/model";
import type { AttachmentMetadata } from "@/attachments/types";
import { patchWorkspaceScripts } from "@/contexts/session-workspace-scripts";
import { useToast } from "@/contexts/toast-context";
import { toErrorMessage } from "@/utils/error-messages";
import { showProviderNoticeToast } from "@/utils/provider-notice-toast";
import { applyCheckoutStatusUpdateFromEvent } from "@/git/checkout-status-cache";
import { useProviderSubagentStore } from "@/subagents/provider-store";
import { revalidateSessionAfterResume } from "@/contexts/session-resume-revalidation";
// Re-export types from session-store and draft-store for backward compatibility
export type { DraftInput } from "@/stores/draft-store";
export type {
MessageEntry,
Agent,
ExplorerEntry,
ExplorerFile,
ExplorerEntryKind,
ExplorerFileKind,
ExplorerEncoding,
AgentFileExplorerState,
} from "@/stores/session-store";
type AudioOutputPayload = Extract<SessionOutboundMessage, { type: "audio_output" }>["payload"];
interface BufferedAudioChunk {
chunkIndex: number;
audio: string;
format: string;
id: string;
}
// COMPAT(selectiveAgentTimeline): added in v0.1.106, remove after 2027-01-12.
function getTimelineDeliveryMode(selectiveAgentTimeline?: boolean): TimelineDeliveryMode {
return selectiveAgentTimeline ? "selective" : "legacy";
}
function decodeBase64Chunk(base64: string): Uint8Array {
return Buffer.from(base64, "base64");
}
function buildAudioPlaybackSource(chunks: BufferedAudioChunk[]): AudioPlaybackSource {
const decodedChunks = chunks.map((chunk) => decodeBase64Chunk(chunk.audio));
const totalSize = decodedChunks.reduce((sum, chunk) => sum + chunk.length, 0);
const output = new Uint8Array(totalSize);
let offset = 0;
for (const chunk of decodedChunks) {
output.set(chunk, offset);
offset += chunk.length;
}
const format = chunks[0]?.format ?? "pcm";
let mimeType: string;
if (format === "pcm") mimeType = "audio/pcm;rate=24000;bits=16";
else if (format === "mp3") mimeType = "audio/mpeg";
else mimeType = `audio/${format}`;
const bytes = output.slice();
return {
size: bytes.byteLength,
type: mimeType,
async arrayBuffer() {
return bytes.buffer.slice(bytes.byteOffset, bytes.byteOffset + bytes.byteLength);
},
};
}
const findLatestAssistantMessageText = (items: StreamItem[]): string | null => {
for (let i = items.length - 1; i >= 0; i -= 1) {
const item = items[i];
if (item.kind === "assistant_message") {
return item.text;
}
}
return null;
};
const getLatestPermissionRequest = (
session: SessionState | undefined,
agentId: string,
): NotificationPermissionRequest | null => {
if (!session) {
return null;
}
let latest: NotificationPermissionRequest | null = null;
for (const pending of session.pendingPermissions.values()) {
if (pending.agentId === agentId) {
latest = pending.request;
}
}
if (latest) {
return latest;
}
const agentPending = session.agents.get(agentId)?.pendingPermissions;
if (agentPending && agentPending.length > 0) {
return agentPending[agentPending.length - 1] as NotificationPermissionRequest;
}
return null;
};
interface AgentAttentionNotificationInput {
notification?: AgentAttentionNotificationPayload;
reason: AgentAttentionReason;
serverId: string;
workspaceId: string | undefined;
agentId: string;
assistantMessage: string | null;
permissionRequest: NotificationPermissionRequest | null;
}
function resolveAgentAttentionNotification(
input: AgentAttentionNotificationInput,
): AgentAttentionNotificationPayload | null {
if (input.notification) {
return input.notification.data.workspaceId ? input.notification : null;
}
if (!input.workspaceId) {
return null;
}
return buildAgentAttentionNotificationPayload({
reason: input.reason,
serverId: input.serverId,
workspaceId: input.workspaceId,
agentId: input.agentId,
assistantMessage: input.reason === "finished" ? input.assistantMessage : null,
permissionRequest: input.reason === "permission" ? input.permissionRequest : null,
});
}
type WorkspaceSetupProgressPayload = Extract<
SessionOutboundMessage,
{ type: "workspace_setup_progress" }
>["payload"];
type SessionStoreActions = ReturnType<typeof useSessionStore.getState>;
type SetInitializingAgents = SessionStoreActions["setInitializingAgents"];
function clearAgentInitializingFlag(
setInitializingAgents: SetInitializingAgents,
serverId: string,
agentId: string,
): void {
setInitializingAgents(serverId, (prev) => {
if (prev.get(agentId) !== true) {
return prev;
}
const next = new Map(prev);
next.set(agentId, false);
return next;
});
}
function handleTimelineError(input: {
result: ProcessTimelineResponseOutput;
agentId: string;
initKey: string;
serverId: string;
setInitializingAgents: SetInitializingAgents;
}): void {
const { result, agentId, initKey, serverId, setInitializingAgents } = input;
if (result.clearInitializing) {
clearAgentInitializingFlag(setInitializingAgents, serverId, agentId);
}
if (result.initResolution === "reject" && result.error) {
rejectInitDeferred(initKey, new Error(result.error));
}
}
function executeTimelineSideEffects(input: {
sideEffects: TimelineReducerSideEffect[];
agentId: string;
recoverTimelineGap: (agentId: string, cursor: { epoch: string; endSeq: number }) => void;
}): void {
const { sideEffects, agentId, recoverTimelineGap } = input;
for (const effect of sideEffects) {
if (effect.type === "catch_up") {
recoverTimelineGap(agentId, effect.cursor);
}
}
}
function finalizeTimelineApplication(input: {
result: ProcessTimelineResponseOutput;
agentId: string;
initKey: string;
serverId: string;
shouldMarkAuthoritativeHistoryApplied: boolean;
setInitializingAgents: SetInitializingAgents;
}): void {
const {
result,
agentId,
initKey,
serverId,
shouldMarkAuthoritativeHistoryApplied,
setInitializingAgents,
} = input;
if (result.clearInitializing) {
clearAgentInitializingFlag(setInitializingAgents, serverId, agentId);
}
if (shouldMarkAuthoritativeHistoryApplied) {
useCreateFlowStore.getState().clearByAgent({ serverId, agentId });
const session = useSessionStore.getState().sessions[serverId];
const agent = session?.agents.get(agentId) ?? session?.agentDetails.get(agentId);
if (agent && agent.status !== "running") {
getHostRuntimeStore().drainQueuedAgentMessage(serverId, agentId);
}
}
if (result.initResolution === "resolve") {
resolveInitDeferred(initKey);
}
}
function applyToolResultToMessages(
toolCallId: string,
result: unknown,
): (prev: MessageEntry[]) => MessageEntry[] {
return (prev) =>
prev.map((msg) =>
msg.type === "tool_call" && msg.id === toolCallId
? { ...msg, result, status: "completed" as const }
: msg,
);
}
function applyToolErrorToMessages(
toolCallId: string,
error: unknown,
): (prev: MessageEntry[]) => MessageEntry[] {
return (prev) =>
prev.map((msg) =>
msg.type === "tool_call" && msg.id === toolCallId
? { ...msg, error, status: "failed" as const }
: msg,
);
}
function notifyVoiceAbortFailure(
data: Extract<SessionOutboundMessage, { type: "activity_log" }>["payload"],
notifyError: (message: string) => void,
): void {
if (data.type === "error" && data.metadata?.voiceAbortFailed === true) {
notifyError(data.content);
}
}
interface SessionProviderSharedProps {
children: ReactNode;
serverId: string;
}
interface SessionProviderClientProps extends SessionProviderSharedProps {
client: DaemonClient;
}
export type SessionProviderProps = SessionProviderClientProps;
function SessionProviderWithClient({ children, serverId, client }: SessionProviderClientProps) {
return (
<SessionProviderInternal serverId={serverId} client={client}>
{children}
</SessionProviderInternal>
);
}
// SessionProvider: Daemon client message handler that updates Zustand store
export function SessionProvider(props: SessionProviderProps) {
return <SessionProviderWithClient {...props} />;
}
function SessionProviderInternal({ children, serverId, client }: SessionProviderClientProps) {
const { t } = useTranslation();
const voiceRuntime = useVoiceRuntimeOptional();
const voiceAudioEngine = useVoiceAudioEngineOptional();
const queryClient = useQueryClient();
const isConnected = useHostRuntimeIsConnected(serverId);
const toast = useToast();
// Zustand store actions
const setIsPlayingAudio = useSessionStore((state) => state.setIsPlayingAudio);
const setMessages = useSessionStore((state) => state.setMessages);
const setCurrentAssistantMessage = useSessionStore((state) => state.setCurrentAssistantMessage);
const setAgentStreamTail = useSessionStore((state) => state.setAgentStreamTail);
const setAgentStreamHead = useSessionStore((state) => state.setAgentStreamHead);
const setAgentStreamState = useSessionStore((state) => state.setAgentStreamState);
const applyAgentTurnLiveness = useSessionStore((state) => state.applyAgentTurnLiveness);
const clearAgentTurnLiveness = useSessionStore((state) => state.clearAgentTurnLiveness);
const clearAgentStreamHead = useSessionStore((state) => state.clearAgentStreamHead);
const setAgentTimelineCursor = useSessionStore((state) => state.setAgentTimelineCursor);
const setInitializingAgents = useSessionStore((state) => state.setInitializingAgents);
const bumpHistorySyncGeneration = useSessionStore((state) => state.bumpHistorySyncGeneration);
const markAgentHistorySynchronized = useSessionStore(
(state) => state.markAgentHistorySynchronized,
);
const applyAgentTimelineResponseState = useSessionStore(
(state) => state.applyAgentTimelineResponseState,
);
const setAgents = useSessionStore((state) => state.setAgents);
const setWorkspaces = useSessionStore((state) => state.setWorkspaces);
const flushAgentLastActivity = useSessionStore((state) => state.flushAgentLastActivity);
const setPendingPermissions = useSessionStore((state) => state.setPendingPermissions);
const updateSessionServerInfo = useSessionStore((state) => state.updateSessionServerInfo);
const setViewedTimelineSync = useSessionStore((state) => state.setViewedTimelineSync);
const upsertWorkspaceSetupProgress = useWorkspaceSetupStore((state) => state.upsertProgress);
// Track focused agent for heartbeat
const focusedAgentId = useSessionStore(
(state) => state.sessions[serverId]?.focusedAgentId ?? null,
);
const focusedTerminalId = useSessionStore(
(state) => state.sessions[serverId]?.focusedTerminalId ?? null,
);
const _sessionStateTimeoutRef = useRef<ReturnType<typeof setTimeout> | null>(null);
const attentionNotifiedRef = useRef<Map<string, number>>(new Map());
const appStateRef = useRef(AppState.currentState);
const viewedTimelineSyncRef = useRef<ViewedTimelineSync | null>(null);
const audioOutputBuffersRef = useRef<Map<string, BufferedAudioChunk[]>>(new Map());
const activeAudioGroupsRef = useRef<Set<string>>(new Set());
const isAppVisible = useAppVisible();
useEffect(() => {
const subscription = AppState.addEventListener("change", (nextState) => {
appStateRef.current = nextState;
});
return () => {
subscription.remove();
};
}, []);
useEffect(() => {
viewedTimelineSyncRef.current?.setActive(isAppVisible);
}, [isAppVisible]);
const recoverTimelineGap = useCallback(
(agentId: string, cursor: { epoch: string; endSeq: number }) => {
viewedTimelineSyncRef.current?.recoverGap(agentId, cursor);
},
[],
);
const handleAppResumed = useCallback(
(awayMs: number) => {
void revalidateSessionAfterResume({
awayMs,
serverId,
bumpHistorySyncGeneration,
refreshDirectories: () => getHostRuntimeStore().refreshDirectories(serverId),
});
},
[bumpHistorySyncGeneration, serverId],
);
// Client activity tracking (heartbeat, push token registration)
useClientActivity({ client, focusedAgentId, focusedTerminalId, onAppResumed: handleAppResumed });
usePushTokenRegistration({ client, serverId });
const notifyAgentAttention = useCallback(
(params: {
agentId: string;
reason: "finished" | "error" | "permission";
timestamp: string;
notification?: AgentAttentionNotificationPayload;
}) => {
const appState = appStateRef.current;
const session = useSessionStore.getState().sessions[serverId];
const attentionFocusedAgentId = session?.focusedAgentId ?? null;
if (params.reason === "error") {
return;
}
const isActivelyVisible = getIsAppActivelyVisible(appState);
const isAwayFromAgent = !isActivelyVisible || attentionFocusedAgentId !== params.agentId;
if (!isAwayFromAgent) {
return;
}
const timestampMs = new Date(params.timestamp).getTime();
const lastNotified = attentionNotifiedRef.current.get(params.agentId);
if (lastNotified && lastNotified >= timestampMs) {
return;
}
attentionNotifiedRef.current.set(params.agentId, timestampMs);
const head = session?.agentStreamHead.get(params.agentId) ?? [];
const tail = session?.agentStreamTail.get(params.agentId) ?? [];
const assistantMessage =
findLatestAssistantMessageText(head) ?? findLatestAssistantMessageText(tail);
const permissionRequest = getLatestPermissionRequest(session, params.agentId);
const workspaceId = session?.agents?.get(params.agentId)?.workspaceId;
const notification = resolveAgentAttentionNotification({
notification: params.notification,
reason: params.reason,
serverId,
workspaceId,
agentId: params.agentId,
assistantMessage,
permissionRequest,
});
if (!notification) {
return;
}
void sendOsNotification({
title: notification.title,
body: notification.body,
data: notification.data,
});
},
[serverId],
);
useEffect(() => {
const serverInfo = client.getLastServerInfoMessage();
if (!serverInfo) {
return;
}
updateSessionServerInfo(serverId, {
serverId: serverInfo.serverId,
hostname: serverInfo.hostname,
version: serverInfo.version,
...(serverInfo.desktopManaged !== undefined
? { desktopManaged: serverInfo.desktopManaged }
: {}),
...(serverInfo.capabilities ? { capabilities: serverInfo.capabilities } : {}),
...(serverInfo.features ? { features: serverInfo.features } : {}),
});
}, [client, serverId, updateSessionServerInfo]);
useEffect(() => {
if (!isConnected) {
return;
}
const serverInfo = client.getLastServerInfoMessage();
if (!serverInfo?.features?.providersSnapshot) {
return;
}
prefetchProvidersSnapshot(serverId, client);
}, [client, isConnected, serverId]);
useEffect(() => {
const unregister = voiceRuntime?.registerSession({
serverId,
setVoiceMode: async (enabled, agentId) => {
if (!client) {
throw new Error(t("common.errors.daemonUnavailable"));
}
await client.setVoiceMode(enabled, agentId);
},
sendVoiceAudioChunk: async (audioData, mimeType) => {
if (!client) {
throw new Error(t("common.errors.daemonUnavailable"));
}
await client.sendVoiceAudioChunk(audioData, mimeType);
},
audioPlayed: async (chunkId) => {
if (!client) {
throw new Error(t("common.errors.daemonUnavailable"));
}
await client.audioPlayed(chunkId);
},
abortRequest: async () => {
if (!client) {
throw new Error(t("common.errors.daemonUnavailable"));
}
await client.abortRequest();
},
setAssistantAudioPlaying: (isPlaying) => {
setIsPlayingAudio(serverId, isPlaying);
},
});
return () => unregister?.();
}, [client, serverId, setIsPlayingAudio, t, voiceRuntime]);
useEffect(() => {
voiceRuntime?.updateSessionConnection(serverId, isConnected);
}, [isConnected, serverId, voiceRuntime]);
// If the client drops mid-initialization, clear pending flags
useEffect(() => {
if (!isConnected) {
flushAgentLastActivity();
setInitializingAgents(serverId, new Map());
}
}, [flushAgentLastActivity, serverId, isConnected, setInitializingAgents]);
useEffect(
() =>
client.subscribeConnectionStatus((connection) => {
if (connection.status === "connected") return;
clearAgentTurnLiveness(serverId);
}),
[clearAgentTurnLiveness, client, serverId],
);
const applyWorkspaceSetupProgress = useCallback(
(payload: WorkspaceSetupProgressPayload) => {
upsertWorkspaceSetupProgress({ serverId, payload });
},
[serverId, upsertWorkspaceSetupProgress],
);
const applyTimelineResponse = useCallback(
(
payload: Extract<
SessionOutboundMessage,
{ type: "fetch_agent_timeline_response" }
>["payload"],
) => {
const agentId = payload.agentId;
const initKey = getInitKey(serverId, agentId);
const shouldMarkAuthoritativeHistoryApplied = isTimelineResumeSnapshotAuthoritative({
direction: payload.direction,
hasNewer: payload.hasNewer,
error: payload.error,
});
// Read current store state
const session = useSessionStore.getState().sessions[serverId];
const isInitializing = session?.initializingAgents.get(agentId) === true;
const activeInitDeferred = getInitDeferred(initKey);
const hasActiveInitDeferred = Boolean(activeInitDeferred);
const timeline = selectAgentTimelineState(session, agentId);
const currentCursor =
timeline.status === "synced" ? (timeline.range ?? undefined) : undefined;
const currentTail = timeline.status === "cold" ? [] : timeline.items;
const currentHead = session?.agentStreamHead.get(agentId) ?? [];
const sendingClientMessageIds = getSendingClientMessageIds(
session?.messageSubmissions.get(agentId),
);
// Call pure reducer
const result = processTimelineResponse({
payload,
currentTail,
currentHead,
currentCursor,
isInitializing,
hasActiveInitDeferred,
initRequestDirection: activeInitDeferred?.requestDirection ?? "tail",
sendingClientMessageIds,
});
if (result.error) {
handleTimelineError({
result,
agentId,
initKey,
serverId,
setInitializingAgents,
});
return;
}
if (result.commit === "discard") {
if (result.acknowledgedClientMessageIds.length > 0) {
setAgentStreamState(serverId, agentId, {
acknowledgedClientMessageIds: result.acknowledgedClientMessageIds,
});
}
markAgentHistorySynchronized(serverId, agentId);
} else {
applyAgentTimelineResponseState(serverId, agentId, {
items: result.tail,
head: result.head,
range: result.cursorChanged ? (result.cursor ?? null) : (currentCursor ?? null),
older: result.older,
synchronized: shouldMarkAuthoritativeHistoryApplied,
acknowledgedClientMessageIds: result.acknowledgedClientMessageIds,
});
}
executeTimelineSideEffects({
sideEffects: result.sideEffects,
agentId,
recoverTimelineGap,
});
finalizeTimelineApplication({
result,
agentId,
initKey,
serverId,
shouldMarkAuthoritativeHistoryApplied,
setInitializingAgents,
});
},
[
applyAgentTimelineResponseState,
markAgentHistorySynchronized,
recoverTimelineGap,
serverId,
setAgentStreamState,
setInitializingAgents,
],
);
useEffect(() => {
const setAgentInitializing = createSetAgentInitializing(serverId, setInitializingAgents);
const initialDeliveryMode = getTimelineDeliveryMode(
client.getLastServerInfoMessage()?.features?.selectiveAgentTimeline,
);
const sync = createViewedTimelineSync({
initialDeliveryMode,
setSubscription: (agentIds) => client.setAgentTimelineSubscription(agentIds),
fetchPage: async (agentId, request) => {
const session = useSessionStore.getState().sessions[serverId];
const initKey = getInitKey(serverId, agentId);
const shouldInitialize = selectAgentTimelineState(session, agentId).status !== "synced";
if (shouldInitialize) {
if (!getInitDeferred(initKey)) {
const deferred = createInitDeferred(initKey, request.direction ?? "tail");
void deferred.promise.catch(() => undefined);
}
refreshAgentInitializationTimeout({
key: initKey,
agentId,
setAgentInitializing,
});
setAgentInitializing(agentId, true);
}
try {
const page = await getHostRuntimeStore().fetchAgentTimeline(serverId, agentId, request);
if (shouldInitialize && getInitDeferred(initKey)) {
refreshAgentInitializationTimeout({ key: initKey, agentId, setAgentInitializing });
}
return page;
} catch (error) {
if (shouldInitialize) {
setAgentInitializing(agentId, false);
rejectInitDeferred(initKey, error instanceof Error ? error : new Error(String(error)));
}
throw error;
}
},
reportError: (error) => {
console.warn("[Session] viewed timeline synchronization failed", { serverId, error });
},
schedule: (task, delayMs) => {
const timeout = setTimeout(task, delayMs);
return () => clearTimeout(timeout);
},
});
viewedTimelineSyncRef.current = sync;
setViewedTimelineSync(serverId, sync);
sync.setActive(getIsAppVisible(appStateRef.current));
return () => {
if (viewedTimelineSyncRef.current === sync) {
viewedTimelineSyncRef.current = null;
}
setViewedTimelineSync(serverId, null);
sync.dispose();
};
}, [client, serverId, setInitializingAgents, setViewedTimelineSync]);
useEffect(() => {
viewedTimelineSyncRef.current?.setConnected(isConnected);
}, [isConnected]);
useEffect(
() =>
getHostRuntimeStore().subscribeAgentStoppedRunning(serverId, (agentId) => {
viewedTimelineSyncRef.current?.reconcileAgent(agentId);
}),
[serverId],
);
// Daemon message handlers - directly update Zustand store
useEffect(() => {
const agentStreamReducerQueue = createSessionAgentStreamReducerQueue({
serverId,
setAgentStreamState,
setAgentTimelineCursor,
recoverTimelineGap,
});
const unsubAgentStream = client.on("agent_stream", (message) => {
if (message.type !== "agent_stream") return;
const { agentId, event, timestamp, seq, epoch } = message.payload;
const parsedTimestamp = new Date(timestamp);
const streamEvent = event;
if (
event.type === "turn_started" ||
event.type === "turn_completed" ||
event.type === "turn_failed" ||
event.type === "turn_canceled"
) {
voiceRuntime?.onTurnEvent(serverId, agentId, event.type);
}
const turnLiveness = deriveAgentStreamTurnLiveness([
{ event: streamEvent, seq, epoch, timestamp: parsedTimestamp },
]);
if (turnLiveness.length > 0) {
applyAgentTurnLiveness(serverId, agentId, turnLiveness);
}
agentStreamReducerQueue.enqueue(agentId, {
event: streamEvent,
seq,
epoch,
timestamp: parsedTimestamp,
});
// NOTE: We don't update lastActivityAt on every stream event to prevent
// cascading rerenders. The agent_update handler updates agent.lastActivityAt
// on status changes, which is sufficient for sorting and display purposes.
});
const unsubAgentAttention = client.onAgentAttentionRequired((notification) => {
if (notification.shouldNotify) {
notifyAgentAttention(notification);
}
});
const unsubAgentTimeline = client.on("fetch_agent_timeline_response", (message) => {
if (message.type !== "fetch_agent_timeline_response") return;
agentStreamReducerQueue.flushAgent(message.payload.agentId);
applyTimelineResponse(message.payload);
});
const unsubProviderSubagentUpdate = client.on("agent.provider_subagents.update", (message) => {
if (message.type !== "agent.provider_subagents.update") return;
useProviderSubagentStore.getState().applyUpdate(serverId, message.payload);
});
const unsubScriptStatusUpdate = client.on("script_status_update", (message) => {
if (message.type !== "script_status_update") return;
setWorkspaces(serverId, (prev) => patchWorkspaceScripts(prev, message.payload));
});
const unsubCheckoutStatusUpdate = client.on("checkout_status_update", (message) => {
if (message.type !== "checkout_status_update") return;
applyCheckoutStatusUpdateFromEvent({ queryClient, serverId, message });
});
const unsubWorkspaceSetupProgress = client.on("workspace_setup_progress", (message) => {
if (message.type !== "workspace_setup_progress") return;
applyWorkspaceSetupProgress(message.payload);
});
const unsubWorkspaceSetupStatusResponse = client.on(
"workspace_setup_status_response",
(message) => {
if (message.type !== "workspace_setup_status_response") return;
const { workspaceId, snapshot } = message.payload;
if (snapshot) {
applyWorkspaceSetupProgress({ workspaceId, ...snapshot });
}
},
);
const unsubStatus = client.on("status", (message) => {
if (message.type !== "status") return;
const serverInfo = parseServerInfoStatusPayload(message.payload);
if (serverInfo) {
viewedTimelineSyncRef.current?.setDeliveryMode(
getTimelineDeliveryMode(serverInfo.features?.selectiveAgentTimeline),
);
updateSessionServerInfo(serverId, {
serverId: serverInfo.serverId,
hostname: serverInfo.hostname,
version: serverInfo.version,
...(serverInfo.desktopManaged !== undefined
? { desktopManaged: serverInfo.desktopManaged }
: {}),
...(serverInfo.capabilities ? { capabilities: serverInfo.capabilities } : {}),
...(serverInfo.features ? { features: serverInfo.features } : {}),
});
return;
}
});
const unsubPermissionRequest = client.on("agent_permission_request", (message) => {
if (message.type !== "agent_permission_request") return;
const { agentId, request } = message.payload;
setPendingPermissions(serverId, (prev) => {
const next = new Map(prev);
const key = derivePendingPermissionKey(agentId, request);
next.set(key, { key, agentId, request });
return next;
});
});
const unsubPermissionResolved = client.on("agent_permission_resolved", (message) => {
if (message.type !== "agent_permission_resolved") return;
const { requestId, agentId } = message.payload;
setPendingPermissions(serverId, (prev) => {
const next = new Map(prev);
const derivedKey = `${agentId}:${requestId}`;
if (!next.delete(derivedKey)) {
for (const [key, pending] of next.entries()) {
if (pending.agentId === agentId && pending.request.id === requestId) {
next.delete(key);
break;
}
}
}
return next;
});
});
const unsubAudioOutput = client.on("audio_output", async (message) => {
if (message.type !== "audio_output") return;
if (!voiceAudioEngine) {
return;
}
const payload: AudioOutputPayload = message.payload;
if (payload.isVoiceMode && voiceRuntime) {
voiceRuntime.handleAudioOutput(serverId, payload);
return;
}
const playbackGroupId = payload.groupId ?? payload.id;
const chunkIndex = payload.chunkIndex ?? 0;
const isFinalChunk = payload.isLastChunk ?? true;
if (!audioOutputBuffersRef.current.has(playbackGroupId)) {
audioOutputBuffersRef.current.set(playbackGroupId, []);
}
const bufferedChunks = audioOutputBuffersRef.current.get(playbackGroupId)!;
bufferedChunks.push({
chunkIndex,
audio: payload.audio,
format: payload.format,
id: payload.id,
});
activeAudioGroupsRef.current.add(playbackGroupId);
setIsPlayingAudio(serverId, true);
if (!isFinalChunk) {
return;
}
bufferedChunks.sort((left, right) => left.chunkIndex - right.chunkIndex);
const chunkIds = bufferedChunks.map((chunk) => chunk.id);
const shouldPlay =
!payload.isVoiceMode || (voiceRuntime?.shouldPlayVoiceAudio(serverId) ?? false);
const audioBlob = buildAudioPlaybackSource(bufferedChunks);
function logAudioPlayedError(error: unknown): void {
console.warn("[Session] Failed to confirm audio playback:", error);
}
const confirmAudioPlayed = async () => {
await Promise.all(
chunkIds.map((chunkId) => client.audioPlayed(chunkId).catch(logAudioPlayedError)),
);
};
let startedVoicePlayback = false;
try {
if (shouldPlay) {
if (payload.isVoiceMode) {
startedVoicePlayback = true;
voiceRuntime?.onAssistantAudioStarted(serverId);
}
await voiceAudioEngine.play(audioBlob);
}
await confirmAudioPlayed();
} catch (error) {
console.error("[Session] Audio playback error:", error);
await confirmAudioPlayed();
} finally {
audioOutputBuffersRef.current.delete(playbackGroupId);
activeAudioGroupsRef.current.delete(playbackGroupId);
setIsPlayingAudio(serverId, activeAudioGroupsRef.current.size > 0);
if (startedVoicePlayback) {
voiceRuntime?.onAssistantAudioFinished(serverId);
}
}
});
const unsubActivity = client.on("activity_log", (message) => {
if (message.type !== "activity_log") return;
const data = message.payload;
if (data.type === "system" && data.content.includes("Transcribing")) {
return;
}
if (data.type === "tool_call" && data.metadata) {
const toolCallId =
typeof data.metadata.toolCallId === "string" ? data.metadata.toolCallId : "";
const toolName = typeof data.metadata.toolName === "string" ? data.metadata.toolName : "";
const args = data.metadata.arguments;
setMessages(serverId, (prev) => [
...prev,
{
type: "tool_call",
id: toolCallId,
timestamp: Date.now(),
toolName,
args,
status: "executing",
},
]);
return;
}
if (data.type === "tool_result" && data.metadata) {
const toolCallId =
typeof data.metadata.toolCallId === "string" ? data.metadata.toolCallId : "";
const result = data.metadata.result;
const applyToolResult = applyToolResultToMessages(toolCallId, result);
setMessages(serverId, applyToolResult);
return;
}
if (data.type === "error" && data.metadata && "toolCallId" in data.metadata) {
const toolCallId =
typeof data.metadata.toolCallId === "string" ? data.metadata.toolCallId : "";
const error = data.metadata.error;
const applyToolError = applyToolErrorToMessages(toolCallId, error);
setMessages(serverId, applyToolError);
}
notifyVoiceAbortFailure(data, toast.error);
let activityType: "system" | "info" | "success" | "error" = "info";
if (data.type === "error") activityType = "error";
if (data.type === "transcript") {
setMessages(serverId, (prev) => [
...prev,
{
type: "user",
id: generateMessageId(),
timestamp: Date.now(),
message: data.content,
},
]);
return;
}
if (data.type === "assistant") {
setMessages(serverId, (prev) => [
...prev,
{
type: "assistant",
id: generateMessageId(),
timestamp: Date.now(),
message: data.content,
},
]);
setCurrentAssistantMessage(serverId, "");
return;
}
setMessages(serverId, (prev) => [
...prev,
{
type: "activity",
id: generateMessageId(),
timestamp: Date.now(),
activityType,
message: data.content,
metadata: data.metadata,
},
]);
});
const unsubChunk = client.on("assistant_chunk", (message) => {
if (message.type !== "assistant_chunk") return;
setCurrentAssistantMessage(serverId, (prev) => prev + message.payload.chunk);
});
const unsubTranscription = client.on("transcription_result", (message) => {
if (message.type !== "transcription_result") return;
const transcriptText = message.payload.text.trim();
voiceRuntime?.onTranscriptionResult(serverId, transcriptText);
if (!transcriptText) {
return;
}
setCurrentAssistantMessage(serverId, "");
});
const unsubVoiceInputState = client.on("voice_input_state", (message) => {
if (message.type !== "voice_input_state") return;
voiceRuntime?.onServerSpeechStateChanged(serverId, message.payload.isSpeaking);
});
const unsubTerminalAttention = client.on("terminal_attention_required", (message) => {
if (message.type !== "terminal_attention_required") {
return;
}
if (!message.payload.shouldNotify) {
return;
}
void sendOsNotification({
title: message.payload.title,
body: message.payload.body,
// serverId + workspaceId + terminalId route a tap to the terminal tab; cwd is
// carried as a fallback identifier when the daemon resolved no workspace.
data: {
serverId: message.payload.serverId ?? serverId,
terminalId: message.payload.terminalId,
cwd: message.payload.cwd,
...(message.payload.workspaceId ? { workspaceId: message.payload.workspaceId } : {}),
},
});
});
return () => {
unsubAgentStream();
unsubAgentTimeline();
unsubProviderSubagentUpdate();
unsubAgentAttention();
unsubScriptStatusUpdate();
unsubCheckoutStatusUpdate();
unsubWorkspaceSetupProgress();
unsubWorkspaceSetupStatusResponse();
unsubStatus();
unsubPermissionRequest();
unsubPermissionResolved();
unsubAudioOutput();
unsubActivity();
unsubChunk();
unsubTranscription();
unsubVoiceInputState();
unsubTerminalAttention();
agentStreamReducerQueue.dispose({ flush: true });
};
}, [
client,
queryClient,
serverId,
setIsPlayingAudio,
setMessages,
setCurrentAssistantMessage,
setAgentStreamTail,
setAgentStreamHead,
setAgentStreamState,
applyAgentTurnLiveness,
clearAgentStreamHead,
setAgentTimelineCursor,
setInitializingAgents,
setAgents,
setWorkspaces,
setPendingPermissions,
notifyAgentAttention,
recoverTimelineGap,
applyWorkspaceSetupProgress,
applyTimelineResponse,
updateSessionServerInfo,
toast,
voiceRuntime,
voiceAudioEngine,
]);
const _cancelAgentRun = useCallback(
(agentId: string) => {
if (!client) {
console.warn("[Session] cancelAgent skipped: daemon unavailable");
return;
}
void client.cancelAgent(agentId).catch((error) => {
console.error("[Session] Failed to cancel agent:", error);
});
},
[client],
);
const _deleteAgent = useCallback(
(agentId: string) => {
if (!client) {
console.warn("[Session] deleteAgent skipped: daemon unavailable");
return;
}
void client.deleteAgent(agentId).catch((error) => {
console.error("[Session] Failed to delete agent:", error);
});
},
[client],
);
const _archiveAgent = useCallback(
(agentId: string) => {
if (!client) {
console.warn("[Session] archiveAgent skipped: daemon unavailable");
return;
}
void client.archiveAgent(agentId).catch((error) => {
console.error("[Session] Failed to archive agent:", error);
});
},
[client],
);
const _restartServer = useCallback(
(reason?: string) => {
if (!client) {
console.warn("[Session] restartServer skipped: daemon unavailable");
return;
}
void client.restartServer(reason).catch((error) => {
console.error("[Session] Failed to restart server:", error);
});
},
[client],
);
const _createAgent = useCallback(
async ({
config,
initialPrompt,
images,
attachments,
git,
worktreeName,
requestId,
}: {
config: AgentSessionConfig;
initialPrompt: string;
images?: AttachmentMetadata[];
attachments?: AgentAttachment[];
git?: GitSetupOptions;
worktreeName?: string;
requestId?: string;
}) => {
if (!client) {
console.warn("[Session] createAgent skipped: daemon unavailable");
return;
}
const trimmedPrompt = initialPrompt.trim();
let imagesData: Array<{ data: string; mimeType: string }> | undefined;
try {
imagesData = await encodeImages(images);
} catch (error) {
console.error("[Session] Failed to prepare images for agent creation:", error);
}
await client.createAgent({
config,
...(trimmedPrompt ? { initialPrompt: trimmedPrompt } : {}),
...(imagesData && imagesData.length > 0 ? { images: imagesData } : {}),
...(attachments && attachments.length > 0 ? { attachments } : {}),
...(git ? { git } : {}),
...(worktreeName ? { worktreeName } : {}),
...(requestId ? { requestId } : {}),
});
},
[client],
);
const _setAgentMode = useCallback(
(agentId: string, modeId: string) => {
if (!client) {
console.warn("[Session] setAgentMode skipped: daemon unavailable");
return;
}
void client
.setAgentMode(agentId, modeId)
.then((notice) => showProviderNoticeToast(toast, notice))
.catch((error) => {
console.error("[Session] Failed to set agent mode:", error);
toast.error(toErrorMessage(error));
});
},
[client, toast],
);
const _setAgentModel = useCallback(
(agentId: string, modelId: string | null) => {
if (!client) {
console.warn("[Session] setAgentModel skipped: daemon unavailable");
return;
}
void client.setAgentModel(agentId, modelId).catch((error) => {
console.error("[Session] Failed to set agent model:", error);
toast.error(toErrorMessage(error));
});
},
[client, toast],
);
const _setAgentThinkingOption = useCallback(
(agentId: string, thinkingOptionId: string | null) => {
if (!client) {
console.warn("[Session] setAgentThinkingOption skipped: daemon unavailable");
return;
}
void client
.setAgentThinkingOption(agentId, thinkingOptionId)
.then((notice) => showProviderNoticeToast(toast, notice))
.catch((error) => {
console.error("[Session] Failed to set agent thinking option:", error);
toast.error(toErrorMessage(error));
});
},
[client, toast],
);
const _respondToPermission = useCallback(
(agentId: string, requestId: string, response: AgentPermissionResponse) => {
if (!client) {
console.warn("[Session] respondToPermission skipped: daemon unavailable");
return;
}
void client.respondToPermission(agentId, requestId, response).catch((error) => {
console.error("[Session] Failed to respond to permission:", error);
});
},
[client],
);
return children;
}