Keep agent timelines stable from submission through resume (#2718)
* Reapply: Stop completed turns from appearing stuck (#2484) * Reapply: Keep older chat history and image previews stable (#2490) * Lock timeline layout stability with user-driven specs Two reds reproduce the reported jank: - new agent, first prompt: the row paints at top=100 then jumps to 144 through authoritative hydration, because the draft tab and the agent panel each render their own AgentStreamView for the same logical row. - long loaded timeline, submit: scrollHeight collapses 10793 -> 9111 mid-turn, because splitOrderedTail reclassifies measured rows into the virtualized segment where the virtualizer falls back to estimateStreamItemHeight. Also restores centered timeline rows: the pagination anchor wrapper introduced in #2490 was a plain block div, which broke alignSelf: center. * Measure the turn footer blink from when the footer appears Bounding the window at the prompt row conflated two things: the footer arriving a frame after the row, and the footer leaving the layout and coming back. Bounded from the footer's first frame, the blink still reproduces, so the footer-ownership race is real rather than a sampling artifact. * Rename the shadowed prompt row binding in the frame recorder * fix(desktop): allow concurrent worktree dev instances Desktop terminals can inherit another instance's profile path and debugging flags. Scope the profile to the current dev root and select an available debugging port so instances do not share global runtime state. * fix(desktop): expose development providers in dev builds Desktop-managed daemons inherited no Paseo runtime mode, so dev builds constructed the production provider registry. Mark unpackaged internal launches as development while continuing to force packaged launches to production. * fix(timeline): stabilize optimistic turn layout Keep initial history loading dormant until user intent and preserve active-turn layout across delayed transport updates. Add painted-frame coverage for optimistic timeline transitions. * fix(timeline): preserve optimistic prompts through hydration Canonical replacement relied on a transport submission record that can settle before a provider echoes the user row. Keep presentation continuity owned by the unreconciled row while destructive history replacement remains governed by active transactions. * fix(timeline): make submitted prompts canonical Submission settlement previously depended on status updates that could arrive after the optimistic row. Persist every accepted prompt in the canonical timeline and keep the footer active until RPC and canonical acknowledgement converge. * fix(timeline): stop older history pages repeating entries Backward pagination counted source sequence rows, so page boundaries inside a merged projected item could return that item repeatedly. Count projected entries and advance from their display anchors instead, with server and browser regressions covering the contract. * fix(timeline): resume agents from the latest history Use a bounded tail probe on resume so long background runs appear atomically instead of replaying every missed page. Preserve unchanged views, reconcile local/live rows, and keep skipped history reachable through backward pagination. * fix(timeline): reconcile turn state after missed updates Turn completion events can be missed while an agent timeline is unsubscribed, leaving footer activity stale even after the directory reports idle. Reconcile the single-owned turn lifecycle from authoritative resume snapshots while preserving newer live evidence. * test(timeline): cover running state across refresh Protect the hydration boundary where directory state must restore consistent tab, composer, and footer activity. Expose the running indicator semantically so the regression checks user-visible state instead of spinner implementation details. * fix(timeline): keep history loading geometry stable Reserve a fixed history-start gutter regardless of pagination state so showing or hiding the loader cannot shift timeline content. * fix(timeline): keep turn activity coherent Derive footer, tab, and composer activity from one lifecycle and submission projection. Idle directory transitions now request an authoritative resume so missed terminal events converge without treating directory delivery order as turn authority. * fix(tests): align checks with runtime boundaries * fix(desktop): stop dev app surviving its terminal Electron was detached from the desktop runner, allowing it to outlive a stopped workspace terminal with broken output pipes and retain the single-instance lock. Keep Electron owned by the runner and preserve explicit process-group cleanup for detached child trees. * fix(timeline): preserve activity through turn handoff Correlate lifecycle events and snapshots with daemon-owned turn identity, and keep optimistic submissions active until their canonical row arrives. This prevents stale updates and RPC timing from exposing a false idle state. * refactor(timeline): remove submission timing state * fix(timeline): restore prompt identity on resume * fix(timeline): exclude attachments from canonical prompt text Structured provider prompts can include hidden history attachments before the user's text. Persist only plain prompt blocks so authoritative user messages match what the composer displayed while provider input remains unchanged. * refactor(agent): remove unused pending-run message identity * fix(desktop): let Chromium allocate the debugging port * refactor(agent): remove impossible finalized-turn tracking * fix(timeline): load history after keyboard scrolling Treat observed upward displacement as user intent so keyboard and scrollbar navigation can wake dormant pagination. * test(server): preserve accepted turn stream order The manager-owned accepted turn boundary intentionally precedes coalesced output. Assert that lifecycle event explicitly while retaining the timeline-before-terminal ordering contract. * test(app): scope sidebar fixtures by identity Shared E2E daemon state may contain unrelated projects. Keep stale-route coverage tied to its named missing project and drag only the two rows seeded by the reorder spec. * fix(timeline): gate prompt revisions by client capability * fix(timeline): finalize canonical prompt delivery Treat provider identity as a single stored-row transition so legacy clients receive one deliverable canonical prompt. Release identity-pending rows before later foreground output or the current terminal while preserving stale-turn isolation. * fix(server): preserve legacy prompt pagination * fix(timeline): keep assistant block identity stable Assign canonical block IDs during live promotion so terminal finalization cannot remount image-bearing rows. * fix(timeline): apply covered revisions during resume append Run the exact canonical revision pass before overlapping resume pages append newer rows, while leaving cursor advancement to the incremental path. * fix(server): preserve queued agent transitions Project each queued live snapshot before awaiting stored metadata so later lifecycle state cannot overwrite an earlier transition. * fix(server): snapshot queued agent updates Capture each live agent payload before it enters the per-agent queue so shared nested state cannot change while an earlier storage read is pending. * docs(app): date agent turn compatibility cleanup * refactor(timeline): mint submitted prompt identity at acceptance Canonical prompt rows now keep one immutable wire identity while provider identity remains internal for rewind translation. This removes the unreleased same-sequence revision and legacy projection contract. * refactor(app): remove prompt revision reconciliation Submitted prompt rows are immutable on the wire, so resume and live reducers no longer need to apply already-covered same-sequence updates. * test(app): cover submitted prompt rewind roundtrip * docs(timeline): describe immutable submitted prompt identity Provider echoes now enrich daemon-internal identity only; wire rows keep their acceptance-time identity and rewind performs the provider translation. * test(server): validate mock rewind targets Reject unknown provider message IDs so app rewind coverage exercises wire-to-provider identity translation. * fix(server): validate rewind before interrupting * fix(server): omit rewind identity for daemon prompts * test(app): guard rewind eligibility and rejection * fix(server): preserve prompt identity across hydration * refactor(timeline): remove stale revision remnants * Revert "fix(server): preserve prompt identity across hydration" This reverts commit 6adcf7e5b0bc8601761af59b461a746cb5e395fa. * fix(timeline): settle missed submission acknowledgements
This commit is contained in:
Mohamed Boudra
@@ -0,0 +1,193 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import {
|
||||
collectRetainedAttachmentIds,
|
||||
retainAttachmentForGarbageCollection,
|
||||
} from "@/attachments/gc-retention";
|
||||
import {
|
||||
createAssistantImageAcquisitionCache,
|
||||
createAssistantImageFileAcquisitionKey,
|
||||
createAssistantImageFilePreviewAttachmentId,
|
||||
createAssistantImageOccurrenceKey,
|
||||
} from "./acquisition-cache";
|
||||
|
||||
describe("assistant image acquisition cache", () => {
|
||||
it("evicts a rejected acquisition so the next request can retry", async () => {
|
||||
const cache = createAssistantImageAcquisitionCache<string>({ capacity: 2 });
|
||||
let attempts = 0;
|
||||
|
||||
await expect(
|
||||
cache.acquire("image", async () => {
|
||||
attempts += 1;
|
||||
throw new Error("first attempt failed");
|
||||
}),
|
||||
).rejects.toThrow("first attempt failed");
|
||||
const recovered = await cache.acquire("image", async () => {
|
||||
attempts += 1;
|
||||
return "recovered";
|
||||
});
|
||||
|
||||
expect({ attempts, recovered, size: cache.size() }).toEqual({
|
||||
attempts: 2,
|
||||
recovered: "recovered",
|
||||
size: 1,
|
||||
});
|
||||
});
|
||||
|
||||
it("bounds successful acquisitions and evicts the least recently used entry", async () => {
|
||||
const cache = createAssistantImageAcquisitionCache<string>({ capacity: 2 });
|
||||
const located: string[] = [];
|
||||
const locate = async (key: string) => {
|
||||
located.push(key);
|
||||
return key;
|
||||
};
|
||||
|
||||
await cache.acquire("a", async () => await locate("a"));
|
||||
await cache.acquire("b", async () => await locate("b"));
|
||||
await cache.acquire("a", async () => await locate("a-again"));
|
||||
await cache.acquire("c", async () => await locate("c"));
|
||||
await cache.acquire("b", async () => await locate("b-again"));
|
||||
|
||||
expect({ located, size: cache.size() }).toEqual({
|
||||
located: ["a", "b", "c", "b-again"],
|
||||
size: 2,
|
||||
});
|
||||
});
|
||||
|
||||
it("reuses an acquired image when the current locator is unavailable", async () => {
|
||||
const cache = createAssistantImageAcquisitionCache<string>({ capacity: 2 });
|
||||
let unavailableCalls = 0;
|
||||
|
||||
await cache.acquire("message:image", async () => "persisted attachment");
|
||||
const cached = await cache.acquire("message:image", async () => {
|
||||
unavailableCalls += 1;
|
||||
throw new Error("daemon disconnected");
|
||||
});
|
||||
|
||||
expect({ cached, unavailableCalls }).toEqual({
|
||||
cached: "persisted attachment",
|
||||
unavailableCalls: 0,
|
||||
});
|
||||
expect(cache.peek("message:image")).toBe("persisted attachment");
|
||||
});
|
||||
|
||||
it("scopes file acquisitions to the rendered message occurrence", () => {
|
||||
const first = createAssistantImageFileAcquisitionKey({
|
||||
serverId: "server",
|
||||
occurrenceKey: "message-1:image-1",
|
||||
cwd: "/workspace",
|
||||
path: "screenshot.png",
|
||||
});
|
||||
const remount = createAssistantImageFileAcquisitionKey({
|
||||
serverId: "server",
|
||||
occurrenceKey: "message-1:image-1",
|
||||
cwd: "/workspace",
|
||||
path: "screenshot.png",
|
||||
});
|
||||
const laterMessage = createAssistantImageFileAcquisitionKey({
|
||||
serverId: "server",
|
||||
occurrenceKey: "message-2:image-1",
|
||||
cwd: "/workspace",
|
||||
path: "screenshot.png",
|
||||
});
|
||||
|
||||
expect(remount).toBe(first);
|
||||
expect(laterMessage).not.toBe(first);
|
||||
});
|
||||
|
||||
it("scopes persisted file previews to the rendered message occurrence", () => {
|
||||
const first = createAssistantImageFilePreviewAttachmentId({
|
||||
serverId: "server-1",
|
||||
occurrenceKey: "message-1:image-1",
|
||||
mimeType: "image/png",
|
||||
path: "/workspace/screenshot.png",
|
||||
size: 512,
|
||||
modifiedAt: "2026-07-27T12:00:00.000Z",
|
||||
contentLength: 512,
|
||||
});
|
||||
const second = createAssistantImageFilePreviewAttachmentId({
|
||||
serverId: "server-1",
|
||||
occurrenceKey: "message-2:image-1",
|
||||
mimeType: "image/png",
|
||||
path: "/workspace/screenshot.png",
|
||||
size: 512,
|
||||
modifiedAt: "2026-07-27T12:00:00.000Z",
|
||||
contentLength: 512,
|
||||
});
|
||||
|
||||
expect(second).not.toBe(first);
|
||||
});
|
||||
|
||||
it("scopes message occurrences to their agent", () => {
|
||||
const first = createAssistantImageOccurrenceKey({ agentId: "agent-1", itemId: "message-1" });
|
||||
const second = createAssistantImageOccurrenceKey({ agentId: "agent-2", itemId: "message-1" });
|
||||
|
||||
expect(second).not.toBe(first);
|
||||
});
|
||||
|
||||
it("retains successful values until their cache entry is evicted", async () => {
|
||||
const retained: string[] = [];
|
||||
const released: string[] = [];
|
||||
const cache = createAssistantImageAcquisitionCache<string>({
|
||||
capacity: 1,
|
||||
onRetain(value) {
|
||||
retained.push(value);
|
||||
return () => released.push(value);
|
||||
},
|
||||
});
|
||||
|
||||
await cache.acquire("first", async () => "attachment-1");
|
||||
expect({ retained, released }).toEqual({ retained: ["attachment-1"], released: [] });
|
||||
|
||||
await cache.acquire("second", async () => "attachment-2");
|
||||
expect({ retained, released }).toEqual({
|
||||
retained: ["attachment-1", "attachment-2"],
|
||||
released: ["attachment-1"],
|
||||
});
|
||||
});
|
||||
|
||||
it("does not evict a value while an active consumer retains it", async () => {
|
||||
const released: string[] = [];
|
||||
const cache = createAssistantImageAcquisitionCache<string>({
|
||||
capacity: 1,
|
||||
onRetain: (value) => () => released.push(value),
|
||||
});
|
||||
|
||||
const first = cache.acquireRetained("first", async () => "attachment-1");
|
||||
await first.promise;
|
||||
const second = cache.acquireRetained("second", async () => "attachment-2");
|
||||
await second.promise;
|
||||
|
||||
expect({ released, size: cache.size() }).toEqual({ released: [], size: 2 });
|
||||
|
||||
first.release();
|
||||
expect({ released, size: cache.size() }).toEqual({
|
||||
released: ["attachment-1"],
|
||||
size: 1,
|
||||
});
|
||||
second.release();
|
||||
});
|
||||
|
||||
it("protects every actively consumed attachment from garbage collection past capacity", async () => {
|
||||
const cache = createAssistantImageAcquisitionCache<{ id: string }>({
|
||||
capacity: 1,
|
||||
onRetain: (attachment) => retainAttachmentForGarbageCollection(attachment.id),
|
||||
});
|
||||
|
||||
const first = cache.acquireRetained("first", async () => ({ id: "mounted-image-1" }));
|
||||
await first.promise;
|
||||
const second = cache.acquireRetained("second", async () => ({ id: "mounted-image-2" }));
|
||||
await second.promise;
|
||||
|
||||
expect(collectRetainedAttachmentIds()).toEqual(new Set(["mounted-image-1", "mounted-image-2"]));
|
||||
|
||||
first.release();
|
||||
expect(collectRetainedAttachmentIds()).toEqual(new Set(["mounted-image-2"]));
|
||||
second.release();
|
||||
await expect(
|
||||
cache.acquire("cleanup", async () => {
|
||||
throw new Error("cleanup");
|
||||
}),
|
||||
).rejects.toThrow("cleanup");
|
||||
expect(collectRetainedAttachmentIds()).toEqual(new Set());
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,159 @@
|
||||
import { createPreviewAttachmentId } from "@/attachments/utils";
|
||||
|
||||
export interface AssistantImageAcquisitionCache<T> {
|
||||
acquire(key: string, locate: () => Promise<T>): Promise<T>;
|
||||
acquireRetained(
|
||||
key: string,
|
||||
locate: () => Promise<T>,
|
||||
): { promise: Promise<T>; value?: T; release: () => void };
|
||||
peek(key: string): T | undefined;
|
||||
size(): number;
|
||||
}
|
||||
|
||||
export function createAssistantImageOccurrenceKey(input: {
|
||||
agentId: string;
|
||||
itemId: string;
|
||||
}): string {
|
||||
return `${input.agentId}:${input.itemId}`;
|
||||
}
|
||||
|
||||
export function createAssistantImageFilePreviewAttachmentId(input: {
|
||||
serverId?: string;
|
||||
occurrenceKey: string;
|
||||
mimeType: string;
|
||||
path: string;
|
||||
size: number;
|
||||
modifiedAt?: string | null;
|
||||
contentLength: number;
|
||||
}): string {
|
||||
return createPreviewAttachmentId({
|
||||
mimeType: input.mimeType,
|
||||
path: input.path,
|
||||
size: input.size,
|
||||
modifiedAt: input.modifiedAt,
|
||||
contentLength: input.contentLength,
|
||||
contentKey: `${input.serverId ?? "unknown-server"}:${input.occurrenceKey}`,
|
||||
});
|
||||
}
|
||||
|
||||
export function createAssistantImageFileAcquisitionKey(input: {
|
||||
serverId?: string;
|
||||
occurrenceKey: string;
|
||||
cwd: string;
|
||||
path: string;
|
||||
}): string {
|
||||
return `file:${input.serverId ?? "unknown-server"}:${input.occurrenceKey}:${input.cwd}:${input.path}`;
|
||||
}
|
||||
|
||||
export function createAssistantImageAcquisitionCache<T>(input: {
|
||||
capacity: number;
|
||||
onRetain?: (value: T) => () => void;
|
||||
}): AssistantImageAcquisitionCache<T> {
|
||||
if (!Number.isInteger(input.capacity) || input.capacity < 1) {
|
||||
throw new Error("Assistant image acquisition cache capacity must be a positive integer.");
|
||||
}
|
||||
interface CacheEntry {
|
||||
pending: Promise<T>;
|
||||
resolved: boolean;
|
||||
value?: T;
|
||||
release: (() => void) | null;
|
||||
activeConsumers: number;
|
||||
}
|
||||
const entries = new Map<string, CacheEntry>();
|
||||
|
||||
const evict = (key: string, entry: CacheEntry) => {
|
||||
if (entries.get(key) === entry) {
|
||||
entries.delete(key);
|
||||
}
|
||||
entry.release?.();
|
||||
entry.release = null;
|
||||
};
|
||||
|
||||
const enforceCapacity = () => {
|
||||
while (entries.size > input.capacity) {
|
||||
let evicted = false;
|
||||
for (const [key, entry] of entries) {
|
||||
if (entry.activeConsumers > 0) {
|
||||
continue;
|
||||
}
|
||||
evict(key, entry);
|
||||
evicted = true;
|
||||
break;
|
||||
}
|
||||
if (!evicted) {
|
||||
return;
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
const acquireEntry = (key: string, locate: () => Promise<T>, retain: boolean): CacheEntry => {
|
||||
const cached = entries.get(key);
|
||||
if (cached) {
|
||||
entries.delete(key);
|
||||
entries.set(key, cached);
|
||||
if (retain) {
|
||||
cached.activeConsumers += 1;
|
||||
}
|
||||
return cached;
|
||||
}
|
||||
const pending = locate();
|
||||
const entry: CacheEntry = {
|
||||
pending,
|
||||
resolved: false,
|
||||
release: null,
|
||||
activeConsumers: retain ? 1 : 0,
|
||||
};
|
||||
entries.set(key, entry);
|
||||
enforceCapacity();
|
||||
void (async () => {
|
||||
try {
|
||||
const value = await pending;
|
||||
const release = input.onRetain?.(value) ?? null;
|
||||
if (entries.get(key) === entry) {
|
||||
entry.value = value;
|
||||
entry.resolved = true;
|
||||
entry.release = release;
|
||||
} else {
|
||||
release?.();
|
||||
}
|
||||
} catch {
|
||||
evict(key, entry);
|
||||
}
|
||||
})();
|
||||
return entry;
|
||||
};
|
||||
|
||||
return {
|
||||
acquire(key, locate) {
|
||||
return acquireEntry(key, locate, false).pending;
|
||||
},
|
||||
acquireRetained(key, locate) {
|
||||
const entry = acquireEntry(key, locate, true);
|
||||
let released = false;
|
||||
return {
|
||||
promise: entry.pending,
|
||||
...(entry.resolved ? { value: entry.value } : {}),
|
||||
release() {
|
||||
if (released) {
|
||||
return;
|
||||
}
|
||||
released = true;
|
||||
entry.activeConsumers = Math.max(0, entry.activeConsumers - 1);
|
||||
enforceCapacity();
|
||||
},
|
||||
};
|
||||
},
|
||||
peek(key) {
|
||||
const entry = entries.get(key);
|
||||
if (!entry?.resolved) {
|
||||
return undefined;
|
||||
}
|
||||
entries.delete(key);
|
||||
entries.set(key, entry);
|
||||
return entry.value;
|
||||
},
|
||||
size() {
|
||||
return entries.size;
|
||||
},
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,53 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import type { AttachmentMetadata } from "@/attachments/types";
|
||||
import {
|
||||
createAssistantImageFileAcquisition,
|
||||
type AssistantImageFileAcquisitionPort,
|
||||
} from "./file-acquisition";
|
||||
|
||||
class MemoryFileAcquisitionPort implements AssistantImageFileAcquisitionPort {
|
||||
readonly reads: Array<{ cwd: string; path: string }> = [];
|
||||
|
||||
async readFile(cwd: string, path: string) {
|
||||
this.reads.push({ cwd, path });
|
||||
return {
|
||||
kind: "image" as const,
|
||||
path,
|
||||
mime: "image/png",
|
||||
size: 4,
|
||||
modifiedAt: "1",
|
||||
bytes: new Uint8Array([1, 2, 3, 4]),
|
||||
};
|
||||
}
|
||||
|
||||
async persist(input: { id: string; mimeType: string; fileName: string | null }) {
|
||||
return {
|
||||
id: input.id,
|
||||
mimeType: input.mimeType,
|
||||
storageType: "web-indexeddb" as const,
|
||||
storageKey: input.id,
|
||||
fileName: input.fileName,
|
||||
byteSize: 4,
|
||||
createdAt: 1,
|
||||
} satisfies AttachmentMetadata;
|
||||
}
|
||||
}
|
||||
|
||||
describe("assistant image file acquisition", () => {
|
||||
it("recreates the same acquisition with a live port after reconnect", async () => {
|
||||
const common = {
|
||||
resolution: { kind: "file_rpc" as const, cwd: "/workspace", path: "reconnect.png" },
|
||||
serverId: "server",
|
||||
occurrenceKey: "agent:message:reconnect-image",
|
||||
unavailableMessage: "Image unavailable",
|
||||
};
|
||||
const disconnected = createAssistantImageFileAcquisition({ ...common, port: null });
|
||||
const connectedPort = new MemoryFileAcquisitionPort();
|
||||
const connected = createAssistantImageFileAcquisition({ ...common, port: connectedPort });
|
||||
|
||||
expect(disconnected?.key).toBe(connected?.key);
|
||||
await expect(disconnected?.locate()).rejects.toThrow("Image unavailable");
|
||||
await expect(connected?.locate()).resolves.toMatchObject({ mimeType: "image/png" });
|
||||
expect(connectedPort.reads).toEqual([{ cwd: "/workspace", path: "reconnect.png" }]);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,67 @@
|
||||
import type { FileReadResult } from "@getpaseo/client/internal/daemon-client";
|
||||
import type { AttachmentMetadata } from "@/attachments/types";
|
||||
import { getFileNameFromPath } from "@/attachments/utils";
|
||||
import type { AssistantImageSourceResolution } from "@/utils/assistant-image-source";
|
||||
import {
|
||||
createAssistantImageFileAcquisitionKey,
|
||||
createAssistantImageFilePreviewAttachmentId,
|
||||
} from "./acquisition-cache";
|
||||
|
||||
export interface AssistantImageFileAcquisitionPort {
|
||||
readFile(cwd: string, path: string): Promise<FileReadResult>;
|
||||
persist(input: {
|
||||
id: string;
|
||||
bytes: Uint8Array;
|
||||
mimeType: string;
|
||||
fileName: string | null;
|
||||
}): Promise<AttachmentMetadata>;
|
||||
}
|
||||
|
||||
export interface AssistantImageAcquisition {
|
||||
key: string;
|
||||
locate: () => Promise<AttachmentMetadata>;
|
||||
}
|
||||
|
||||
export function createAssistantImageFileAcquisition(input: {
|
||||
port: AssistantImageFileAcquisitionPort | null;
|
||||
resolution: AssistantImageSourceResolution | null;
|
||||
serverId?: string;
|
||||
occurrenceKey: string;
|
||||
unavailableMessage: string;
|
||||
}): AssistantImageAcquisition | null {
|
||||
if (input.resolution?.kind !== "file_rpc") {
|
||||
return null;
|
||||
}
|
||||
const { port, resolution } = input;
|
||||
return {
|
||||
key: createAssistantImageFileAcquisitionKey({
|
||||
serverId: input.serverId,
|
||||
occurrenceKey: input.occurrenceKey,
|
||||
cwd: resolution.cwd,
|
||||
path: resolution.path,
|
||||
}),
|
||||
locate: async () => {
|
||||
if (!port) {
|
||||
throw new Error(input.unavailableMessage);
|
||||
}
|
||||
const file = await port.readFile(resolution.cwd, resolution.path);
|
||||
if (file.kind !== "image") {
|
||||
throw new Error(input.unavailableMessage);
|
||||
}
|
||||
return await port.persist({
|
||||
id: createAssistantImageFilePreviewAttachmentId({
|
||||
serverId: input.serverId,
|
||||
occurrenceKey: input.occurrenceKey,
|
||||
mimeType: file.mime,
|
||||
path: file.path || resolution.path,
|
||||
size: file.size,
|
||||
modifiedAt: file.modifiedAt,
|
||||
contentLength: file.bytes.byteLength,
|
||||
}),
|
||||
bytes: file.bytes,
|
||||
mimeType: file.mime,
|
||||
fileName: getFileNameFromPath(file.path || resolution.path),
|
||||
});
|
||||
},
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,120 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { createAssistantImageLifecycle, transitionAssistantImageLifecycle } from "./lifecycle";
|
||||
|
||||
describe("assistant image lifecycle", () => {
|
||||
it("keeps preview URL recreation in the public loading state", () => {
|
||||
const loading = transitionAssistantImageLifecycle(createAssistantImageLifecycle(), {
|
||||
type: "preview_created",
|
||||
uri: "blob:first",
|
||||
aspectRatio: null,
|
||||
});
|
||||
const loaded = transitionAssistantImageLifecycle(loading, {
|
||||
type: "image_loaded",
|
||||
uri: "blob:first",
|
||||
aspectRatio: 1.5,
|
||||
});
|
||||
|
||||
const recreating = transitionAssistantImageLifecycle(loaded, {
|
||||
type: "preview_released",
|
||||
});
|
||||
|
||||
expect(recreating).toEqual({ status: "loading", uri: null, aspectRatio: null });
|
||||
});
|
||||
|
||||
it("publishes loaded only after the recreated URL loads", () => {
|
||||
const recreating = transitionAssistantImageLifecycle(createAssistantImageLifecycle(), {
|
||||
type: "preview_created",
|
||||
uri: "blob:recreated",
|
||||
aspectRatio: 1.5,
|
||||
});
|
||||
|
||||
expect(recreating).toEqual({
|
||||
status: "loading",
|
||||
uri: "blob:recreated",
|
||||
aspectRatio: 1.5,
|
||||
});
|
||||
|
||||
const recreated = transitionAssistantImageLifecycle(recreating, {
|
||||
type: "image_loaded",
|
||||
uri: "blob:recreated",
|
||||
aspectRatio: 0.75,
|
||||
});
|
||||
|
||||
expect(recreated).toEqual({
|
||||
status: "loaded",
|
||||
uri: "blob:recreated",
|
||||
aspectRatio: 0.75,
|
||||
});
|
||||
});
|
||||
|
||||
it("does not reset a loaded image when the same preview is reported again", () => {
|
||||
const loading = transitionAssistantImageLifecycle(createAssistantImageLifecycle(), {
|
||||
type: "preview_created",
|
||||
uri: "blob:current",
|
||||
aspectRatio: null,
|
||||
});
|
||||
const loaded = transitionAssistantImageLifecycle(loading, {
|
||||
type: "image_loaded",
|
||||
uri: "blob:current",
|
||||
aspectRatio: 1.5,
|
||||
});
|
||||
|
||||
const repeated = transitionAssistantImageLifecycle(loaded, {
|
||||
type: "preview_created",
|
||||
uri: "blob:current",
|
||||
aspectRatio: 1.5,
|
||||
});
|
||||
|
||||
expect(repeated).toBe(loaded);
|
||||
});
|
||||
|
||||
it("publishes failed for a terminal image failure on the current URI", () => {
|
||||
const loading = transitionAssistantImageLifecycle(createAssistantImageLifecycle(), {
|
||||
type: "preview_created",
|
||||
uri: "blob:current",
|
||||
aspectRatio: null,
|
||||
});
|
||||
const failed = transitionAssistantImageLifecycle(loading, {
|
||||
type: "failed",
|
||||
uri: "blob:current",
|
||||
message: "Unable to load image preview.",
|
||||
});
|
||||
|
||||
expect(failed).toEqual({
|
||||
status: "failed",
|
||||
message: "Unable to load image preview.",
|
||||
});
|
||||
});
|
||||
|
||||
it("ignores a stale load callback from a replaced URI", () => {
|
||||
const current = transitionAssistantImageLifecycle(createAssistantImageLifecycle(), {
|
||||
type: "preview_created",
|
||||
uri: "blob:current",
|
||||
aspectRatio: 1.25,
|
||||
});
|
||||
|
||||
const afterStaleLoad = transitionAssistantImageLifecycle(current, {
|
||||
type: "image_loaded",
|
||||
uri: "blob:released",
|
||||
aspectRatio: 2,
|
||||
});
|
||||
|
||||
expect(afterStaleLoad).toEqual(current);
|
||||
});
|
||||
|
||||
it("ignores a stale error callback from a replaced URI", () => {
|
||||
const current = transitionAssistantImageLifecycle(createAssistantImageLifecycle(), {
|
||||
type: "preview_created",
|
||||
uri: "blob:current",
|
||||
aspectRatio: null,
|
||||
});
|
||||
|
||||
const afterStaleError = transitionAssistantImageLifecycle(current, {
|
||||
type: "failed",
|
||||
uri: "blob:released",
|
||||
message: "Image unavailable",
|
||||
});
|
||||
|
||||
expect(afterStaleError).toEqual(current);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,46 @@
|
||||
export type AssistantImageLifecycle =
|
||||
| { status: "loading"; uri: string | null; aspectRatio: number | null }
|
||||
| { status: "loaded"; uri: string; aspectRatio: number }
|
||||
| { status: "failed"; message: string };
|
||||
|
||||
export type AssistantImageLifecycleEvent =
|
||||
| { type: "preview_created"; uri: string; aspectRatio: number | null }
|
||||
| { type: "preview_released" }
|
||||
| { type: "image_loaded"; uri: string; aspectRatio: number }
|
||||
| { type: "failed"; uri: string; message: string };
|
||||
|
||||
export function createAssistantImageLifecycle(): AssistantImageLifecycle {
|
||||
return { status: "loading", uri: null, aspectRatio: null };
|
||||
}
|
||||
|
||||
export function transitionAssistantImageLifecycle(
|
||||
state: AssistantImageLifecycle,
|
||||
event: AssistantImageLifecycleEvent,
|
||||
): AssistantImageLifecycle {
|
||||
if (event.type === "image_loaded") {
|
||||
if (state.status !== "loading" || state.uri !== event.uri) {
|
||||
return state;
|
||||
}
|
||||
return {
|
||||
status: "loaded",
|
||||
uri: event.uri,
|
||||
aspectRatio: event.aspectRatio,
|
||||
};
|
||||
}
|
||||
if (event.type === "failed") {
|
||||
if (state.status === "failed" || state.uri !== event.uri) {
|
||||
return state;
|
||||
}
|
||||
return { status: "failed", message: event.message };
|
||||
}
|
||||
if (event.type === "preview_created") {
|
||||
if (state.status === "loaded" && state.uri === event.uri) {
|
||||
return state;
|
||||
}
|
||||
return { status: "loading", uri: event.uri, aspectRatio: event.aspectRatio };
|
||||
}
|
||||
if (event.type === "preview_released") {
|
||||
return { status: "loading", uri: null, aspectRatio: null };
|
||||
}
|
||||
return state;
|
||||
}
|
||||
@@ -0,0 +1,48 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import {
|
||||
type AssistantImageRenderedDimensionsReader,
|
||||
resolveAssistantImageLoadDimensions,
|
||||
} from "./load-dimensions";
|
||||
|
||||
class MemoryRenderedDimensionsReader implements AssistantImageRenderedDimensionsReader {
|
||||
constructor(private readonly dimensions: { width: number; height: number } | null) {}
|
||||
|
||||
read(): { width: number; height: number } | null {
|
||||
return this.dimensions;
|
||||
}
|
||||
}
|
||||
|
||||
describe("assistant image load dimensions", () => {
|
||||
it("prefers native source dimensions", () => {
|
||||
const dimensions = resolveAssistantImageLoadDimensions({
|
||||
source: { width: 640, height: 320 },
|
||||
target: { naturalWidth: 800, naturalHeight: 400 },
|
||||
renderedImage: null,
|
||||
renderedDimensions: new MemoryRenderedDimensionsReader({ width: 900, height: 600 }),
|
||||
});
|
||||
|
||||
expect(dimensions).toEqual({ width: 640, height: 320 });
|
||||
});
|
||||
|
||||
it("uses browser event dimensions when native dimensions are absent", () => {
|
||||
const dimensions = resolveAssistantImageLoadDimensions({
|
||||
source: { width: 0, height: 0 },
|
||||
target: { naturalWidth: 800, naturalHeight: 400 },
|
||||
renderedImage: null,
|
||||
renderedDimensions: new MemoryRenderedDimensionsReader({ width: 900, height: 600 }),
|
||||
});
|
||||
|
||||
expect(dimensions).toEqual({ width: 800, height: 400 });
|
||||
});
|
||||
|
||||
it("falls back to the rendered image adapter", () => {
|
||||
const renderedImage = { id: "rendered-image" };
|
||||
const dimensions = resolveAssistantImageLoadDimensions({
|
||||
target: null,
|
||||
renderedImage,
|
||||
renderedDimensions: new MemoryRenderedDimensionsReader({ width: 900, height: 600 }),
|
||||
});
|
||||
|
||||
expect(dimensions).toEqual({ width: 900, height: 600 });
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,45 @@
|
||||
export interface AssistantImageDimensions {
|
||||
width: number;
|
||||
height: number;
|
||||
}
|
||||
|
||||
interface AssistantImageDimensionCandidate {
|
||||
width?: unknown;
|
||||
height?: unknown;
|
||||
}
|
||||
|
||||
export interface AssistantImageRenderedDimensionsReader {
|
||||
read(renderedImage: unknown): AssistantImageDimensions | null;
|
||||
}
|
||||
|
||||
function readPositiveDimensions(
|
||||
candidate: AssistantImageDimensionCandidate | null | undefined,
|
||||
): AssistantImageDimensions | null {
|
||||
if (
|
||||
typeof candidate?.width !== "number" ||
|
||||
typeof candidate.height !== "number" ||
|
||||
candidate.width <= 0 ||
|
||||
candidate.height <= 0
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
return { width: candidate.width, height: candidate.height };
|
||||
}
|
||||
|
||||
export function resolveAssistantImageLoadDimensions(input: {
|
||||
source?: AssistantImageDimensionCandidate | null;
|
||||
target?: { naturalWidth?: unknown; naturalHeight?: unknown } | null;
|
||||
renderedImage: unknown;
|
||||
renderedDimensions: AssistantImageRenderedDimensionsReader;
|
||||
}): AssistantImageDimensions | null {
|
||||
const source = readPositiveDimensions(input.source);
|
||||
if (source) {
|
||||
return source;
|
||||
}
|
||||
|
||||
const target = readPositiveDimensions({
|
||||
width: input.target?.naturalWidth,
|
||||
height: input.target?.naturalHeight,
|
||||
});
|
||||
return target ?? input.renderedDimensions.read(input.renderedImage);
|
||||
}
|
||||
@@ -0,0 +1,49 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { runAssistantImageOperationWithRetry } from "./retry";
|
||||
|
||||
describe("assistant image retry", () => {
|
||||
it("recovers from transient failures while the image remains mounted", async () => {
|
||||
const waits: number[] = [];
|
||||
let attempts = 0;
|
||||
|
||||
const result = await runAssistantImageOperationWithRetry({
|
||||
operation: async () => {
|
||||
attempts += 1;
|
||||
if (attempts < 3) {
|
||||
throw new Error("transient");
|
||||
}
|
||||
return "loaded";
|
||||
},
|
||||
delaysMs: [10, 20, 30],
|
||||
wait: async (delayMs) => {
|
||||
waits.push(delayMs);
|
||||
},
|
||||
});
|
||||
|
||||
expect({ attempts, waits, result }).toEqual({
|
||||
attempts: 3,
|
||||
waits: [10, 20],
|
||||
result: "loaded",
|
||||
});
|
||||
});
|
||||
|
||||
it("stops retrying after the image is released", async () => {
|
||||
let stopped = false;
|
||||
let attempts = 0;
|
||||
|
||||
await expect(
|
||||
runAssistantImageOperationWithRetry({
|
||||
operation: async () => {
|
||||
attempts += 1;
|
||||
throw new Error("transient");
|
||||
},
|
||||
delaysMs: [10, 20],
|
||||
shouldStop: () => stopped,
|
||||
wait: async () => {
|
||||
stopped = true;
|
||||
},
|
||||
}),
|
||||
).rejects.toThrow("transient");
|
||||
expect(attempts).toBe(1);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,31 @@
|
||||
export const ASSISTANT_IMAGE_RETRY_DELAYS_MS = [100, 400, 1_200] as const;
|
||||
|
||||
export async function runAssistantImageOperationWithRetry<T>(input: {
|
||||
operation: () => Promise<T>;
|
||||
delaysMs?: readonly number[];
|
||||
shouldStop?: () => boolean;
|
||||
wait?: (delayMs: number) => Promise<void>;
|
||||
}): Promise<T> {
|
||||
const delays = input.delaysMs ?? ASSISTANT_IMAGE_RETRY_DELAYS_MS;
|
||||
const shouldStop = input.shouldStop ?? (() => false);
|
||||
const wait =
|
||||
input.wait ??
|
||||
(async (delayMs: number) => {
|
||||
await new Promise<void>((resolve) => setTimeout(resolve, delayMs));
|
||||
});
|
||||
|
||||
for (let attempt = 0; ; attempt += 1) {
|
||||
try {
|
||||
return await input.operation();
|
||||
} catch (error) {
|
||||
const delay = delays[attempt];
|
||||
if (delay === undefined || shouldStop()) {
|
||||
throw error;
|
||||
}
|
||||
await wait(delay);
|
||||
if (shouldStop()) {
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,544 @@
|
||||
import {
|
||||
useCallback,
|
||||
useEffect,
|
||||
useLayoutEffect,
|
||||
useMemo,
|
||||
useReducer,
|
||||
useRef,
|
||||
useState,
|
||||
} from "react";
|
||||
import type { ImageLoadEvent } from "react-native";
|
||||
import { useTranslation } from "react-i18next";
|
||||
import type { DaemonClient } from "@getpaseo/client/internal/daemon-client";
|
||||
import type { AttachmentMetadata } from "@/attachments/types";
|
||||
import { isWeb } from "@/constants/platform";
|
||||
import { useStableEvent } from "@/hooks/use-stable-event";
|
||||
import { retainAttachmentForGarbageCollection } from "@/attachments/gc-retention";
|
||||
import {
|
||||
persistAttachmentFromBytes,
|
||||
persistAttachmentFromDataUrl,
|
||||
releaseAttachmentPreviewUrl,
|
||||
resolveAttachmentPreviewUrl,
|
||||
} from "@/attachments/service";
|
||||
import { createPreviewAttachmentId, parseImageDataUrl } from "@/attachments/utils";
|
||||
import {
|
||||
getAssistantImageMetadata,
|
||||
setAssistantImageMetadata,
|
||||
} from "@/utils/assistant-image-metadata";
|
||||
import { resolveAssistantImageSource } from "@/utils/assistant-image-source";
|
||||
import { createAssistantImageAcquisitionCache } from "./acquisition-cache";
|
||||
import {
|
||||
createAssistantImageFileAcquisition,
|
||||
type AssistantImageAcquisition,
|
||||
type AssistantImageFileAcquisitionPort,
|
||||
} from "./file-acquisition";
|
||||
import {
|
||||
createAssistantImageLifecycle,
|
||||
transitionAssistantImageLifecycle,
|
||||
type AssistantImageLifecycle,
|
||||
type AssistantImageLifecycleEvent,
|
||||
} from "./lifecycle";
|
||||
import {
|
||||
type AssistantImageRenderedDimensionsReader,
|
||||
resolveAssistantImageLoadDimensions,
|
||||
} from "./load-dimensions";
|
||||
import { runAssistantImageOperationWithRetry } from "./retry";
|
||||
|
||||
interface AssistantImageRenderBinding {
|
||||
uri: string;
|
||||
onRef: (instance: unknown) => void;
|
||||
onLoad: (event: ImageLoadEvent) => void;
|
||||
onError: () => void;
|
||||
}
|
||||
|
||||
const renderedDimensions: AssistantImageRenderedDimensionsReader = {
|
||||
read(renderedImage) {
|
||||
if (!isWeb || !(renderedImage instanceof HTMLElement)) {
|
||||
return null;
|
||||
}
|
||||
const image = renderedImage.querySelector("img");
|
||||
return image && image.naturalWidth > 0 && image.naturalHeight > 0
|
||||
? { width: image.naturalWidth, height: image.naturalHeight }
|
||||
: null;
|
||||
},
|
||||
};
|
||||
|
||||
export type AssistantImageResult =
|
||||
| {
|
||||
status: "loading";
|
||||
binding: AssistantImageRenderBinding | null;
|
||||
aspectRatio: number | null;
|
||||
}
|
||||
| {
|
||||
status: "loaded";
|
||||
binding: AssistantImageRenderBinding;
|
||||
aspectRatio: number;
|
||||
}
|
||||
| { status: "failed"; message: string };
|
||||
|
||||
interface UseAssistantImageInput {
|
||||
source: string;
|
||||
occurrenceKey: string;
|
||||
client?: DaemonClient | null;
|
||||
workspaceRoot?: string;
|
||||
serverId?: string;
|
||||
}
|
||||
|
||||
type PreviewUrlState =
|
||||
| { status: "waiting" }
|
||||
| { status: "loading" }
|
||||
| { status: "loaded"; uri: string }
|
||||
| { status: "failed"; error: unknown };
|
||||
|
||||
type AttachmentAcquisitionState =
|
||||
| { status: "waiting" }
|
||||
| { status: "loading" }
|
||||
| { status: "loaded"; attachment: AttachmentMetadata }
|
||||
| { status: "failed"; error: unknown };
|
||||
|
||||
interface DataImage {
|
||||
mimeType: string;
|
||||
base64: string;
|
||||
cacheKey: string;
|
||||
}
|
||||
|
||||
const attachmentAcquisitionCache = createAssistantImageAcquisitionCache<AttachmentMetadata>({
|
||||
capacity: 500,
|
||||
onRetain: (attachment) => retainAttachmentForGarbageCollection(attachment.id),
|
||||
});
|
||||
|
||||
interface CachedPreviewUrl {
|
||||
attachment: AttachmentMetadata;
|
||||
uri: string;
|
||||
}
|
||||
|
||||
const previewUrlCache = createAssistantImageAcquisitionCache<CachedPreviewUrl>({
|
||||
capacity: 500,
|
||||
onRetain:
|
||||
({ attachment, uri }) =>
|
||||
() => {
|
||||
void releaseAttachmentPreviewUrl({ attachment, url: uri });
|
||||
},
|
||||
});
|
||||
|
||||
const LOADED_IMAGE_CACHE_CAPACITY = 500;
|
||||
const loadedImageCache = new Map<string, number>();
|
||||
|
||||
function getLoadedImageAspectRatio(uri: string): number | null {
|
||||
const aspectRatio = loadedImageCache.get(uri);
|
||||
if (aspectRatio === undefined) {
|
||||
return null;
|
||||
}
|
||||
loadedImageCache.delete(uri);
|
||||
loadedImageCache.set(uri, aspectRatio);
|
||||
return aspectRatio;
|
||||
}
|
||||
|
||||
function rememberLoadedImage(uri: string, aspectRatio: number): void {
|
||||
loadedImageCache.delete(uri);
|
||||
loadedImageCache.set(uri, aspectRatio);
|
||||
if (loadedImageCache.size <= LOADED_IMAGE_CACHE_CAPACITY) {
|
||||
return;
|
||||
}
|
||||
const leastRecentlyUsedUri = loadedImageCache.keys().next().value;
|
||||
if (leastRecentlyUsedUri !== undefined) {
|
||||
loadedImageCache.delete(leastRecentlyUsedUri);
|
||||
}
|
||||
}
|
||||
|
||||
function useAttachmentAcquisition(
|
||||
acquisition: AssistantImageAcquisition | null,
|
||||
): AttachmentAcquisitionState {
|
||||
const acquisitionKey = acquisition?.key ?? null;
|
||||
const [entry, setEntry] = useState<{
|
||||
key: string | null;
|
||||
state: AttachmentAcquisitionState;
|
||||
}>(() => {
|
||||
const cached = acquisitionKey ? attachmentAcquisitionCache.peek(acquisitionKey) : undefined;
|
||||
return {
|
||||
key: acquisitionKey,
|
||||
state: cached ? { status: "loaded", attachment: cached } : { status: "waiting" },
|
||||
};
|
||||
});
|
||||
|
||||
useEffect(() => {
|
||||
let disposed = false;
|
||||
let releaseCurrent: (() => void) | null = null;
|
||||
if (!acquisition || !acquisitionKey) {
|
||||
setEntry({ key: null, state: { status: "waiting" } });
|
||||
return;
|
||||
}
|
||||
|
||||
const acquireCurrent = () => {
|
||||
releaseCurrent?.();
|
||||
const retained = attachmentAcquisitionCache.acquireRetained(
|
||||
acquisitionKey,
|
||||
acquisition.locate,
|
||||
);
|
||||
releaseCurrent = retained.release;
|
||||
return retained;
|
||||
};
|
||||
const initial = acquireCurrent();
|
||||
if (initial.value) {
|
||||
setEntry({ key: acquisitionKey, state: { status: "loaded", attachment: initial.value } });
|
||||
return () => {
|
||||
disposed = true;
|
||||
releaseCurrent?.();
|
||||
};
|
||||
}
|
||||
|
||||
setEntry({ key: acquisitionKey, state: { status: "loading" } });
|
||||
void (async () => {
|
||||
let firstAttempt: ReturnType<typeof acquireCurrent> | null = initial;
|
||||
try {
|
||||
const attachment = await runAssistantImageOperationWithRetry({
|
||||
operation: async () => {
|
||||
const retained = firstAttempt ?? acquireCurrent();
|
||||
firstAttempt = null;
|
||||
return await retained.promise;
|
||||
},
|
||||
shouldStop: () => disposed,
|
||||
});
|
||||
if (!disposed) {
|
||||
setEntry({ key: acquisitionKey, state: { status: "loaded", attachment } });
|
||||
}
|
||||
} catch (error) {
|
||||
if (!disposed) {
|
||||
setEntry({ key: acquisitionKey, state: { status: "failed", error } });
|
||||
}
|
||||
}
|
||||
})();
|
||||
return () => {
|
||||
disposed = true;
|
||||
releaseCurrent?.();
|
||||
};
|
||||
}, [acquisition, acquisitionKey]);
|
||||
|
||||
if (!acquisitionKey) {
|
||||
return { status: "waiting" };
|
||||
}
|
||||
const cached = attachmentAcquisitionCache.peek(acquisitionKey);
|
||||
if (cached) {
|
||||
return { status: "loaded", attachment: cached };
|
||||
}
|
||||
return entry.key === acquisitionKey ? entry.state : { status: "waiting" };
|
||||
}
|
||||
|
||||
function createDataImageAcquisition(input: {
|
||||
source: string;
|
||||
dataImage: DataImage | null;
|
||||
}): AssistantImageAcquisition | null {
|
||||
if (!input.dataImage) {
|
||||
return null;
|
||||
}
|
||||
const { dataImage, source } = input;
|
||||
return {
|
||||
key: dataImage.cacheKey,
|
||||
locate: async () =>
|
||||
await persistAttachmentFromDataUrl({
|
||||
id: createPreviewAttachmentId({
|
||||
mimeType: dataImage.mimeType,
|
||||
contentLength: dataImage.base64.length,
|
||||
contentKey: dataImage.cacheKey,
|
||||
}),
|
||||
dataUrl: source,
|
||||
mimeType: dataImage.mimeType,
|
||||
}),
|
||||
};
|
||||
}
|
||||
|
||||
function usePreviewUrl(attachment: AttachmentMetadata | null | undefined): PreviewUrlState {
|
||||
const id = attachment?.id;
|
||||
const storageType = attachment?.storageType;
|
||||
const storageKey = attachment?.storageKey;
|
||||
const mimeType = attachment?.mimeType;
|
||||
const previewKey =
|
||||
id && storageType && storageKey && mimeType
|
||||
? `${id}:${storageType}:${storageKey}:${mimeType}`
|
||||
: null;
|
||||
const [entry, setEntry] = useState<{ key: string | null; state: PreviewUrlState }>(() => {
|
||||
return {
|
||||
key: previewKey,
|
||||
state: { status: "waiting" },
|
||||
};
|
||||
});
|
||||
const getCurrentAttachment = useStableEvent(() => attachment ?? null);
|
||||
|
||||
useLayoutEffect(() => {
|
||||
let disposed = false;
|
||||
let releaseCurrent: (() => void) | null = null;
|
||||
const current = getCurrentAttachment();
|
||||
|
||||
if (!current || !previewKey) {
|
||||
setEntry({ key: null, state: { status: "waiting" } });
|
||||
return;
|
||||
}
|
||||
|
||||
const acquireCurrent = () => {
|
||||
releaseCurrent?.();
|
||||
const retained = previewUrlCache.acquireRetained(previewKey, async () => ({
|
||||
attachment: current,
|
||||
uri: await resolveAttachmentPreviewUrl(current),
|
||||
}));
|
||||
releaseCurrent = retained.release;
|
||||
return retained;
|
||||
};
|
||||
const initial = acquireCurrent();
|
||||
if (initial.value) {
|
||||
setEntry({ key: previewKey, state: { status: "loaded", uri: initial.value.uri } });
|
||||
return () => {
|
||||
disposed = true;
|
||||
releaseCurrent?.();
|
||||
};
|
||||
}
|
||||
|
||||
setEntry({ key: previewKey, state: { status: "loading" } });
|
||||
void (async () => {
|
||||
let firstAttempt: ReturnType<typeof acquireCurrent> | null = initial;
|
||||
try {
|
||||
const preview = await runAssistantImageOperationWithRetry({
|
||||
operation: async () => {
|
||||
const retained = firstAttempt ?? acquireCurrent();
|
||||
firstAttempt = null;
|
||||
return await retained.promise;
|
||||
},
|
||||
shouldStop: () => disposed,
|
||||
});
|
||||
if (!disposed) {
|
||||
setEntry({ key: previewKey, state: { status: "loaded", uri: preview.uri } });
|
||||
}
|
||||
} catch (error) {
|
||||
if (!disposed) {
|
||||
setEntry({ key: previewKey, state: { status: "failed", error } });
|
||||
}
|
||||
}
|
||||
})();
|
||||
|
||||
return () => {
|
||||
disposed = true;
|
||||
releaseCurrent?.();
|
||||
};
|
||||
}, [getCurrentAttachment, previewKey]);
|
||||
|
||||
if (!previewKey) {
|
||||
return { status: "waiting" };
|
||||
}
|
||||
return entry.key === previewKey ? entry.state : { status: "waiting" };
|
||||
}
|
||||
|
||||
function lifecycleReducer(
|
||||
state: AssistantImageLifecycle,
|
||||
event: AssistantImageLifecycleEvent,
|
||||
): AssistantImageLifecycle {
|
||||
return transitionAssistantImageLifecycle(state, event);
|
||||
}
|
||||
|
||||
function errorMessage(error: unknown, fallback: string): string {
|
||||
return error instanceof Error ? error.message : fallback;
|
||||
}
|
||||
|
||||
function getAcquisitionFailure(input: {
|
||||
hasResolution: boolean;
|
||||
isFileSource: boolean;
|
||||
isDataImage: boolean;
|
||||
fileAttachment: AttachmentAcquisitionState;
|
||||
dataImageAttachment: AttachmentAcquisitionState;
|
||||
preview: PreviewUrlState;
|
||||
hasDirectUri: boolean;
|
||||
fallbackMessage: string;
|
||||
}): AssistantImageResult | null {
|
||||
if (!input.hasResolution) {
|
||||
return { status: "failed", message: input.fallbackMessage };
|
||||
}
|
||||
if (input.isFileSource && input.fileAttachment.status === "failed") {
|
||||
return {
|
||||
status: "failed",
|
||||
message: errorMessage(input.fileAttachment.error, input.fallbackMessage),
|
||||
};
|
||||
}
|
||||
if (input.isDataImage && input.dataImageAttachment.status === "failed") {
|
||||
return {
|
||||
status: "failed",
|
||||
message: errorMessage(input.dataImageAttachment.error, input.fallbackMessage),
|
||||
};
|
||||
}
|
||||
if (!input.hasDirectUri && input.preview.status === "failed") {
|
||||
return {
|
||||
status: "failed",
|
||||
message: errorMessage(input.preview.error, input.fallbackMessage),
|
||||
};
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
export function useAssistantImage({
|
||||
source,
|
||||
occurrenceKey,
|
||||
client,
|
||||
workspaceRoot,
|
||||
serverId,
|
||||
}: UseAssistantImageInput): AssistantImageResult {
|
||||
const { t } = useTranslation();
|
||||
const resolution = useMemo(
|
||||
() => resolveAssistantImageSource({ source, workspaceRoot }),
|
||||
[source, workspaceRoot],
|
||||
);
|
||||
const dataImage = useMemo(() => parseImageDataUrl(source), [source]);
|
||||
const fileAcquisition = useMemo(() => {
|
||||
const port: AssistantImageFileAcquisitionPort | null = client
|
||||
? {
|
||||
readFile: async (cwd, path) => await client.readFile(cwd, path),
|
||||
persist: persistAttachmentFromBytes,
|
||||
}
|
||||
: null;
|
||||
return createAssistantImageFileAcquisition({
|
||||
port,
|
||||
resolution,
|
||||
serverId,
|
||||
occurrenceKey,
|
||||
unavailableMessage: t("message.attachments.imagePreviewUnavailable"),
|
||||
});
|
||||
}, [client, occurrenceKey, resolution, serverId, t]);
|
||||
const dataImageAcquisition = useMemo(
|
||||
() => createDataImageAcquisition({ source, dataImage }),
|
||||
[dataImage, source],
|
||||
);
|
||||
const fileAttachment = useAttachmentAcquisition(fileAcquisition);
|
||||
const dataImageAttachment = useAttachmentAcquisition(dataImageAcquisition);
|
||||
const filePreview = usePreviewUrl(
|
||||
fileAttachment.status === "loaded" ? fileAttachment.attachment : null,
|
||||
);
|
||||
const dataImagePreview = usePreviewUrl(
|
||||
dataImageAttachment.status === "loaded" ? dataImageAttachment.attachment : null,
|
||||
);
|
||||
const directUri = resolution?.kind === "direct" && !dataImage ? resolution.uri : null;
|
||||
const preview = dataImage ? dataImagePreview : filePreview;
|
||||
const previewUri = preview.status === "loaded" ? preview.uri : null;
|
||||
const uri = directUri ?? previewUri;
|
||||
const cachedMetadata = useMemo(
|
||||
() => getAssistantImageMetadata({ source, workspaceRoot, serverId }),
|
||||
[serverId, source, workspaceRoot],
|
||||
);
|
||||
const [lifecycle, dispatchLifecycle] = useReducer(
|
||||
lifecycleReducer,
|
||||
uri,
|
||||
(initialUri): AssistantImageLifecycle => {
|
||||
if (initialUri) {
|
||||
const aspectRatio = getLoadedImageAspectRatio(initialUri);
|
||||
if (aspectRatio !== null) {
|
||||
return { status: "loaded", uri: initialUri, aspectRatio };
|
||||
}
|
||||
}
|
||||
return createAssistantImageLifecycle();
|
||||
},
|
||||
);
|
||||
const dispatch = useCallback((event: AssistantImageLifecycleEvent) => {
|
||||
if (event.type === "image_loaded") {
|
||||
rememberLoadedImage(event.uri, event.aspectRatio);
|
||||
} else if (event.type === "failed" && event.uri) {
|
||||
loadedImageCache.delete(event.uri);
|
||||
}
|
||||
dispatchLifecycle(event);
|
||||
}, []);
|
||||
const renderedImageRef = useRef<unknown>(null);
|
||||
const handleImageRef = useCallback((instance: unknown) => {
|
||||
renderedImageRef.current = instance;
|
||||
}, []);
|
||||
|
||||
useEffect(() => {
|
||||
if (!uri) {
|
||||
dispatch({ type: "preview_released" });
|
||||
return;
|
||||
}
|
||||
|
||||
dispatch({
|
||||
type: "preview_created",
|
||||
uri,
|
||||
aspectRatio: cachedMetadata?.aspectRatio ?? null,
|
||||
});
|
||||
}, [cachedMetadata, dispatch, uri]);
|
||||
|
||||
const handleImageError = useCallback(() => {
|
||||
if (uri) {
|
||||
dispatch({
|
||||
type: "failed",
|
||||
uri,
|
||||
message: t("message.attachments.imageUnavailable"),
|
||||
});
|
||||
}
|
||||
}, [dispatch, t, uri]);
|
||||
const handleImageLoad = useCallback(
|
||||
(event: ImageLoadEvent) => {
|
||||
if (!uri) {
|
||||
return;
|
||||
}
|
||||
const nativeEvent = event.nativeEvent as ImageLoadEvent["nativeEvent"] & {
|
||||
target?: { naturalWidth?: unknown; naturalHeight?: unknown };
|
||||
};
|
||||
const dimensions = resolveAssistantImageLoadDimensions({
|
||||
source: nativeEvent.source,
|
||||
target: nativeEvent.target,
|
||||
renderedImage: renderedImageRef.current,
|
||||
renderedDimensions,
|
||||
});
|
||||
const metadata = dimensions
|
||||
? setAssistantImageMetadata({ source, workspaceRoot, serverId }, dimensions)
|
||||
: null;
|
||||
const aspectRatio = metadata?.aspectRatio ?? cachedMetadata?.aspectRatio ?? null;
|
||||
if (!aspectRatio) {
|
||||
dispatch({
|
||||
type: "failed",
|
||||
uri,
|
||||
message: t("message.attachments.imageUnavailable"),
|
||||
});
|
||||
return;
|
||||
}
|
||||
dispatch({ type: "image_loaded", uri, aspectRatio });
|
||||
},
|
||||
[cachedMetadata, dispatch, serverId, source, t, uri, workspaceRoot],
|
||||
);
|
||||
|
||||
const acquisitionFailure = getAcquisitionFailure({
|
||||
hasResolution: resolution !== null,
|
||||
isFileSource: resolution?.kind === "file_rpc",
|
||||
isDataImage: dataImage !== null,
|
||||
fileAttachment,
|
||||
dataImageAttachment,
|
||||
preview,
|
||||
hasDirectUri: directUri !== null,
|
||||
fallbackMessage: t("message.attachments.imagePreviewLoadFailed"),
|
||||
});
|
||||
if (acquisitionFailure) {
|
||||
return acquisitionFailure;
|
||||
}
|
||||
const hasCurrentLifecycleUri = lifecycle.status !== "failed" && lifecycle.uri === uri;
|
||||
let binding: AssistantImageRenderBinding | null = null;
|
||||
if (hasCurrentLifecycleUri && lifecycle.uri) {
|
||||
binding = {
|
||||
uri: lifecycle.uri,
|
||||
onRef: handleImageRef,
|
||||
onLoad: handleImageLoad,
|
||||
onError: handleImageError,
|
||||
};
|
||||
}
|
||||
if (lifecycle.status === "loaded" && lifecycle.uri === uri) {
|
||||
return {
|
||||
status: "loaded",
|
||||
binding: {
|
||||
uri: lifecycle.uri,
|
||||
onRef: handleImageRef,
|
||||
onLoad: handleImageLoad,
|
||||
onError: handleImageError,
|
||||
},
|
||||
aspectRatio: lifecycle.aspectRatio,
|
||||
};
|
||||
}
|
||||
if (lifecycle.status === "failed") {
|
||||
return lifecycle;
|
||||
}
|
||||
return {
|
||||
status: "loading",
|
||||
binding,
|
||||
aspectRatio: hasCurrentLifecycleUri ? lifecycle.aspectRatio : null,
|
||||
};
|
||||
}
|
||||
Reference in New Issue
Block a user