Keep agent timelines stable from submission through resume (#2718)

* Reapply: Stop completed turns from appearing stuck (#2484)

* Reapply: Keep older chat history and image previews stable (#2490)

* Lock timeline layout stability with user-driven specs

Two reds reproduce the reported jank:
- new agent, first prompt: the row paints at top=100 then jumps to 144 through
  authoritative hydration, because the draft tab and the agent panel each render
  their own AgentStreamView for the same logical row.
- long loaded timeline, submit: scrollHeight collapses 10793 -> 9111 mid-turn,
  because splitOrderedTail reclassifies measured rows into the virtualized segment
  where the virtualizer falls back to estimateStreamItemHeight.

Also restores centered timeline rows: the pagination anchor wrapper introduced in
#2490 was a plain block div, which broke alignSelf: center.

* Measure the turn footer blink from when the footer appears

Bounding the window at the prompt row conflated two things: the footer arriving a
frame after the row, and the footer leaving the layout and coming back. Bounded from
the footer's first frame, the blink still reproduces, so the footer-ownership race is
real rather than a sampling artifact.

* Rename the shadowed prompt row binding in the frame recorder

* fix(desktop): allow concurrent worktree dev instances

Desktop terminals can inherit another instance's profile path and debugging flags. Scope the profile to the current dev root and select an available debugging port so instances do not share global runtime state.

* fix(desktop): expose development providers in dev builds

Desktop-managed daemons inherited no Paseo runtime mode, so dev builds constructed the production provider registry. Mark unpackaged internal launches as development while continuing to force packaged launches to production.

* fix(timeline): stabilize optimistic turn layout

Keep initial history loading dormant until user intent and preserve active-turn layout across delayed transport updates. Add painted-frame coverage for optimistic timeline transitions.

* fix(timeline): preserve optimistic prompts through hydration

Canonical replacement relied on a transport submission record that can settle before a provider echoes the user row. Keep presentation continuity owned by the unreconciled row while destructive history replacement remains governed by active transactions.

* fix(timeline): make submitted prompts canonical

Submission settlement previously depended on status updates that could arrive after the optimistic row. Persist every accepted prompt in the canonical timeline and keep the footer active until RPC and canonical acknowledgement converge.

* fix(timeline): stop older history pages repeating entries

Backward pagination counted source sequence rows, so page boundaries inside a merged projected item could return that item repeatedly. Count projected entries and advance from their display anchors instead, with server and browser regressions covering the contract.

* fix(timeline): resume agents from the latest history

Use a bounded tail probe on resume so long background runs appear atomically instead of replaying every missed page. Preserve unchanged views, reconcile local/live rows, and keep skipped history reachable through backward pagination.

* fix(timeline): reconcile turn state after missed updates

Turn completion events can be missed while an agent timeline is unsubscribed, leaving footer activity stale even after the directory reports idle. Reconcile the single-owned turn lifecycle from authoritative resume snapshots while preserving newer live evidence.

* test(timeline): cover running state across refresh

Protect the hydration boundary where directory state must restore consistent tab, composer, and footer activity. Expose the running indicator semantically so the regression checks user-visible state instead of spinner implementation details.

* fix(timeline): keep history loading geometry stable

Reserve a fixed history-start gutter regardless of pagination state so showing or hiding the loader cannot shift timeline content.

* fix(timeline): keep turn activity coherent

Derive footer, tab, and composer activity from one lifecycle and submission projection. Idle directory transitions now request an authoritative resume so missed terminal events converge without treating directory delivery order as turn authority.

* fix(tests): align checks with runtime boundaries

* fix(desktop): stop dev app surviving its terminal

Electron was detached from the desktop runner, allowing it to outlive a stopped workspace terminal with broken output pipes and retain the single-instance lock. Keep Electron owned by the runner and preserve explicit process-group cleanup for detached child trees.

* fix(timeline): preserve activity through turn handoff

Correlate lifecycle events and snapshots with daemon-owned turn identity, and keep optimistic submissions active until their canonical row arrives. This prevents stale updates and RPC timing from exposing a false idle state.

* refactor(timeline): remove submission timing state

* fix(timeline): restore prompt identity on resume

* fix(timeline): exclude attachments from canonical prompt text

Structured provider prompts can include hidden history attachments before the user's text. Persist only plain prompt blocks so authoritative user messages match what the composer displayed while provider input remains unchanged.

* refactor(agent): remove unused pending-run message identity

* fix(desktop): let Chromium allocate the debugging port

* refactor(agent): remove impossible finalized-turn tracking

* fix(timeline): load history after keyboard scrolling

Treat observed upward displacement as user intent so keyboard and scrollbar navigation can wake dormant pagination.

* test(server): preserve accepted turn stream order

The manager-owned accepted turn boundary intentionally precedes coalesced output. Assert that lifecycle event explicitly while retaining the timeline-before-terminal ordering contract.

* test(app): scope sidebar fixtures by identity

Shared E2E daemon state may contain unrelated projects. Keep stale-route coverage tied to its named missing project and drag only the two rows seeded by the reorder spec.

* fix(timeline): gate prompt revisions by client capability

* fix(timeline): finalize canonical prompt delivery

Treat provider identity as a single stored-row transition so legacy clients receive one deliverable canonical prompt. Release identity-pending rows before later foreground output or the current terminal while preserving stale-turn isolation.

* fix(server): preserve legacy prompt pagination

* fix(timeline): keep assistant block identity stable

Assign canonical block IDs during live promotion so terminal finalization cannot remount image-bearing rows.

* fix(timeline): apply covered revisions during resume append

Run the exact canonical revision pass before overlapping resume pages append newer rows, while leaving cursor advancement to the incremental path.

* fix(server): preserve queued agent transitions

Project each queued live snapshot before awaiting stored metadata so later lifecycle state cannot overwrite an earlier transition.

* fix(server): snapshot queued agent updates

Capture each live agent payload before it enters the per-agent queue so shared nested state cannot change while an earlier storage read is pending.

* docs(app): date agent turn compatibility cleanup

* refactor(timeline): mint submitted prompt identity at acceptance

Canonical prompt rows now keep one immutable wire identity while provider identity remains internal for rewind translation. This removes the unreleased same-sequence revision and legacy projection contract.

* refactor(app): remove prompt revision reconciliation

Submitted prompt rows are immutable on the wire, so resume and live reducers no longer need to apply already-covered same-sequence updates.

* test(app): cover submitted prompt rewind roundtrip

* docs(timeline): describe immutable submitted prompt identity

Provider echoes now enrich daemon-internal identity only; wire rows keep their acceptance-time identity and rewind performs the provider translation.

* test(server): validate mock rewind targets

Reject unknown provider message IDs so app rewind coverage exercises wire-to-provider identity translation.

* fix(server): validate rewind before interrupting

* fix(server): omit rewind identity for daemon prompts

* test(app): guard rewind eligibility and rejection

* fix(server): preserve prompt identity across hydration

* refactor(timeline): remove stale revision remnants

* Revert "fix(server): preserve prompt identity across hydration"

This reverts commit 6adcf7e5b0bc8601761af59b461a746cb5e395fa.

* fix(timeline): settle missed submission acknowledgements
This commit is contained in:
Mohamed Boudra
2026-08-01 14:13:47 +02:00
committed by GitHub
parent 8b87299ce1
commit 8021d2b6c9
161 changed files with 13757 additions and 2523 deletions
+9 -1
View File
@@ -229,6 +229,11 @@ New session RPCs use dotted names with `.request` and `.response` suffixes, such
- `agent_permission_request` / `agent_permission_resolved` — Tool-call permission flow
- `agent_deleted`, `agent_archived`, `agent_status`, `agent_list`
- `checkout_status_update`, `checkout_diff_update`, and the full `checkout_*` request/response set for git operations
Agent snapshots optionally carry the daemon-owned active turn identity, and turn lifecycle stream events
optionally carry the same `turnId`. New clients use these fields when present and normalize an old daemon's
status once at the directory boundary rather than maintaining a second activity model.
- Terminal subscribe/input/capture commands
- Voice/dictation streaming events (`dictation_stream_*`, `assistant_chunk`, `audio_output`, `transcription_result`)
- Request/response pairs for fetch, list, create, etc., correlated by `requestId`; failures use `rpc_error`
@@ -294,7 +299,10 @@ initializing → idle ⇄ running
`ManagedAgent` is a discriminated union over those lifecycle tags. Notes:
- **AgentManager** is the source of truth for agent state and broadcasts updates to all subscribers
- Timeline is append-only with epochs (each run starts a new epoch). Storage uses sequence numbers for client-side dedup; the default fetch page is 200 items
- Timeline sequence allocation is append-only with epochs (each run starts a new epoch). The one
permitted in-place enrichment adds a provider message id to the manager-owned row for an accepted
prompt; it preserves the row's sequence, content, and timestamp. Storage uses sequence numbers for
client-side dedup; the default fetch page is 200 items.
- Timeline row `timestamp` values are canonical daemon-owned timestamps. Providers may supply original replay timestamps, but clients must not guess timestamp trust or hide time UI based on local clock heuristics.
- Events stream to connected clients in real time; correctness is backed by authoritative timeline fetches and paged-to-completion catch-up.
- Agent state persists to `$PASEO_HOME/agents/{cwd-with-dashes}/{agent-id}.json` (timeline rows live alongside the record). That storage path is derived from `cwd`, not from workspace id.
+22 -6
View File
@@ -19,6 +19,10 @@ Root checkout dev is intentionally split across terminals:
- `npm run dev:app` runs Expo on `http://localhost:8081` and connects to the dev daemon.
- `npm run dev:desktop` runs its own Electron-flavored Expo server on the first free port from `8082` through `8089`. It never claims port `8081`.
Desktop dev launches its desktop-managed daemon with `PASEO_NODE_ENV=development`,
so development-only providers such as Mock Load Test are available. Packaged
desktop launches always force the daemon to production mode.
`npm run dev` is only a shorthand for `npm run dev:server`. Keep `127.0.0.1:6767` for the packaged app and production-style `~/.paseo` state.
## Nix desktop package
@@ -111,10 +115,21 @@ The iOS simulator shares the Mac's loopback, so `localhost:<port>` reaches the h
### Desktop renderer profiling
`npm run dev:desktop` starts Electron with Chromium remote debugging enabled on
`http://127.0.0.1:9223` so renderer CPU profiles can be captured through CDP.
It launches its own Electron-flavored Expo server and passes that URL to Electron.
Override the CDP port with `PASEO_ELECTRON_REMOTE_DEBUGGING_PORT` when `9223` is busy.
`npm run dev:desktop` starts Electron with Chromium remote debugging enabled so
renderer CPU profiles can be captured through CDP. By default it passes
`--remote-debugging-port=0`, so Chromium atomically asks the OS for an available
port and prints the selected DevTools endpoint. Set
`PASEO_ELECTRON_REMOTE_DEBUGGING_PORT` when a QA workflow requires a validated,
fixed port.
Desktop dev also scopes Electron `userData` to the current dev root. This prevents
desktop-only environment inherited by terminals opened inside Paseo from coupling
a new worktree instance to the parent desktop instance's profile or single-instance
lock.
Electron remains in the desktop dev runner's process group. Closing or stopping the
workspace-script terminal must terminate Electron with Metro; detaching Electron
leaves an orphan holding the worktree's single-instance lock and broken output pipes.
With desktop dev running, verify the real BrowserWindow, titlebar clearance, fullscreen
transition, and 751-pixel settings split with:
@@ -124,8 +139,9 @@ npm run verify:electron-cdp --workspace=@getpaseo/desktop
```
The verifier reads the same `EXPO_PORT` and
`PASEO_ELECTRON_REMOTE_DEBUGGING_PORT` environment names as desktop dev. Set both when
testing an isolated instance on non-default ports.
`PASEO_ELECTRON_REMOTE_DEBUGGING_PORT` environment names as desktop dev. Set an
explicit remote-debugging port for verifier runs, and set both when testing an
isolated instance on non-default ports.
When running a dedicated Electron QA instance against a non-default Expo port, set
`EXPO_DEV_URL` explicitly. Desktop main defaults to `http://localhost:8081`, so
+4 -2
View File
@@ -42,9 +42,11 @@ OpenCode MCP injection is dynamic and session-scoped. Call OpenCode's `mcp.add`
OpenCode owns user message IDs. Do not pass Paseo-generated IDs to OpenCode prompt APIs; let OpenCode create `msg*` IDs and record the user timeline item from the `message.updated` event.
Every provider adapter owns its canonical user-message timeline rows. When a foreground prompt is accepted, the adapter must emit exactly one `user_message` timeline item for that submitted prompt, using the same message ID it gives to or receives from the provider runtime. Optimistic client messages are UI-only and provider transcript echoes are optional; neither is allowed to be the only source of truth. If the provider later echoes the same submitted user message, dedupe it only within the active turn. Prefer provider-visible message IDs, but ACP runtimes may omit that ID or replace it with a provider-owned one; in that case suppress only echo chunks whose accumulated text is a prefix of the active submitted prompt. Do not perform global transcript text dedupe.
`AgentManager` owns the one canonical timeline row for a foreground prompt carrying a Paseo `clientMessageId`. It records that row when `startTurn` accepts, with the wire `messageId` set to the same value. Provider adapters still emit their native user-message echo with the same `clientMessageId` when available; the manager records its provider identity on the internal row without changing or redispatching the wire item. If an adapter emits the echo before `startTurn` resolves, the manager records the provider identity with the row at acceptance. Provider adapters continue to own externally initiated user rows that have no Paseo client identity. Do not perform global transcript text dedupe.
Submitted user-message rows preserve both identities: `messageId` is the provider-visible ID and the optional `clientMessageId` is the Paseo ID from `AgentRunOptions`. Attach `clientMessageId` only to the canonical row for that foreground submission; provider history and externally initiated user rows do not have a Paseo client ID.
Rewind accepts the canonical wire `messageId` and resolves it to the provider identity before calling the adapter. A submitted prompt cannot be rewound until its provider echo supplies that identity.
Submitted user-message wire items carry the same Paseo ID in `messageId` and `clientMessageId`. Provider adapters attach `clientMessageId` only to the echo for that foreground submission; provider history and externally initiated user rows do not have a Paseo client ID.
Draft metadata lookups should avoid creating provider sessions when the upstream provider has top-level APIs for that metadata. Prefer `AgentClient.fetchCatalog`, `listCommands`, or `listFeatures` over creating a scratch `AgentSession`; scratch sessions can show up as empty native sessions in provider import/history UIs. `fetchCatalog` is the single discovery API for models and modes — provider implementations may use one process, separate upstream calls, or static data internally, but callers outside the provider do not get separate runtime model/mode probes. Draft feature and command listing must use the explicit draft model only; if no model is selected yet, return no metadata instead of resolving a default model through catalog discovery.
+116 -14
View File
@@ -5,9 +5,11 @@ Agent chat delivery has two paths:
1. **Live stream** — `agent_stream` WebSocket messages for immediacy. These may be delta-shaped lifecycle updates.
2. **Authoritative history** — `fetch_agent_timeline_request` for correctness. This always returns full projected timeline items, never lifecycle deltas.
The invariant is:
The invariants are:
> If the daemon has committed timeline rows for an agent, any connected client that opens or resumes that agent eventually displays every row through the daemon's current tail.
> A continuously subscribed client applies every committed row in order. Opening or resuming an
> agent establishes the daemon's current tail in one bounded request, with older history reachable
> through backward pagination.
Tool output is bounded before it enters either delivery path. Canonical shell tool output is sliced
to 64 KiB, and the same bounded item is used for durable timeline rows and live stream events.
@@ -25,17 +27,26 @@ Client heartbeat reports presence:
Heartbeat is used for notification routing. It must not be used as a correctness gate for `agent_stream` delivery. A stale mobile focus heartbeat may affect whether the user gets notified; it must not make timeline rows disappear from the live stream.
## Catch-up is paged but complete
## Gap recovery is paged but complete
Large unbounded timeline responses can exceed relay frame limits, so catch-up uses bounded pages. Bounded does not mean partial.
Page limits are projected-item targets. A tool call lifecycle is one projected item even if it spans many source sequence numbers, and assistant/reasoning chunks are merged before counting. The response carries `seqStart`, `seqEnd`, `sourceSeqRanges`, and `collapsed` so clients can advance sequence cursors without rendering delta rows.
When the app fetches `direction: "after"` and the daemon responds with `hasNewer: true`, the app must immediately fetch the next page from `endCursor`. The catch-up is complete only when `hasNewer: false`.
When live delivery detects a sequence gap, the app fetches `direction: "after"`. If the daemon
responds with `hasNewer: true`, the app immediately fetches the next page from `endCursor`. Gap
recovery is complete only when `hasNewer: false`.
Initialization timeouts guard lack of catch-up progress, not the full multi-page sync. A successful page that queues the next `after` page refreshes the watchdog.
The first load of an agent without a local cursor is different: it fetches a bounded latest tail page. Older history remains user-driven by scrolling upward.
Opening or resuming an agent fetches one bounded latest tail page. Older history remains
user-driven by scrolling upward.
Reaching the history-start threshold loads one older page and preserves the visible content anchor.
Cursor progress does not trigger another page. The user must leave and return to the threshold unless
the anchored page still leaves the viewport at history start, as with short or compacted content; in
that case pagination continues as one loading operation until the page fills the viewport or history
is exhausted.
## Durable item anchors
@@ -47,9 +58,21 @@ The daemon validates that the epoch is current and the exact source sequence sti
## Resume behavior
When a client resumes with a known cursor, it catches up after that cursor to completion. It does not replace the view with a latest tail page, because tail pagination can skip the middle of a long background run.
Opening, reconnecting, or revisiting after selective-delivery grace fetches the latest tail page.
Focus alone does not mutate timeline state; the tail response is compared with the local
authoritative range first.
When a client resumes without a cursor, it fetches the latest tail page.
- The same epoch and `window.maxSeq` is an exact display no-op. The app advances synchronization
bookkeeping without replacing timeline arrays, preserving an upward-scrolled viewport.
- When the page overlaps or is adjacent to the local end cursor, only projected items newer than
that cursor are applied. Already-covered rows are not replayed.
- A true middle gap, epoch change, or rewind atomically replaces stale canonical history with the
latest tail. The replacement reconciles positioned live rows beyond its coverage and unresolved
local submissions; it never retains two discontiguous canonical ranges.
The installed tail carries `hasOlder`, so history skipped by a replacement remains reachable through
ordinary backward pagination. A backward page is accepted only when it is adjacent to the current
history start; a response requested from a pre-replacement range is stale and is discarded.
## Client replica lifetime
@@ -61,6 +84,24 @@ recomposition while the runtime still owns the same directory snapshot and timel
Removing the host from the registry is the destructive boundary: it stops the runtime and clears the
session and host-scoped setup state together.
The durable replica cache is a display cache, not a synchronization checkpoint. Its timeline record
contains only the focused `agentId` and a truncated item tail. It never persists a cursor, epoch,
older-history availability, authority status, or sync generation because those facts would describe
the complete source dataset rather than the truncated display dataset.
Restoring that cache produces a painted timeline: the items may render immediately, but the first
daemon timeline request is still `tail`. A successful tail response atomically establishes canonical
items, range, and older-history availability. Live rows received between cache paint and that tail
response stay in the separate live head, do not advance a cursor or trigger gap recovery, and are
reconciled with the authoritative tail and subsequent catch-up.
Every daemon-derived live item carries its timeline epoch and sequence position. Bootstrap
replacement keeps only positioned rows newer than the page it installs, while unresolved local
user presentations identify themselves by having client identity without a timeline position. This
prevents a page from duplicating rows it already covers without coupling display continuity to the
shorter-lived submission registry. Unreconciled local presentations are not persisted in the durable
replica cache.
## Selective and legacy delivery
The app chooses one delivery policy from `server_info.features.selectiveAgentTimeline`:
@@ -87,14 +128,75 @@ its completion advances `seqEnd`, followed by a merged assistant message. The ap
remaining page through the existing stream reducer. It must not append full projected text to a
live prefix.
Optimistic user prompts occupy stable timeline slots. Catch-up never extracts, delays, or reinserts
them. A canonical user row replaces its matching slot in place; an unmatched prompt stays exactly
where the user submitted it. Other canonical rows are applied after the already-present timeline
instead of relocating visible user messages around newly fetched history.
Every path that sends a message to an agent — composer send, dictation accept-and-send, queued
send-now, and the automatic queue drain in `HostRuntime` — goes through
`dispatchComposerAgentMessage` with a submission writer. There is no second transport for the same
product action: calling `client.sendAgentMessage` directly skips the submitted row and the pending
footer, and permanently drops attachments because the daemon does not echo them back.
Canonical submitted user rows carry the provider's `messageId` and Paseo's optional
`clientMessageId`. Clients reconcile optimistic prompts by `clientMessageId`. Content matching is
limited to the dated compatibility path for daemon timelines created before that field existed.
A submitted prompt is one `UserMessageItem` row. That row is the authoritative local presentation:
its stable identity, text, timestamp, images, and attachments do not change when the provider
acknowledges it. Submission lifecycle is a separate record keyed by agent, not another row shape.
The transaction registry records two independent settlement facts: canonical acknowledgement and RPC
settlement. Canonical acknowledgement retires optimistic activity immediately. When both facts are
known, whichever arrives second deletes the record. A canonical acknowledgement that arrives first
prevents a later transport error from rolling back a prompt already observed.
The daemon's accepted response waits for the correlated run start and guarantees that the canonical
submitted row has been recorded. It publishes the accepted turn's liveness before that row, so the
client applies authoritative activity before canonical acknowledgement retires optimistic activity.
Timeline render batching does not delay lifecycle application. Directory status never settles a
submission. Overlapping sends settle independently rather than collapsing to one newest pending
message.
Daemons advertising `server_info.features.canonicalSubmittedPrompts` guarantee that every accepted
prompt carrying a client message id is recorded and streamed as a canonical `user_message` with that
same id. This includes daemon-handled commands that do not allocate a foreground turn; their submitted
row is recorded before handler output. The app tracks submission transactions only for hosts with this
capability. Older hosts keep the shipped untracked optimistic-row behavior and roll that row back on RPC
rejection.
Turn activity has one client-side replica. Lifecycle events can attach `turnId`, and agent snapshots can
expose `activeTurn: { turnId, startedAt } | null`. An identified terminal cannot close a different
identified turn; an unnamed legacy terminal can close the current turn. Snapshots, stream events,
cancellation requests, and every visible surface update the same per-agent record. The snapshot covers
both user-started foreground turns and autonomous provider turns;
foreground control ownership remains a separate daemon concern. Cancellation request identity is stored
with that record rather than in a React component, so an old request cannot clear a newer one. Submissions
remain a separate pre-turn registry and retire on canonical acknowledgement.
The compatibility boundary for older daemons is snapshot normalization: running/idle status becomes an
anonymous active turn or idle state once, and downstream code consumes the same activity shape. The app
does not combine anonymous lifecycle events, timestamps, timeline rows, and resume coverage to infer a
second running state. Disconnect and replica removal remain destructive close boundaries. Elapsed time
comes only from turn liveness, never from submission records or whichever timeline rows happen to be
mounted.
The daemon records one canonical submitted user row at acceptance. Its wire `messageId` is the
submission's `clientMessageId`, so the row is born with its final identity and remains immutable on
the wire. A correlated provider echo records the provider's native identity internally without
dispatching another timeline event. Rewind resolves the wire identity to that provider identity at
the provider boundary. Daemon-handled prompts follow the same identity rule.
Content matching is limited to the dated compatibility path for daemon timelines created before
that field existed. Canonical ingestion may match only an explicit unreconciled local candidate;
the draft-create handoff is the one boundary that also permits the legacy canonical twin to have
arrived first. Generic reducers and consumers do not reimplement message identity matching.
Ordinary bootstrap, same-epoch reset, and catch-up replacement preserve unmatched locally submitted
rows because a provider may never echo them. A known epoch change or rewind replaces history and
drops acknowledged local rows omitted by the new canonical epoch; every transaction not yet
acknowledged by the provider, and no other local row, crosses that destructive boundary. A cold
reset without an existing epoch is destructive because the client has no continuity anchor.
Tail rows are positioned history, so an unmatched local presentation is appended after the
canonical replacement rather than ordered by timestamps from different machines. The head is a
live overlay: cursorless items stay there during continuity replacement until canonical positions
arrive, while a destructive replacement retains only active submission transactions.
Canonical replacement owns both timeline lanes. A matching local row keeps its presentation ID and
payload while taking the canonical row's ordered position. If a live assistant head is the
canonical assistant prefix, it stays in the head lane. No row may be returned in both lanes.
## Relevant code