Mohamed BoudraandGitHub
72b67f48e3
* Eliminate spiky terminal lag under load
Terminal output stuttered during heavy output and when the daemon was
busy. Two compounding causes, both confirmed by measurement:
- Every 256KB of output, the daemon dropped pending frames and re-sent a
full cell-grid snapshot (cloned across IPC, stringified to JSON) — a
10MB build did this ~40 times. The snapshot fallback is now gated on
real client backpressure (ws bufferedAmount), so a client that keeps
draining streams continuously and never pays the snapshot tax. This
also removes the periodic GC hitch the churn caused.
- Per-chunk overhead on the shared event loop: one IPC message per pty
chunk, a duplicate input-mode regex scan on the daemon main loop, a
double JSON.stringify per outbound message, and 16KB string realloc
per chunk. Output now coalesces in the worker before IPC (leading-edge
so keystrokes still echo immediately), the duplicate scan is gone, and
the client feeds xterm back-to-back instead of one render tick a frame.
Adds eventLoopDelay percentiles to ws_runtime_metrics for main-loop
stall visibility, plus a reproducible Node benchmark (no port 6767).
Measured: echo p50 7.7ms to 2.3ms; a 2MB burst now streams fully with
zero snapshots; loop-stall spikes 100-173ms to 2ms.
* Guard trace-level check against partial logger stubs
The isLevelEnabled gate added for the emit() perf fix crashed every
session test that injects a hand-rolled logger stub (10 of them omit
isLevelEnabled). Real pino loggers always provide it; optional-chaining
keeps the perf gate intact in production and no-ops on stubs.
* Make recent-output exit-summary test deterministic
The added test self-exited the child immediately after writing 3000
lines, then asserted the newest line was in the exit summary — but the
summary reads the headless xterm buffer, which parses writes
asynchronously, so a loaded CI runner saw a stale buffer (line-2707 not
line-2999). Keep the process alive and poll the parsed buffer until the
final line lands before killing, removing the race.
* Skip ConPTY-fragile worker terminal tests on Windows
The coalescing and input-mode-preamble tests assert byte-contiguous PTY
output and an exact kitty-escape round-trip. Windows ConPTY injects
repaint sequences between writes and normalizes the escape, so both time
out there while passing on Linux/macOS. Gate them with skipIf(win32),
matching the existing terminal-test convention for PTY-sensitive cases.
* Apply terminal barriers immediately when no writes are pending
A barrier op (snapshot/restore/clear) only needs the sentinel-write gate
to wait out plain writes still parsing in xterm's buffer. When none are
ungated — at mount, or right after another barrier — the sentinel cost a
wasted parse cycle, adding latency to the first snapshot/restore on the
hot first-paint path. Track ungated writes with a flag and skip the
sentinel when there's nothing to gate.
* Fix mobile startup saved-host e2e
* Unskip Windows worker terminal coverage
* Tighten worker coalescing assertion
* Reset terminal ungated writes on unmount
* Fix terminal regressions found in adversarial review
Three issues this PR's terminal changes introduced:
- Worker snapshot could duplicate coalesced output. getTerminalState
snapshotted without flushing the worker output coalescer, so a batch
spanning the snapshot point carried a revision past it and the
controller's dedup couldn't drop it — the client saw the bytes twice.
Flush before snapshotting.
- Relay clients lost backpressure protection. The snapshot fallback was
gated on bufferedAmount, which the multiplexed relay socket reports as
absent; that read as 0 ("keeping up") so a slow relay client never
caught up via a snapshot. Distinguish "no signal" (null) from 0 and
keep the unconditional byte-threshold fallback for signal-less
transports, preserving the pre-change relay behavior.
- Recent-output buffer was no longer a hard cap. A single chunk larger
than the limit was retained whole; slice its tail.
Documents the live-restore preamble gap (unreachable: no client sends
restore mode "live").
* Skip input-mode preamble test on Windows ConPTY
CI confirmed Windows ConPTY normalizes away the kitty keyboard escape
the child writes, so it never reaches the worker's input-mode tracker
and the preamble stays empty — the test times out. ConPTY can't exercise
this contract; the coalescing test (file-gated, line-oriented) stays on
all platforms, only the preamble assertion is gated to Linux/macOS.
83 lines
2.8 KiB
TypeScript
83 lines
2.8 KiB
TypeScript
import type { SubscribeTerminalRequest } from "@getpaseo/protocol/messages";
|
|
import {
|
|
TerminalStreamOpcode,
|
|
encodeTerminalSnapshotPayload,
|
|
encodeTerminalStreamFrame,
|
|
} from "@getpaseo/protocol/binary-frames/index";
|
|
import { renderTerminalSnapshotToAnsi } from "@getpaseo/protocol/terminal-snapshot";
|
|
import type { TerminalStateSnapshot, TerminalStateSnapshotOptions } from "./terminal.js";
|
|
|
|
export const MAX_TERMINAL_OUTPUT_FRAME_BYTES = 256 * 1024;
|
|
|
|
// A client is only forced onto the snapshot catch-up path once its transport is
|
|
// genuinely backed up — measured by the socket's bufferedAmount, not by how much
|
|
// the terminal has produced. Set well above any normal in-flight burst (a
|
|
// keeping-up client drains continuously and never approaches this) but low
|
|
// enough that a stalled socket crosses it long before a large burst finishes.
|
|
export const MAX_CLIENT_BUFFERED_BYTES = 4 * 1024 * 1024;
|
|
|
|
const DEFAULT_VISIBLE_RESTORE_SCROLLBACK_LINES = 200;
|
|
const MAX_VISIBLE_RESTORE_SCROLLBACK_LINES = 500;
|
|
|
|
export type TerminalRestoreOptions = NonNullable<SubscribeTerminalRequest["restore"]>;
|
|
|
|
export type TerminalSubscriptionSnapshotMode = "state" | "ready";
|
|
|
|
export function resolveTerminalSubscriptionSnapshotMode(
|
|
restore: TerminalRestoreOptions | undefined,
|
|
): TerminalSubscriptionSnapshotMode {
|
|
return restore ? "ready" : "state";
|
|
}
|
|
|
|
export function resolveRestoreAfterOutputOverflow(
|
|
restore: TerminalRestoreOptions | undefined,
|
|
): TerminalRestoreOptions | undefined {
|
|
if (restore?.mode === "live") {
|
|
return { mode: "visible-snapshot" };
|
|
}
|
|
return restore;
|
|
}
|
|
|
|
export function resolveTerminalRestoreSnapshotOptions(
|
|
restore: TerminalRestoreOptions,
|
|
): TerminalStateSnapshotOptions | null | undefined {
|
|
if (restore.mode === "live") {
|
|
return null;
|
|
}
|
|
if (restore.mode === "visible-snapshot") {
|
|
return {
|
|
scrollbackLines: resolveVisibleRestoreScrollbackLines(restore.scrollbackLines),
|
|
};
|
|
}
|
|
return undefined;
|
|
}
|
|
|
|
export function encodeLegacyTerminalSnapshotFrame(input: {
|
|
slot: number;
|
|
snapshot: TerminalStateSnapshot;
|
|
}): Uint8Array {
|
|
return encodeTerminalStreamFrame({
|
|
opcode: TerminalStreamOpcode.Snapshot,
|
|
slot: input.slot,
|
|
payload: encodeTerminalSnapshotPayload(input.snapshot.state),
|
|
});
|
|
}
|
|
|
|
export function encodeTerminalRestoreFrame(input: {
|
|
slot: number;
|
|
snapshot: TerminalStateSnapshot;
|
|
}): Uint8Array {
|
|
return encodeTerminalStreamFrame({
|
|
opcode: TerminalStreamOpcode.Restore,
|
|
slot: input.slot,
|
|
payload: renderTerminalSnapshotToAnsi(input.snapshot.state),
|
|
});
|
|
}
|
|
|
|
function resolveVisibleRestoreScrollbackLines(value: number | undefined): number {
|
|
if (typeof value !== "number") {
|
|
return DEFAULT_VISIBLE_RESTORE_SCROLLBACK_LINES;
|
|
}
|
|
return Math.min(Math.max(0, value), MAX_VISIBLE_RESTORE_SCROLLBACK_LINES);
|
|
}
|