Improve mobile terminal input, selection, and resizing (#1607)

* Improve mobile terminal typing and selection * Address mobile terminal review feedback * Remove dead native terminal input branch * Replay bracketed paste in terminal preamble * Stop per-key terminal resize claims * Stabilize terminal resize browser test * Use tracked terminal mode for native paste * Restore terminal toolbar focus handling * Adapt native terminal to active visibility state * Stabilize native mobile terminal interactions * Add native terminal fallback boundary * test(app): expect explicit terminal resize reclaim * fix(app): stabilize terminal buffer coordinates * fix(app): restrict terminal resize ownership * fix(terminal): gate restored input modes * fix(app): recover stalled terminal WebViews * chore(app): remove obsolete terminal harnesses * fix(app): make terminal resize claims explicit * test(app): stabilize native terminal device flows * test(app): track absolute scrollback coordinates * fix(app): isolate terminal WebView streams * fix(terminal): arbitrate size ownership across clients The daemon now distinguishes explicit ownership claims from passive geometry updates. This keeps the active client authoritative while allowing its split, viewport, and keyboard changes to resize the PTY without idle clients stealing it. * test(server): make worker resize assertion portable Query the worker's authoritative terminal snapshot instead of invoking a Unix-only shell command, so the resize regression exercises the same boundary on Windows and Unix.
2026-08-03 19:01:34 +08:00
parent 2948a435ed
commit 04fe30a3e2
119 changed files with 9843 additions and 1032 deletions
@@ -70,6 +70,10 @@ definition, no longer eligible to begin.

 - Callers request semantic targets through `panel-store`; they never write shared values.
 - Gesture behavior comes from the four explicit hooks in `mobile-panels/gestures.ts`.
+- A focused interactive surface may block panel-opening gestures through
+  `useBlockMobilePanelOpenGestures`. Register only while the conflicting interaction is active and
+  unregister when the surface is hidden or unfocused. The blocker never disables gestures that close
+  an already-open panel.
 - Keep `SidebarModelProvider` outside `MobileGestureWrapper`. The provider shares sidebar derivation
  across consumers, while Gesture Handler requires the wrapper's direct child to be a native `View`
  so its injected `collapsable={false}` reaches Android/Fabric.
@@ -1,6 +1,47 @@
 # Mobile Testing

-## Maestro
+## Agent Device
+
+Agent Device `.ad` scripts are the primary mobile E2E format. An agent discovers a working flow interactively, saves the successful commands, then the replay runner executes the same typed plan locally or in CI.
+
+Record a flow while driving the app normally:
+
+```bash
+agent-device open sh.paseo.debug \
+  --platform ios \
+  --session terminal-author \
+  --save-script ./packages/app/e2e/mobile/agent-device/terminal.ios.ad
+agent-device snapshot -i --session terminal-author
+agent-device press 'id="workspace-header-menu-trigger"' --session terminal-author
+# Continue the flow and verify its result with wait/get/is/find.
+agent-device close --session terminal-author
+```
+
+`close` writes the script. Keep selectors based on stable app IDs. Keep assertions as `wait`, `get`, `is`, or `find` commands; screenshots are evidence, not assertions.
+
+Run the Paseo mobile suite:
+
+```bash
+npm run test:e2e:mobile
+```
+
+The runner uses an isolated Agent Device state directory, verifies or starts Metro for this checkout, prewarms the iOS runner, discovers each script's platform from its `context` header, and cleans its sessions, runner lease, daemon, and any Metro process it started. Attempt results, timings, logs, and failure artifacts go under `.dev/agent-device-artifacts`.
+
+Set `PASEO_MOBILE_E2E_METRO_PORT` when this worktree already has Metro on a non-default port:
+
+```bash
+PASEO_MOBILE_E2E_METRO_PORT=62093 npm run test:e2e:mobile
+```
+
+[native-terminal-basic.ios.ad](../packages/app/e2e/mobile/agent-device/native-terminal-basic.ios.ad) and [native-terminal-basic.android.ad](../packages/app/e2e/mobile/agent-device/native-terminal-basic.android.ad) are the smallest examples. Each opens a fresh terminal, types a command at zero delay, submits it, and asserts its distinct output. The app must be connected to a daemon with an active workspace.
+
+When replay diverges, read its ranked selector suggestions. Edit the script deliberately and rerun it from the beginning. `--update` is retained for compatibility but no longer rewrites scripts.
+
+## Maestro compatibility
+
+Existing Maestro flows live in `packages/app/maestro/`. Agent Device can execute its supported subset with `agent-device test <path> --maestro`, which provides a migration path while `.ad` coverage replaces these flows.
+
+### Existing flows

 Maestro flows live in `packages/app/maestro/`. Reusable sub-flows live in `packages/app/maestro/flows/`.

@@ -25,6 +25,7 @@ Terminal frames share the daemon main event loop with all agent traffic. The `ev
 - **The input-mode tracker runs once per process boundary, not per hop.** The worker owns the authoritative tracker; the daemon caches the replay preamble from `getTerminalState` responses and `snapshotReady` messages. Do not reintroduce a per-chunk `feed()` on the daemon main loop.
 - **Snapshot catch-up is backpressure-gated.** A stream falls back to a full snapshot only when `outputBytesSinceSnapshot > MAX_TERMINAL_OUTPUT_FRAME_BYTES` (256KB) **and** the client transport reports `bufferedAmount > MAX_CLIENT_BUFFERED_BYTES` (4MB). A client that keeps draining streams continuously, no matter how much output is produced. Before this gate existed, every 256KB of build output dropped a frame and forced a full JSON cell-grid snapshot (~200k objects across IPC) — the historical source of spiky lag and GC hitches.
 - **Client output writes are not serialized per frame.** The emulator runtime drains contiguous plain writes straight into xterm (which buffers internally). Only barrier ops (`clear`, `snapshot`, `suppressInput` writes) wait — behind a zero-length sentinel write — so resets can't interleave with in-flight output.
+- **Terminal size has one daemon-owned claimant.** Focus and direct interaction send a `claim`; later geometry changes from that connection send `update`. A claim transfers ownership even when the dimensions are unchanged, while an update from any other connection is ignored. This lets an owning pane follow splits and keyboard insets without allowing an idle phone or browser to steal the PTY size.

 ## Measuring