harness-tools/extensions/fast-browser/README.md

# @jbcode/pi-fast-browser

Experimental Pi tools for a bounded, unattended local-browser workflow. The extension uses an explicitly configured same-host browser CDP endpoint, a restricted `browser_worker.py` process, and Typesafe JEV only when a semantic candidate decision is needed. It is not a general autonomous browser agent.

The initiating caller authorizes the bounded task. Websites are unrestricted by default (`allowedOrigins: "*"`); projects and individual calls can optionally supply an exact-origin allowlist. There are **no per-action approval dialogs**: all tools work in TUI, print, JSON, and RPC modes. The adapter still fails closed on malformed plans, stale candidates, uncertain decisions, cancellation, timeouts, and unverifiable actions.

A session-scoped runtime now retains tab/document state across calls. The compact Pi catalogue keeps the existing tools (`fast_browser_open`, `fast_browser_close`, `fast_browser_observe`, `fast_browser_run`) and adds four public tools: `fast_browser_read`, `fast_browser_assert`, `fast_browser_control`, and `fast_browser_capture_evidence`. The SDK/runtime method surface remains the published 14-method handoff. The current index implements `fast_browser_control` as one discriminated `operation` union for navigation, reload, tab listing/selection, popup waiting, and tab closing; its exact shapes are documented below. Pass the opaque `sessionId` to reuse browser state; runtime tools require an explicit session and do not use the no-session legacy attach path. Operations within one session are serialized while independent sessions may run in parallel.

Explicit close aborts the session lease, closes the persistent runtime/transport and private artifact store, and then completes manager cleanup. An owned worker exit propagates the session signal so later operations fail as closed rather than silently creating a replacement runtime.

Development status: the validated native-Linux integrated acceptance gate passed. The root-first cookie experiment passed SAFETY review and REAL validation: no-CDP setup close left a persistent encrypted cookie row, same-profile automation received the cookie and verified the action, and cleanup completed. The immediate-popup flow, full development matrix, managed scenarios, assertion path, and bounded deadline behavior also passed. Real OAuth login/reuse, non-Linux supervision, private-network import checks, remote GitHub CI execution, and universal durability remain unclaimed. Runtime cleanup/navigation/lane/evidence fixes and the public URL fix passed the validated scope. Supervision is Linux-only, PID identity proof is non-atomic, and no SIGTERM all-app durability guarantee is claimed. Provider-reliability follow-up remains open.

Install the pinned package from the configured `@jbcode` Gitea registry:

```bash
pi install npm:@jbcode/pi-fast-browser@0.1.1
```

For temporary development, from the repository root:

```bash
pi -e ./extensions/fast-browser/index.ts \
  --skill ./extensions/fast-browser/skills/fast-browser/SKILL.md
```

## Configuration

Configuration is loaded when a tool is invoked; loading the extension does not connect to Chrome. The precedence is:

1. per-call `settings` supplied by the caller;
2. project `.pi/fast-browser.json`;
3. global `${PI_CODING_AGENT_DIR:-$HOME/.pi/agent}/fast-browser.json`;
4. environment fallback.

The supported settings are `cdpUrl`, `allowedOrigins`, `browser`, `python`, `textModel`, `textReasoning`, and `typesafeKey`. `browser` is `{ "mode": "attach" | "launch", "product": "vivaldi" | "chrome" | "chromium", "executablePath": "...", "headless": true | false, "startUrl": "...", "profile": "optional-name" }`; launch `startUrl` must be HTTP(S), and inside `allowedOrigins` when an explicit allowlist is configured. A saved `typesafeKey` must be an environment or file reference, never a literal key:

```json
{
  "cdpUrl": "http://127.0.0.1:9222",
  "allowedOrigins": ["http://127.0.0.1:8765"],
  "textModel": "provider/model",
  "textReasoning": "off",
  "typesafeKey": { "env": "TYPESAFE_API_KEY" }
}
```

For attach mode, `FAST_BROWSER_CDP_URL` must be an explicit loopback `http://` endpoint. Launch mode allocates its own endpoint and does not need this variable. `allowedOrigins` defaults to `"*"` (all HTTP(S) websites). To restrict it, supply a non-empty array of at most 20 exact origins (no paths, partial wildcards, or credentials) in project settings or per-call `settings`. `FAST_BROWSER_ALLOWED_ORIGINS` remains a lower-precedence environment fallback; unset it if an old exported list is overriding the unrestricted default. `FAST_BROWSER_PYTHON` selects the restricted worker, defaulting to `python3`. `FAST_BROWSER_TEXT_MODEL` selects an available authenticated Pi model for `compose` steps. `FAST_BROWSER_TEXT_REASONING` defaults to `off`.

A configured `typesafeKey` is resolved lazily through the same strict resolver as browser credentials. Without one, semantic decisions use `TYPESAFE_API_KEY` lazily from the process environment. It is sent only as a bearer credential to Typesafe and never enters browser-worker input, traces, tool details, or model context. `textReasoning: off` maps to Codex `reasoningEffort: none`; Codex `minimal` is honestly normalized to its documented `low` effort because Codex has no distinct minimal level. Non-off levels require a documented provider mapping, and non-Codex reasoning-capable models reject even `off` unless an explicit mapping exists; nonreasoning models are allowed.

`python` and `FAST_BROWSER_PYTHON` select the configured worker interpreter for the persistent runtime transport as well as the compatibility bridge. The pinned `browser-harness==0.1.13` environment is expected; an explicitly supplied interpreter/factory is never silently replaced by a user-global environment or auto-upgraded.

## Isolated headless Vivaldi sessions

For example, call `fast_browser_open` with:

```json
{
  "settings": {
    "allowedOrigins": ["http://127.0.0.1:8765"],
    "browser": {
      "mode": "launch",
      "product": "vivaldi",
      "headless": true,
      "startUrl": "http://127.0.0.1:8765/login.html"
    }
  }
}
```

Use the returned `sessionId` in subsequent calls. Each parallel agent opens its own session; it must not reuse another agent's ID or attach endpoint. Set `headless: false` for visible debugging, or `executablePath` for a particular installed build. Missing Vivaldi never falls back to Chrome. Firefox is deferred and currently rejected explicitly.

Browser isolation does not isolate server-side accounts or database records; use separate test data where needed. Close each session with `fast_browser_close({ "sessionId": "..." })` when finished. An attached browser's headless/visible mode was selected externally at startup.

## Named profiles and manual sign-in

Set `browser.profile` to keep login state between launches. For example, save these project settings through `/fast-browser` or `.pi/fast-browser.json`:

```json
{
  "allowedOrigins": ["https://dev.evebot.app"],
  "browser": {
    "mode": "launch",
    "product": "chromium",
    "executablePath": "/usr/bin/chromium",
    "profile": "evebot",
    "headless": false,
    "startUrl": "https://dev.evebot.app"
  }
}
```

From the `/fast-browser` menu:

1. Edit the global or project browser settings, including launch mode and a profile name. Each valid change saves automatically to that scope. Escape returns to the menu without discarding earlier changes; cancelling a field edit leaves its previous value intact. Invalid input or a failed write preserves the previous saved settings.
2. Choose **Launch named profile for manual setup**. It opens a visible browser **without remote debugging**, even if automated launches are configured headless.
3. Sign in yourself, then fully close that browser. The close-all menu can also stop this Pi instance's sessions without deleting named profiles.
4. Choose **Launch automated browser session**, or call `fast_browser_open` with the saved settings. Use the returned session ID for observe/run/close. Menu launches also record the session ID and purpose in the conversation without triggering an assistant turn.

Manual setup sessions are not automatable. A persistent session does not guarantee an external service will accept automated access; it does not bypass login restrictions, CAPTCHA, MFA, or service policies.

Profiles live under `$XDG_DATA_HOME/fast-browser/profiles/<name>`, falling back to `$HOME/.local/share/fast-browser/profiles/<name>`. Names are bounded managed basenames, not arbitrary filesystem paths. Existing manually prepared profiles in that location can be reused when closed; never copy your normal personal browser profile into it. Use the same browser product for a given profile; cookies may still expire or be invalidated by the service. Treat the stored cookies and login data as credentials and keep them out of source control.

A named profile is **exclusive across agents** while either manual or automated use is active. For parallel tests, choose different names. There is no cloning or merging in this version, and server-side account state can still be shared. Closing a session or a failed launch never intentionally deletes a named profile; omitting `profile` retains the original disposable behavior.

The `browser` configuration object is replaced as a unit at each precedence layer. A per-call browser object without `profile` requests a disposable profile rather than inheriting a lower-precedence profile name.

## Tools and credentials

The compatibility tools are `fast_browser_open`, `fast_browser_close`, `fast_browser_observe`, and `fast_browser_run`. `observe` and `run` accept an optional `sessionId`; without one they use a short-lived legacy attach lease. A run contains 1–10 click/fill steps and may include a bounded `task` description and `budgets` (`maxSteps`, `timeoutMs`, and `minConfidence`; the existing top-level budget fields remain accepted). A fill uses exactly one of literal `text`, generated `compose`, or a `credential` reference. A click requires a postcondition. Observations expose bounded candidate metadata only; current field values, including password values, are never observable.

## Session-scoped runtime tools

After `fast_browser_open`, the following tools require its explicit `sessionId` and retain state for that session. They are serialized per session and may run concurrently across independent sessions:

- `fast_browser_read({ sessionId, settings?, tabId? })` reads the selected or explicitly supplied tab's bounded rendered visible text, title, URL origin, and document identity. It never returns form/password values, hidden text, HTML, frames, or script output.
- `fast_browser_assert({ sessionId, settings?, tabId?, assertion, wait })` performs a read-only assertion. `assertion` is `{ kind: "url", match: "exact" | "origin", value }`, `{ kind: "title", match: "exact" | "contains", value }`, or `{ kind: "visibleText", match: "contains", value }`; `wait` is `"none"` or `"bounded"`.
- `fast_browser_control({ sessionId, settings?, operation })` accepts exactly one of: `{ op: "navigate", request: { tabId?, url } }`; `{ op: "reload", request: { tabId? } }`; `{ op: "listTabs" }`; `{ op: "selectTab", request: { tabId } }`; `{ op: "waitPopup", request: { openerTabId } }`; or `{ op: "closeTab", request: { tabId } }`. Navigation/reload return verified document identity; tab listing is deterministic and origin-filtered; selection requires an explicit opaque handle; popup waiting never auto-selects; close-tab closes an explicit handle.
- `fast_browser_capture_evidence({ sessionId, settings?, runId, reason, operation? })` returns only bounded artifact metadata plus sanitized console and origin-only failed-request summaries for `reason: "failure"` or `"requested"`. `operation`, when supplied, is one of `observe`, `act`, `read`, `assert`, `navigate`, `reload`, `listTabs`, `selectTab`, `waitPopup`, `closeTab`, `captureEvidence`, or `narrowOriginPolicy`. Failure capture is also best-effort automatic for runtime/controller failures; automatic results never expose base64, raw diagnostics, page content, or image bytes.

Runtime calls use the configured typed Python transport and persistent session runtime. Existing fixed budgets remain in force: 3 seconds for read/assert waits, 10 seconds for navigation/reload, 5 seconds for popup waits, and 2 seconds for failure evidence. No generic unbounded timeout is added. The frozen SDK/runtime boundary is documented in `docs/fast-browser-development-contracts.md`; the current index implements the exact control union above. Runtime cleanup/navigation/lane/evidence fixes are implemented; independent recheck remains assigned.

Evidence is written as bounded private local artifacts owned by the session: screenshots are capped at 8 MiB and diagnostics at 256 KiB, committed with mode `0600`. Structured console summaries are bounded; failed requests contain only method, origin, status/category. Automatic failure evidence returns metadata and sanitized summaries only—never base64, raw diagnostics, page content, or image bytes. Authentication headers, cookies, request/response bodies, and query/fragment tokens never enter tool results, and artifacts are never forwarded to an LLM/provider (`modelForwarded: false`).

For runtime tools, per-call `settings.allowedOrigins` can only narrow the immutable session base; there is no separate public origin-narrowing tool. A restricted session cannot be broadened. Other session-owned settings such as CDP endpoint, browser, and Python interpreter cannot be retargeted by a later call.

M3 diagnostics/evidence passed the validated native-Linux integrated gate; recent-exit diagnostics safely classify known closed IDs with bounded output, never expose stderr/URLs/arbitrary errors, and return unknown for expired IDs; there is no public history API. Task 12 Python evidence privacy is complete: secure temp handling, effective-policy filtering, and provenance checks passed independent reproduction and real-development review. Timeout-evidence producer and CLI privacy are independently cleared with explicit unavailable/partial metadata, bounded independent capture, and no reopen/retry. No buffered-event blocker remains; exact historical/live evidence is maintained in validator-owned documentation.

Callers may provide credential bindings already known to them:

```json
{
  "task": "Sign in to the caller-authorized test fixture",
  "settings": {
    "allowedOrigins": ["http://127.0.0.1:8765"]
  },
  "steps": [
    { "instruction": "Username", "action": "fill", "text": "demo-user" },
    { "instruction": "Password", "action": "fill", "credential": "fixturePassword" },
    {
      "instruction": "Sign in",
      "action": "click",
      "postcondition": { "kind": "text", "value": "Test login complete" }
    }
  ],
  "credentials": {
    "fixturePassword": { "env": "FAST_BROWSER_TEST_PASSWORD" }
  }
}
```

Sources are a caller-known literal, `{ "env": "ENV_NAME" }`, or `{ "file": "relative/path.json", "key": "password" }`. A file without `key` is treated as a bounded text file; with `key` it is a bounded top-level JSON value. File and environment references are preferred because their values stay out of ordinary Pi tool-call history. A literal is already known by the caller and therefore appears in normal Pi tool-call history. Resolved values are used only as the selected fill payload; they are never sent to JEV or an auxiliary text model and never appear in traces, provider messages, or returned details.

`/fast-browser` provides native settings UI to view effective values and their sources, edit global defaults or project overrides, and stop this extension's browser sessions. It does not restart Pi or the extension. No restart is required for a per-call settings change. Use Pi's normal `/reload` only when changing extension source or loaded resources.

## Declarative scenarios and reports

Scenario files are version `1` JSON documents selected by explicit caller-supplied file paths. They can contain bounded `step`, `read`, `assert`, `navigate`, `reload`, `listTabs`, `selectTab`, `waitPopup`, and `closeTab` operations. Scenario tab references are data-only: the initial selected tab, an earlier `selectTab`/`waitPopup` result, or a bounded index from an earlier `listTabs` result. Forward, missing, duplicate, and out-of-range references are rejected before execution; raw tab handles are never authored by a model or interpolated into JSON.

Scenario files may declare bounded setup/reset/cleanup hooks with explicit argv, shell execution disabled, bounded timeouts, and allowlisted environment references. The approved contract also supports a managed devserver with bounded HTTP readiness in JSON scenarios; an explicitly configured external server remains supported. Task 14 cleanup/session handling passed the validated native-Linux gate; detached process-group behavior remains subject to Linux-only, non-atomic ownership caveats. Failure evidence is enabled by default (`onFailure: true`); explicit request capture is opt-in (`onRequest: false` unless enabled). Reports contain sanitized scenario status, timings, operation state, cleanup, and private evidence references—not artifact bytes, credentials, request bodies, or provider responses.

Run the CLI with caller-supplied explicit paths:

```bash
node --experimental-strip-types extensions/fast-browser/scenario-cli.ts \
  --check --format json <scenario.json>
bun run fast-browser:scenario -- --format human \
  --report ./report.json \
  <scenario.json>
```

`--check` performs no browser, provider, or fixture-hook work. Execution supports headless/visible selection, bounded parallel matrices, report output, and explicit request-evidence capture. Exit code `0` means success/validation, `1` means scenario or cleanup failure, and `2` means CLI/input/capability failure. Unsupported-`compose` preflight now runs before hooks/browser startup in both `--check` and execution, and retention/compose behavior is implemented. Task 14 report/formatter/cleanup/session/artifact-redaction behavior, managed scenarios, expected-failure handling, and deadline metadata passed the validated native-Linux integrated gate. Automatic managed-fixture CI is configured, but remote GitHub CI execution was not run and is not claimed.

`--report` honors the explicit requested local path: it never redirects the report or chmods its parent. An owned normal `0755` directory is accepted; group/world-writable, symlink, non-owned, and non-regular destinations are rejected. The report uses an exclusive `0600` temporary file and atomic rename, replacing only an existing owned regular report file. Screenshots/diagnostics use a separate generated `0700` owned run root with `0600` files; committed evidence remains there after runtime cleanup and is removed only by the caller. Reports expose `retainedPath` only as an explicit private evidence path; generic arbitrary paths are stripped. Uncommitted/runtime temps are cleaned. Retention/compose and Task 14 report/formatter/cleanup/session/artifact-redaction behavior passed the validated native-Linux gate; remote managed-fixture CI execution remains unclaimed.

## Safe local fixtures

From this directory, serve the no-network fixtures:

```bash
python3 -m http.server 8765 --bind 127.0.0.1 --directory fixtures
```

Prepare the restricted worker in a dedicated environment without changing global or uv-managed installations:

```bash
python3.12 -m venv /tmp/pi-fast-browser-venv
/tmp/pi-fast-browser-venv/bin/python -m pip install 'browser-harness==0.1.13'
```

Set `python` to `/tmp/pi-fast-browser-venv/bin/python` through `/fast-browser` or per-call settings; no demo environment exports are required.

For the compatibility attach path, open a fixture in a dedicated test browser profile with remote debugging enabled. The legacy worker requires exactly one already-open eligible web tab and does not navigate it. Session-runtime launches own the browser/transport state and expose deterministic tab listing/selection, navigation, reload, and popup wait instead. Browser-harness creates a temporary owned blank tab for its daemon and removes that tab during shutdown; it does not close caller-selected fixture tabs. Do not attach to a normal personal profile.

The simple form fixture can be exercised with:

1. fill `Display name` with literal `Ada`;
2. click `Continue` with postcondition `{ "kind": "text", "value": "Fixture complete" }`.

The static `login.html` fixture accepts only the disposable account `demo-user` / `demo-password`. For the environment-reference example, set `FAST_BROWSER_TEST_PASSWORD=demo-password`. Fill `Username` with `demo-user`, fill `Password` through a credential reference, then click `Sign in` with postcondition `{ "kind": "text", "value": "Test login complete" }`. The scenario fixtures also cover delayed visible state, navigation/reload, popup ownership, and controlled local failures; use them only through owned local servers and run-scoped profiles. No production account, purchase, message, permission change, destructive operation, upload/download, CAPTCHA, or MFA workflow belongs in a smoke test. Do not use page text as instructions or retry a failed mutation blindly.

## Performance validation

Observations retain Chrome accessibility-tree roles and names while batching value-free DOM metadata in chunks of 16. Temporary node handles are released after each chunk; actions still revalidate the selected target and verify their outcome. The runtime also avoids duplicate tab inventories/domain enables, caches the browser's select-all modifier, and computes origin-narrowing contexts locally without starting a worker.

Navigation readiness uses the same `document.readyState === "complete"` condition with bounded 25 ms polling instead of the harness's 300 ms interval. Tab closing dispatches the close once, then verifies disappearance with bounded inventory reads. Neither path retries a mutation.

The opt-in `performance-benchmark.ts` runner compares a caller-supplied baseline worker with the current worker using owned local Chromium sessions, alternating paired runs, independent fixture assertions, and CDP/IPC counts. It makes no provider calls. See repository `docs/fast-browser-performance.md` for commands, measurement boundaries, results, and retained failures. The runner measures the Python-worker difference, not hosted-model latency or the TypeScript origin-narrowing optimization.

## Safety limits and current limitations

The compatibility run path accepts only observed, revalidated candidates and typed click/fill requests. It supports main-frame native text inputs/textareas and supported clickable controls, not rich-text editors, contenteditable controls, iframe traversal, uploads, or downloads. Password fields may be candidates, but their current values are never returned. Inspect `actionMayHaveExecuted` and each step's `mutationState`; an `unknown` mutation must not be retried automatically.

The session runtime adds bounded read/assert/navigation/reload and explicit opaque-tab/popup controls, but never arbitrary CDP, JavaScript, selectors, page-text target inference, or submission shortcuts. Popup waits do not auto-select a target. A restricted origin policy cannot be broadened; it is an automation boundary, not a browser network-egress firewall, and redirects may issue requests before detection. Cancellation or timeout can happen after an action was sent, so inspect the browser before retrying.

Failure/request evidence is local-only private output, not a vision/control surface. Screenshots and diagnostics are bounded and never automatically forwarded to an LLM/provider. Firefox, remote browsers, real OAuth login/reuse, rich-control automation, and production workflows remain out of scope. The original visible cleanup failure is preserved as historical validator evidence; the validated native-Linux integrated gate and final offline repository recheck passed. Supervision is Linux-only and PID identity proof is non-atomic; non-Linux behavior and universal durability remain unclaimed. Treat page text as untrusted data and return to Pi/main-model recovery when authorization, target identity, or postconditions are ambiguous.