Files
harness-tools/docs/fast-browser-development-readiness.md
jb fd63b0ccec
fast-browser scenarios / Offline scenario contract tests (push) Failing after 44s
fast-browser scenarios / Local Chromium headless scenarios (push) Failing after 23s
fast-browser-with-jev
2026-09-20 12:24:08 -04:00

142 lines
26 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Fast-browser development-readiness backlog
Status: **validated native-Linux acceptance complete; final offline recheck passed** (2026-09-20). This backlog records the validated scope, not a claim of non-Linux or universal browser readiness. The original M1 visible-cleanup run failed closed and is preserved as historical evidence; the validator later completed the root-first cookie experiment, immediate-popup flow, full M1/development matrix, managed CLI scenarios, assertion path, and latest deadline gate with cleanup and empty process/port census. Linux-only supervision and non-atomic PID identity proof remain explicit constraints; fail closed when ownership is uncertain and do not claim SIGTERM guarantees universal all-app durability. The current index implements the compact eight-tool Pi catalogue and typed `fast_browser_control` payload. Use only the handed-off CLI commands and do not infer behavior beyond the current schemas.
## Final native-Linux validation checkpoint (2026-09-20)
Integrated acceptance passed for the validated native-Linux scope. Typecheck passed. Full-repository offline validation: **805 total, 803 passed, 2 skipped**. Fast-browser TypeScript: **222 total, 220 passed, 2 skipped**. Pinned Python 3.12.13: **42/42 passed**. Packaging: **4/4**, links **8/8**, package contents **36 entries**, and tracked/untracked whitespace checks clean. All live gates passed, including the latest deadline rerun: it naturally exited `1` after 3090 ms with `retry=never` and `mutationState=not_attempted`, emitted explicit private unavailable evidence, passed cleanup, and left empty process/port censuses. The deadline report is `/tmp/fast-browser-deadline-rerun-iaE8de/reports/overall-deadline-no-popup-rerun.json`; the older failed report remains preserved.
`imports:check` was **not run** because it requires private network access. Remote GitHub CI was **not run**; automatic CI remains configured, and equivalent managed-CLI live validation passed locally. These are scope caveats, not failures of the native-Linux acceptance gate. Non-Linux supervision remains unsupported, PID identity proof is non-atomic, and no universal SIGTERM flush/all-app durability guarantee is claimed.
## Baseline and guardrails
Observed reusable behavior includes `fast_browser_open`/`close` session ownership, exclusive persistent named-profile locks, disposable launch profiles, explicit headless launch, visible manual setup without CDP, per-session serialization, origin `"*"` as the default, autosaved global/project settings with scope boundaries, stale observation rejection, exact fill verification, click URL/visible-text postconditions, bounded worker startup/action polling, bridge request deadlines, and unknown-mutation/no-blind-retry reporting. This inventory comes from `extensions/fast-browser/browser-session.ts`/`profiles.ts`, `browser_worker.py`/`bridge.ts`, `controller.ts`, `contracts.ts`, and their offline tests; `fixtures/form.html` and `fixtures/login.html` are the current owned pages. Existing waits are concrete: browser spawn/readiness 10s, worker daemon readiness 5s, bridge requests 10s, worker action verification 3s, and controller runs 60s by default/120s maximum. The readiness work must extend those boundaries rather than replace working waits or weaken them.
Every live check in this backlog uses an owned local HTTP fixture and a unique run-scoped test profile (for example, a temporary `XDG_DATA_HOME` plus `fb-readiness-<run>-<worker>` names). It must use per-call/project-temporary settings, never edit global configuration, never use the active-user `harness-test` profile, and never log in to EVE or any other real service. Credential tests use only the disposable fixture account and environment/file references; values must not appear in traces, evidence, reports, or model requests. Cleanup is required on pass, failure, cancellation, and process interruption.
## Architecture checkpoint (scout findings)
The architecture scout traced `index.ts` → `browser-session.ts` → `browser_worker.py` → `controller.ts`, with configuration in `config.ts`, leases in `profiles.ts`, IPC in `bridge.ts`, and public result shaping in `index.ts`. Final contracts are published in `extensions/fast-browser/contracts.ts` with the companion `docs/fast-browser-development-contracts.md`; scoped typecheck passed. This closes the contract-publication checkpoint; M1–M5 implementation and the integrated native-Linux live acceptance gate are complete, with the final offline repository recheck now passed. The historical scout snapshot recorded `Observation`, `Postcondition`, `StepTrace`, and `RunResult`, an exactly-one-tab worker, and no public navigation/tab/popup/evidence API; the current SDK/runtime handoff has since added bounded session controls and evidence plumbing. The scenario source/CLI handoff and retention/unsupported-`compose` preflight are complete; Task 14 report/formatter/cleanup/session/artifact-redaction behavior passed the integrated native-Linux gate. These historical and current facts are not proposed APIs.
The main task assigned `fb-development-contracts`, which published the SDK/runtime contracts before parallel implementation. The remaining review must preserve readiness phase diagnostics (spawn, DevTools, worker, eligible-tab selection, origin approval, page/action readiness), same- versus cross-document navigation semantics, and popup/new-tab ownership. New read/assert/control/evidence features use the session-scoped runtime because the former per-operation worker lifecycle did not retain tab handles or console history. Review must preserve bounded post-close exit diagnostics and scenario/report privacy without exposing full URLs, page content, field values, authentication headers, request bodies, or provider responses. The current runtime index implements the compact eight-tool Pi catalogue and its discriminated control union; no public origin-narrowing tool exists because `settings.allowedOrigins` is per-call. Task 13 runtime cleanup/navigation/lane/evidence fixes passed the integrated native-Linux gate; the shared `BROWSER_EVIDENCE_OPERATIONS` contract is published. Task 14 CLI report/formatter/cleanup/session/artifact-redaction fixes and the managed-fixture examples passed the integrated gate; the final rerun carries real deadline metadata.
## Known failures to carry forward
- The successful historical live demo used the old attach-only, approval-based contract; it remains historical context and is not evidence for the new unattended launcher. The current native-Linux launcher is covered by the completed live gate.
- Two managed visible Chromium launches crashed with SIGSEGV under an older flag/environment combination. A minimal user-data/CDP/no-first-run baseline worked in a diagnostic, but the integrated launcher and the cause of the old crash are not validated.
- A browser crash was historically reported only as `Unknown browser session`; startup/worker paths had generic or empty-looking errors, including a no-approved-tab condition. The worker now has bounded sanitized startup IPC/origin diagnostics, and recent-exit diagnostics are implemented/unit-tested with safe known-closed-ID reasons; the native-Linux live gate passed.
- The historical compatibility worker required exactly one eligible web tab and had no navigation, reload, popup, screenshot, console, or failed-request evidence surface. The current SDK/runtime handoff adds bounded session controls and evidence plumbing; the compact public wiring and final remediation passed the native-Linux gate. These historical gaps are not reasons to invent unbounded waits.
- The historical auxiliary-model failure (unsupported Codex Spark entitlement) is an environment/provider issue, not browser reliability evidence; live readiness defaults to exact-name/literal fixture checks.
## Early-review acceptance status (historical checkpoints; 2026-09-20)
These historical code/test handoffs preceded the final native-Linux acceptance gate; their counts remain dated checkpoint evidence, not moving completion totals:
- `fb-transport-evidence` delivered wire/details/parent-abort and artifact-store fixes in a 22m31 handoff; 19 targeted TypeScript tests plus 35 Python tests pass.
- `fb-runtime-tools` delivered the runtime factory/tools/controller handoff in 4m39; 48 targeted tests plus bridge coverage pass.
- The typed wire mismatch, structured-error parser loss, dropped parent signal, and post-dispatch `not_attempted` mutation-state findings are implemented and unit-tested.
- Bounded opaque-tab references are implemented with static validation/resolution, closed/missing checks, popup JSON coverage, 20 focused tests, clean typecheck, and passing `--check` examples. The CLI handoff now provides `node --experimental-strip-types extensions/fast-browser/scenario-cli.ts --check --format json <scenario.json>` and `bun run fast-browser:scenario -- --format human --report ./report.json <scenario.json>`. Unsupported-`compose` preflight now runs before hooks/browser startup in both `--check` and execution; Task 14 report fixes have an implementation checkpoint with 24 tests (not final). Defensive human/JSON formatters, cleanup/session handling, artifact metadata redaction, usable retained paths, explicit atomic `0600` reports in owned `0755` parents, private artifact roots, and post-cleanup retention are covered. Managed CLI implementation is complete with three `--check` examples; all three managed CLI examples and the intentional failed-assert path have REAL pass/expected-failure proofs, automatic managed-fixture CI is configured, and the final rerun includes real deadline metadata.
- The artifact-root chmod issue is fixed with private owned roots and tests; independent review found no remaining chmod/shared-parent bypass. The separate provider-reliability TODO remains open.
- Consolidated lifecycle status: the original M1 visible-cleanup run failed closed and preserved `/tmp/fast-browser-live-validation-5HvmBr`; the validator safely terminated all 11 verified owned processes, left no target-profile processes, and preserved `evidence/owned-process-cleanup.json` as historical evidence. PID/helper safety independently cleared; the root-first cookie experiment, full M1/development matrix, and latest deadline gate passed with complete cleanup and empty process/port census. Supervision is Linux-only; PID identity proof is non-atomic, so ownership uncertainty must fail closed. This evidence does not claim SIGTERM guarantees universal all-app durability.
- Task 12 Python evidence privacy is complete: independent reproduction and six real-development cases pass; no buffered-event blocker remains, and exact historical/live results remain owned by the validator documentation. Timeout-evidence producer and CLI privacy are independently cleared with explicit unavailable/partial metadata, bounded independent capture, and no reopen/retry. Task 13 runtime cleanup/navigation/lane/evidence fixes and the public URL fix passed the integrated native-Linux gate; the shared `BROWSER_EVIDENCE_OPERATIONS` contract is published. The immediate-popup fix passed independent review and the REAL immediate flow without a delay workaround. Task 14 CLI report fixes passed the integrated gate. Do not treat these native-Linux results as non-Linux or remote-CI claims.
## Root widget delivery sequence
These are the user-approved work items; all five feature milestones passed the integrated native-Linux LIVE gate. Task 1 is the published contract/documentation checkpoint; Tasks 2–6 have live acceptance evidence. The final offline repository recheck also passed; remote GitHub CI was not run and remains an explicitly unclaimed execution path.
1. **Docs/contracts — published checkpoint (`fb-development-contracts`):** final contracts/backlog are published and scoped typecheck passed.
2. **Live profiles/headless/parallel validation — complete for validated native Linux (`fb-live-baseline`):** PID/helper safety, root-first cookie persistence, full M1/development coverage, latest deadline behavior, cleanup, and empty process/port census passed. Retain the historical failed-cleanup evidence; Linux-only supervision and non-atomic PID identity proof remain constraints.
3. **Bounded page reading/assertions — complete for validated native Linux (`fb-python-development`).**
4. **Failure evidence/diagnostics — complete for validated native Linux (`fb-transport-evidence`).**
5. **Navigation, reload, tabs, and popups — complete for validated native Linux (`fb-runtime-tools`).**
6. **Declarative scenarios, fixture resets, reports, and headless CI — complete for validated native Linux (`fb-scenarios-ci`).**
7. **Independent review and final offline check — complete:** runtime/public-URL review, validator freeze, full live matrix, and final offline repository recheck passed for the validated native-Linux scope.
8. **M1 acceptance closeout — complete for validated native Linux:** historical failures remain preserved and non-Linux/universal durability claims remain out of scope.
13. **Final TypeScript/runtime remediation review — complete for validated native Linux:** registry, abort/error-shaping, schema-alignment, navigation, lane, evidence, and public-URL fixes passed; exact historical validation results remain owned by the validator documentation.
14. **CLI report/formatter remediation — complete for validated native Linux:** defensive human/JSON formatting, cleanup/session/artifact metadata redaction, retained paths, report permissions/retention, managed scenarios, and expected-failure behavior passed; remote GitHub CI execution was not run.
## Prioritized milestones
### M1 — real profiles, headless, and parallel live validation (**complete for validated native Linux**)
**Goal:** prove the new session manager against real installed browser processes, not only fake children.
**Acceptance criteria**
- Two simultaneous sessions use distinct unique named profiles, processes, CDP ports, and browser state. Closing or crashing one cannot close, unlock, or alter the other.
- A named profile can be closed and reopened for the same local fixture without losing its test marker; profile locks reject contention and stale DevTools metadata is handled only after ownership is acquired.
- At least one installed Chromium-family browser passes a headless login-fixture smoke test and a visible diagnostic smoke test. Vivaldi is tested when installed and never silently substituted; unavailable products produce an explicit skip/evidence record.
- The check exercises literal/exact-name steps by default, so the readiness run needs no provider call. It records browser/version, profile/port ownership, cleanup, and any exit reason without credentials.
**Current live evidence:** `docs/fast-browser-live-validation.md` is validator-owned and preserves the exact historical results, including the failed original visible-cleanup run at `/tmp/fast-browser-live-validation-5HvmBr` and the report of 10 owned Chrome processes. The validator subsequently safely terminated all 11 verified owned processes, left no target-profile processes, and preserved `evidence/owned-process-cleanup.json`. The root-first cookie experiment, full M1/development matrix, managed scenarios, assertion path, and latest deadline rerun passed with cleanup and empty process/port census. The latest deadline rerun naturally exited `1` after 3090 ms with `retry=never` and `mutationState=not_attempted`, emitted explicit private unavailable evidence, and is not a live-gate failure. Its report is `/tmp/fast-browser-deadline-rerun-iaE8de/reports/overall-deadline-no-popup-rerun.json`; the older failed report remains preserved. Linux-only supervision and non-atomic PID identity proof remain constraints, and no SIGTERM all-app durability guarantee is claimed.
**Tests/dependencies:** retain existing `browser-session`, `profiles`, bridge, and index tests. The owned live-validation workflow and final offline repository recheck passed for native Linux; preserve the historical failed-cleanup evidence and require complete cleanup on future runs.
### M2 — bounded visible-page reads and assertions (**complete for validated native Linux — `fb-python-development`**)
**Goal:** add a read-only, caller-bounded way to inspect/assert visible fixture state without turning page text into instructions.
**Published contract:** the session runtime exposes typed `read` and `assert` operations with bounded visible text, URL/title/visible-text assertions, and session-scoped target identity. Implementation must preserve limits for characters, nodes, elapsed wait, origin, and result shape; reuse the existing typed bridge and 3-second postcondition/read-assert wait model rather than inventing another. Do not expose arbitrary CDP, JavaScript, DOM handles, hidden text, current input/password values, or page HTML.
**Acceptance criteria**
- Reads return only bounded visible text/metadata from the approved main page and redact/truncate deterministically. Hidden, disabled, password-value, iframe, and rich-control content cannot leak through the result.
- Assertions support at least the fixture's visible text and URL-origin outcomes, fail closed on ambiguity/staleness, and distinguish unmet from timed-out/unknown outcomes. Existing action postcondition polling and run/request deadlines are reused and documented, not duplicated.
- Offline fake-backend tests cover visibility, limits, navigation/frame identity changes, cancellation, and no credential/page-value leakage; local live tests prove the same results on owned fixtures.
**Dependencies:** M1 session/fixture runner and the published runtime contract.
### M3 — evidence and crash/exit diagnostics (**complete for validated native Linux — `fb-transport-evidence`**)
**Goal:** make failures actionable without leaking page data.
**Acceptance criteria**
- A bounded failure evidence bundle is captured automatically on failure and can also be requested explicitly. Local-only screenshot artifacts are the default; the bundle may also include sanitized console errors and sanitized failed-request summaries, tagged to session/step/phase. Successful runs do not capture evidence unless requested.
- Evidence stays in bounded, local, private artifacts with explicit size, retention, and permissions. Structured console/request evidence never includes authentication headers, request bodies, credential/field values, query/fragment tokens, or unbounded page text. A screenshot is rendered fixture pixels, so it remains local/private and bounded; it is never automatically forwarded to a model or provider.
- Startup/worker failures distinguish no eligible tab, ambiguous tabs, origin rejection, missing/mismatched harness, timeout, and browser exit. Recent-exit diagnostics now safely classify known closed IDs with a 30-second TTL and 32-entry cap, omit stderr/URLs/arbitrary errors, return unknown for expired IDs, and expose no public history API; this is implemented/unit-tested but still needs live validation.
- Screenshot/console/request capture is failure evidence, not an autonomous vision/control surface. Evidence creation never retries an action or changes the browser state.
**Current M3 acceptance status:** Task 12 Python evidence privacy and the timeout-evidence producer/CLI privacy review passed. Explicit unavailable/partial metadata, bounded independent capture, and no reopen/retry were validated. No buffered-event blocker remains; exact historical/live evidence is maintained in the validator-owned documentation. Recent-exit diagnostics remain implemented/unit-tested with safe bounded output; the native-Linux live gate and final offline repository recheck passed.
**Tests/dependencies:** contract tests for redaction/size/permissions and crash races; induced local-fixture console/request failures; controlled browser-exit diagnostics; and M1's real session IDs passed within the validated native-Linux gate. Keep generic extension-facing errors safe while retaining the worker's already-bounded startup detail.
### M4 — navigation, reload, tab selection, and popups (**complete for validated native Linux — `fb-runtime-tools`**)
**Goal:** replace the current exactly-one-eligible-tab limitation with explicit, caller-authorized lifecycle controls.
**Published SDK/runtime and Pi contract:** typed `navigate`, `reload`, `listTabs`, `selectTab`, `waitPopup`, and `closeTab` methods use opaque session-scoped tab identities; popup waits never auto-select. The public `fast_browser_control` payload is a discriminated `operation` union: `navigate`/`reload` carry optional `tabId` (and `navigate` carries `url`), `listTabs` carries no request, `selectTab` carries `tabId`, `waitPopup` carries `openerTabId`, and `closeTab` carries `tabId`; all are wrapped with `sessionId` and optional `settings`. Per-call `settings.allowedOrigins` narrows policy; there is no public narrow-origin tool. Implementation/review must preserve selection lifetime, document identity, bounded completion/failure semantics, popup ownership, and explicit target choice. A model must not infer or invent a target from page text.
**Acceptance criteria**
- Navigation accepts only bounded HTTP(S) destinations allowed by the session policy; reload has a bounded completion result; both invalidate observations and report redirects/out-of-origin transitions safely.
- Tab listing/selection is deterministic and origin-filtered. Multiple eligible tabs remain an error until the caller selects one; no arbitrary tab or personal browser tab is chosen.
- A fixture-created popup/new tab can be listed, selected, asserted, and closed only when owned by the session. Closing a session leaves externally attached browser processes untouched. The immediate-popup fix has passed independent review and a REAL immediate flow; no delay workaround is part of the contract.
- Offline tests cover stale tab/document identities, concurrent session lanes, popup races, cancellation, and no cross-profile/tab leakage; live tests use only local fixtures.
**Dependencies:** M1 session ownership, M2 assertion/read result limits, and M3 diagnostics for failed navigation/popup startup.
### M5 — reusable scenarios, fixture/reset hooks, reports, and headless CI (**complete for validated native Linux — `fb-scenarios-ci`**)
**Goal:** turn the checks above into repeatable regression evidence.
**Current M5 status and acceptance boundary:** the user-approved scenario contract includes a managed devserver with bounded HTTP readiness in JSON scenarios; an external-server option remains supported. The managed CLI examples and intentional failed-assert scenario have REAL pass/expected-failure proofs; the final deadline rerun includes real deadline metadata, explicit private unavailable evidence, no retry, and cleanup pass. Retention, unsupported-`compose` preflight, report/formatter/cleanup/session/redaction behavior, and the integrated native-Linux acceptance gate passed. Automatic managed-fixture CI is configured, but remote GitHub CI execution was not run and is not claimed.
**Acceptance criteria**
- A small local fixture suite covers form/login, navigation/reload, a popup, delayed visible state, console error, and controlled failed request. Each scenario has explicit setup, bounded steps/assertions, reset, and cleanup hooks; reset is test-only and never deletes a user-managed named profile.
- JSON scenarios may provision an owned managed devserver and wait for bounded HTTP readiness; an explicitly configured external server remains supported. Managed CLI implementation, helper integration, automatic CI configuration, and the listed REAL proofs passed within the validated native-Linux scope.
- Reusable tests are declarative JSON scenario definitions supplied through explicit user-provided scenario file paths and executed by a CLI runner, with schema validation before launch. The runner can execute one scenario or a parallel matrix with unique profile/port/temporary-data names in visible and headless modes. It is network-free except for the owned local fixture and never requires provider credentials.
- Reports have human-readable and machine-readable forms with scenario/status/timings, browser/profile ownership, evidence references, failure phase, and cleanup outcome—but no credential values, field values, raw URLs with secrets, or unbounded page text.
- CI is configured to run deterministic offline tests and headless JSON-scenario CLI workflows for relevant fast-browser PR, push, dependency, and workflow changes, with a manual trigger as well; this validation did not execute remote GitHub CI. Missing optional Vivaldi is reported as a skip, not a fallback. Firefox remains deferred.
**Dependencies:** M1–M4 contracts and evidence, fixture server/reset design, the available CLI commands/hooks/reports source, automatic CI workflows with isolated temporary directories that execute the real headless JSON-scenario runner against the managed fixture, and the final native-Linux acceptance evidence. All five milestones are complete for the validated scope; remote GitHub CI execution remains unclaimed.
## Explicit non-goals and deferred scope
Firefox needs a separate backend and remains deferred; reject it honestly rather than substituting Chromium. Rich-text/contenteditable controls, iframe traversal, uploads/downloads, arbitrary JavaScript/CDP, remote browsers, screenshots as vision input, broad autonomous planning, blind retries, and production/real-EVE workflows are out of scope unless a later approved milestone specifically requires them. Parallel browser processes do not isolate server-side accounts. No speed or arbitrary-site reliability claim is made by this backlog.
## First milestone: actionable tasks
1. Reload the extension in a clean process and run the existing opt-in local fixture preflight with a temporary config/data root.
2. Add the real-browser runner's unique profile/port allocator, process/version preflight, signal-safe cleanup trap, and machine-readable pass/fail/skip record.
3. Run two named-profile sessions concurrently (headless plus visible where available) against the local login fixture using only exact-name literal/fixture credentials.
4. Reopen one profile, verify its local test marker, close it, and prove the other session remains usable; capture bounded exit/cleanup diagnostics for any failure.
5. Record browser availability, commands, sanitized errors, and evidence in the readiness report; do not edit global config or use active-user/real-service profiles.