fast-browser-with-jev
This commit is contained in:
@@ -0,0 +1,206 @@
|
||||
# Local browser automation with Pi and Typesafe JEV
|
||||
|
||||
Status: feasibility review; proposed architecture, not an implementation decision.
|
||||
Reviewed: 2026-09-18. No browser tasks or inference benchmarks were run. No API keys were used or changed.
|
||||
|
||||
## Verdict
|
||||
|
||||
**Worth a bounded prototype for DOM-accessible workflows; not a demonstrated replacement for a general browser agent.** Browser-harness already supports local Chrome control without Browser Use Cloud. JEV can plausibly reduce the cost and latency of semantic choices between known actions. Its usefulness depends on how reliably code can expose the correct action and verify its outcome.
|
||||
|
||||
The most important correction: **JEV is text-only, not a screenshot-to-action model.** Use accessibility/DOM observations, not screenshots, on the JEV fast path. A vision-capable Pi model would be a separate fallback for visually grounded tasks.
|
||||
|
||||
A skill alone can teach Pi how to use the browser but does not remove main-model turns. The candidate design is a **Pi extension containing a bounded decision loop, with browser-harness as its local execution backend and a companion skill for workflow guidance**. A Chrome extension is not necessary for the initial experiment.
|
||||
|
||||
## What the sources establish
|
||||
|
||||
### Browser-harness
|
||||
|
||||
Inspected upstream commit `afbcc381b963040c19627d788e40c7e7663171ee`, package version `0.1.13`, MIT license.
|
||||
|
||||
- Local Chrome/Chromium is supported; cloud browsers are optional. The package requires Python >=3.11; its setup prompt recommends Python 3.12.
|
||||
- A persistent local daemon owns the CDP connection. Python helpers provide browser operations; the CLI executes Python, and an optional stdio MCP server exposes helpers as tools.
|
||||
- Upstream recommends accessibility-tree grounding using `Accessibility.getFullAXTree`, `backendDOMNodeId`, box-model coordinates, and outcome verification. Screenshots are a fallback when layout or imagery matters.
|
||||
- The default local daemon has one mutable current tab. Concurrent tab switching and actions can race. A Pi adapter needs serialized browser operations, not just per-request locking.
|
||||
- `fill_input()` emits key events per character; `type_text()` uses `Input.insertText`. Long-text entry can dominate latency even with a fast text model. Framework-managed inputs require event handling and exact-value verification; bulk insertion is not a universal replacement.
|
||||
- Arbitrary Python/JS and self-written helpers are powerful but inappropriate as the automatic JEV action interface. Expose vetted typed operations instead; keep any helper authoring in the supervised slow path.
|
||||
|
||||
Local-only configuration needs care:
|
||||
|
||||
- Set `BH_TELEMETRY=0`: telemetry is otherwise enabled by default.
|
||||
- Set `BH_UPDATE_CHECK=0`: the CLI otherwise checks PyPI for updates.
|
||||
- **Unset `BU_AUTOSPAWN`**, rather than setting it to `0`: source checks whether the environment value is nonempty.
|
||||
- Explicitly bind to the approved local browser. Do not expose remote-browser start, profile sync, cloud authentication, or proxy-fetch helpers in the fast adapter.
|
||||
- Do not supply a Browser Use key to this worker. `http_get()` selects a hosted fetch proxy when `BROWSER_USE_API_KEY` is present. Treat environment and workspace `.env` inheritance as part of the audit.
|
||||
- Keep recordings off unless requested (`BH_RECORD=0` is an explicit override). Treat saved DOM, screenshots, and logs as sensitive.
|
||||
|
||||
This establishes that hosted browser infrastructure is unnecessary, not that all dependencies have been exhaustively audited for outbound traffic. An implementation should verify network egress explicitly.
|
||||
|
||||
### Typesafe JEV
|
||||
|
||||
The documented API is `POST https://api.typesafe.ai/v1/systemone`, authenticated with a bearer key. Requests contain `state`, `model`, and named `questions`.
|
||||
|
||||
- Current documented version: `jev-1.13.0`. Pin a version when calibrating thresholds; `jev-latest` can move.
|
||||
- Inputs: text or structured JSON; no images, audio, or video.
|
||||
- Outputs: `Choice` among supplied options, `Noul` yes-probability, or `Score` against supplied criteria. This is not arbitrary prose, code, or a general agent action stream.
|
||||
- Multiple questions about the same state are evaluated independently in parallel. A question cannot rely on another question's answer in the same request.
|
||||
- Published limits: 64k tokens for the request and 32k for state plus the longest question. These are ceilings, not recommended browser-state sizes.
|
||||
- Published price: $0.042 per million input tokens, with output tokens free. At 2,000 input tokens per request, 1,000 decisions would cost approximately $0.084 for JEV alone, assuming that token count matches billing. Planner/text calls and retries are additional.
|
||||
- Typesafe says most queries complete in about 100 ms. **This is a provider claim, not measured local end-to-end latency or a browser-task guarantee.**
|
||||
- The docs say customer requests/responses are not used for training. They mention enterprise ZDR separately; do not assume a normal API account has zero retention.
|
||||
- The JavaScript SDK is `@typesafe-ai/sdk` (Node.js >=20), using `TypeSafeClient.systemOne()` and `TYPESAFE_API_KEY`. Keep this as an independent API adapter, not a fake chat model.
|
||||
- Typesafe explicitly documents weakness with numeric precision, counting, date comparison, multi-hop indirection, irrelevant context, and adversarial input. Parse/count/compare in code; give JEV semantic labels and short, direct questions.
|
||||
- The documented adversarial-content limitation is especially relevant to websites: page content can steer classifications. Instructions and confidence thresholds alone are not adequate protection.
|
||||
- Choice confidence and Noul probabilities are not interchangeable. Calibrate each gate separately; a relative Choice can pick a winner even when every option is unsuitable.
|
||||
|
||||
Local browser execution still sends selected page-derived text to Typesafe and generated-text context to the configured Pi model provider. Redact passwords, tokens, cookies, and unrelated page content before those calls.
|
||||
|
||||
## Proposed execution split
|
||||
|
||||
```text
|
||||
User goal -> Pi main model: bounded subgoal and success criteria
|
||||
|
|
||||
Pi extension
|
||||
|
|
||||
observe AX/DOM, filter candidates
|
||||
|
|
||||
exact deterministic match available?
|
||||
yes: use code no: JEV Choice
|
||||
|
|
||||
policy + freshness + confidence checks
|
||||
|
|
||||
typed local browser-harness bridge
|
||||
|
|
||||
execute -> verify -> repeat
|
||||
|
|
||||
finished / uncertain / budget exhausted
|
||||
|
|
||||
return to Pi
|
||||
|
||||
Text needed: supplied literal -> insert directly
|
||||
new prose -> configured small Pi model -> validate -> insert
|
||||
Visual or novel reasoning needed -> return to Pi, optionally with screenshot
|
||||
```
|
||||
|
||||
### Responsibilities
|
||||
|
||||
| Component | Responsibility |
|
||||
| --- | --- |
|
||||
| Pi main model | Interpret novel goals, supply bounded subgoals, handle recovery requiring reasoning |
|
||||
| Extension controller | Budgets, state machine, permissions, cancellation, model routing, traces |
|
||||
| DOM/AX adapter | Extract relevant controls and labels; generate candidate IDs tied to tab/frame/document |
|
||||
| JEV | Rank/select candidates, classify known states, flag uncertainty, assess narrow semantic predicates |
|
||||
| Small Pi model | Generate genuinely new text within supplied facts and constraints |
|
||||
| Browser bridge | Execute allowlisted operations; verify target identity and resulting browser state |
|
||||
| Companion skill | Explain usage, escalation rules, and reusable site-specific workflows |
|
||||
|
||||
JEV should **replace a main-model microdecision**, not be added before an unchanged main-model turn. The deterministic path should bypass JEV entirely when no semantic judgment is needed.
|
||||
|
||||
Candidate sets must include `none_of_the_above`/`needs_more_context`. Selecting the best of an incomplete set can still be wrong. Keep candidate descriptions grounded in visible labels, roles, nearby headings and form structure; filter hidden/disabled elements and preserve frame identity. Treat the correct-target inclusion rate as a first-class metric.
|
||||
|
||||
Run independent page-state questions together, but do not confuse speculative decisions with permission to execute alternative actions in parallel. Reobserve after navigation or meaningful DOM changes. Invalidate stale candidates before any click; do not cache coordinates across page transitions.
|
||||
|
||||
For literals such as a supplied email address, search phrase, or exact message, **no text-model call is necessary**. A generation model is only justified for composition or rewriting. Validate length, exact required facts, and field-specific constraints. Do not regenerate text when retrying insertion; verify the actual value and framework state.
|
||||
|
||||
### Scope and limitations
|
||||
|
||||
Good first targets: locating a labeled control, navigating a known menu, choosing a relevant result from a shortlist, filling conventional forms, recognizing a success/error message, and repeating established workflows.
|
||||
|
||||
Weak targets: canvas-only applications, image-only controls, novel workflows that require broad planning, CAPTCHAs, ambiguous authorization, and long tasks where the correct action is not represented in the candidate set. Shadow DOM, cross-origin frames, rich-text editors, and changing layouts need explicit coverage rather than assumptions.
|
||||
|
||||
## Pi integration findings
|
||||
|
||||
The installed Pi API supports this split without changing the main conversation model:
|
||||
|
||||
- Register typed tools with `pi.registerTool`; tool execution receives an abort signal and extension context.
|
||||
- Resolve the auxiliary text model with `ctx.modelRegistry.find(provider, id)` and check `getAvailable()` / `hasConfiguredAuth(model)` at runtime.
|
||||
- Call `ctx.modelRegistry.complete(model, context, { signal })` for independent text generation. This uses Pi's model registry rather than duplicating provider credential handling.
|
||||
- Keep Typesafe behind its own SDK/HTTP adapter; its constrained decision API does not need a Pi chat-provider registration.
|
||||
- Cancellation must propagate to model calls and the local worker. Stopping a tool does not undo an already executed browser action.
|
||||
|
||||
Sanitized cached model metadata includes these **candidates, not verified live availability or measured speed rankings**:
|
||||
|
||||
| Candidate | Potential role |
|
||||
| --- | --- |
|
||||
| `anthropic/claude-haiku-4-5` | Small text-generation baseline; metadata also advertises image input |
|
||||
| `anthropic/claude-haiku-4-5-20251001` | Pinned Haiku alternative for reproducibility |
|
||||
| `openai-codex/gpt-5.3-codex-spark` | Text-only alternative to benchmark if runtime authorization permits |
|
||||
|
||||
Start by comparing Haiku and Spark on short, constrained field-generation tasks; do not select a winner from model names alone. A local LM Studio provider is another supported integration route, but no particular loaded local model was verified.
|
||||
|
||||
Relevant precedents:
|
||||
|
||||
- Installed Pi `examples/extensions/qna.ts` and `summarize.ts`: independent registry completions and cancellation.
|
||||
- `extensions/lm-studio/index.ts`: dynamic local OpenAI-compatible model discovery.
|
||||
- `extensions/subagents/src/backends/pi.ts`: registry reuse and abort lifecycle handling.
|
||||
- Adjacent project `/srv/work/personal/code/eden/src/lib/server/assistant/intent.ts` and `tests/intent.test.ts`: JEV Choice routing with pinned version, response validation, confidence gating and failure-closed tests. This is a reference, not a proposed runtime dependency.
|
||||
|
||||
Browser-harness exposes Python/CLI/MCP interfaces; no existing TypeScript-native bridge was identified in this investigation. A restricted local Python bridge is the proposed adapter, with the existing MCP server as an alternative transport to evaluate. Keep the controller independent of transport so this choice can change without rewriting decision logic. Measure worker startup/IPC before committing to an additional always-running service.
|
||||
|
||||
## Safety and reliability requirements
|
||||
|
||||
- Code, not model confidence, enforces allowed domains and actions. Treat all page text as untrusted data, never instructions for the controller.
|
||||
- Stop for approval before purchases, sending messages, publishing, destructive changes, granting access, or other consequential submissions unless the exact action is already authorized.
|
||||
- Keep credentials out of model inputs. Stop for passwords, MFA, and ambiguous account selection; prefer an explicitly approved logged-in test profile.
|
||||
- Bind candidates and approval to the precise browser/tab/frame, target identity, intended action and payload. Revalidate immediately before execution.
|
||||
- Do not blindly retry non-idempotent actions after timeouts. Inspect the resulting state first: a cancelled request may already have clicked or submitted.
|
||||
- Bound steps, elapsed time, model requests, and recovery attempts. Detect repeated states/no progress, release the browser lane on stop, and surface a useful partial result.
|
||||
- Fail closed on malformed responses, missing targets, expired observations, and permission uncertainty. Semantic confidence is not a security boundary.
|
||||
- Keep the browser debug endpoint local/private. Starting a browser on a remote Pi host does not automatically attach to a desktop browser elsewhere; deployment location needs confirmation before a prototype.
|
||||
|
||||
## Benchmark before committing to the architecture
|
||||
|
||||
Use a shared local browser backend and comparable observations, test fixtures, warm/cold conditions, and safety policy. Pin model versions and prompts. No live purchases, messages, or destructive tasks in the initial suite.
|
||||
|
||||
Compare four paths:
|
||||
|
||||
1. Main Pi model chooses each browser action: conventional baseline.
|
||||
2. Small Pi model chooses from the same bounded candidate list.
|
||||
3. JEV chooses from that list; small model only generates text when needed.
|
||||
4. Deterministic/site-specific workflow where applicable: establishes whether AI adds value at all.
|
||||
|
||||
Use 30–50 task instances across navigation, result selection, conventional forms, generated text, and recovery. Repeat each several times, randomize run order, reset initial state, and report uncertainty. This is a screening study, not proof of general-web reliability. Use held-out pages/tasks for confirmation.
|
||||
|
||||
Measure:
|
||||
|
||||
- Verified task success, critical misactions, and human interventions.
|
||||
- End-to-end p50/p95 latency and time-to-verified-success, including failed runs/timeouts rather than reporting only successful fast runs.
|
||||
- Observe/filter time, payload size, API RTT, action time, DOM settling, verification, text insertion, and escalation latency separately.
|
||||
- Candidate recall, selected-action accuracy, no-progress loops, stale-target failures, and fallback frequency.
|
||||
- Main-model calls eliminated, token usage and total cost per verified successful task.
|
||||
- Confidence versus empirical action accuracy, by task type. Tune thresholds on calibration data and evaluate on held-out data; do not assume a probability cutoff guarantees browser reliability.
|
||||
|
||||
Illustrative latency budget, **not a measurement**:
|
||||
|
||||
`T_task = planning + Σ(observe + decide + act + settle + verify) + text generation + recovery`
|
||||
|
||||
Replacing a 1,500 ms decision with 200 ms when all other step work takes 500 ms changes a step from 2,000 ms to 700 ms (~2.9× faster). If the site instead spends 3,000 ms loading, the same replacement gives only ~1.4×. Retry and escalation costs can erase either gain.
|
||||
|
||||
Suggested promotion gate for discussion: roughly 2× median improvement over the main-model baseline on the target workflows, no observed critical misactions, task success comparable to baseline, and a material advantage over the small-model-only alternative. Tail latency and fallback cost must remain acceptable. These are proposed targets, not established outcomes or statistical guarantees.
|
||||
|
||||
## Suggested next experiment
|
||||
|
||||
Build only after agreeing on the architecture and deployment target:
|
||||
|
||||
1. An offline AX/DOM fixture adapter and JEV-vs-small-model action-selection benchmark. First establish candidate recall and decision quality without any side effects.
|
||||
2. A bounded local execution prototype using the same candidate schema, with literal form filling and deterministic postconditions.
|
||||
3. Add generated-text calls and supervised escalation only if the first two stages show a useful advantage.
|
||||
|
||||
Do not begin with a general autonomous browser agent, self-modifying helpers, Chrome extension, hosted browser, or custom JEV chat-provider emulation.
|
||||
|
||||
## Sources
|
||||
|
||||
- [Browser-harness README](https://github.com/browser-use/browser-harness/blob/afbcc381b963040c19627d788e40c7e7663171ee/README.md)
|
||||
- [Browser workflow and local concurrency](https://github.com/browser-use/browser-harness/blob/afbcc381b963040c19627d788e40c7e7663171ee/src/browser_harness/SKILL.md)
|
||||
- [CDP and input helpers](https://github.com/browser-use/browser-harness/blob/afbcc381b963040c19627d788e40c7e7663171ee/src/browser_harness/helpers.py)
|
||||
- [Telemetry controls](https://github.com/browser-use/browser-harness/blob/afbcc381b963040c19627d788e40c7e7663171ee/src/browser_harness/telemetry.py), [cloud autospawn](https://github.com/browser-use/browser-harness/blob/afbcc381b963040c19627d788e40c7e7663171ee/src/browser_harness/run.py), [update checks](https://github.com/browser-use/browser-harness/blob/afbcc381b963040c19627d788e40c7e7663171ee/src/browser_harness/admin.py)
|
||||
- [Typesafe workflow architecture](https://docs.typesafe.ai/concepts/how-to-build-with-system-one)
|
||||
- [JEV models, pricing, limits and data handling](https://docs.typesafe.ai/models)
|
||||
- [Text-only state contract](https://docs.typesafe.ai/concepts/state)
|
||||
- [Typesafe HTTP API](https://docs.typesafe.ai/api)
|
||||
- [JEV 1.13 documented limitations](https://docs.typesafe.ai/model-jaggedness/jev-1.13)
|
||||
- [Typesafe JavaScript SDK](https://docs.typesafe.ai/sdk/javascript)
|
||||
- Installed Pi API reference: `/home/jb/.bun/install/global/node_modules/@earendil-works/pi-coding-agent/dist/core/model-registry.d.ts`
|
||||
- Installed Pi extension documentation: `/home/jb/.bun/install/global/node_modules/@earendil-works/pi-coding-agent/docs/extensions.md`
|
||||
- Installed Pi auxiliary-model examples: `/home/jb/.bun/install/global/node_modules/@earendil-works/pi-coding-agent/examples/extensions/{qna,summarize}.ts`
|
||||
|
||||
Research inputs were cached under `/tmp/browser-efficacy/.firecrawl/`; the inspected upstream checkout is `/tmp/browser-efficacy/browser-harness/`. These temporary caches are not deliverables or durable repository dependencies.
|
||||
@@ -0,0 +1,217 @@
|
||||
# Fast-browser development contracts
|
||||
|
||||
Status: frozen implementation boundary, 2026-09-19. The TypeScript source of truth is
|
||||
`extensions/fast-browser/contracts.ts`; this document explains ownership and safety
|
||||
rules, not a second protocol.
|
||||
|
||||
## Published exports
|
||||
|
||||
`BrowserBridge` remains the prototype observe/act/close contract. New work must
|
||||
implement `BrowserRuntime`, a persistent, session-owned extension of that
|
||||
contract. The published operation names are:
|
||||
|
||||
`observe`, `act`, `read`, `assert`, `navigate`, `reload`, `listTabs`,
|
||||
`selectTab`, `waitPopup`, `closeTab`, `captureEvidence`, `narrowOriginPolicy`,
|
||||
and `close`.
|
||||
|
||||
`EvidenceRequest.operation` uses the exported
|
||||
`BROWSER_EVIDENCE_OPERATIONS`/`BrowserEvidenceOperationName` allowlist. It
|
||||
contains every supported capture label except session `close`; `closeTab`
|
||||
remains a supported distinct label.
|
||||
|
||||
`BrowserOperation` is the explicit JSONL/CLI request union. `ScenarioOperation`
|
||||
wraps the existing bounded `Step` as `{op:"step",step}` and reuses the fixed
|
||||
read/assert/control request arms. It has no JavaScript, selector, raw CDP, or
|
||||
arbitrary command arm. `FAST_BROWSER_SCENARIO_FILE_VERSION` and
|
||||
`FAST_BROWSER_REPORT_VERSION` are both `1`; scenario owners may add their
|
||||
validated setup/reset/cleanup details around this union. Scenario input is an
|
||||
explicit caller-provided file path; it is not restricted to fixture IDs.
|
||||
|
||||
Assertions are finite and read-only: URL `exact` or `origin`, title `exact` or
|
||||
`contains`, and visible-text `contains`. `wait:"bounded"` uses the fixed
|
||||
3-second read/assertion budget; `wait:"none"` performs one read. A failed or
|
||||
stale read is an operation error, never a mutation or retry. `read` returns only
|
||||
bounded rendered visible text, title, and URL metadata; it excludes input/value
|
||||
properties, password values, hidden text, HTML, frames, and script output.
|
||||
|
||||
Tab IDs are opaque runtime-issued handles. A handle remains valid across calls
|
||||
for the same target; `documentId` changes on navigation/reload. `listTabs` is
|
||||
origin-filtered and deterministic. `waitPopup` returns an owned popup handle
|
||||
with `autoSwitched:false`; only `selectTab` changes the selected target. No
|
||||
operation infers a target from page text.
|
||||
|
||||
## Scenario-only tab references
|
||||
|
||||
Runtime and transport calls still use opaque `BrowserTabId` values. JSON scenario
|
||||
files use the bounded, data-only `ScenarioTabReference` instead; the CLI resolves
|
||||
it to an opaque ID at execution time. References are either the initial selected
|
||||
tab, an earlier operation's `selectedTab`/`popup` result, or an earlier
|
||||
`listTabs` result at `ScenarioTabIndex` 0–31. Operation references may use an
|
||||
explicit prior numeric `index` (0–31) or prior unique `id`; forward, missing,
|
||||
duplicate, and out-of-range references are rejected before execution.
|
||||
|
||||
A list operation may include an exact-origin or bounded-index `select` predicate.
|
||||
An origin predicate must match exactly one listed tab. Listing never changes the
|
||||
runtime selection; a later explicit `selectTab` operation does that. Popup waits
|
||||
also never auto-switch.
|
||||
|
||||
Example:
|
||||
|
||||
```json
|
||||
{
|
||||
"version": 1,
|
||||
"id": "popup-check",
|
||||
"operations": [
|
||||
{"id":"start","op":"read","request":{"tab":{"source":"initial","target":"selected"}}},
|
||||
{"id":"tabs","op":"listTabs","request":{"select":{"kind":"origin","origin":"http://127.0.0.1:4310"}}},
|
||||
{"id":"selected","op":"selectTab","request":{"tab":{"source":"operation","operation":{"id":"tabs"},"result":"listedTab","index":0}}},
|
||||
{"id":"popup","op":"waitPopup","request":{"opener":{"source":"operation","operation":{"id":"selected"},"result":"selectedTab"}}}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
`selectedTab` is produced by `selectTab`, `popup` by `waitPopup`, and `listedTab`
|
||||
by `listTabs`; there is no interpolation, templating, selector, or script arm.
|
||||
|
||||
## Scenario fixture/server lifecycle
|
||||
|
||||
`ScenarioDefinition.fixture` is optional. When present, `baseUrl` is an explicit
|
||||
credential-free HTTP(S) URL and `server` is either:
|
||||
|
||||
- `{"mode":"managed","start":...,"readiness":...}`: the CLI starts one
|
||||
long-lived child with trusted argv (`shell:false`), optional cwd, and existing
|
||||
env-reference shape, then owns that process until the scenario `finally` path;
|
||||
- `{"mode":"external","readiness":...}`: no child is started. The caller
|
||||
supplies the explicit already-running readiness URL.
|
||||
|
||||
`ScenarioHttpReadiness` is GET-only and contains only `url`, optional expected
|
||||
status, and a bounded timeout. It has no headers, body, shell, or page-derived
|
||||
command fields. Setup/reset/cleanup hooks remain one-shot hooks and are distinct
|
||||
from the managed server child. The CLI's existing scenario-wide
|
||||
`budgets.timeoutMs` starts before any hook or server startup; no second global
|
||||
deadline knob is added. The runner polls readiness within the published
|
||||
`FAST_BROWSER_SCENARIO_SERVER_BOUNDS`, keeps the managed child owned through all
|
||||
operations, and performs bounded shutdown in `finally`. Existing definitions
|
||||
without a managed server remain valid when the CLI/user config provides an
|
||||
explicit external readiness URL.
|
||||
|
||||
Example execution order:
|
||||
|
||||
```json
|
||||
{
|
||||
"version": 1,
|
||||
"id": "fixture-login",
|
||||
"fixture": {
|
||||
"baseUrl": "http://127.0.0.1:4310",
|
||||
"server": {
|
||||
"mode": "managed",
|
||||
"start": {"command":"npm","args":["run","fixture"]},
|
||||
"readiness": {"url":"http://127.0.0.1:4310/health","expectedStatus":200}
|
||||
}
|
||||
},
|
||||
"operations": [{"op":"step","step":{"instruction":"Continue","action":"click","postcondition":{"kind":"text","value":"Done"}}}]
|
||||
}
|
||||
```
|
||||
|
||||
Order is: start the existing global scenario budget → setup hook → managed
|
||||
server start and bounded GET readiness (or external readiness) → reset hook →
|
||||
browser operations → finally close browser/runtime, run cleanup while the owned
|
||||
server is still available, bounded server stop, and close uncommitted artifact
|
||||
resources while retaining committed artifacts. A readiness failure stops before
|
||||
browser operations and never becomes a page-derived command.
|
||||
|
||||
## Policy, budgets, and errors
|
||||
|
||||
`RuntimeOriginPolicy` preserves the current `"*"` default or a non-empty exact
|
||||
origin list. `narrowOriginPolicy` computes a `RuntimeOperationContext` for an
|
||||
effective request, constrained by the immutable session base; it never
|
||||
irreversibly tightens later calls. Pass that context to the scoped calls, or
|
||||
omit it on a later call to restore the session base. A restricted base never
|
||||
widens. Navigation accepts only bounded HTTP(S) URLs without credentials whose
|
||||
origin is approved. A runtime owns one session's CDP/worker state and
|
||||
serializes its calls; distinct sessions may run in parallel. Its lifetime is
|
||||
the browser-session lifetime, not the first tool call. Cancellation must stop
|
||||
the worker/daemon and release no resource until teardown is confirmed. Close is
|
||||
idempotent and best effort.
|
||||
|
||||
`FAST_BROWSER_BUDGETS` publishes defaults and hard ceilings for startup (10s),
|
||||
worker readiness (5s), bridge request (10s/120s ceiling for the existing
|
||||
compatibility seam), action verification (3s), read/assertion wait (3s),
|
||||
navigation/reload (10s), popup wait (5s), screenshot (10s/60s ceiling), failure
|
||||
evidence (2s), and a run (60s/120s). Do not add a generic unbounded timeout
|
||||
options object. Failure capture is bounded and best effort; it must never mask
|
||||
the original operation result or retry a mutation.
|
||||
|
||||
All safe failures use `BrowserErrorDetails`/`BrowserOperationError`, with an
|
||||
operation, phase, safe message, error code, retry classification, and mandatory
|
||||
`mutationState` (`not_attempted`, `verified`, or `unknown`). Raw exceptions are
|
||||
not reportable.
|
||||
|
||||
## Transport boundary
|
||||
|
||||
`BrowserTransport` is the typed JSONL boundary. Its exact API is
|
||||
`dispatch<Operation extends BrowserOperation>(request: BrowserTransportRequest<Operation>,
|
||||
signal): Promise<BrowserTransportResultFor<Operation>>` and
|
||||
`close(): Promise<void>`. `BrowserTransportRequest` contains one
|
||||
`BrowserOperation` plus an optional scoped `RuntimeOperationContext`;
|
||||
`BrowserTransportResultFor<Operation>` is the exact result mapping for every
|
||||
`BrowserOperation` variant, and `BrowserTransportResult` is its complete union.
|
||||
Evidence transport uses `BrowserTransportArtifactPayload`: JSONL
|
||||
`encoding:"base64"`, `dataBase64`, declared `byteLength`, content type, kind,
|
||||
and truncation flag. The runtime decodes those bytes and writes them to the
|
||||
parent `PrivateArtifactStore`; no caller path crosses into the worker.
|
||||
`FAST_BROWSER_ARTIFACT_LIMITS` caps screenshots at 8 MiB and diagnostics at
|
||||
256 KiB. `BrowserTransportFactory.create({cdpUrl, allowedOrigins, python?},
|
||||
parentSignal?)` owns worker framing, validation, deadlines, and teardown.
|
||||
`BrowserRuntime` owns persistent tab/policy state and delegates to this
|
||||
transport; these layers must not both implement the JSONL protocol.
|
||||
|
||||
`BrowserRuntimeOptions.python` and `BrowserTransportOptions.python` are the
|
||||
explicit interpreter-selection seam for the pinned environment. If omitted,
|
||||
the existing configured/default bridge resolution remains responsible for
|
||||
choosing Python; no worker may silently drop to a user-global environment when
|
||||
an explicit interpreter/factory is supplied.
|
||||
|
||||
## Evidence and artifact ownership
|
||||
|
||||
`PrivateArtifactStore.open(owner, kind)` is the only write API. Evidence
|
||||
requests carry the caller/scenario `runId`; the runtime creates the operation
|
||||
identity. There is no caller-selected path. The session-owned store creates
|
||||
private directories and atomically commits `0600` files under its private root,
|
||||
conceptually `<root>/<sessionId>/<runId>/<artifactId>.(png|json)`. The returned
|
||||
`PrivateArtifactMetadata` records owner, path, byte count, content type, and
|
||||
`localOnly:true/modelForwarded:false`. Runtime close closes the store. Screenshots
|
||||
are local failure/request artifacts and are never automatically sent to an LLM
|
||||
or provider.
|
||||
|
||||
`captureEvidence` may return a partial/unavailable bundle. Console summaries are
|
||||
bounded level/message/count records. The pinned browser-harness 0.1.13 source
|
||||
supports `drain_events()` and enables Page/DOM/Runtime/Network on tab attach;
|
||||
implementers may consume buffered `Runtime.consoleAPICalled` and
|
||||
`Network.loadingFailed` events only if present. Do not invent a helper method or
|
||||
claim complete console history. Network summaries contain method, origin-only
|
||||
URL, status/category—never query/fragment tokens, headers, cookies, or bodies.
|
||||
Console text can still contain arbitrary page content; local/private and bounded
|
||||
is not the same as secret-free, so public shaping must redact it.
|
||||
|
||||
## File ownership / update order
|
||||
|
||||
- `contracts.ts` — shared API owner (this change); additions must remain typed.
|
||||
- `bridge.ts` — later JSONL transport validation and request deadlines; no new
|
||||
public operation names without updating `contracts.ts` first.
|
||||
- `browser_worker.py` — later pinned browser-harness adapter and persistent
|
||||
target/diagnostic state; use only observed harness capabilities.
|
||||
- `browser-session.ts` — browser process/profile/CDP ownership and parallel
|
||||
session isolation; it does not own tab protocol state.
|
||||
- `controller.ts` — later bounded action orchestration and automatic
|
||||
failure-evidence call; preserve stale/no-blind-retry mutation semantics.
|
||||
- `index.ts` — later public redaction, tool/CLI wiring, and origin narrowing;
|
||||
never expose private artifact contents or raw URL paths.
|
||||
- `fb-scenarios-ci` — scenario file parsing, trusted argv/config hooks, CLI,
|
||||
reports, and CI examples. Composition requires an explicitly supplied model
|
||||
adapter and returns a capability error when absent; never silently selects a
|
||||
provider.
|
||||
|
||||
Existing test mocks that implement only `BrowserBridge` need no compatibility
|
||||
aliases; runtime mocks must implement every `BrowserRuntime` method. The
|
||||
scenario worker should import the exports above rather than define alternatives.
|
||||
@@ -0,0 +1,141 @@
|
||||
# Fast-browser development-readiness backlog
|
||||
|
||||
Status: **validated native-Linux acceptance complete; final offline recheck passed** (2026-09-20). This backlog records the validated scope, not a claim of non-Linux or universal browser readiness. The original M1 visible-cleanup run failed closed and is preserved as historical evidence; the validator later completed the root-first cookie experiment, immediate-popup flow, full M1/development matrix, managed CLI scenarios, assertion path, and latest deadline gate with cleanup and empty process/port census. Linux-only supervision and non-atomic PID identity proof remain explicit constraints; fail closed when ownership is uncertain and do not claim SIGTERM guarantees universal all-app durability. The current index implements the compact eight-tool Pi catalogue and typed `fast_browser_control` payload. Use only the handed-off CLI commands and do not infer behavior beyond the current schemas.
|
||||
|
||||
## Final native-Linux validation checkpoint (2026-09-20)
|
||||
|
||||
Integrated acceptance passed for the validated native-Linux scope. Typecheck passed. Full-repository offline validation: **805 total, 803 passed, 2 skipped**. Fast-browser TypeScript: **222 total, 220 passed, 2 skipped**. Pinned Python 3.12.13: **42/42 passed**. Packaging: **4/4**, links **8/8**, package contents **36 entries**, and tracked/untracked whitespace checks clean. All live gates passed, including the latest deadline rerun: it naturally exited `1` after 3090 ms with `retry=never` and `mutationState=not_attempted`, emitted explicit private unavailable evidence, passed cleanup, and left empty process/port censuses. The deadline report is `/tmp/fast-browser-deadline-rerun-iaE8de/reports/overall-deadline-no-popup-rerun.json`; the older failed report remains preserved.
|
||||
|
||||
`imports:check` was **not run** because it requires private network access. Remote GitHub CI was **not run**; automatic CI remains configured, and equivalent managed-CLI live validation passed locally. These are scope caveats, not failures of the native-Linux acceptance gate. Non-Linux supervision remains unsupported, PID identity proof is non-atomic, and no universal SIGTERM flush/all-app durability guarantee is claimed.
|
||||
|
||||
## Baseline and guardrails
|
||||
|
||||
Observed reusable behavior includes `fast_browser_open`/`close` session ownership, exclusive persistent named-profile locks, disposable launch profiles, explicit headless launch, visible manual setup without CDP, per-session serialization, origin `"*"` as the default, autosaved global/project settings with scope boundaries, stale observation rejection, exact fill verification, click URL/visible-text postconditions, bounded worker startup/action polling, bridge request deadlines, and unknown-mutation/no-blind-retry reporting. This inventory comes from `extensions/fast-browser/browser-session.ts`/`profiles.ts`, `browser_worker.py`/`bridge.ts`, `controller.ts`, `contracts.ts`, and their offline tests; `fixtures/form.html` and `fixtures/login.html` are the current owned pages. Existing waits are concrete: browser spawn/readiness 10s, worker daemon readiness 5s, bridge requests 10s, worker action verification 3s, and controller runs 60s by default/120s maximum. The readiness work must extend those boundaries rather than replace working waits or weaken them.
|
||||
|
||||
Every live check in this backlog uses an owned local HTTP fixture and a unique run-scoped test profile (for example, a temporary `XDG_DATA_HOME` plus `fb-readiness-<run>-<worker>` names). It must use per-call/project-temporary settings, never edit global configuration, never use the active-user `harness-test` profile, and never log in to EVE or any other real service. Credential tests use only the disposable fixture account and environment/file references; values must not appear in traces, evidence, reports, or model requests. Cleanup is required on pass, failure, cancellation, and process interruption.
|
||||
|
||||
## Architecture checkpoint (scout findings)
|
||||
|
||||
The architecture scout traced `index.ts` → `browser-session.ts` → `browser_worker.py` → `controller.ts`, with configuration in `config.ts`, leases in `profiles.ts`, IPC in `bridge.ts`, and public result shaping in `index.ts`. Final contracts are published in `extensions/fast-browser/contracts.ts` with the companion `docs/fast-browser-development-contracts.md`; scoped typecheck passed. This closes the contract-publication checkpoint; M1–M5 implementation and the integrated native-Linux live acceptance gate are complete, with the final offline repository recheck now passed. The historical scout snapshot recorded `Observation`, `Postcondition`, `StepTrace`, and `RunResult`, an exactly-one-tab worker, and no public navigation/tab/popup/evidence API; the current SDK/runtime handoff has since added bounded session controls and evidence plumbing. The scenario source/CLI handoff and retention/unsupported-`compose` preflight are complete; Task 14 report/formatter/cleanup/session/artifact-redaction behavior passed the integrated native-Linux gate. These historical and current facts are not proposed APIs.
|
||||
|
||||
The main task assigned `fb-development-contracts`, which published the SDK/runtime contracts before parallel implementation. The remaining review must preserve readiness phase diagnostics (spawn, DevTools, worker, eligible-tab selection, origin approval, page/action readiness), same- versus cross-document navigation semantics, and popup/new-tab ownership. New read/assert/control/evidence features use the session-scoped runtime because the former per-operation worker lifecycle did not retain tab handles or console history. Review must preserve bounded post-close exit diagnostics and scenario/report privacy without exposing full URLs, page content, field values, authentication headers, request bodies, or provider responses. The current runtime index implements the compact eight-tool Pi catalogue and its discriminated control union; no public origin-narrowing tool exists because `settings.allowedOrigins` is per-call. Task 13 runtime cleanup/navigation/lane/evidence fixes passed the integrated native-Linux gate; the shared `BROWSER_EVIDENCE_OPERATIONS` contract is published. Task 14 CLI report/formatter/cleanup/session/artifact-redaction fixes and the managed-fixture examples passed the integrated gate; the final rerun carries real deadline metadata.
|
||||
|
||||
## Known failures to carry forward
|
||||
|
||||
- The successful historical live demo used the old attach-only, approval-based contract; it remains historical context and is not evidence for the new unattended launcher. The current native-Linux launcher is covered by the completed live gate.
|
||||
- Two managed visible Chromium launches crashed with SIGSEGV under an older flag/environment combination. A minimal user-data/CDP/no-first-run baseline worked in a diagnostic, but the integrated launcher and the cause of the old crash are not validated.
|
||||
- A browser crash was historically reported only as `Unknown browser session`; startup/worker paths had generic or empty-looking errors, including a no-approved-tab condition. The worker now has bounded sanitized startup IPC/origin diagnostics, and recent-exit diagnostics are implemented/unit-tested with safe known-closed-ID reasons; the native-Linux live gate passed.
|
||||
- The historical compatibility worker required exactly one eligible web tab and had no navigation, reload, popup, screenshot, console, or failed-request evidence surface. The current SDK/runtime handoff adds bounded session controls and evidence plumbing; the compact public wiring and final remediation passed the native-Linux gate. These historical gaps are not reasons to invent unbounded waits.
|
||||
- The historical auxiliary-model failure (unsupported Codex Spark entitlement) is an environment/provider issue, not browser reliability evidence; live readiness defaults to exact-name/literal fixture checks.
|
||||
|
||||
## Early-review acceptance status (historical checkpoints; 2026-09-20)
|
||||
|
||||
These historical code/test handoffs preceded the final native-Linux acceptance gate; their counts remain dated checkpoint evidence, not moving completion totals:
|
||||
|
||||
- `fb-transport-evidence` delivered wire/details/parent-abort and artifact-store fixes in a 22m31 handoff; 19 targeted TypeScript tests plus 35 Python tests pass.
|
||||
- `fb-runtime-tools` delivered the runtime factory/tools/controller handoff in 4m39; 48 targeted tests plus bridge coverage pass.
|
||||
- The typed wire mismatch, structured-error parser loss, dropped parent signal, and post-dispatch `not_attempted` mutation-state findings are implemented and unit-tested.
|
||||
- Bounded opaque-tab references are implemented with static validation/resolution, closed/missing checks, popup JSON coverage, 20 focused tests, clean typecheck, and passing `--check` examples. The CLI handoff now provides `node --experimental-strip-types extensions/fast-browser/scenario-cli.ts --check --format json <scenario.json>` and `bun run fast-browser:scenario -- --format human --report ./report.json <scenario.json>`. Unsupported-`compose` preflight now runs before hooks/browser startup in both `--check` and execution; Task 14 report fixes have an implementation checkpoint with 24 tests (not final). Defensive human/JSON formatters, cleanup/session handling, artifact metadata redaction, usable retained paths, explicit atomic `0600` reports in owned `0755` parents, private artifact roots, and post-cleanup retention are covered. Managed CLI implementation is complete with three `--check` examples; all three managed CLI examples and the intentional failed-assert path have REAL pass/expected-failure proofs, automatic managed-fixture CI is configured, and the final rerun includes real deadline metadata.
|
||||
- The artifact-root chmod issue is fixed with private owned roots and tests; independent review found no remaining chmod/shared-parent bypass. The separate provider-reliability TODO remains open.
|
||||
- Consolidated lifecycle status: the original M1 visible-cleanup run failed closed and preserved `/tmp/fast-browser-live-validation-5HvmBr`; the validator safely terminated all 11 verified owned processes, left no target-profile processes, and preserved `evidence/owned-process-cleanup.json` as historical evidence. PID/helper safety independently cleared; the root-first cookie experiment, full M1/development matrix, and latest deadline gate passed with complete cleanup and empty process/port census. Supervision is Linux-only; PID identity proof is non-atomic, so ownership uncertainty must fail closed. This evidence does not claim SIGTERM guarantees universal all-app durability.
|
||||
- Task 12 Python evidence privacy is complete: independent reproduction and six real-development cases pass; no buffered-event blocker remains, and exact historical/live results remain owned by the validator documentation. Timeout-evidence producer and CLI privacy are independently cleared with explicit unavailable/partial metadata, bounded independent capture, and no reopen/retry. Task 13 runtime cleanup/navigation/lane/evidence fixes and the public URL fix passed the integrated native-Linux gate; the shared `BROWSER_EVIDENCE_OPERATIONS` contract is published. The immediate-popup fix passed independent review and the REAL immediate flow without a delay workaround. Task 14 CLI report fixes passed the integrated gate. Do not treat these native-Linux results as non-Linux or remote-CI claims.
|
||||
|
||||
|
||||
## Root widget delivery sequence
|
||||
|
||||
These are the user-approved work items; all five feature milestones passed the integrated native-Linux LIVE gate. Task 1 is the published contract/documentation checkpoint; Tasks 2–6 have live acceptance evidence. The final offline repository recheck also passed; remote GitHub CI was not run and remains an explicitly unclaimed execution path.
|
||||
|
||||
1. **Docs/contracts — published checkpoint (`fb-development-contracts`):** final contracts/backlog are published and scoped typecheck passed.
|
||||
2. **Live profiles/headless/parallel validation — complete for validated native Linux (`fb-live-baseline`):** PID/helper safety, root-first cookie persistence, full M1/development coverage, latest deadline behavior, cleanup, and empty process/port census passed. Retain the historical failed-cleanup evidence; Linux-only supervision and non-atomic PID identity proof remain constraints.
|
||||
3. **Bounded page reading/assertions — complete for validated native Linux (`fb-python-development`).**
|
||||
4. **Failure evidence/diagnostics — complete for validated native Linux (`fb-transport-evidence`).**
|
||||
5. **Navigation, reload, tabs, and popups — complete for validated native Linux (`fb-runtime-tools`).**
|
||||
6. **Declarative scenarios, fixture resets, reports, and headless CI — complete for validated native Linux (`fb-scenarios-ci`).**
|
||||
7. **Independent review and final offline check — complete:** runtime/public-URL review, validator freeze, full live matrix, and final offline repository recheck passed for the validated native-Linux scope.
|
||||
8. **M1 acceptance closeout — complete for validated native Linux:** historical failures remain preserved and non-Linux/universal durability claims remain out of scope.
|
||||
13. **Final TypeScript/runtime remediation review — complete for validated native Linux:** registry, abort/error-shaping, schema-alignment, navigation, lane, evidence, and public-URL fixes passed; exact historical validation results remain owned by the validator documentation.
|
||||
14. **CLI report/formatter remediation — complete for validated native Linux:** defensive human/JSON formatting, cleanup/session/artifact metadata redaction, retained paths, report permissions/retention, managed scenarios, and expected-failure behavior passed; remote GitHub CI execution was not run.
|
||||
|
||||
## Prioritized milestones
|
||||
|
||||
### M1 — real profiles, headless, and parallel live validation (**complete for validated native Linux**)
|
||||
|
||||
**Goal:** prove the new session manager against real installed browser processes, not only fake children.
|
||||
|
||||
**Acceptance criteria**
|
||||
- Two simultaneous sessions use distinct unique named profiles, processes, CDP ports, and browser state. Closing or crashing one cannot close, unlock, or alter the other.
|
||||
- A named profile can be closed and reopened for the same local fixture without losing its test marker; profile locks reject contention and stale DevTools metadata is handled only after ownership is acquired.
|
||||
- At least one installed Chromium-family browser passes a headless login-fixture smoke test and a visible diagnostic smoke test. Vivaldi is tested when installed and never silently substituted; unavailable products produce an explicit skip/evidence record.
|
||||
- The check exercises literal/exact-name steps by default, so the readiness run needs no provider call. It records browser/version, profile/port ownership, cleanup, and any exit reason without credentials.
|
||||
|
||||
**Current live evidence:** `docs/fast-browser-live-validation.md` is validator-owned and preserves the exact historical results, including the failed original visible-cleanup run at `/tmp/fast-browser-live-validation-5HvmBr` and the report of 10 owned Chrome processes. The validator subsequently safely terminated all 11 verified owned processes, left no target-profile processes, and preserved `evidence/owned-process-cleanup.json`. The root-first cookie experiment, full M1/development matrix, managed scenarios, assertion path, and latest deadline rerun passed with cleanup and empty process/port census. The latest deadline rerun naturally exited `1` after 3090 ms with `retry=never` and `mutationState=not_attempted`, emitted explicit private unavailable evidence, and is not a live-gate failure. Its report is `/tmp/fast-browser-deadline-rerun-iaE8de/reports/overall-deadline-no-popup-rerun.json`; the older failed report remains preserved. Linux-only supervision and non-atomic PID identity proof remain constraints, and no SIGTERM all-app durability guarantee is claimed.
|
||||
|
||||
**Tests/dependencies:** retain existing `browser-session`, `profiles`, bridge, and index tests. The owned live-validation workflow and final offline repository recheck passed for native Linux; preserve the historical failed-cleanup evidence and require complete cleanup on future runs.
|
||||
|
||||
### M2 — bounded visible-page reads and assertions (**complete for validated native Linux — `fb-python-development`**)
|
||||
|
||||
**Goal:** add a read-only, caller-bounded way to inspect/assert visible fixture state without turning page text into instructions.
|
||||
|
||||
**Published contract:** the session runtime exposes typed `read` and `assert` operations with bounded visible text, URL/title/visible-text assertions, and session-scoped target identity. Implementation must preserve limits for characters, nodes, elapsed wait, origin, and result shape; reuse the existing typed bridge and 3-second postcondition/read-assert wait model rather than inventing another. Do not expose arbitrary CDP, JavaScript, DOM handles, hidden text, current input/password values, or page HTML.
|
||||
|
||||
**Acceptance criteria**
|
||||
- Reads return only bounded visible text/metadata from the approved main page and redact/truncate deterministically. Hidden, disabled, password-value, iframe, and rich-control content cannot leak through the result.
|
||||
- Assertions support at least the fixture's visible text and URL-origin outcomes, fail closed on ambiguity/staleness, and distinguish unmet from timed-out/unknown outcomes. Existing action postcondition polling and run/request deadlines are reused and documented, not duplicated.
|
||||
- Offline fake-backend tests cover visibility, limits, navigation/frame identity changes, cancellation, and no credential/page-value leakage; local live tests prove the same results on owned fixtures.
|
||||
|
||||
**Dependencies:** M1 session/fixture runner and the published runtime contract.
|
||||
|
||||
### M3 — evidence and crash/exit diagnostics (**complete for validated native Linux — `fb-transport-evidence`**)
|
||||
|
||||
**Goal:** make failures actionable without leaking page data.
|
||||
|
||||
**Acceptance criteria**
|
||||
- A bounded failure evidence bundle is captured automatically on failure and can also be requested explicitly. Local-only screenshot artifacts are the default; the bundle may also include sanitized console errors and sanitized failed-request summaries, tagged to session/step/phase. Successful runs do not capture evidence unless requested.
|
||||
- Evidence stays in bounded, local, private artifacts with explicit size, retention, and permissions. Structured console/request evidence never includes authentication headers, request bodies, credential/field values, query/fragment tokens, or unbounded page text. A screenshot is rendered fixture pixels, so it remains local/private and bounded; it is never automatically forwarded to a model or provider.
|
||||
- Startup/worker failures distinguish no eligible tab, ambiguous tabs, origin rejection, missing/mismatched harness, timeout, and browser exit. Recent-exit diagnostics now safely classify known closed IDs with a 30-second TTL and 32-entry cap, omit stderr/URLs/arbitrary errors, return unknown for expired IDs, and expose no public history API; this is implemented/unit-tested but still needs live validation.
|
||||
- Screenshot/console/request capture is failure evidence, not an autonomous vision/control surface. Evidence creation never retries an action or changes the browser state.
|
||||
|
||||
**Current M3 acceptance status:** Task 12 Python evidence privacy and the timeout-evidence producer/CLI privacy review passed. Explicit unavailable/partial metadata, bounded independent capture, and no reopen/retry were validated. No buffered-event blocker remains; exact historical/live evidence is maintained in the validator-owned documentation. Recent-exit diagnostics remain implemented/unit-tested with safe bounded output; the native-Linux live gate and final offline repository recheck passed.
|
||||
|
||||
**Tests/dependencies:** contract tests for redaction/size/permissions and crash races; induced local-fixture console/request failures; controlled browser-exit diagnostics; and M1's real session IDs passed within the validated native-Linux gate. Keep generic extension-facing errors safe while retaining the worker's already-bounded startup detail.
|
||||
|
||||
### M4 — navigation, reload, tab selection, and popups (**complete for validated native Linux — `fb-runtime-tools`**)
|
||||
|
||||
**Goal:** replace the current exactly-one-eligible-tab limitation with explicit, caller-authorized lifecycle controls.
|
||||
|
||||
**Published SDK/runtime and Pi contract:** typed `navigate`, `reload`, `listTabs`, `selectTab`, `waitPopup`, and `closeTab` methods use opaque session-scoped tab identities; popup waits never auto-select. The public `fast_browser_control` payload is a discriminated `operation` union: `navigate`/`reload` carry optional `tabId` (and `navigate` carries `url`), `listTabs` carries no request, `selectTab` carries `tabId`, `waitPopup` carries `openerTabId`, and `closeTab` carries `tabId`; all are wrapped with `sessionId` and optional `settings`. Per-call `settings.allowedOrigins` narrows policy; there is no public narrow-origin tool. Implementation/review must preserve selection lifetime, document identity, bounded completion/failure semantics, popup ownership, and explicit target choice. A model must not infer or invent a target from page text.
|
||||
|
||||
**Acceptance criteria**
|
||||
- Navigation accepts only bounded HTTP(S) destinations allowed by the session policy; reload has a bounded completion result; both invalidate observations and report redirects/out-of-origin transitions safely.
|
||||
- Tab listing/selection is deterministic and origin-filtered. Multiple eligible tabs remain an error until the caller selects one; no arbitrary tab or personal browser tab is chosen.
|
||||
- A fixture-created popup/new tab can be listed, selected, asserted, and closed only when owned by the session. Closing a session leaves externally attached browser processes untouched. The immediate-popup fix has passed independent review and a REAL immediate flow; no delay workaround is part of the contract.
|
||||
- Offline tests cover stale tab/document identities, concurrent session lanes, popup races, cancellation, and no cross-profile/tab leakage; live tests use only local fixtures.
|
||||
|
||||
**Dependencies:** M1 session ownership, M2 assertion/read result limits, and M3 diagnostics for failed navigation/popup startup.
|
||||
|
||||
### M5 — reusable scenarios, fixture/reset hooks, reports, and headless CI (**complete for validated native Linux — `fb-scenarios-ci`**)
|
||||
|
||||
**Goal:** turn the checks above into repeatable regression evidence.
|
||||
|
||||
**Current M5 status and acceptance boundary:** the user-approved scenario contract includes a managed devserver with bounded HTTP readiness in JSON scenarios; an external-server option remains supported. The managed CLI examples and intentional failed-assert scenario have REAL pass/expected-failure proofs; the final deadline rerun includes real deadline metadata, explicit private unavailable evidence, no retry, and cleanup pass. Retention, unsupported-`compose` preflight, report/formatter/cleanup/session/redaction behavior, and the integrated native-Linux acceptance gate passed. Automatic managed-fixture CI is configured, but remote GitHub CI execution was not run and is not claimed.
|
||||
|
||||
**Acceptance criteria**
|
||||
- A small local fixture suite covers form/login, navigation/reload, a popup, delayed visible state, console error, and controlled failed request. Each scenario has explicit setup, bounded steps/assertions, reset, and cleanup hooks; reset is test-only and never deletes a user-managed named profile.
|
||||
- JSON scenarios may provision an owned managed devserver and wait for bounded HTTP readiness; an explicitly configured external server remains supported. Managed CLI implementation, helper integration, automatic CI configuration, and the listed REAL proofs passed within the validated native-Linux scope.
|
||||
- Reusable tests are declarative JSON scenario definitions supplied through explicit user-provided scenario file paths and executed by a CLI runner, with schema validation before launch. The runner can execute one scenario or a parallel matrix with unique profile/port/temporary-data names in visible and headless modes. It is network-free except for the owned local fixture and never requires provider credentials.
|
||||
- Reports have human-readable and machine-readable forms with scenario/status/timings, browser/profile ownership, evidence references, failure phase, and cleanup outcome—but no credential values, field values, raw URLs with secrets, or unbounded page text.
|
||||
- CI is configured to run deterministic offline tests and headless JSON-scenario CLI workflows for relevant fast-browser PR, push, dependency, and workflow changes, with a manual trigger as well; this validation did not execute remote GitHub CI. Missing optional Vivaldi is reported as a skip, not a fallback. Firefox remains deferred.
|
||||
|
||||
**Dependencies:** M1–M4 contracts and evidence, fixture server/reset design, the available CLI commands/hooks/reports source, automatic CI workflows with isolated temporary directories that execute the real headless JSON-scenario runner against the managed fixture, and the final native-Linux acceptance evidence. All five milestones are complete for the validated scope; remote GitHub CI execution remains unclaimed.
|
||||
|
||||
## Explicit non-goals and deferred scope
|
||||
|
||||
Firefox needs a separate backend and remains deferred; reject it honestly rather than substituting Chromium. Rich-text/contenteditable controls, iframe traversal, uploads/downloads, arbitrary JavaScript/CDP, remote browsers, screenshots as vision input, broad autonomous planning, blind retries, and production/real-EVE workflows are out of scope unless a later approved milestone specifically requires them. Parallel browser processes do not isolate server-side accounts. No speed or arbitrary-site reliability claim is made by this backlog.
|
||||
|
||||
## First milestone: actionable tasks
|
||||
|
||||
1. Reload the extension in a clean process and run the existing opt-in local fixture preflight with a temporary config/data root.
|
||||
2. Add the real-browser runner's unique profile/port allocator, process/version preflight, signal-safe cleanup trap, and machine-readable pass/fail/skip record.
|
||||
3. Run two named-profile sessions concurrently (headless plus visible where available) against the local login fixture using only exact-name literal/fixture credentials.
|
||||
4. Reopen one profile, verify its local test marker, close it, and prove the other session remains usable; capture bounded exit/cleanup diagnostics for any failure.
|
||||
5. Record browser availability, commands, sanitized errors, and evidence in the readiness report; do not edit global config or use active-user/real-service profiles.
|
||||
@@ -0,0 +1,48 @@
|
||||
# Fast Browser live demo
|
||||
|
||||
The real Pi `fast_browser_observe` and `fast_browser_run` tools completed the harmless local form demo, using Typesafe JEV for semantic control selection and `openai-codex/gpt-5.6-luna` for greeting composition. The auxiliary Codex request explicitly used `reasoningEffort: none`. Every mutation received actual Pi UI confirmation.
|
||||
|
||||
## Environment
|
||||
|
||||
- Dedicated Chrome profile, local CDP endpoint `http://127.0.0.1:9227`.
|
||||
- Local fixture `http://127.0.0.1:8767/form.html`; only that origin allowed.
|
||||
- Separate project-local Python environment with browser-harness0.1.13.
|
||||
- No hosted browser service. JEV and Pi-model inference used remote APIs.
|
||||
|
||||
## First attempt and recovery
|
||||
|
||||
JEV selected the display-name input and the approved literal `Ada` fill was verified. JEV then selected the greeting field, but Codex Spark rejected generation because that model was unsupported with the current ChatGPT-account authentication. No greeting fill or Continue click occurred in that failed attempt.
|
||||
|
||||
The user explicitly selected GPT5.6 Luna with reasoning disabled. The launcher and auxiliary-call reasoning policy were updated; the remaining two steps were resumed without repeating the name fill.
|
||||
|
||||
## Successful remaining two-step run
|
||||
|
||||
| Stage | Greeting fill | Continue click |
|
||||
| --- | ---: | ---: |
|
||||
| Observation | 148 ms | 11 ms |
|
||||
| JEV decision | 374 ms | 207 ms |
|
||||
| JEV confidence | 0.90 | 0.86 |
|
||||
| Luna text generation | 2,042 ms | — |
|
||||
| User approval | 1,594 ms | 1,692 ms |
|
||||
| Action and verification | 11 ms | 8 ms |
|
||||
|
||||
Total elapsed for these **remaining two steps**: **6,087 ms**, including **3,286 ms** of human approval time. Excluding approval wait gives approximately2,801ms, but this is not a separate autonomous benchmark. It excludes the earlier name fill, failed Spark attempt, setup and diagnostics.
|
||||
|
||||
Results:
|
||||
- Generated greeting field value verified.
|
||||
- Continue clicked once, then previously absent visible text `Fixture complete` verified.
|
||||
- Both mutations reported `mutationState: verified`; no uncertain action outcome remained.
|
||||
- No blind retries or silent model substitution.
|
||||
|
||||
The generated greeting was displayed for user approval and in the dedicated browser. This report intentionally does not duplicate field contents.
|
||||
|
||||
## Conclusions and next work
|
||||
|
||||
This validates the end-to-end integration: local observation, constrained JEV selection, separate Pi-model composition, exact-action approval, local execution and deterministic postcondition checks. It does not establish reliability on arbitrary websites, population-level confidence calibration or a speed advantage over another agent.
|
||||
|
||||
Next priorities:
|
||||
1. Replace environment-only non-secret settings with project-local/runtime configuration so model changes do not require restarting Pi.
|
||||
2. Compare JEV and a small-model-only selector on identical candidate sets and tasks.
|
||||
3. Measure repeated-run success, p50/p95 latency, startup/observation overhead and fallback cost, separating human approval waits.
|
||||
|
||||
Demo setup and owned resource IDs are in ignored `.pi/fast-browser-demo/README.md`; ask the user before closing the dedicated browser.
|
||||
@@ -0,0 +1,125 @@
|
||||
# Fast-browser live validation (milestone 1)
|
||||
|
||||
`extensions/fast-browser/live-validation.ts` is an opt-in, local-only integration harness. It starts an HTTP fixture on a random loopback port, creates a unique temporary profile root and CDP lease root, and uses only the typed `BrowserSessionManager`, `BrowserBridge`, and `runSteps` APIs. It never loads normal user profiles, reads existing manual sessions, contacts JEV/provider APIs, or uses external accounts. Literal accessible names are required; the injected decider throws if a semantic decision is attempted.
|
||||
|
||||
The fixture's `/setup` response sets a synthetic, run-unique persistent cookie (`Max-Age=3600`). Its page makes a bounded `/ready?case=setup` request; the server resolves readiness only after receiving that cookie, so this is evidence that Chromium processed the seed response—not a manual credential login. The visible setup launch is asserted to have no CDP or headless flags. Automation, when unblocked, reopens the same named profile and verifies retention with a visible postcondition after a literal fill and click. Headless launch, two concurrent isolated named profiles, same-profile contention, closing one while the other remains alive, and cleanup are covered independently.
|
||||
|
||||
## Integration command
|
||||
|
||||
From the repository root, with the existing pinned environment:
|
||||
|
||||
```bash
|
||||
node --experimental-strip-types extensions/fast-browser/live-validation.ts
|
||||
```
|
||||
|
||||
The focused test is opt-in and does not start a browser during ordinary test runs. It currently fails honestly on the visible-close blocker above until the production lifecycle fix lands:
|
||||
|
||||
```bash
|
||||
FAST_BROWSER_LIVE=1 node --test --experimental-strip-types \
|
||||
extensions/fast-browser/live-validation.test.ts
|
||||
```
|
||||
|
||||
The harness requires `/usr/bin/chromium` and `.pi/fast-browser-demo/venv/bin/python` with `browser-harness==0.1.13`. It uses finite startup, action, readiness, worker, and close deadlines. Cleanup reports owned process/root leftovers rather than terminating anything it did not own.
|
||||
|
||||
## Validation evidence (2026-09-19)
|
||||
|
||||
Runtime versions: Chromium `152.0.7977.75 Arch Linux`; Python `3.12.13`; `browser-harness 0.1.13`; Node `v22.23.2`.
|
||||
|
||||
- **Blocked:** visible manual setup seeded the cookie and `/ready` proved it. `manager.close()` returned, establishing that the manager's tracked Chromium root had exited, but owned GPU descendant PID `2870508` still carried the temporary profile `--user-data-dir`; `SingletonLock` was still present and the next `manager.open()` rejected the profile. Visible automation was skipped rather than bypassing the lock. The PID is from this disposable test profile only.
|
||||
- **Passed:** headless launch with bounded literal fill/click and verified visible postcondition.
|
||||
- **Passed:** two different headless named profiles concurrently, server-verified cookie/state isolation, same-profile contention rejection, and profile B remaining alive after profile A closed.
|
||||
- **Passed:** final owned cleanup; no test processes or temporary root remained.
|
||||
|
||||
This is a production lifecycle blocker, not a fixture workaround. The minimal proposed fix is for the session manager to own/wait for the complete Chromium process tree (including GPU/renderer descendants) and confirm the profile lock is gone before releasing a named-profile lease, within its existing bounded close deadline. No production module or package manifest was changed in this milestone.
|
||||
|
||||
## Controlled Linux rerun (2026-09-20)
|
||||
|
||||
The Linux-only PID review was cleared for this controlled run. Preflight used the real `/usr/bin/chromium` and a real positive version-check PID `3222628` (parent `3222621`, not the validator process); the pinned environment remained Python `3.12.13` with `browser-harness 0.1.13`. No fake process or PID seam, user profile, or external origin was used.
|
||||
|
||||
The original M1 command was bounded with `timeout -k 5s 180s`. Cookie seed/readiness reached the visible setup close boundary, but close failed closed with `browser process-group ownership could not be confirmed; preserving session state`; automation reuse was skipped. Headless bounded fill/click and concurrent profile isolation/contention passed. Cleanup correctly preserved the uncertain owned root `/tmp/fast-browser-live-validation-5HvmBr` rather than deleting it. Ten owned Chromium processes remained at capture: `3222696, 3222709, 3222710, 3222712, 3222738, 3222740, 3222752, 3222761, 3222770, 3222771`; the root process was `3222696` and GPU descendant `3222738`. This run exited `124` from the finite timeout and is evidence for the lifecycle blocker, not a release claim.
|
||||
|
||||
The canonical development validator then ran with a bounded `240s` timeout. All six cases passed: input-safe read/assert/navigation/reload, popup/no-auto-switch/targeted-read selection, A→B evidence isolation, narrowed peer-origin filtering with base restoration, private PNG+diagnostics capture, and honest provenance handling (`Evidence provenance unavailable: false`). Its private artifacts lived under a unique `/tmp/fast-browser-runtime-live-*` root and were removed after owned cleanup; no canonical-run process/root remained. The report artifact is this document; non-Linux/global release clearance remains pending.
|
||||
|
||||
## Isolated managed-CLI batch (2026-09-20, no rerun after this evidence)
|
||||
|
||||
After the independent broker/provenance review cleared, a single sequential native-Linux batch ran with `/usr/bin/chromium` `152.0.7977.75`, Python `3.12.13`, `browser-harness 0.1.13`, and Node `v22.23.2`. `HOME`, `PI_CODING_AGENT_DIR`, XDG config/data/cache/state/runtime roots, `TMPDIR`, cwd, scenario JSON copies, and report roots were unique and owned. The four loopback ports were unique and all managed servers were started only by `scenario-cli`; no active `harness-test` profile, provider key, JEV call, or external origin was used. Synthetic `FAST_BROWSER_TEST_PASSWORD=demo-password` was local-only and did not appear in reports, stdout, or diagnostics.
|
||||
|
||||
The M1 validator reported: manual setup reached cookie readiness and `manager.close()` returned, but its stale tombstone assertion expected `/Unknown browser session/`; the manager intentionally returned `Browser session is no longer available: session closed`. Headless launch and the two-profile case then failed closed on process-group ownership, and the finite command exited `124`. The validator assertion is now corrected; the fixture now sends `Max-Age=3600`, but no post-change browser rerun has been made.
|
||||
|
||||
The preserved setup profile from that batch was inspected without launching a browser. The fixture source used at capture time had only `Path=/; SameSite=Lax` and no `Expires` or `Max-Age`, so it created a session cookie rather than a persistent cookie. HTTP receipt is nevertheless proven: `/ready?case=setup` only resolves after the server sees the exact cookie value, and the M1 run reached that readiness. After the setup close, `Default/Cookies` was SQLite `journal_mode=delete` with zero rows, `Cookies-journal` was zero bytes, no `Cookies-wal` existed, and `SingletonLock` was absent; no pending WAL/checkpoint explains the absence. The tracked root had closed and no live process for the named profile remained. Therefore this batch does **not** prove profile-cookie reuse: the zero row is explained by the old validator fixture's session-cookie attribute, while the independent root-exit-before-first-census lifecycle blocker still prevented the automation phase.
|
||||
|
||||
The three managed examples ran from disposable JSON copies. `local-form` and `local-login` completed all browser operations, but their reports were failed because owned browser cleanup did not complete before the bounded outer timeout (`124`). The corrected-port `local-popup` navigation passed, then its click failed because the example's postcondition (`Scenario fixture ready`) already held before the click; the worker correctly refused to treat that as proof of a mutation. The intentional failed-assert scenario reached `assertion timed_out` after a credential fill, retained a real PNG and diagnostics JSON, and produced a nonzero failed report. Its report was mode `0600` in an owned mode-`0755` directory; artifact files were `0600` under mode-`0700` roots; diagnostics contained `password=[redacted]` and the synthetic secret was absent.
|
||||
|
||||
Every browser and crashpad group left by the timed-out commands was later cleaned only after verifying current PID/start-time/UID/executable/argv/ancestry/PGID/SID ownership against the unique batch roots. All managed ports were closed and no batch-owned Chromium or crashpad process remained. The M1 cleanup proof is retained at `/tmp/fast-browser-next-validation-uqoyor/m1-cleanup.json`; per-scenario cleanup proofs and the complete summary are under `/tmp/fast-browser-next-validation-uqoyor/` (`batch-summary.json`). Disposable profile and retained-evidence roots remain preserved for diagnosis. This batch is validation evidence, not release clearance.
|
||||
|
||||
## Follow-up M1 gate rerun (2026-09-20, Linux; CLI batch intentionally not started)
|
||||
|
||||
A final ownership review cleared another single native-Linux run with fresh roots, `/usr/bin/chromium`, the pinned Python environment, isolated HOME/XDG/TMPDIR/cwd/config, and no provider or active user-profile state. The original M1 ran first and was **not** a full pass: visible manual setup seeded the synthetic cookie and closed cleanly, but visible automation failed its retained-session click with `Postcondition not verified; action was not retried`. Headless bounded fill/click and the two named-profile isolation/contention cases passed. The validator reported owned cleanup passed and exited `1` solely for the automation case.
|
||||
|
||||
Because M1 did not pass, the gate correctly stopped before launching any managed CLI examples or intentional-failure scenario. A post-run `/proc` census found no Chromium or crashpad process carrying the fresh batch roots, and no subsequent CLI ports existed. The batch root is `/tmp/fast-browser-final-validation-ewubUW`; its bounded M1 output is `m1.stdout`. Chromium left only disposable `org.chromium.Chromium.*` socket directories under the isolated TMPDIR; these contain no live owned process and were preserved rather than removed blindly.
|
||||
|
||||
The stale recent-exit tombstone assertion and validator cookie are now corrected in `live-validation.ts`: closed-session lookup expects `session closed`, and `/setup` sends `Max-Age=3600`. This rerun therefore does not reproduce the earlier session-cookie explanation, but it also does not prove profile-cookie reuse: the named profile was removed after owned cleanup and no retained profile database remained for inspection. The remaining failure is separate from the validator assertion and broker census fixes: Chromium processed the seed request, yet the post-close automation session did not reach the retained postcondition. No production spec was changed and no release or cookie-reuse claim is made.
|
||||
|
||||
## Independent managed-CLI rerun after the M1 gate (2026-09-20)
|
||||
|
||||
The M1 cleanup gate passed even though retained-session automation did not, so the independent CLI proof was run separately with fresh isolated state at `/tmp/fast-browser-cli-final-OqHfA6`. The environment again used only `/usr/bin/chromium`, the pinned Python environment, unique loopback ports, isolated HOME/XDG/TMPDIR/cwd/config roots, synthetic local credentials, and no provider keys or user profiles.
|
||||
|
||||
Successful managed examples:
|
||||
|
||||
- `local-form`: exit `0`; all two operations passed; cleanup passed.
|
||||
- `local-login`: exit `0`; all three operations passed using the synthetic environment credential; cleanup passed.
|
||||
- `local-popup`: the original immediate-popup copy reached the click but timed out in `waitPopup` (exit `1`, cleanup passed). The failure is retained as a production target-inventory race finding: the click postcondition passed, but the popup was not discovered by the opener-bound wait. A disposable diagnostic copy with a 100 ms `window.open` delay then passed navigation, click, popup discovery, and read with exit `0`; that delay is diagnostic evidence only, not popup clearance. The source fixture was restored to immediate popup behavior with its status-change fix preserved.
|
||||
|
||||
The intentional failed-assert scenario was changed to `wait: "none"` so the expected assertion failure remains non-aborted and evidence capture can complete. It exited `1` promptly (~506 ms), reported `assertion not_satisfied`, retained a real PNG and diagnostics JSON, and cleaned owned browser/fixture resources. The report is `0600` in the owned `0755` reports directory; artifact files are `0600` under `0700` roots. Diagnostics contain `password=[redacted]`; the synthetic credential is absent from report and stdout. Final census found no batch-owned Chromium/crashpad processes and no open managed ports.
|
||||
|
||||
The immediate-popup failure produced no retained evidence because the runtime operation aborted before the CLI's failure capture could run; cleanup still passed and the CLI removed its disposable profile. This is reported as a production/runtime evidence-retention finding, not hidden by treating cleanup success as scenario success. The validator now preserves failed M1 roots even when process cleanup succeeds and records automation cookie/outcome evidence for any future rerun.
|
||||
|
||||
## Diagnostic retained-profile rerun (2026-09-20)
|
||||
|
||||
With process cleanup verified and no popup workaround retained, only the retained-profile prerequisite and automation case were run from a fresh isolated root `/tmp/fast-browser-retained-run-gizlJ0`. No headless, isolation, popup, or managed-CLI case was launched.
|
||||
|
||||
- Setup readiness passed, named setup closed, the profile remained, and no profile process remained.
|
||||
- Setup persistence metadata was directly inspected at `.../Default/Cookies`: 20,480-byte database, zero rows, zero journal bytes, zero WAL bytes. The final post-automation metadata was identical. There was therefore no persisted Chromium cookie row to reuse; no cookie value or token was logged.
|
||||
- Automation readiness passed, but server telemetry recorded `cookieHeaderPresent: false`, `cookieReceived: false`, and `retained: false`.
|
||||
- The exact fill of `Milestone` verified. The exact click dispatched once with `mutationState: unknown` / `actionMayHaveExecuted: true`; it was never retried. Expected text was `Retained session verified`; the fixture's actual bounded outcome was `Fixture action did not meet its bounded condition`, and the controller returned `Postcondition not verified; action was not retried`.
|
||||
- Cleanup passed with no remaining owned processes. The failed diagnostic root was deliberately preserved mode `0700`, with summary mode `0600` at `/tmp/fast-browser-retained-run-gizlJ0/tmp/fast-browser-retained-diagnostic-0Rkywk/retained-diagnostic-summary.json`.
|
||||
|
||||
This proves the retained-session failure is an actual missing-cookie condition in this isolated run, not merely an unrecorded postcondition mismatch. It does not establish cross-platform or global release clearance.
|
||||
|
||||
## Immediate-popup managed rerun (2026-09-20)
|
||||
|
||||
The restored immediate-popup fixture was run alone from fresh isolated state at `/tmp/fast-browser-popup-final-8qElTH`, using native Linux `/usr/bin/chromium`, the pinned Python environment, an explicit unique loopback port, and isolated HOME/XDG/TMPDIR/cwd/config roots. The disposable scenario verified the full sequence `click → waitPopup → selectTab → read` without exposing tab IDs to the model.
|
||||
|
||||
The managed CLI exited `0` in 530 ms. Navigate, click (`Popup opened` postcondition), `waitPopup`, explicit `selectTab`, and popup read all passed; cleanup passed. Final ownership census found no batch-owned Chromium/crashpad processes and no open managed port. No popup delay workaround was used, and no timeout-metadata clearance claim is made.
|
||||
|
||||
## Controlled persistent-cookie M1 comparison (2026-09-20)
|
||||
|
||||
The current leader-first implementation was tested once on a fresh isolated Linux profile at `/tmp/fast-browser-persist-run-9J6jRV/tmp/fast-browser-persistent-cookie-m1-NKPozp`. It used `/usr/bin/chromium`, the pinned Python environment, a disposable loopback fixture, and the explicit `Max-Age=3600; Path=/; SameSite=Lax` cookie. Setup was visible and no-CDP: its process argv had no `--headless` or remote-debugging flag.
|
||||
|
||||
- Setup readiness and close passed; the server received the synthetic cookie and the named profile survived with no remaining browser process.
|
||||
- Post-close SQLite metadata showed one persistent row for the loopback host, `hasExpires: true`, `isPersistent: true`, `expiresUtc` populated, and encrypted value storage; raw cookie values were not recorded.
|
||||
- Same-profile automation received the cookie (`cookieHeaderPresent: true`, `cookieReceived: true`, `retained: true`). Fill and retained-session click both verified; actual outcome was `Retained session verified`; no action retry occurred.
|
||||
- Final metadata still showed the one persistent row. Cleanup passed with no owned Chromium/crashpad processes; the failed-case-safe root was preserved mode `0700` and the summary is mode `0600`.
|
||||
|
||||
Conclusion: root-first fixes persistent-cookie reuse in this controlled current Linux run. This is scoped evidence only; it is not global or non-Linux release clearance.
|
||||
|
||||
## Final live integration pass (2026-09-20)
|
||||
|
||||
The current tree was validated with fresh isolated Linux roots and no user profiles, providers, or fixture timing workaround.
|
||||
|
||||
- Full original M1: exit `0`; visible setup persistence, retained-session automation, headless bounded action, and two-profile isolation/contention all passed. Cleanup passed.
|
||||
- Canonical development matrix: exit `0`; all six cases passed with `Evidence provenance unavailable: false`.
|
||||
- Managed `local-form`: exit `0`; cleanup passed.
|
||||
- Managed `local-login`: exit `0`; cleanup passed.
|
||||
- Managed immediate `local-popup`: exit `0` in 531 ms; navigate, click, `waitPopup`, explicit `selectTab`, and read passed; cleanup passed.
|
||||
- Intentional assertion failure: exit `1` in 558 ms with `assertion not_satisfied`, `retry: never`; private PNG and diagnostics were retained. Report mode was `0600` in a `0755` report directory, artifacts were `0600` under `0700` roots, and diagnostics retained `password=[redacted]` only.
|
||||
|
||||
The real overall-deadline case used a no-popup `waitPopup` with a 3,000 ms scenario budget, below the operation's bounded wait. It exited nonzero (`1`) naturally in 3,066 ms; navigation passed and `waitPopup` aborted with `mutationState: not_attempted`, `retry: never`. Evidence was explicitly represented as private unavailable evidence (`status: failed`, error `browser session is unavailable`). The exact report is `/tmp/fast-browser-final-batch-boijm3/reports/overall-deadline-no-popup.json`; its scenario cleanup is `{ status: "failed", detail: "browser session cleanup failed" }`, and the top-level cleanup is `{ status: "failed", detail: "one or more owned resources failed cleanup" }`. This is the handoff finding for the CLI owner. Validation stopped immediately as required. No force-kill or mutation retry was used. A final ownership census still found no batch-owned Chromium/crashpad processes and no open managed ports, but this does not convert the reported cleanup failure into a pass.
|
||||
|
||||
The failed batch root remains preserved at `/tmp/fast-browser-final-batch-boijm3`; the complete summary is `/tmp/fast-browser-final-batch-boijm3/final-validation-summary.json`. No production or validation-source changes were needed for this pass.
|
||||
|
||||
## Deadline cleanup-race rerun after manager fix (2026-09-20)
|
||||
|
||||
After the manager/CLI cleanup-race fix, only the real overall-deadline/no-popup case was rerun from fresh native-Linux isolated state at `/tmp/fast-browser-deadline-rerun-iaE8de`; the earlier failure report was not overwritten. The no-popup fixture used a 3,000 ms scenario budget below the bounded popup wait.
|
||||
|
||||
The new report is `/tmp/fast-browser-deadline-rerun-iaE8de/reports/overall-deadline-no-popup-rerun.json`. It exited `1` naturally in 3,090 ms: navigation passed, `waitPopup` aborted, and the report preserved `retry: never`, `mutationState: not_attempted`, and private unavailable evidence (`browser session is unavailable`). Cleanup now passed with `owned runtime, artifacts, browser session, and fixture resources cleaned`. Final census found no rerun-owned Chromium/crashpad process and no open port. The rerun root remains preserved mode `0700`, and the report is mode `0600`.
|
||||
@@ -0,0 +1,46 @@
|
||||
# Fast Browser named profiles
|
||||
|
||||
## User decisions
|
||||
|
||||
- Support both manual setup/sign-in and automated launches from `/fast-browser`.
|
||||
- Use an exclusive profile lock. Parallel agents must choose different names; copying/cloning profiles is not part of this change.
|
||||
- Keep profile data on close. There is no automatic logout, profile reset, or deletion action.
|
||||
|
||||
## Configuration and storage
|
||||
|
||||
Add optional `browser.profile`, a 1–64 character managed name starting with a letter/digit and containing letters, digits, dots, underscores or hyphens. It is not an arbitrary filesystem path and is valid only for launch mode. Without it, the existing disposable profile behavior stays unchanged.
|
||||
|
||||
Resolve profiles below `$XDG_DATA_HOME/fast-browser/profiles/<name>`, or `$HOME/.local/share/fast-browser/profiles/<name>` when XDG_DATA_HOME is unset. This matches the manual `evebot` profile command already given to the user. Adopt existing manually prepared profiles without deleting their contents, provided they are not in use. Keep directories private and reject symlink/path traversal.
|
||||
|
||||
The `browser` object continues to replace lower-precedence browser settings as a unit. An explicit per-call browser object without `profile` therefore requests a disposable profile, not an inherited persistent one.
|
||||
|
||||
## Session lifecycle
|
||||
|
||||
**Manual setup:** launch the named browser visibly with its dedicated user-data directory, but without CDP/remote-debugging flags. This mode is for the user to sign in. It does not create an automation connection, expose a page to JEV, or claim that a particular external service will accept subsequent automation. An explicit setup-session ID identifies only the owned process lifetime; observe/run cannot automate it.
|
||||
|
||||
**Automated launch:** open the same named profile with CDP enabled and return the normal automation session ID. The manual browser must be fully closed first. Browser profile locks and process ownership are distinct from CDP endpoint leases, and both must be respected.
|
||||
|
||||
Acquire a cross-process lock before either launch. Reject active unmanaged Chromium profile use as well, rather than forwarding a new command into an existing unowned browser. Do not blindly remove stale SingletonLock files. Ignore/remove stale DevToolsActivePort metadata only after obtaining exclusive profile access, so a previous session's port cannot be mistaken for the newly launched browser.
|
||||
|
||||
Closing a session terminates its owned process and releases locks only after exit is confirmed. Disposable profiles are cleaned up; named profiles survive both normal close and startup failures. Preserve retryable ownership bookkeeping if termination or cleanup fails. Shutdown handles manual and automated sessions.
|
||||
|
||||
## Launch baseline from live testing
|
||||
|
||||
The initial visible `/usr/bin/chromium` launch with extra disable/password/keychain flags and a reduced desktop environment crashed twice with SIGSEGV. User-confirmed normal launch worked. A diagnostic launch also worked with only:
|
||||
|
||||
- `--user-data-dir=<private directory>`
|
||||
- `--remote-debugging-address=127.0.0.1` and `--remote-debugging-port=0` for automation
|
||||
- `--no-first-run` and `--no-default-browser-check`
|
||||
- the requested URL
|
||||
|
||||
That diagnostic retained `XDG_SESSION_TYPE` and other non-secret desktop context. Adopt this minimal baseline for the managed launcher, omitting the two debugging flags for manual setup. Do not force a different password-store/keychain mode between setup and automation. This establishes a working combination, not which individual old flag or omitted variable caused the crash.
|
||||
|
||||
## UI
|
||||
|
||||
Add a profile row to the native subdued-value settings view. An unset name means disposable; inherited names must remain visibly inherited. Add explicit manual-setup and automated-launch menu actions. Use current saved settings and report the returned session ID/profile. Following user feedback, every valid scope-local settings edit saves automatically; Escape returns without discarding earlier edits. Cancelling a field edit or failing validation/write preserves its previous value. Launch remains an explicit separate action. Existing close-all acts only on this Pi instance's sessions and never deletes a named profile.
|
||||
|
||||
## Verification
|
||||
|
||||
Final integration checks passed: `bun run test:fast-browser` (111 TypeScript tests and 21 Python tests), `bun run typecheck`, `bun run test:packages`, `bun run test`, and `git diff --check`.
|
||||
|
||||
Offline coverage includes name validation, config roundtrips, cross-manager contention, adoption, persistent sentinel-file retention, forced-visible setup without CDP, stale-port handling, failure cleanup, menu callbacks, conversation session metadata and unchanged disposable behavior. Real cookies were not used. No external login attempt or resumption of the cancelled Evebot test occurred during development; a live named-profile setup remains a user-driven check after reload.
|
||||
@@ -0,0 +1,36 @@
|
||||
# Unrestricted web-origin defaults
|
||||
|
||||
## User decision
|
||||
|
||||
The user requested no default website restriction. Origin policy is now intended to be optional, not a mandatory permission dialog or fixed test-origin list.
|
||||
|
||||
- Omitted `allowedOrigins` defaults to `"*"`: any valid HTTP(S) website.
|
||||
- Explicit `allowedOrigins: "*"` selects unrestricted web origins at a particular configuration scope.
|
||||
- A nonempty array retains the existing exact-origin allowlist behavior. Projects and individual runs can opt into restrictions.
|
||||
- Empty arrays and partial hostname wildcards are not alternate spellings for unrestricted access.
|
||||
- Internal browser URLs, local file URLs, malformed URLs and credential-bearing URLs remain unsupported.
|
||||
- A session created with an explicit restricted policy cannot be widened by a later run. An unrestricted session can be narrowed for one call.
|
||||
|
||||
Unrestricted origin policy does not authorize unrelated actions: the initiating user's task, bounded steps, mutation verification and no-blind-retry behavior still apply. It also does not add tab management: multiple eligible web tabs remain an explicit ambiguity rather than permission to choose an arbitrary tab.
|
||||
|
||||
## Retired demo environment
|
||||
|
||||
The active Pi process had inherited four variables from `.pi/fast-browser-demo/start-pi.sh`: a fixed CDP endpoint, the old localhost fixture origin, a pinned Python interpreter and the Luna auxiliary model.
|
||||
|
||||
At the user's request:
|
||||
|
||||
- Removed the four exports from that ignored helper and explicitly unset inherited copies before its `exec pi`.
|
||||
- Moved the useful Python interpreter and model settings into global `fast-browser.json`, with reasoning off, preserving existing settings and credential references.
|
||||
- Did not migrate the fixture origin or fixed CDP endpoint.
|
||||
- Did not delete browser profiles, the pinned Python environment, or provider credentials.
|
||||
- Verified the helper with an offline stub Pi executable: all four variables were absent in the new process.
|
||||
|
||||
A child shell cannot unset its parent Pi process's environment. A **clean restart through the updated helper**, rather than `/reload` alone, is needed to remove values already inherited by the current process. Deliberately configured environment defaults remain supported after this cleanup.
|
||||
|
||||
## Verification
|
||||
|
||||
Passed `bun run test:fast-browser` (126 TypeScript tests and 23 Python tests), `bun run typecheck`, `bun run test:packages`, `bun run test`, and `git diff --check`.
|
||||
|
||||
Coverage includes missing/unrestricted/restricted policies across config, native settings, session lifecycle, TypeScript bridge and Python worker. Restricted sessions cannot be widened, non-web/credential-bearing/wildcard-host URLs remain unsupported, and the settings view exposes default provenance. Tests isolate agent directories and project paths instead of reading the user's real global configuration or provider keys.
|
||||
|
||||
A read-only configuration check with the four legacy variables removed confirmed the actual project resolves origins to `"*"` from defaults, Python/model from global JSON, and retains its `harness-test` profile. No browser was launched or external login attempted.
|
||||
@@ -0,0 +1,27 @@
|
||||
# Fast Browser prototype
|
||||
|
||||
Historical first-slice plan. The [unattended-session plan](fast-browser-unattended-plan.md) supersedes its approval, credential and configuration requirements.
|
||||
|
||||
Branch: `feat/fast-browser`. Package/extension directory: `extensions/fast-browser`.
|
||||
User-approved scope: bounded local-browser Pi extension, same-host Chrome, JEV decisions, literal filling and optional small-model text generation. See `browser-use-jev-feasibility.md` for research.
|
||||
|
||||
Status: first slice implemented and offline-validated. See [validation results and remaining live-test work](fast-browser-prototype-validation.md).
|
||||
|
||||
## First slice
|
||||
|
||||
- Typed `observe`, `act`, and `close` bridge operations; no arbitrary model-generated Python/JS/CDP.
|
||||
- Explicit loopback Chrome endpoint, dedicated test profile recommended. Do not auto-open or connect to a user's default browser.
|
||||
- Extract actionable candidates with stable observation IDs and tab/frame/document identity. Revalidate before action and reject stale/changed targets.
|
||||
- A supplied sequence of bounded click/fill steps, rather than an unconstrained autonomous agent. JEV picks a candidate or abstains. Literal text bypasses generation; composition uses the configured Pi auxiliary model.
|
||||
- Every live mutation requires a Pi UI confirmation of the exact action/payload. Noninteractive sessions fail closed. Offline controller tests inject approval. We will not silently trade away approval overhead for speed.
|
||||
- TypeScript controller owns step/time budgets, cancellation, strict JEV response validation, approval, postconditions and compact timings. No blind action retries.
|
||||
- Browser-harness runs through a restricted Python worker. No cloud APIs, telemetry, update checks, recording, profile sync, arbitrary network fetch or raw execution tools exposed.
|
||||
- Companion skill and README describe experimental limits and setup. No automatic global installation/config edits.
|
||||
|
||||
## Validation
|
||||
|
||||
Offline TypeScript tests with fake browser/JEV/text adapters, Python bridge tests without Chrome, package checks and typecheck. Local live smoke tests only after explicit safe endpoint/test fixture setup; live inference only with approved use of credentials. Record actual failures encountered in `docs/todos.md` when not fixed immediately.
|
||||
|
||||
## Deferred
|
||||
|
||||
General autonomous planning, screenshot/vision fallback, automatic site-specific helper writing, DOM-independent actions, remote browser hosts, multi-browser concurrency, uploads/downloads, submission/Enter shortcuts, implicit bulk approval, and any efficacy claim without benchmarks.
|
||||
@@ -0,0 +1,63 @@
|
||||
# Fast Browser prototype validation
|
||||
|
||||
This records the original approval-based prototype and demo. Subsequent user requirements and implementation are tracked in [the unattended-session plan](fast-browser-unattended-plan.md).
|
||||
|
||||
Branch: `feat/fast-browser`. Validation date: 2026-09-19.
|
||||
|
||||
## Implemented
|
||||
|
||||
- `fast_browser_observe` and `fast_browser_run` Pi tools, lazily loaded config and per-extension serialized browser lane.
|
||||
- Bounded click/fill plans (up to 10 steps), exact-name deterministic shortcut, JEV semantic candidate selection, optional registry-backed composition, and per-action UI approval.
|
||||
- Literal input bypasses text generation. JEV is lazy: exact-name runs need no Typesafe key.
|
||||
- JEV requests contain the task, action, origin and bounded candidate labels, not page URL paths, query strings, fragments, page titles or field values. Control labels still intentionally go to the model and may contain private page-derived information; this is not universal content redaction.
|
||||
- Isolated Python worker using the installed, pinned `browser-harness==0.1.13` daemon. The worker owns its child process instead of using the harness's detached/retrying/auto-launch path. It attaches to exactly one already-open approved-origin tab, never navigates to choose a tab, and shuts down its own daemon/temporary directories.
|
||||
- Only main-frame native text inputs/textareas and supported clickable controls are in scope. No arbitrary model-authored code, screenshots, frame traversal, uploads, downloads, or submit/Enter shortcut API.
|
||||
- Every mutation revalidates tab/document/candidate identity and invalidates previous observations. Clicks require a previously unsatisfied URL/text postcondition. Fills verify exact current field value and any optional postcondition. Verification does not prove all internal application state or external side effects.
|
||||
- `mutationState` and `actionMayHaveExecuted` distinguish an unverified dispatched mutation from a run that never attempted one. Cancellation cannot undo an already dispatched browser action. Unknown outcomes require inspection, not blind retry.
|
||||
|
||||
## Validation performed
|
||||
|
||||
- Final `bun run test` passed the complete repository suite, including **56 fast-browser TypeScript tests and 18 Python tests**. `bun run typecheck` and `git diff --check` also passed.
|
||||
- Offline TypeScript tests cover controller budgets, cancellation races, deterministic selection, approval, JEV request/response validation, model routing, config, IPC limits and lifecycle.
|
||||
- Python tests cover candidate filtering, stale/changed targets, replay prevention, postconditions, origin changes, helper call contracts, owned daemon startup/shutdown, strict version guard, and non-sensitive protocol errors.
|
||||
- A real subprocess smoke check using this host's existing uv interpreter confirmed that installed browser-harness **0.1.8 is rejected before importing browser helpers or connecting to Chrome**. No upgrade was performed on that installation.
|
||||
- Workspace typecheck passes in the local dependency environment. The lockfile changes only add the new workspace; no dependency-version churn.
|
||||
- Package checks verify that runtime Python/TypeScript files, skill and fixture are included and tests/bytecode are excluded.
|
||||
- The earlier full repository run passed existing suites but caught a new confirmation-test expectation during a concurrent approval-format edit. That test was corrected to expect safely quoted display values; the final full rerun passed.
|
||||
|
||||
## Review findings fixed
|
||||
|
||||
- Removed an implicit import from a temporary source checkout.
|
||||
- Stopped forwarding the Pi environment and provider API keys into Python/daemon children.
|
||||
- Replaced detached daemon startup with explicit process ownership; handled cancellation during startup and waited for worker exit before releasing the lane.
|
||||
- Fixed candidate-name/role consistency, main-frame checks, click hit testing, and focused bulk input rather than per-character typing.
|
||||
- Rejected preexisting click postconditions, out-of-origin post-action states, and replay through other old snapshots.
|
||||
- Tightened probability normalization and rejected a selected JEV option that contradicts the reported winner. Confidence remains a separate provider-reported value with an uncalibrated prototype threshold, not a security boundary.
|
||||
- Made streamed model responses abortable and sanitized their failures; bounded direct JEV input calls as well as controller inputs.
|
||||
- Limited action evidence to fixed non-sensitive messages and exposed uncertain action outcomes in machine-readable form.
|
||||
|
||||
The review's suggestion that the controller must itself understand DOM/frame identity was not adopted: that identity stays private in the real bridge, whose `act` contract and tests enforce freshness immediately before mutation. Local approval retains the exact escaped URL/payload for informed permission; those confirmation values are not JEV request context.
|
||||
|
||||
## Not validated / next step
|
||||
|
||||
The original prototype validation did not include live mutation/inference. The subsequent demo partially exercised those paths, as recorded below. No comparative task-success or latency benchmark has been run. Initial validation found no Chrome on PATH and an older default harness installation.
|
||||
|
||||
### Follow-up: live demo preparation
|
||||
|
||||
At the user's request, Chrome was subsequently found at `/opt/google/chrome/chrome` (version150). A separate ignored project-local environment was prepared with browser-harness0.1.13 and a dedicated Chrome profile. The local fixture now includes a `Demo greeting` textarea for the requested small-model composition demo.
|
||||
|
||||
A **real read-only bridge smoke test passed**: it returned exactly `Display name`, `Demo greeting`, and `Continue`, excluding the password and disabled controls. After bridge shutdown, a CDP tab-list check confirmed that only the original fixture tab remained; the harness-owned blank tab was cleaned up. No API credentials were used and no fields were filled or clicked.
|
||||
|
||||
After restarting through `.pi/fast-browser-demo/start-pi.sh --resume`, the actual Pi tools ran. JEV selected `Display name` (confidence0.99,433ms); after UI approval, filling `Ada` was verified in14ms. It next selected `Demo greeting` (confidence0.89,324ms), but Spark generation failed before any greeting-field mutation. The full partial run took13,240ms, including11,552ms of human approval wait. Continue was not clicked.
|
||||
|
||||
A separate tool-free Pi diagnostic confirmed the provider error: `gpt-5.3-codex-spark` is not supported with this ChatGPT-account authentication. The user selected `openai-codex/gpt-5.6-luna` with reasoning off instead. The launcher was updated, and Codex auxiliary calls now explicitly send `reasoningEffort: none`; only the remaining two steps should be resumed after restart. Safe generation errors are now retained in tool results. Latest tests:58 TypeScript+18 Python and workspace typecheck pass.
|
||||
|
||||
During this follow-up, one no-key adapter test accidentally inherited the live Typesafe environment and issued a synthetic fixture request. The test helper now clears all fast-browser settings/key before each test, and all adapter tests prohibit HTTP. No actual browser page data or field values were sent by that test. This finding and the remaining environment-only reconfiguration limitation are recorded in `docs/todos.md`. Preparation/resume state and cleanup resource IDs remain in ignored `.pi/fast-browser-demo/README.md`.
|
||||
|
||||
No speedup or general-browser reliability claim is established by these tests. After the approved live demo, compare identical tasks against a small-model-only baseline.
|
||||
|
||||
The origin allowlist controls where this adapter operates; it is not a browser network firewall. A redirect or action may already have caused a network request before the adapter detects it. The JS/DOM checks are not atomic against a malicious page changing between check and input. Use a trusted, dedicated test profile for this prototype.
|
||||
|
||||
## Tooling failure observed
|
||||
|
||||
The browser-bridge subagent failed twice with a provider WebSocket error after writing partial files. The parent resumed once, then completed and tested the implementation directly. Reproduction evidence and a follow-up remain in `docs/todos.md`.
|
||||
@@ -0,0 +1,32 @@
|
||||
# Fast Browser: unattended, caller-configured test sessions
|
||||
|
||||
This supersedes the original prototype's mandatory per-action approval and password-field exclusion. The user explicitly requested an automation tool controlled by an initiating model such as Astra, including local test-account login, configuration files/TUI, parallel isolated browsers and headless execution.
|
||||
|
||||
## Contract
|
||||
|
||||
- The initiating model supplies the task, bounded steps, allowed origins and optional settings/credential bindings. Execution has no interactive approval layer, including in print mode.
|
||||
- Preserve operational correctness: step/time budgets, current target/document checks, exact fill verification, explicit click postconditions, cancellation and unknown-outcome reporting. These are not user permission prompts.
|
||||
- Credentials may be caller-provided literals, environment references, plain local files, or named top-level JSON file entries. Resolve only for the selected fill. Never insert resolved values into JEV state, generation prompts or result traces. Literal arguments remain present in the initiating Pi session's ordinary tool-call history; references avoid adding the value there.
|
||||
- Password inputs are supported like other native inputs. No blanket login/test-account ban. Hidden controls and unsupported file upload/rich-text mechanics remain capability limits, not authentication policy.
|
||||
|
||||
## Configuration
|
||||
|
||||
Precedence: per-call settings > project `.pi/fast-browser.json` > global agent-directory `fast-browser.json` > legacy environment defaults. File settings are reread for the next invocation. `/fast-browser` uses Pi's native selection/input UI to inspect/edit settings and choose project/global save scope; changes do not require restarting Pi.
|
||||
|
||||
Keep actual provider API keys out of settings. A Typesafe key may be selected through a local file/environment reference; ordinary Pi model authentication continues to use the model registry. Runtime endpoint/executable/model fields and declared credential references are caller-controlled, never selected by website content.
|
||||
|
||||
## Browser sessions
|
||||
|
||||
- Add explicit open/observe/run/close session lifecycle. The initiating agent can retain an opaque session ID across calls, so login and navigation state survive until cleanup.
|
||||
- Each launched session gets a distinct browser process, temporary user-data directory, CDP endpoint and browser-harness controller. Tabs in a shared profile are not treated as isolation.
|
||||
- Support Vivaldi, Chrome and Chromium using the current CDP/browser-harness backend. A configured product never silently falls back to another product.
|
||||
- A caller may override the executable path. Headless launch is explicit (`browser.headless: true`); visible launch remains available for debugging. An attached browser's display mode was determined externally when it started.
|
||||
- Attach to an explicit compatible endpoint when requested, without closing the externally owned browser. Hold an exclusive endpoint lease for the session to reject competing automation on that same endpoint; attachment does not isolate cookies or local storage from other clients.
|
||||
- Independent sessions can operate concurrently. Serialize operations within each session, not globally. Close only the requested owned session; extension shutdown cleans up owned resources.
|
||||
- Firefox is explicitly deferred by the user. Reject unsupported engine selection honestly; a future Firefox backend must not pretend CDP compatibility.
|
||||
|
||||
Separate browser processes do not isolate application database records, accounts or server-side sessions. Parallel suites must choose suitable test accounts/data as well.
|
||||
|
||||
## Validation boundaries
|
||||
|
||||
Use offline controller, credential, native-UI, process-lifecycle and worker tests first. Do not use modified repository code to bypass the currently loaded tool's mandatory action UI; live unattended checks require a future session with the updated tool contract. No real credentials, browsers or provider calls are required for this implementation's regression suite.
|
||||
@@ -0,0 +1,42 @@
|
||||
# Unattended Fast Browser validation
|
||||
|
||||
Implementation follows [the unattended-session plan](fast-browser-unattended-plan.md). The original [live demo](fast-browser-live-demo.md) remains historical evidence for the prior attach-only, UI-approved implementation, not for the new launcher.
|
||||
|
||||
## Implemented and covered offline
|
||||
|
||||
- No controller approval callback or adapter UI authorization gate; literal/credential flows also work in noninteractive modes.
|
||||
- Literal, environment, plain-file and top-level JSON credential references, bounded file reads, cancellation, sanitized errors and password input support. Resolved values are not inserted into JEV input, text-generation prompts or result traces.
|
||||
- Global defaults, project overrides and per-call settings; provenance; strict validation; atomic mode-0600 writes; native `/fast-browser` editing of scope-local partial values. Clearing an override means inheritance, not secretly disabling lower-precedence configuration.
|
||||
- Optional auxiliary-model lookup is lazy. An unavailable configured model does not break literal fills. Codex reasoning-off is explicit; unsupported reasoning mappings fail rather than silently select another provider/model.
|
||||
- Explicit open/observe/run/close lifecycle, opaque session IDs, per-session busy guards and concurrent independent sessions. Caller-supplied settings cannot retarget an existing session or broaden its origin policy.
|
||||
- Vivaldi/Chrome/Chromium executable selection, explicit headless flags, unique process/profile/endpoint per launched session and credential-free child environment. Attach sessions lease the configured endpoint without owning the external browser process.
|
||||
- Independent close, cancellation, post-ready browser exit, startup failure, graceful/forced termination, pending-open shutdown races and retryable failed cleanup.
|
||||
|
||||
## Issues found and fixed during development
|
||||
|
||||
- Scope saves originally serialized merged effective values, copying project-only settings into global defaults. The editor now edits and saves only the selected scope's partial object; tests preserve the global/project boundary.
|
||||
- Closing an attached session initially rejected it because the adapter confused browser-process ownership with lease ownership. Closing now releases the session lease without killing the external browser.
|
||||
- Forced process termination originally allowed profile/lease cleanup before confirmed exit. Cleanup now retains ownership when termination cannot be confirmed and supports retry.
|
||||
- Failed startup cleanup could lose ownership bookkeeping, and pending-entry failures could disappear before shutdown inspected them. Retained failed-start resources and pending snapshots are regression-tested.
|
||||
- Earlier prototype tests once inherited a real Typesafe key and issued one synthetic fixture request, without field values or page contents. Adapter tests were changed to isolate configuration and prohibit HTTP. The current suite requires no live browser, provider or network requests.
|
||||
- The auxiliary model catalogue listed Codex Spark, but the actual provider rejected it for ChatGPT-account authentication. Model catalogue availability is not API entitlement; safe errors are retained and no automatic model fallback is used.
|
||||
|
||||
## Final checks
|
||||
|
||||
Passed after integration and review fixes:
|
||||
|
||||
- `bun run test:fast-browser`: **93 TypeScript tests and 19 Python tests**.
|
||||
- `bun run typecheck`.
|
||||
- `bun run test:packages`.
|
||||
- `bun run test` (full repository suite).
|
||||
- `git diff --check`.
|
||||
|
||||
## Settings persistence follow-up
|
||||
|
||||
The original native editor kept an explicit-save draft. User feedback clarified that leaving the editor should retain changes, as Pi's own settings do. The corrected workflow saves each valid scope-local edit immediately, makes Escape a back/exit action, and preserves previous saved values when a field is cancelled, invalid, or cannot be written. It must not copy project overrides into global defaults. This supersedes earlier explicit-save/cancel instructions.
|
||||
|
||||
## Remaining live validation
|
||||
|
||||
After `/reload`, exercise the new tool contract against the local login fixture using two headless browser sessions, including Vivaldi where installed. Verify independent login state and cleanup with actual browser builds. Offline fake-process tests do not prove a particular installed browser accepts every flag.
|
||||
|
||||
This work did not change the user's global settings, launch a new live browser, or invoke the old loaded approval-based browser tools to bypass their contract. Firefox remains explicitly deferred. Follow-up items are tracked in [todos.md](todos.md).
|
||||
@@ -1,5 +1,49 @@
|
||||
# Todos
|
||||
|
||||
## Fast-browser development readiness
|
||||
|
||||
Status: **validated native-Linux integrated acceptance complete; final offline recheck passed** (2026-09-20). The original M1 visible-cleanup failure is preserved as historical evidence. The root-first cookie experiment, immediate-popup flow, full M1/development matrix, managed CLI scenarios, assertion path, and latest deadline gate passed with cleanup and empty process/port census. Linux-only supervision and non-atomic PID identity proof remain explicit constraints, and no SIGTERM guarantee of universal all-app durability is claimed. The compact public `fast_browser_control` surface and runtime cleanup/navigation/lane/evidence fixes passed the validated scope. Remote GitHub CI was not run; automatic CI remains configured and equivalent managed-CLI live validation passed locally. Exact historical validation results remain in the validator-owned live-validation documentation. See [fast-browser-development-readiness.md](fast-browser-development-readiness.md) and [fast-browser-live-validation.md](fast-browser-live-validation.md); provider-reliability follow-up remains open.
|
||||
|
||||
### Final native-Linux validation checkpoint (2026-09-20)
|
||||
|
||||
Typecheck passed. Full-repository offline validation: **805 total, 803 passed, 2 skipped**. Fast-browser TypeScript: **222 total, 220 passed, 2 skipped**. Pinned Python 3.12.13: **42/42 passed**. Packaging: **4/4**, links **8/8**, package contents **36 entries**, and tracked/untracked whitespace checks clean. `imports:check` was not run because it requires private network access. Remote GitHub CI was not run; no remote CI execution claim is made. All live gates passed, including the latest deadline rerun, which naturally exited `1` after 3090 ms with `retry=never` and `mutationState=not_attempted`, explicit private unavailable evidence, cleanup passed, and empty process/port census.
|
||||
|
||||
## Confirm the minimal Chromium launch baseline after reload
|
||||
|
||||
During the cancelled Evebot test, the managed visible `/usr/bin/chromium` launches crashed twice with SIGSEGV (coredump records PIDs 2579240 and 2580031). A user-confirmed successful diagnostic used a fresh profile, only the user-data/CDP/no-first-run/no-default-browser-check flags, and preserved desktop environment including XDG_SESSION_TYPE. Named-profile work adopts that minimal baseline for both setup and automation (without CDP for setup). Verify the integrated launcher after reload; the individual cause among removed flags/environment differences is not yet isolated. Do not assume Chromium itself is broken or claim EVE accepts automated login.
|
||||
|
||||
Also preserve a bounded, non-secret exit reason for recently closed session IDs: after the SIGSEGV, `fast_browser_observe` reported only `Unknown browser session`, requiring an OS crash-log check to distinguish a crash from an ownership/lifetime bug.
|
||||
|
||||
The same test exposed generic startup errors hiding the actual no-approved-tab condition. Fixed the Python worker to return sanitized startup errors through bounded IPC; it can now report web origins only, without URL paths/query tokens. The app was redirecting to a Pangolin gateway, which the user then disabled. The Characters page was reached, but no character connection or asset-value query occurred before the user cancelled.
|
||||
|
||||
## Live-validate isolated Vivaldi/Chromium test sessions after reload
|
||||
|
||||
The new session launcher and unattended adapter are being regression-tested offline; the earlier real Chrome demo tested the original attach-only, UI-approved implementation. After loading the new extension contract, run two simultaneous isolated sessions against the disposable login fixture, including headless mode and configured Vivaldi where installed. Verify separate profiles/endpoints, independent login state and cleanup, and no credential values in result traces or JEV requests. Do not treat fake-process tests as evidence that a particular installed browser build supports every launch flag. Do not bypass the old loaded tool's approval contract to run this check.
|
||||
|
||||
## Add a Firefox backend to fast-browser
|
||||
|
||||
Explicitly deferred by the user while Vivaldi/Chrome/Chromium configuration and isolated sessions are implemented. Firefox is not compatible with the current CDP/browser-harness backend; add and test a separate backend rather than silently substituting Chromium. Preserve per-session isolation, headless operation and the existing bounded-run contract.
|
||||
|
||||
## Investigate interrupted subagent WebSocket recovery
|
||||
|
||||
During the `feat/fast-browser` prototype on 2026-09-18–19, the `browser-bridge` Pi worker failed after 12m21s with `WebSocket error` / provider auto-retry exhausted. Four implementation/test files were present but no completion report was delivered. The parent resumed the same named session with `subagent_message` and reviewed the partial artifacts rather than assuming they were finished. The resumed worker failed again after 8m41s with the same error; the parent then finished the bridge directly and added missing lifecycle/real-adapter contract tests. Reproduce provider-stream interruption after file writes; verify durable failure reporting, same-session resume, and preservation of uncommitted work. This is an orchestration/provider failure, not a browser/JEV inference failure; no live browser or JEV call was involved.
|
||||
|
||||
The `fast-browser-settings` worker later hit the same provider error after 13m40s while implementing config/TUI changes. Its partial files were retained and the parent sent a same-session resume with explicit file ownership and offline-validation instructions. This recurrence also occurred without a live browser/provider test call.
|
||||
|
||||
On 2026-09-19, `fb-transport-evidence` failed after 6m02s with provider `WebSocket error` / auto-retry exhausted. Main resumed the same agent to preserve partial work. The incident is **recovered with a code/test handoff**: wire/details/parent-abort and artifact-store fixes were delivered with 19 targeted TypeScript plus 35 Python tests passing. The underlying provider-reliability follow-up remains open.
|
||||
|
||||
A subsequent `fb-scenarios-ci` run failed after 23m08s with WebSocket close 1011 / auto-retry exhausted. Main resumed the same session while preserving partial work. The incident is **recovered with a concrete CLI code/test handoff**: tab-reference parsing/static validation/resolution and closed/missing checks, popup JSON coverage, 20 focused tests, typecheck, and all `--check` examples passed. At that historical checkpoint, scenario commands/hooks/reports handoff was still being finalized and tab references remained pending live/review validation; the later native-Linux acceptance gate superseded that checkpoint. The underlying provider-reliability follow-up remains open.
|
||||
|
||||
A later `fb-runtime-tools` run failed after 25m57s with provider `WebSocket error` / auto-retry exhausted. Main resumed the same session while preserving partial runtime/tools work. The incident is **recovered with a code/test handoff**: runtime factory/tools/controller fixes were delivered with 48 targeted tests plus bridge coverage passing. The underlying provider-reliability follow-up remains open.
|
||||
|
||||
A later `fb-browser-lifecycle-fix` run failed after 38m07s with provider `WebSocket error` / retries exhausted. Main resumed the existing agent while preserving the partial shutdown fix. This historical incident is superseded by the recovered lifecycle handoff below; preserve it as incident history rather than current status. The earlier transport/runtime recoveries are backed by concrete test handoffs.
|
||||
|
||||
The original M1 visible-cleanup run failed closed and preserved `/tmp/fast-browser-live-validation-5HvmBr`; the validator safely terminated all 11 verified owned processes, left no target-profile processes, and preserved `evidence/owned-process-cleanup.json` as historical evidence. PID/helper safety independently cleared. The root-first cookie experiment, full M1/development matrix, managed scenarios, assertion path, and latest deadline rerun passed with complete cleanup and empty process/port census. The deadline rerun naturally exited `1` after 3090 ms with `retry=never` and `mutationState=not_attempted`, explicit private unavailable evidence, and no retry. The old lifecycle assertion failure is closed. Linux-only supervision and non-atomic PID identity proof remain constraints; no SIGTERM guarantee of universal all-app durability is claimed. Task 13 runtime cleanup/navigation/lane/evidence fixes and Task 14 CLI report/formatter/cleanup/session/artifact-redaction fixes passed the validated native-Linux gate. Provider WebSocket failures from lifecycle-fix, lifecycle-review, and runtime URL-fix work remain under the existing provider-reliability TODO; the runtime owner has resumed the narrow-URL fix.
|
||||
|
||||
A related artifact-store test attempted `chmod /tmp` and hit permission denied. This is **fixed in source/tests** using private owned child roots; no chmod of shared parents or sandbox bypass is allowed, and independent review found no remaining chmod issue.
|
||||
|
||||
Task 12 Python evidence privacy is complete: independent reproductions and six real-development cases pass the privacy review; no buffered-event blocker remains. Timeout-evidence producer and CLI privacy are independently cleared with explicit unavailable/partial metadata, bounded independent capture, and no reopen/retry. Recent-exit diagnostics are implemented/unit-tested with safe known-closed-ID reasons, 30-second TTL, 32-entry cap, sanitized output, and unknown expired IDs; no public history API exists. Exact historical/live results remain in the validator-owned live-validation documentation.
|
||||
|
||||
## Retire the Bash cwd compatibility override after a usable Pi upgrade
|
||||
|
||||
Pi 0.85.0 corrects built-in tools that captured the process directory instead of using the live execution `ctx.cwd`, but it still does not declare or honor a per-call `cwd` argument. Its public `@earendil-works/pi-coding-agent` entry point also imports `@earendil-works/pi-server` without declaring that runtime dependency, so importing the package after a normal Bun install fails with `ERR_MODULE_NOT_FOUND`. Keep `@jbcode/pi-bash-cwd` until an upstream release exposes equivalent validated per-call cwd behavior and loads cleanly for extension consumers. Then verify the hidden-worktree root, nested-package, relative-path, and fail-before-execution cases against the built-in tool before removing the override package.
|
||||
|
||||
Reference in New Issue
Block a user