From 6fba62800ac9ad613466234292cba7b84ce52606 Mon Sep 17 00:00:00 2001 From: JB Date: Mon, 21 Sep 2026 12:58:21 -0400 Subject: [PATCH] docs: approve fast-browser performance implementation plan --- docs/plans/fast-browser-performance.md | 118 +++++++++++++++++++++++++ 1 file changed, 118 insertions(+) create mode 100644 docs/plans/fast-browser-performance.md diff --git a/docs/plans/fast-browser-performance.md b/docs/plans/fast-browser-performance.md new file mode 100644 index 0000000..1837000 --- /dev/null +++ b/docs/plans/fast-browser-performance.md @@ -0,0 +1,118 @@ +# Fast-browser performance workflow record + +## Status + +- Status: active +- Current phase: approved plan; isolated worktree established, before product edits. +- Next safe action: commit the approved record and dispatch A/B/D with exclusive ownership. + +## Request and scope + +Optimize fast-browser for local development testing and validation, borrowing proven round-trip reductions from jev-ultrafast. The parent orchestrates; subagents implement exclusive slices. Optimize browser/runtime overhead first, with measured correctness-preserving gains rather than a speculative autonomous-agent rewrite. + +In scope: redundant browser calls; AX-preserving batched observation metadata; local origin-context calculation; a provider-free paired benchmark; focused regression tests and performance documentation. + +Out of scope: new actions, autonomous planning, broader model context, changed model/provider, dependency changes, public API changes, uploads/downloads, personal browser profiles, production access, publishing, and speculative page-state/decision caches. No product UI change. Synthetic fixture HTML is validation material, not a new UI feature. + +## Git identity + +- Source branch: `main` at `773c628512fdfdf5a139ae1b27833bea97d74255`. +- Remote: `origin/main`, fetched and identical to local main. +- Feature branch: `feature/fast-browser-performance` (must be absent before creation). +- Worktree: `/home/jb/code/harness-tools/.worktrees/fast-browser-performance` (must be absent before creation). +- Omitted source changes: untracked `docs/local-decision-model-feasibility.md`, preserved untouched. No tracked source modifications observed. +- Add `/.worktrees/` to the repository-local Git exclude file before worktree creation. This is local Git housekeeping, not a product change. +- Record path after approval: `docs/plans/fast-browser-performance.md` in the new worktree. +- Reuse existing root node_modules and the pinned Python environment `/home/jb/code/harness-tools/.pi/fast-browser-demo/venv/bin/python`; no installation or lockfile modification. Any worktree-local links are ignored and explicitly owned. + +## Acceptance criteria + +1. Existing public tools, operation schemas, candidate ordering/limit, field-label privacy, fresh action revalidation, occlusion checks, outcome verification, budgets, session isolation, and no-blind-retry semantics remain intact. +2. Browser hot-path call counts fall deterministically: one target inventory per refresh; no redundant domain enables after the pinned harness attach; browser platform detection at most once per backend; batched observations remove per-candidate partial-AX queries and repeated evaluate/release cycles. +3. Chrome AX data remains authoritative for observation roles and click names. Fill labels retain the existing value-free DOM extraction and never fall back to AX value-bearing names. No field values, raw DOM, selectors, or arbitrary scripts enter observations, provider input, or benchmark reports. +4. Batch resources exist only for one bounded observation chunk and are released on success/failure. Fresh single-target metadata remains the pre-mutation authority. Missing/failed batch data uses the existing safe per-node path where compatible; no blind mutation retry. +5. Origin narrowing remains validated and session-lifetime/serialization aware but does not send a redundant JSONL operation. The effective context still travels with every subsequent browser operation; omitted context restores the session base, and restricted sessions cannot broaden. +6. Benchmark results cover unchanged baseline and optimized code with the same runner/fixture/browser, independent outcome assertions, all attempts retained, startup separate from warm operation timing, p50/p95 and protocol counts. Keep optimizations only when call counts improve and live results show no material regression; investigate noisy or mixed results before claiming gains. +7. Full repository tests/typecheck, focused regressions, real Chromium runtime validation, and two independent review axes pass for the final committed diff. Any environment blocker or unsupported claim remains explicit. + +## Approved implementation plan and gates + +Plan approval: user selected `approve` (Approve and implement) through ask_user on 2026-09-21. Approval covers the exact slices and source-file omission below, local benchmark browsers/fixtures, local commits, and eventual safe fast-forward integration. It does not authorize dependency changes, production access, API/security-posture changes, or publication. + +### Slice A: provider-free measurement and regression fixture + +Owner `performance-benchmark` (worker). Exclusive paths: `extensions/fast-browser/performance-benchmark.ts`, `extensions/fast-browser/performance-benchmark.test.ts`, `extensions/fast-browser/performance_probe.py`, `extensions/fast-browser/fixtures/performance.html`. + +Build a bounded opt-in runner using existing browser-session/runtime/transport seams. Accept explicit worker/baseline paths and pinned Python/browser configuration rather than copying an old implementation into production. Count CDP/IPC methods and JSONL operations locally without recording arguments, URLs, values, text, or screenshots. A test-only Python probe may wrap the selected worker's helper methods for counting; no product instrumentation switch is required. + +Use realistic local account/settings or search/filter form workflows with 0/10/60 eligible controls, fill replacement, immediate/delayed click outcomes, and popup discovery. Assert actual final state independently of action postconditions. Exercise warm repeated observations and complete verified flows; measure cold startup separately. Alternate baseline/optimized arms, aim for at least 20 paired warm samples, retain failures and cleanup results. Deterministic model stubs may exercise orchestration without provider cost; clearly exclude real provider latency and semantic accuracy from claims. Focused tests cover statistics/report accounting, input bounds, failure inclusion, and report privacy. No wall-clock thresholds in unit tests. + +### Slice B: remove proven browser-call duplication + +Owner `browser-hotpath` (worker). Exclusive paths: `extensions/fast-browser/browser_worker.py`, `extensions/fast-browser/test_browser_worker.py`. + +Use a single `Target.getTargets` response for page inventory and opener metadata, preserving filtering and target identity. Remove duplicate Page/DOM/Runtime/Network enables only where pinned browser-harness 0.1.13 already guarantees attempts on attachment. Cache browser-version-derived select-all modifiers per backend lifetime. Reuse the already obtained tab result during fill URL verification. Add exact call-count and behavioral regressions. Do not lengthen polling intervals as a speculative throughput optimization or cache changing document/target state across operations. + +### Slice C: AX-preserving observation batching + +Owner `browser-hotpath`, serialized after B, same exclusive paths. Use an internal Python-owned script constant to avoid new runtime assets and packaging changes. + +Introduce a private optional batch-metadata seam with the legacy backend/fake fallback. Keep full AX candidate discovery and ordering. Resolve eligible backend node IDs into a bounded temporary object group; evaluate existing value-free DOM metadata for a small chunk together, using full AX snapshot role/name rather than redundant partial AX reads; release the object group in finally. Never retain remote node handles or geometry across observations. Keep fresh existing single-node AX/DOM revalidation before act. Cover distinct AX/DOM labels, password/value privacy, disabled/hidden/disconnected nodes, duplicate candidates, limits, malformed/partial responses, navigation/detachment and cleanup. Measure against the baseline before retaining. If the compatible batch path cannot demonstrate safe benefit, retain B and document the blocked experiment instead of weakening contracts. + +### Slice D: local origin-context calculation + +Owner `runtime-hotpath` (worker). Exclusive paths: `extensions/fast-browser/runtime.ts`, `extensions/fast-browser/runtime.test.ts`. + +Compute `narrowOriginPolicy` locally, retaining runtime closed/aborted behavior, serialized session semantics and current validation/error shaping. Subsequent transport/worker operations must still enforce the returned context. Preserve compatibility for direct legacy transport requests. Test no transport dispatch for narrowing, context propagation, rejected broadening, base restoration, closed/aborted sessions and concurrency behavior. Do not remove transport validation, serialized lanes, or change error/mutation semantics to save time. + +### Orchestration and documentation + +Parent owns `docs/plans/fast-browser-performance.md`, `docs/fast-browser-performance.md`, and the performance section of `extensions/fast-browser/README.md`. No code implementation is assigned to the parent. Record measured boundaries, exact commands, baseline identity, limitations and retained results. Add `docs/todos.md` evidence only for a newly encountered tool defect that cannot be resolved in scope. + +Each mutating assignment has a 15-minute deadline. B and C are separate assignments to the same owner; A and D may run in parallel because paths are disjoint. Workers do not commit, delegate mutation, alter dependencies, modify other owners' files, or integrate branches. Parent records before/after Git evidence and inspects every returned diff. A worker hitting its deadline must stop and report its partial state; reassignment requires proven completion. + +## Ownership + +| Assignment | Owner identity | Deadline | Exclusive paths | Before/after Git evidence | Outcome | +| --- | --- | --- | --- | --- | --- | +| Research browser | browser-performance-research / scout | complete | none; read-only | source unchanged | delivered | +| Research runtime | runtime-performance-research / scout | complete | none; read-only | source unchanged | delivered | +| A benchmark | performance-benchmark / worker | 15 minutes | four listed new files | capture before dispatch | pending | +| B browser duplication | browser-hotpath / worker | 15 minutes | worker Python + Python tests | capture before dispatch | pending | +| C observation batching | browser-hotpath / worker | 15 minutes after B | same Python files | capture before dispatch | pending | +| D origin context | runtime-hotpath / worker | 15 minutes | runtime.ts + runtime.test.ts | capture before dispatch | pending | +| Docs/integration | parent | phase bounded | listed docs/README only | record checkpoints | pending | + +## Verification matrix + +| Surface | Command | Exit code | Outcome | +| --- | --- | --- | --- | +| Baseline typecheck | `bun run typecheck` | 0 | Passed; an accidental nonexistent chained test command then failed, unrelated to repository correctness. | +| Baseline full tests | `bun run test` | 0 | Passed. | +| Baseline fast-browser | `bun run test:fast-browser` | 0 | 220 TS passed, 2 skipped; 42 Python passed. | +| Focused Python | pinned Python `-m unittest discover -s extensions/fast-browser -p 'test_*.py'` | pending | Include counting and stale/privacy regressions. | +| Focused runtime/benchmark | `node --test --experimental-strip-types extensions/fast-browser/runtime.test.ts extensions/fast-browser/performance-benchmark.test.ts` | pending | Run after implementation. | +| Integrated static/offline | `bun run typecheck && bun run test` | pending | Existing root suite; no new dependency. | +| Scenario validation | scenario-cli `--check --format json` for existing local-form/login/popup examples | pending | All parse/preflight. | +| Real runtime | `FAST_BROWSER_DEVELOPMENT_LIVE=1 node --test --experimental-strip-types extensions/fast-browser/development-live-validation.test.ts` | pending | Owned local Chromium, private artifacts, cleanup. | +| Paired benchmark | new runner CLI, exact command recorded after runner exists | pending | At least 20 paired warm attempts, cold separately; baseline from Git, not copied production code. | +| Hygiene/package | `git diff --check`; root package tests | pending | Ensure any new runtime resource ships. | + +## Review + +Review fixed point: final verified product commits against the fetched source base. Specification is the approved committed record. Standards sources are repository/global AGENTS.md plus fast-browser contracts. + +Two new read-only Git-capable `claim-verifier` sessions independently review standards and specification. All authors, parent, and worker identities are excluded. Parent pins base, product HEAD, clean status, tree, commit list and SHA-256 of `git diff --binary --full-index --find-renames ... --`. Reviewers reproduce the digest. Accepted fixes go back to an implementer, followed by verification and fresh independent review. + +## Integration and rollback + +Commit checked slices locally, fetch source again and rebase only if needed, reverify any changed tree, then review. Fast-forward integration only. Existing untracked source material stays untouched; if the source checkout is still dirty, pause integration rather than moving or deleting it. Approval of omission does not authorize removing the unrelated file. + +After safe integration, make one record-only closure commit, prove reachability, and remove only the clean owned worktree/branch. No push or publish. Rollback is a normal revert of focused optimization commits; retain benchmark tooling if useful. If batching fails acceptance, drop only its uncommitted owned changes through a reviewed non-destructive patch, preserving other slices and benchmark evidence. + +## Checkpoints and blockers + +- Last committed valid checkpoint: initial approved record is being committed before product mutation. +- Blockers: none for implementation. Source still has only the approved unrelated untracked file. +- Worktree creation verified at the specified base and path. `/.worktrees/` was already in local Git excludes; no exclude change was needed. +- Next safe action: dispatch A/B/D concurrently after the record commit.