docs(fast-browser): record measured performance and verification evidence

This commit is contained in:
jb
2026-09-21 13:47:35 -04:00
parent 2dc359c48d
commit 54303e0b5d
3 changed files with 89 additions and 10 deletions
+13 -10
View File
@@ -3,8 +3,8 @@
## Status
- Status: active
- Current phase: correcting verified close/readiness bottlenecks after deterministic reproduction.
- Next safe action: bounded read-only close verification and shorter readiness polling, followed by original live benchmark and full verification.
- Current phase: final verified product ready for independent review.
- Next safe action: pin the clean committed diff, run independent standards/specification reviews, then integrate only through the approved safe source-checkout gate.
## Request and scope
@@ -90,13 +90,13 @@ Each mutating assignment has a 15-minute deadline. B and C are separate assignme
| Baseline typecheck | `bun run typecheck` | 0 | Passed; an accidental nonexistent chained test command then failed, unrelated to repository correctness. |
| Baseline full tests | `bun run test` | 0 | Passed. |
| Baseline fast-browser | `bun run test:fast-browser` | 0 | 220 TS passed, 2 skipped; 42 Python passed. |
| Focused Python | pinned Python `-m unittest discover -s extensions/fast-browser -p 'test_*.py'` | pending | Include counting and stale/privacy regressions. |
| Focused runtime/benchmark | `node --test --experimental-strip-types extensions/fast-browser/runtime.test.ts extensions/fast-browser/performance-benchmark.test.ts` | pending | Run after implementation. |
| Integrated static/offline | `bun run typecheck && bun run test` | pending | Existing root suite; no new dependency. |
| Scenario validation | scenario-cli `--check --format json` for existing local-form/login/popup examples | pending | All parse/preflight. |
| Real runtime | `FAST_BROWSER_DEVELOPMENT_LIVE=1 node --test --experimental-strip-types extensions/fast-browser/development-live-validation.test.ts` | pending | Owned local Chromium, private artifacts, cleanup. |
| Paired benchmark | new runner CLI, exact command recorded after runner exists | pending | At least 20 paired warm attempts, cold separately; baseline from Git, not copied production code. |
| Hygiene/package | `git diff --check`; root package tests | pending | Ensure any new runtime resource ships. |
| Focused Python | `.pi/fast-browser-demo/venv/bin/python -m unittest discover -s extensions/fast-browser -p 'test_*.py'` | 0 | 61 passed, including counting, batching, stale/privacy, delayed-close and readiness fake-clock regressions. |
| Focused runtime/benchmark | individual `node --test --experimental-strip-types` runs for runtime.test.ts and performance-benchmark.test.ts | 0 | Parent verified 12 runtime and 7 benchmark tests. |
| Integrated static/offline | `bun run typecheck && bun run test` | 0 | Full repository suite passed. Fast-browser: 230 TS passed, 2 opt-in skips; 61 Python passed. No dependency installation. |
| Scenario validation | scenario-cli `--check --format json` for existing local-form/login/popup examples | 0 | All three validated; no browser started by preflight. |
| Real runtime | `FAST_BROWSER_DEVELOPMENT_LIVE=1 node --test --experimental-strip-types extensions/fast-browser/development-live-validation.test.ts` | 0 | Passed with no skip on baseline, batched worker and final corrected worker. Includes private evidence and cleanup. |
| Paired benchmark | command in `docs/fast-browser-performance.md`, 20 pairs plus 3 cold samples per arm | 0 | Final all 46 attempts passed, counters complete, cleanup passed. Warm p50 702 to 279ms; p95 729.05 to 315.30ms; CDP 17,791 to 7,407. Earlier failures retained. |
| Hygiene/package | `git diff --check`; root package tests in full suite | 0 | Passed. Metadata script is an internal Python constant; no external runtime resource needed. |
## Review
@@ -125,4 +125,7 @@ After safe integration, make one record-only closure commit, prove reachability,
- Timing probe committed 12a6bb6; parent reran 7 benchmark tests and full typecheck. Diagnostic `/tmp/fast-browser-performance-4-pairs-diagnostic-20260921.json` passed all attempts and confirms navigate ~335ms versus observe ~223 to ~88ms. Action times ~110ms remain similar. No provider latency is measured.
- Parent reproduced close tests RED: 4 focused tests, 1 expected failure and 1 expected error for delayed/permanent target presence, 2 passed for inventory/cancellation handling. Test patch `/tmp/pi-present/performance-close-red.patch`.
- Browser-hotpath correction assignment: 15 minutes, same Python worker/test paths; before HEAD 12a6bb6, only owned red tests dirty plus parent doc. Implement single-dispatch close followed by bounded inventory reads under existing ACTION_TIMEOUT_SECONDS. Replace pinned helper's 300ms readiness sleep with <=25ms bounded monotonic polling for the exact same document.readyState == complete predicate and existing timeout, retaining error propagation. Add readiness regression tests before the fix. This changes private polling cadence, not API, security posture, postconditions or budgets. Before patch `/tmp/pi-present/performance-close-red.patch`.
- Next safe action: inspect corrected worker delivery, run final stable paired benchmark and full verification. Preserve every failed result.
- Corrected close/readiness worker committed 2dc359c; parent inspected patch `/tmp/pi-present/performance-readiness-close-fix.patch` and reran 61 Python tests. Final report `/tmp/fast-browser-performance-20-pairs-final.json` passed all 46 attempts; hash and exact environment/commands are in performance documentation.
- Final verification background task bg_vjl_mubjba9s_12 exited 0 after typecheck, complete repository suite, pinned Python suite, real Chromium runtime test, scenario preflight and diff check. No failures; opt-in general-profile test remains skipped in the default suite, while the development live gate was explicitly run.
- Source fetched again: origin/main still 773c628512fdfdf5a139ae1b27833bea97d74255, no rebase needed. Source checkout still has only the unrelated untracked feasibility document; final integration requires a bounded decision or a clean checkout.
- Next safe action: commit final documentation, pin review bytes, launch independent read-only reviewers. Preserve every failed result.