Files
harness-tools/docs/fast-browser-live-demo.md
jb fd63b0ccec
fast-browser scenarios / Offline scenario contract tests (push) Failing after 44s
fast-browser scenarios / Local Chromium headless scenarios (push) Failing after 23s
fast-browser-with-jev
2026-09-20 12:24:08 -04:00

3.1 KiB

Fast Browser live demo

The real Pi fast_browser_observe and fast_browser_run tools completed the harmless local form demo, using Typesafe JEV for semantic control selection and openai-codex/gpt-5.6-luna for greeting composition. The auxiliary Codex request explicitly used reasoningEffort: none. Every mutation received actual Pi UI confirmation.

Environment

  • Dedicated Chrome profile, local CDP endpoint http://127.0.0.1:9227.
  • Local fixture http://127.0.0.1:8767/form.html; only that origin allowed.
  • Separate project-local Python environment with browser-harness0.1.13.
  • No hosted browser service. JEV and Pi-model inference used remote APIs.

First attempt and recovery

JEV selected the display-name input and the approved literal Ada fill was verified. JEV then selected the greeting field, but Codex Spark rejected generation because that model was unsupported with the current ChatGPT-account authentication. No greeting fill or Continue click occurred in that failed attempt.

The user explicitly selected GPT5.6 Luna with reasoning disabled. The launcher and auxiliary-call reasoning policy were updated; the remaining two steps were resumed without repeating the name fill.

Successful remaining two-step run

Stage Greeting fill Continue click
Observation 148 ms 11 ms
JEV decision 374 ms 207 ms
JEV confidence 0.90 0.86
Luna text generation 2,042 ms —
User approval 1,594 ms 1,692 ms
Action and verification 11 ms 8 ms

Total elapsed for these remaining two steps: 6,087 ms, including 3,286 ms of human approval time. Excluding approval wait gives approximately2,801ms, but this is not a separate autonomous benchmark. It excludes the earlier name fill, failed Spark attempt, setup and diagnostics.

Results:

  • Generated greeting field value verified.
  • Continue clicked once, then previously absent visible text Fixture complete verified.
  • Both mutations reported mutationState: verified; no uncertain action outcome remained.
  • No blind retries or silent model substitution.

The generated greeting was displayed for user approval and in the dedicated browser. This report intentionally does not duplicate field contents.

Conclusions and next work

This validates the end-to-end integration: local observation, constrained JEV selection, separate Pi-model composition, exact-action approval, local execution and deterministic postcondition checks. It does not establish reliability on arbitrary websites, population-level confidence calibration or a speed advantage over another agent.

Next priorities:

  1. Replace environment-only non-secret settings with project-local/runtime configuration so model changes do not require restarting Pi.
  2. Compare JEV and a small-model-only selector on identical candidate sets and tasks.
  3. Measure repeated-run success, p50/p95 latency, startup/observation overhead and fallback cost, separating human approval waits.

Demo setup and owned resource IDs are in ignored .pi/fast-browser-demo/README.md; ask the user before closing the dedicated browser.