49 lines
3.1 KiB
Markdown
49 lines
3.1 KiB
Markdown
# Fast Browser live demo
|
|
|
|
The real Pi `fast_browser_observe` and `fast_browser_run` tools completed the harmless local form demo, using Typesafe JEV for semantic control selection and `openai-codex/gpt-5.6-luna` for greeting composition. The auxiliary Codex request explicitly used `reasoningEffort: none`. Every mutation received actual Pi UI confirmation.
|
|
|
|
## Environment
|
|
|
|
- Dedicated Chrome profile, local CDP endpoint `http://127.0.0.1:9227`.
|
|
- Local fixture `http://127.0.0.1:8767/form.html`; only that origin allowed.
|
|
- Separate project-local Python environment with browser-harness0.1.13.
|
|
- No hosted browser service. JEV and Pi-model inference used remote APIs.
|
|
|
|
## First attempt and recovery
|
|
|
|
JEV selected the display-name input and the approved literal `Ada` fill was verified. JEV then selected the greeting field, but Codex Spark rejected generation because that model was unsupported with the current ChatGPT-account authentication. No greeting fill or Continue click occurred in that failed attempt.
|
|
|
|
The user explicitly selected GPT5.6 Luna with reasoning disabled. The launcher and auxiliary-call reasoning policy were updated; the remaining two steps were resumed without repeating the name fill.
|
|
|
|
## Successful remaining two-step run
|
|
|
|
| Stage | Greeting fill | Continue click |
|
|
| --- | ---: | ---: |
|
|
| Observation | 148 ms | 11 ms |
|
|
| JEV decision | 374 ms | 207 ms |
|
|
| JEV confidence | 0.90 | 0.86 |
|
|
| Luna text generation | 2,042 ms | — |
|
|
| User approval | 1,594 ms | 1,692 ms |
|
|
| Action and verification | 11 ms | 8 ms |
|
|
|
|
Total elapsed for these **remaining two steps**: **6,087 ms**, including **3,286 ms** of human approval time. Excluding approval wait gives approximately2,801ms, but this is not a separate autonomous benchmark. It excludes the earlier name fill, failed Spark attempt, setup and diagnostics.
|
|
|
|
Results:
|
|
- Generated greeting field value verified.
|
|
- Continue clicked once, then previously absent visible text `Fixture complete` verified.
|
|
- Both mutations reported `mutationState: verified`; no uncertain action outcome remained.
|
|
- No blind retries or silent model substitution.
|
|
|
|
The generated greeting was displayed for user approval and in the dedicated browser. This report intentionally does not duplicate field contents.
|
|
|
|
## Conclusions and next work
|
|
|
|
This validates the end-to-end integration: local observation, constrained JEV selection, separate Pi-model composition, exact-action approval, local execution and deterministic postcondition checks. It does not establish reliability on arbitrary websites, population-level confidence calibration or a speed advantage over another agent.
|
|
|
|
Next priorities:
|
|
1. Replace environment-only non-secret settings with project-local/runtime configuration so model changes do not require restarting Pi.
|
|
2. Compare JEV and a small-model-only selector on identical candidate sets and tasks.
|
|
3. Measure repeated-run success, p50/p95 latency, startup/observation overhead and fallback cost, separating human approval waits.
|
|
|
|
Demo setup and owned resource IDs are in ignored `.pi/fast-browser-demo/README.md`; ask the user before closing the dedicated browser.
|