jb/paseo
1
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

Surface codex provider tool events live

Browse Source
This commit is contained in:
Mohamed Boudra
2025-11-15 19:41:40 +01:00
parent 9dcc8314b2
commit 8e6508f01f
3 changed files with 286 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
plan.md
+2 -1
View File
@@ -74,7 +74,8 @@
- Split OpenAI TTS responses into chunked `audio_output` payloads via a streaming `TTSManager` pipeline (`packages/server/src/server/agent/tts-manager.ts`, `packages/server/src/server/agent/tts-openai.ts`) that tracks per-chunk acknowledgements, adds utterance IDs, and resolves once every chunk finishes. The app’s session context now keeps audio groups active until the last chunk (or an error) lands so playback begins as soon as each chunk arrives without waiting for the full clip (`packages/app/src/contexts/session-context.tsx`). `npm run typecheck --workspace=@voice-dev/server` and `npm run typecheck --workspace=@voice-dev/app` still fail in the pre-existing cross-workspace stream harness files (see earlier tasks); no new regressions were introduced.
- [x] Add telemetry for barge-in latency (speech start → playback stop, chunk arrival → LLM abort) to measure improvements per milestone.
- `RealtimeProvider` now records when speech detection interrupts active playback and logs `[Telemetry] barge_in.playback_stop_latency` once the app reports audio stopped, while the server's `Session.handleRealtimeSpeechStart` logs `[Telemetry] barge_in.llm_abort_latency` after aborting live LLM runs—giving us measurable client + server latency metrics per turn.
- [ ] Codex tool calls aren’t visible in the live stream (they only show up after hydrating/refreshing); fix the reducer/rendering path so Codex tool events appear in real time the same way Claude’s do.
- [x] Codex tool calls aren’t visible in the live stream (they only show up after hydrating/refreshing); fix the reducer/rendering path so Codex tool events appear in real time the same way Claude’s do.
- Stream reducer now inspects Codex `provider_event` payloads and converts command/file/MCP/web search items into agent tool entries immediately, so the UI shows the pills while the turn is running instead of waiting for hydration; `stream.test.ts` gained coverage proving provider events alone still surface command + MCP calls, and `npm run test --workspace=@voice-dev/app -- src/types/stream.test.ts` passes.
- [ ] Update the AgentInput “Cancel” control to use a square stop icon button (consistent with media players) so interrupting an agent is visually distinct from other actions.
- [ ] When the agent input has text and we’re not in the cancellable state, replace both the mic (Dictate) and Realtime buttons with a single Send button—only surface Dictate/Realtime when the input is empty or the agent is running.
- [ ] Remove the agent status pill next to the permission selector; instead show a “working” indicator inside the chat scroll view itself (three bouncing dots animation) whenever the agent is busy.