jb/paseo
1
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

feat: keep agents running during orchestrator speech

Browse Source
This commit is contained in:
Mohamed Boudra
2025-11-15 19:21:57 +01:00
parent 24a245bd17
commit 4bd36c0808
2 changed files with 13 additions and 19 deletions
Download Patch File Download Diff File Expand all files Collapse all files
plan.md
+2 -1
View File
@@ -62,7 +62,8 @@
- Session context now tracks a `focusedAgentId` that each agent screen sets/clears on mount, and the realtime provider looks at that value to send a `cancel_agent_request` before issuing voice aborts so the next spoken prompt doesn't pile onto a running turn. Attempted `npm run typecheck --workspace=@voice-dev/app` but it still fails in the pre-existing stream harness/tests complaining about `parsed*` fields.
- [x] Extend focused-agent tracking to orchestrator mode so realtime interruptions cancel the agent that’s actually active, even when no agent detail screen is open.
- Session context now keeps a derived auto-focus pointing at the most recent running agent whenever no explicit agent screen is selected, so realtime speech aborts send `cancel_agent_request` for that agent before streaming audio; agent screens continue to override the focus via `setFocusedAgentId`.
- [ ] Keep agent runs alive while speaking to the orchestrator: realtime speech should **not** cancel the focused agent by default. Instead, only interrupt when the user explicitly addresses that agent again or issues a cancel command.
- [x] Keep agent runs alive while speaking to the orchestrator: realtime speech should **not** cancel the focused agent by default. Instead, only interrupt when the user explicitly addresses that agent again or issues a cancel command.
- Removed the automatic `cancel_agent_request` that fired on every realtime speech start, so orchestrator dictation no longer tears down whichever agent is in focus; users can still stop runs explicitly via the agent cancel control while the orchestrator abort logic (`abort_request`) remains intact.
- [x] Harden playback stop/queue clearing so speech detection purges suppressed audio and the server stops emitting TTS while the user is talking.
- Speech detection now aborts any in-flight TTS generation, clears buffered realtime segments (pending chunks plus partial buffers), and cancels pending playbacks before delegating to the existing abort flow, so no additional audio_output events fire once the user starts talking.
- [x] Prototype chunked audio input: emit smaller PCM frames with `isLast=false`, update `handleAudioChunk` to buffer and trigger STT once the minimum duration is reached.