From 67877297d01f05745b76c7129fd39fe39c8da4a9 Mon Sep 17 00:00:00 2001 From: JB Date: Wed, 2 Sep 2026 21:07:11 -0400 Subject: [PATCH] docs(dictate): plan configurable ASR backends --- .../dictate/configurable-asr-backends-plan.md | 415 ++++++++++++++++++ 1 file changed, 415 insertions(+) create mode 100644 docs/dictate/configurable-asr-backends-plan.md diff --git a/docs/dictate/configurable-asr-backends-plan.md b/docs/dictate/configurable-asr-backends-plan.md new file mode 100644 index 0000000..7b6dd6a --- /dev/null +++ b/docs/dictate/configurable-asr-backends-plan.md @@ -0,0 +1,415 @@ +# Configurable Dictate ASR backends + +Status: proposed plan, 01SEP2026 + +## Decisions + +- Keep microphone capture and every ASR implementation in the existing host-side `pi-dictate-service` boundary. +- Keep provider settings and API keys in host-owned, mode-`0600` configuration outside Pi's nono sandbox. The Pi extension must never read or transmit cloud credentials. +- Select one backend in host configuration and apply the change by restarting the socket-activated service. Do not add per-session or per-generation provider selection to the Pi protocol. +- Support three backend profiles: + - local NeMo/Parakeet; + - Meta Model API Muse Voice Transcribe; + - OpenAI Realtime transcription. +- Stream microphone audio to both cloud providers while recording. Preserve the current push-to-start, push-to-stop interaction and paste only one finalized transcript after stop. +- Keep local as the migration default. Cloud audio and cost require an explicit backend selection and credential setup. +- Default new OpenAI configuration to `gpt-live-transcribe`. Permit a configurable OpenAI transcription model, and document `gpt-transcribe` as the commit-oriented alternative. Do not make `whisper-1` the default: current OpenAI guidance recommends the GPT transcription models, and `whisper-1` is not the selected realtime path. + +## Goals + +1. A user can configure one local, Meta, or OpenAI ASR backend without changing extension source. +2. Local mode retains its offline runtime, pinned-model verification, coding dictionary, and network-denied systemd sandbox. +3. Meta and OpenAI modes connect before recording, stream bounded PCM without writing audio files, finalize on stop, and return one bounded transcript. +4. The extension continues to see only bounded state, level, result, and error events over `/run/user/1000/pi-dictate.sock`. +5. API keys, provider wire events, partial hypotheses, and raw audio never cross into Pi. +6. Switching backend is fail-closed: configuration, installed dependency profile, and systemd network hardening must agree before the service starts. + +## Non-goals + +- Automatic failover between providers. +- Sending one recording to multiple providers. +- A runtime backend picker in Pi. +- Partial text in Pi's editor. +- Voice-activity-controlled submission; `Alt+M` remains the turn boundary. +- Diarized editor output, timestamps, confidence scores, or translation. +- Arbitrary local inference runtimes in the first slice. The local adapter remains NeMo-compatible; its model archive, checksum, terms file, and boost can become configuration. +- Reusing credentials from Pi's provider auth store. Dictate gets dedicated host credentials so the service does not need access to all of `~/.pi/agent/auth.json`. + +## Current architecture and constraints + +- `extensions/dictate/index.ts` is a strict protocol-v2 Unix-socket client. It owns no audio, model, or network capability and pastes only a current finalized result. +- `services/dictate/src/pi_dictate_service/server.py` serializes one generation globally and currently assumes a lazy `TranscriptionModel` with `load()` followed by `transcribe(samples)` after recording. +- `services/dictate/src/pi_dictate_service/recorder.py` launches `pw-record` at 16 kHz mono PCM16, drains stdout/stderr in threads, retains at most five minutes of PCM, and derives the local level meter. +- `services/dictate/src/pi_dictate_service/model.py` supports one explicit `.nemo` archive, verifies its SHA-256, applies the coding dictionary, and runs offline on CUDA. +- `systemd/user/pi-dictate.service` denies all IP networking and limits address families to `AF_UNIX`. That must remain true in local mode but cannot be shared unchanged by cloud mode. +- `services/dictate/install.sh` always installs the full NeMo/CUDA environment and pinned model. Cloud-only installation should not require either. +- `profiles/nono/pi-local.json` already grants only the exact Dictate Unix socket and explicitly denies the installed local runtime. Add a deny for `~/.config/pi-dictate` so choosing that directory as `$WORKDIR` cannot expose provider keys; no new Pi-side network, credential, PipeWire, model, or device grant is required. + +## External API contract baseline + +The implementation must turn the current provider documentation into checked fixtures before production code is merged, because Muse Voice Transcribe was released on the date of this plan and its contract may still move. + +### Meta Muse Voice Transcribe + +Primary references: + +- [Meta announcement](https://research.meta.ai/blog/introducing-muse-voice-transcribe) +- [Meta speech-to-text guide](https://ai.developer.meta.com/docs/speech-to-text) +- [Meta realtime voice API reference](https://ai.developer.meta.com/docs/api-reference/voice/realtime) +- [Meta voice event schemas](https://ai.developer.meta.com/docs/api-reference/voice/schemas) +- [Meta Model API authentication](https://ai.developer.meta.com/docs/getting-started/authentication) +- [Meta voice cookbook](https://github.com/meta-models/meta-model-cookbook/blob/main/06_muse_voice/01_voice_api_fundamentals/README.md) + +Current documented shape: + +- WebSocket endpoint: `wss://api.meta.ai/v1/asr/realtime`, with an optional `sessionId` query parameter. +- Model: `muse-voice-transcribe-1.0`; no omitted-model default is documented. +- The first text frame is a handshake requiring `authorization.accessToken`, `audioEncoding`, and `model`. The key value is `Bearer ` inside that object; an HTTP authorization header or API-key query parameter is ignored. +- The public API accepts signed little-endian mono PCM16 as binary frames. Use native `PCM_24KHZ` (48,000 bytes/second); `PCM_16KHZ` is also accepted and server-resampled. +- `PUSH_TO_TALK` is the mode matching Dictate's one explicit turn. Defaults also include `partialMode: "CUMULATIVE"` and `emitAudioProgress: true`. +- Wait for the server's `{ "sessionId": "..." }` acknowledgement before sending audio. Frames carry no headers, timestamps, sequence numbers, or semantic boundaries; approximately 80 ms pacing is guidance rather than a required frame size. +- End input with the text frame `{ "type": "endStream" }`, continue reading, require a `transcript` event with `final: true`, then accept normal close code `1000`. +- Cumulative partials replace the prior hypothesis. Dictate ignores partials, turn boundaries, endpointing, and speaker events and retains only the explicit bounded final transcript. +- `keywords` and `languageBias` are handshake-only hints. The existing coding dictionary can feed `keywords`; no free-form prompt field is documented. +- Meta documents a 60-minute session, at most five seconds of audio backlog with approximately realtime pacing, eight concurrent streams, and 1,000 streams/hour. Dictate's own five-minute cap remains stricter. +- Treat close codes `1008` (invalid request/pacing), `1011` (backend or duration), and `1013` (rate limit), plus bounded `error` events, as provider failures. + +The contract spike must freeze these fields as sanitized fixtures and confirm them against a live non-sensitive sample, especially because Muse Voice Transcribe was released on the date of this plan. Do not copy the undocumented `shortwave.facebook.com` Muse Code endpoint used by third-party extensions. + +### OpenAI Realtime transcription + +Primary references: + +- [Realtime transcription guide](https://developers.openai.com/api/docs/guides/realtime-transcription) +- [Realtime WebSocket guide](https://developers.openai.com/api/docs/guides/realtime-websocket) +- [Speech-to-text guide](https://developers.openai.com/api/docs/guides/speech-to-text) +- [`gpt-live-transcribe` model](https://developers.openai.com/api/docs/models/gpt-live-transcribe) + +Current documented shape: + +- Use the GA server-side WebSocket connection `wss://api.openai.com/v1/realtime?intent=transcription` with `Authorization: Bearer `. Do not mix this with the deprecated `/v1/realtime/transcription_sessions` schema or put the transcription model in the connection query. +- After `session.created`, send the nested GA `session.update` shape with `type: "transcription"`, the configured transcription model, 24 kHz mono PCM16, and `turn_detection: null`; wait for `session.updated` before capture. +- PCM is base64 encoded in bounded `input_audio_buffer.append` JSON events. The API's per-append limit is 15 MiB, but Dictate must use much smaller realtime chunks. +- Stop sends `input_audio_buffer.commit`; it starts transcription without creating an assistant response. +- Partial text arrives as `conversation.item.input_audio_transcription.delta`; final text arrives as `conversation.item.input_audio_transcription.completed` and is correlated by `item_id`. Transcription-specific failures use `conversation.item.input_audio_transcription.failed`; general protocol failures use `error`. +- `gpt-live-transcribe` is the recommended low-latency model and supports prompt, keywords, languages, and delay tuning. +- `gpt-transcribe` can use the same Realtime session but begins transcription after commit and can return detected languages. +- `whisper-1` remains a file-transcription model for older integrations, translation, and timestamp use cases; its model page says realtime transcription is unsupported. +- OpenAI documents a 60-minute Realtime session. Dictate's own five-minute cap remains stricter. +- The current guide recommends `delay` for `gpt-live-transcribe`, while one API reference says it is supported only by `gpt-realtime-whisper`; Phase 0 must verify actual GA behavior and omit `delay` if the selected model rejects it. + +## Proposed host configuration + +Add `services/dictate/src/pi_dictate_service/config.py` with strict, versioned parsing. The installed authority is: + +```text +~/.config/pi-dictate/ + config.json + credentials/ + meta-api-key + openai-api-key +``` + +Illustrative schema (field names may be refined during implementation): + +```json +{ + "version": 1, + "backend": "openai", + "idleSeconds": 900, + "local": { + "adapter": "nemo-parakeet", + "modelPath": "~/.local/share/pi-dictate/models/parakeet.nemo", + "modelSha256": "...", + "termsPath": "~/.local/share/pi-dictate/config/coding-terms.txt", + "boostAlpha": 1.0 + }, + "meta": { + "model": "muse-voice-transcribe-1.0", + "apiKeyFile": "~/.config/pi-dictate/credentials/meta-api-key", + "termsPath": "~/.local/share/pi-dictate/config/coding-terms.txt" + }, + "openai": { + "model": "gpt-live-transcribe", + "apiKeyFile": "~/.config/pi-dictate/credentials/openai-api-key", + "delay": "medium", + "prompt": "Coding dictation", + "termsPath": "~/.local/share/pi-dictate/config/coding-terms.txt", + "languages": ["en"] + } +} +``` + +Configuration rules: + +- Reject unknown root and selected-backend keys, malformed JSON, unsupported versions, invalid numeric bounds, and incompatible options. +- Read only the selected backend's credential. Never require or expose credentials for inactive backends. +- Credential files must be regular files owned by the service user, not symlinks, with no group/other permission bits and a small byte limit. Trim one trailing newline; reject empty or multi-line values. +- Never accept literal keys on command lines, in systemd environment variables, in the Pi protocol, or in logs. +- Keep provider endpoints fixed in production. Inject transports in tests rather than adding a credential-exfiltration-prone endpoint override. If a localhost override is needed for an opt-in integration harness, gate it behind an explicit test-only setting. +- Keep the current pinned Parakeet revision as the generated local default. A custom local archive must supply an explicit checksum and use a supported NeMo adapter. +- Validate coding terms against the existing byte/count limits before mapping them to provider keyword limits. Provider-specific filtering must reject unsupported characters instead of silently altering terms. +- Configuration is read at process startup. Changes require the documented configure operation and service restart. + +## Backend-neutral service design + +Replace the model-only assumption with a backend that creates one prepared turn: + +```text +TranscriptionBackend + descriptor # bounded backend/model diagnostics + start_turn() -> Turn # load/reuse local model or connect/auth cloud session + +TranscriptionTurn + sample_rate_hz + retain_pcm + enqueue_pcm(chunk) # nonblocking, bounded + finish(capture) -> text # infer locally or commit/await cloud final + poll_error() + cancel() + close() +``` + +`start_turn()` and `finish()` run outside the socket loop. A local backend persists its loaded model across turns; each cloud turn owns its own WebSocket and provider workers. + +Exact Python protocols can differ, but the following invariants are mandatory: + +1. `start_turn()` completes before `pw-record` starts. The UI must never say `listening` before provider auth/handshake succeeds. +2. The backend declares the recorder rate: 16 kHz for the existing local adapter and 24 kHz for Meta/OpenAI. +3. Local mode retains bounded PCM for post-stop inference. Cloud mode streams through a bounded queue and does not retain the whole recording. +4. The recorder drain thread must never perform network I/O. It enqueues PCM quickly; a provider worker drains the queue. +5. Queue overflow or a stalled provider is a terminal error. Stop capture and discard the generation; never silently drop audio and return a plausible but incomplete transcript. +6. On stop, signal `pw-record`, drain stdout to EOF, wait for child close, then enqueue the provider end/commit marker after all PCM. This preserves the existing no-truncated-tail guarantee. +7. Ignore partial hypotheses at the service boundary. Only an explicit provider final event may produce the protocol `result`. +8. Every provider event, transcript, close reason, and error is byte-bounded before parsing or logging. +9. Cloud transports use normal certificate and hostname verification, fixed production endpoints, no credential-bearing redirects, and no ambient proxy environment unless a separately reviewed proxy feature is added. +10. Cancellation/disconnect tombstones delivery immediately, stops capture, closes the provider turn, joins worker threads with bounded timeouts, and suppresses every later final event. +11. The service remains globally single-generation. Idle exit is suspended during local load/inference and cloud connect/stream/finalization. + +### Lifecycle + +The existing extension states remain valid: + +```text +idle -> connecting -> recording -> stopping -> transcribing -> idle +``` + +Service interpretation becomes backend-neutral: + +- `loading`: preparing the selected backend (local model load or cloud connect/auth/session setup); +- `listening`: recorder and, for cloud mode, provider stream are both ready; +- `stopping`: recorder shutdown and complete PCM drain; +- `transcribing`: local inference or cloud commit/end-of-input plus final-result wait; +- `idle`: generation resources are closed and no result remains deliverable. + +Change extension text from `loading model...` to `preparing transcription...`. + +## Provider adapters + +### Local NeMo/Parakeet + +Refactor `NemoParakeetModel` behind `LocalNemoBackend` without changing its observable transcription behavior: + +- lazy checksum-verified load; +- 16 kHz PCM16 capture and current five-minute/9.6 MB bound; +- offline environment and CUDA behavior; +- coding-term phrase boosting; +- post-stop conversion and inference; +- warm model reuse until idle service exit. + +This slice must pass the existing service tests before either cloud adapter is introduced. + +### Meta + +Add `backends/meta.py` and a small bounded WebSocket transport: + +- read and validate the dedicated Meta key; +- connect only to the public `api.meta.ai` realtime endpoint; +- send the documented first-frame bearer handshake for `muse-voice-transcribe-1.0`, `PCM_24KHZ`, `PUSH_TO_TALK`, and `CUMULATIVE` partials; +- wait for the `sessionId` acknowledgement before starting capture; +- send recorder bytes as ordered, approximately realtime-paced binary frames and enforce Meta's five-second backlog limit locally; +- consume but do not expose cumulative partial hypotheses; +- send `{ "type": "endStream" }` after recorder EOF; +- require a `transcript` event with `final: true`, then consume the normal `1000` close; +- map bounded provider error events and close codes `1008`, `1011`, and `1013`, plus auth, timeout, malformed-event, and premature-close failures, to stable `meta_*` service error codes without including the key or raw provider payload. + +Diarization and endpointing events are parsed defensively and ignored for editor delivery in this feature. + +### OpenAI + +Add `backends/openai.py`: + +- connect to the GA `intent=transcription` URL with the standard key in the `Authorization` header, never the URL; +- wait for `session.created`, send the nested GA `session.update` configuration with 24 kHz PCM and `turn_detection: null`, then wait for `session.updated` before capture; +- emit bounded `input_audio_buffer.append` events with base64 PCM, far below the provider's 15 MiB event limit; +- commit only after recorder EOF; +- correlate final completion to the committed item and ignore stale/unmatched events; +- accept configurable model, prompt, keywords, languages, and provider-supported delay values; +- default to `gpt-live-transcribe`; document and test `gpt-transcribe`'s commit-oriented behavior, and contract-test whether `delay` is accepted; +- map `conversation.item.input_audio_transcription.failed`, general `error` events, auth/rate limits, timeouts, malformed events, and premature closes to stable `openai_*` errors without logging request bodies or keys. + +Do not build around the OpenAI SDK unless it materially simplifies Realtime event handling without hiding cancellation, queue bounds, or frame limits. A direct WebSocket adapter is likely smaller and easier to fixture-test. + +## Pi protocol and extension work + +Preserve protocol version 2 unless the implementation discovers a required breaking frame change. The existing state/result contract already represents all three backends. + +Additive changes are sufficient: + +- Allow `pong` to report bounded `backend` and `model` strings for diagnostics. Never include paths, endpoints with query data, or credential state. +- Define `ready` as whether the selected backend can begin immediately without preparation; the extension must not depend on this field for state transitions. +- Add `/dictate-status` to establish a short hello/ping when idle and report selected backend/model. It is read-only and does not expose host config or credentials. +- Keep `Alt+M`, `Alt+N`, `/dictate-cancel`, stale-generation rejection, transcript bounds, and `pasteToEditor()` behavior unchanged. +- Do not add a Pi-side configuration writer. Backend changes are privileged host operations outside the agent sandbox. + +If additive diagnostic fields are not worth the extra command/client complexity, omit them rather than bumping the protocol solely for display. + +## Installer, packaging, and systemd + +### Dependency profiles + +Split `services/dictate/pyproject.toml` into a small core plus locked optional profiles: + +- core: protocol, recorder, configuration, and service code; +- local extra: NeMo, NumPy, PyTorch, and current CUDA stack; +- cloud extra: the selected maintained WebSocket client and its locked dependencies. + +Cloud-only installation must not download the Parakeet archive or install the multi-gigabyte local CUDA stack. Local installation remains reproducible and checksum pinned. + +### Configure operation + +Extend `services/dictate/install.sh` (or add an installer-owned companion command) with explicit operations such as: + +```text +install.sh install --backend local [--model-source PATH] +install.sh install --backend meta --api-key-file PATH +install.sh install --backend openai --api-key-file PATH +install.sh configure --backend ... +install.sh check +``` + +- `--api-key-file` copies from a file into the fixed credential directory with mode `0600`; never accept a literal secret argument. +- Write config and generated systemd hardening atomically, run validation, daemon-reload, and restart the service/socket as one operation. +- Preserve existing config, keys, coding dictionary, and valid model archives on reinstall unless an explicit replacement option is supplied. +- Make uninstall's treatment of `~/.config/pi-dictate` explicit; prefer preserving credentials by default with a separate `--purge-config` confirmation path. +- Resolve the existing installer model-reuse todo while this flow is being changed: checksum and reuse a valid installed archive before attempting a network download. + +### Backend-specific network hardening + +Do not weaken local mode to make cloud mode work. + +- Keep common hardening (`NoNewPrivileges`, private temp, core/swap limits, system protection, read-only home, and socket ownership) in the base service unit. Clear ambient `HTTP_PROXY`, `HTTPS_PROXY`, and `ALL_PROXY` variables unless proxy support is separately designed and reviewed. +- Generate/install an offline drop-in for local mode containing `RestrictAddressFamilies=AF_UNIX` and `IPAddressDeny=any`. +- Generate/install a cloud drop-in permitting only `AF_UNIX`, `AF_INET`, and `AF_INET6` and no inbound listener beyond the inherited Unix socket. DNS and outbound TLS must be tested under the final unit. +- Stamp the expected network mode in the generated unit environment. At startup, compare it with the selected backend and refuse to run on mismatch. This prevents a hand-edited local config from silently running under cloud-capable hardening, and a cloud backend from failing mysteriously under the offline profile. +- Restrict service reads to the installed runtime, selected model/dictionary paths, and `~/.config/pi-dictate` where practical. +- Document that cloud mode necessarily grants the host transcription process general outbound IP connectivity; systemd cannot safely express a stable hostname allowlist for changing provider addresses. + +Update `profiles/nono/pi-local.json` only to deny `~/.config/pi-dictate`, mirroring the existing runtime deny. The resolved policy must add no capability: Pi still reaches only `/run/user/1000/pi-dictate.sock` and cannot read the host config or keys even if launched with that directory as `$WORKDIR`. + +## Implementation phases + +### Phase 0: Freeze provider contracts and run opt-in spikes + +1. Save sanitized request/event fixtures from the current public Meta and OpenAI docs. +2. With explicit user consent and test keys, run a short prerecorded, non-sensitive PCM fixture against each API outside Pi. +3. Confirm handshake, framing, finalization, close/error behavior, first/final latency, model availability, and documented billing/rate limits. +4. Record exact model IDs and API doc dates. Treat a changed or unavailable Meta contract as a planning gate, not a reason to use an undocumented endpoint. + +Completion: one minimal client per provider produces an exact final transcript and no unresolved required schema field remains. + +### Phase 1: Add strict host config and preserve local behavior + +1. Add the versioned config and credential readers with permission, ownership, size, and redaction checks. +2. Move local model path/checksum/terms/boost/idle settings from environment assumptions into typed config. +3. Introduce backend/turn protocols and wrap the current NeMo path without changing recorder or protocol behavior. +4. Add config and local-regression tests. + +Completion: local Dictate passes all existing tests and runs with generated local config under the same offline unit restrictions. + +### Phase 2: Generalize capture for realtime sinks + +1. Make recorder sample rate and byte-duration cap explicit and tested. +2. Add a nonblocking PCM sink plus fixed queue/backpressure limits while retaining local-buffer mode. +3. Add ordered finish, provider-error polling, cancellation, and bounded worker joins to the service core. +4. Test 16/24 kHz commands, exact five-minute limits, stalled consumers, queue overflow, final PCM ordering, cancellation in every phase, and no thread/resource leaks. + +Completion: a fake realtime backend receives byte-for-byte ordered PCM through stop and returns one final result; local behavior remains unchanged. + +### Phase 3: Implement Meta + +1. Implement the public Muse handshake and event parser from Phase 0 fixtures. +2. Integrate 24 kHz binary streaming, end-of-input, final-only delivery, timeout, and cancellation. +3. Map the coding dictionary into validated Meta bias terms. +4. Add fake-server contract tests and an opt-in live smoke test. + +Completion: Meta mode starts recording only after handshake, streams bounded audio, pastes exactly one final transcript, and leaves no deliverable result after cancel/disconnect. + +### Phase 4: Implement OpenAI + +1. Implement authenticated Realtime transcription session setup. +2. Integrate base64 appends, explicit commit, item correlation, model/context settings, final-only delivery, timeout, and cancellation. +3. Test `gpt-live-transcribe` as default and `gpt-transcribe` as the documented commit-oriented alternative. +4. Add fake-server contract tests and an opt-in live smoke test. + +Completion: OpenAI mode satisfies the same user-visible and cleanup contract as Meta and local mode. + +### Phase 5: Make installation and hardening backend-aware + +1. Add locked local/cloud dependency profiles and backend-specific provisioning. +2. Add atomic install/configure/check operations and secure key copying. +3. Add local/cloud systemd hardening generation plus startup mismatch checks. +4. Following the repository's nono-profile workflow, draft and strictly validate the explicit deny for `~/.config/pi-dictate`, review the resolved policy, and obtain approval for the exact profile diff before copying it into `profiles/nono/pi-local.json`. +5. Verify the resolved profile adds no capability, then test fresh install, every switch direction, reinstall preservation, invalid config, missing/insecure key files, local network denial, cloud DNS/TLS, idle exit, reactivation, and uninstall. + +Completion: selecting a backend installs only its required runtime, applies matching hardening, and reactivates through the same Unix socket after restart. + +### Phase 6: Extension diagnostics and documentation + +1. Use backend-neutral status wording and optionally expose bounded backend/model diagnostics. +2. Add `/dictate-status` only if it can remain a simple read-only protocol operation. +3. Update `docs/dictate/protocol.md`, `extensions/dictate/README.md`, root `README.md`, attribution, privacy, troubleshooting, and operations sections. +4. Document exactly when cloud audio begins leaving the host, credential storage, provider retention/terms links, billing, five-minute cap, no partial editor insertion, and local offline guarantees. + +Completion: docs let a new user install any one backend, switch safely, understand data flow/cost, diagnose failures, and return to local offline mode. + +### Phase 7: Validation and independent review + +1. Run focused TypeScript/Python tests, typecheck, package tests, and the full repository test suite. +2. Manually test cold/warm local starts and fresh cloud sessions, stop/cancel/disconnect/shutdown in every state, empty speech, provider auth/rate-limit/outage, network loss, duration limit, two competing Pi clients, service idle exit, and socket reactivation. +3. Confirm the local unit cannot resolve/connect to the Internet and the cloud unit cannot expose a TCP listener. +4. Confirm Pi's resolved nono policy has no new grants and Pi cannot read Dictate config/keys or reach either provider directly. +5. Have an independent reviewer audit credential handling, audio retention, network hardening, provider event validation, cancellation, and installer migration. + +## Test matrix + +| Area | Local | Meta | OpenAI | +| --- | --- | --- | --- | +| Prepare before recorder | lazy model load | connect/auth/handshake | connect/auth/session ack | +| Capture | 16 kHz PCM16 retained | 24 kHz PCM16 queued/binary | 24 kHz PCM16 queued/base64 JSON | +| Stop | drain, convert, infer | drain, end marker, final | drain, commit, matching completion | +| Context terms | NeMo phrase boost | Meta bias terms | keywords/prompt/languages | +| Cancellation | tombstone load/infer | close stream, suppress final | close session, suppress final | +| Hardening | AF_UNIX, IP denied | outbound TLS enabled | outbound TLS enabled | +| Main result | bounded string | bounded explicit final | bounded matching final | +| Live smoke | GPU/model opt-in | key/cost opt-in | key/cost opt-in | + +Every provider also needs tests for malformed/oversized events, stale results, partial-only close, empty final, invalid UTF-8/JSON where applicable, auth failure, rate limit, timeout, queue overflow, recorder failure, service shutdown, and redacted errors. + +## Acceptance criteria + +- One strict host configuration selects local, Meta, or OpenAI and takes effect only after a validated service restart. +- Existing users remain local until they explicitly opt into a cloud backend. +- The local model path, checksum, terms, and boost are configurable without accepting an unverified archive. +- Meta uses only the public `api.meta.ai` Muse Voice Transcribe endpoint and documented bearer contract. +- OpenAI defaults to `gpt-live-transcribe`; the model is configurable and `whisper-1` is not silently treated as Realtime-compatible. +- Provider connection/session readiness precedes recorder startup. +- Cloud PCM is streamed in order through a fixed bounded queue; no audio is silently dropped, retried to another provider, written to disk, or exposed to Pi. +- Stop drains the recorder completely before provider finalization. +- Only one explicit bounded final result for the active generation can change the editor. +- Partial events, cancellation, stale generations, disconnects, provider failures, and cleanup never insert text. +- API keys live only in dedicated host credential files, pass strict file checks, never cross the Unix protocol, and never appear in logs, errors, command lines, URLs, redirects, or ambient proxies. +- Local mode retains IP denial; cloud mode's broader outbound capability is explicit, tested, and cannot be selected under mismatched hardening. +- Cloud-only installation does not provision NeMo/CUDA/model artifacts. +- The Pi nono profile gains no new filesystem, socket, network, PipeWire, GPU, or credential capability. +- Focused and full tests, typecheck, host smoke tests, and independent security review pass.