Compare commits

...
5 Commits
47 changed files with 7699 additions and 500 deletions
+24 -19
View File
@@ -11,7 +11,7 @@ Personal [Pi](https://github.com/earendil-works/pi-mono) extensions, skills, glo
- the `manage-pi-links` and `nono-profiles` project-only skills under `.pi/skills/`; - the `manage-pi-links` and `nono-profiles` project-only skills under `.pi/skills/`;
- no global Pi extension symlinks. Active extensions are installed separately as pinned Gitea npm packages. - no global Pi extension symlinks. Active extensions are installed separately as pinned Gitea npm packages.
The repository contains `copy-all`, `dictate`, `effort-controls`, `file-search`, `interactive-subagents`, `lm-studio`, `native-hooks`, `prompt-snippets`, `subagents`, `todo-herdr`, `update`, and the private `workflow` extension. The link manifest does not install any of them globally. The `pptx`, `stop-slop`, `subagents`, and `xlsx` skills are also intentionally excluded. Archived resources under `__archived/` are never installed. Update `config/pi-links.json` deliberately if that policy changes. The repository contains `copy-all`, `dictate` (Linux host-service dictation), `effort-controls`, `file-search`, `interactive-subagents`, `lm-studio`, `native-hooks`, `prompt-snippets`, `subagents`, `todo-herdr`, `update`, and the private `workflow` extension. The link manifest does not install any of them globally. The `pptx`, `stop-slop`, `subagents`, and `xlsx` skills are also intentionally excluded. Archived resources under `__archived/` are never installed. Update `config/pi-links.json` deliberately if that policy changes.
## Fresh-machine setup ## Fresh-machine setup
@@ -91,13 +91,13 @@ Use `bun install --frozen-lockfile` after pulling a lockfile change and plain `b
Preview the exact changes first, then apply them: Preview the exact changes first, then apply them:
```bash ```bash
cd /srv/data/code/harness-tools cd /srv/data/code/jb-pi/harness-tools
./scripts/pi-links.mjs sync --dry-run ./scripts/pi-links.mjs sync --dry-run
./scripts/pi-links.mjs sync ./scripts/pi-links.mjs sync
./scripts/pi-links.mjs check ./scripts/pi-links.mjs check
``` ```
A successful check currently reports 41 links and zero local package entries. The manager: A successful check currently reports 39 links and zero local package entries. The manager:
- preserves unrelated global resources; - preserves unrelated global resources;
- refuses to overwrite regular files, directories, or links owned outside this repository; - refuses to overwrite regular files, directories, or links owned outside this repository;
@@ -126,7 +126,7 @@ Inside Pi:
The currently published and deployed extension set is: The currently published and deployed extension set is:
- `@jbcode/pi-dictate@0.1.0`; - `@jbcode/pi-dictate@0.2.0`;
- `@jbcode/pi-effort-controls@0.1.0`; - `@jbcode/pi-effort-controls@0.1.0`;
- `@jbcode/pi-prompt-snippets@0.1.0`; - `@jbcode/pi-prompt-snippets@0.1.0`;
- `@jbcode/pi-interactive-subagents@3.7.3`; - `@jbcode/pi-interactive-subagents@3.7.3`;
@@ -140,7 +140,7 @@ Configure the scoped Gitea npm registry and install only the packages you want.
``` ```
```bash ```bash
pi install npm:@jbcode/pi-dictate@0.1.0 pi install npm:@jbcode/pi-dictate@0.2.0
pi install npm:@jbcode/pi-effort-controls@0.1.0 pi install npm:@jbcode/pi-effort-controls@0.1.0
pi install npm:@jbcode/pi-prompt-snippets@0.1.0 pi install npm:@jbcode/pi-prompt-snippets@0.1.0
pi install npm:@jbcode/pi-interactive-subagents@3.7.3 pi install npm:@jbcode/pi-interactive-subagents@3.7.3
@@ -435,38 +435,43 @@ Do not place real provider keys in files readable by sandboxed Pi if the goal is
## Optional Dictate service ## Optional Dictate service
Dictate is Linux-only and requires: Dictate is a Linux-only host service. Every backend requires:
- PipeWire with `pw-record`; - PipeWire with `pw-record`;
- systemd user services; - systemd user services;
- an NVIDIA GPU with a working CUDA runtime;
- `uv`; - `uv`;
- network access for the initial locked dependency and model download; - access to the exact `/run/user/1000/pi-dictate.sock`, unless the socket path is changed for another UID.
- UID 1000 unless the hardcoded socket path is changed first.
Install the host service from the repository root: The default backend is local NeMo/Parakeet. Local installation additionally requires an NVIDIA GPU, a working CUDA runtime, network access for initial provisioning, and several gigabytes for the locked environment and pinned model. Meta and OpenAI are explicit cloud opt-ins. Cloud-only installation does not install NeMo, CUDA, or model weights, but it requires a dedicated host credential and outbound DNS/TLS access from the service.
Install one backend from the repository root:
```bash ```bash
cd /srv/data/code/harness-tools cd /srv/data/code/harness-tools
./services/dictate/install.sh ./services/dictate/install.sh install --backend local
# or:
./services/dictate/install.sh install --backend meta --api-key-file /path/to/meta-key
./services/dictate/install.sh install --backend openai --api-key-file /path/to/openai-key
```
`--api-key-file` copies a regular source file into the host-owned mode-`0600` credential directory. Never pass a literal key. Configure an installed backend or validate it with:
```bash
./services/dictate/install.sh configure --backend local
./services/dictate/install.sh check
systemctl --user status pi-dictate.socket pi-dictate.service systemctl --user status pi-dictate.socket pi-dictate.service
``` ```
The installer provisions the locked Python environment, verifies the pinned model, installs host-owned systemd units, and enables socket activation. Initial storage is several gigabytes. To reuse an existing model archive or preview the operation: Local mode remains offline after provisioning and uses the checksum-pinned Parakeet archive. Cloud mode streams bounded PCM to the selected provider only after its session is authenticated and ready. Backend changes are privileged host operations, require validation and a service restart, and fail closed if systemd network hardening does not match the selection. The service does not write audio files or expose provider credentials to Pi. Cloud provider retention, terms, and billing policies apply to audio and configured context sent with a cloud session.
```bash Restart a sandboxed Pi after profile or socket changes. In Pi, press `Alt+M` to start or stop dictation, `Alt+N` or `/dictate-cancel` to cancel, and `/dictate-status` to display bounded backend/model diagnostics. See [`extensions/dictate/README.md`](extensions/dictate/README.md) for configuration, provider links, privacy, dictionary, model, and troubleshooting details.
./services/dictate/install.sh --model-source /path/to/parakeet-tdt-0.6b-v3.nemo
./services/dictate/install.sh --dry-run --model-source /path/to/model.nemo
```
Restart a sandboxed Pi after profile or socket changes. In Pi, press `Alt+M` to start or stop dictation and `Alt+N` to cancel. See [`extensions/dictate/README.md`](extensions/dictate/README.md) for privacy, dictionary, model, and troubleshooting details.
## Maintenance ## Maintenance
After pulling repository changes: After pulling repository changes:
```bash ```bash
cd /srv/data/code/harness-tools cd /srv/data/code/jb-pi/harness-tools
git pull --rebase git pull --rebase
bun install --frozen-lockfile bun install --frozen-lockfile
./scripts/pi-links.mjs sync --dry-run ./scripts/pi-links.mjs sync --dry-run
+1 -1
View File
@@ -27,7 +27,7 @@
}, },
"extensions/dictate": { "extensions/dictate": {
"name": "@jbcode/pi-dictate", "name": "@jbcode/pi-dictate",
"version": "0.1.0", "version": "0.2.0",
"peerDependencies": { "peerDependencies": {
"@earendil-works/pi-coding-agent": "catalog:pi-peer", "@earendil-works/pi-coding-agent": "catalog:pi-peer",
}, },
@@ -0,0 +1,415 @@
# Configurable Dictate ASR backends
Status: implemented and validated, 03SEP2026
## Decisions
- Keep microphone capture and every ASR implementation in the existing host-side `pi-dictate-service` boundary.
- Keep provider settings and API keys in host-owned, mode-`0600` configuration outside Pi's nono sandbox. The Pi extension must never read or transmit cloud credentials.
- Select one backend in host configuration and apply the change by restarting the socket-activated service. Do not add per-session or per-generation provider selection to the Pi protocol.
- Support three backend profiles:
- local NeMo/Parakeet;
- Meta Model API Muse Voice Transcribe;
- OpenAI Realtime transcription.
- Stream microphone audio to both cloud providers while recording. Preserve the current push-to-start, push-to-stop interaction and paste only one finalized transcript after stop.
- Keep local as the migration default. Cloud audio and cost require an explicit backend selection and credential setup.
- Default new OpenAI configuration to `gpt-live-transcribe`. Permit a configurable OpenAI transcription model, and document `gpt-transcribe` as the commit-oriented alternative. Do not make `whisper-1` the default: current OpenAI guidance recommends the GPT transcription models, and `whisper-1` is not the selected realtime path.
## Goals
1. A user can configure one local, Meta, or OpenAI ASR backend without changing extension source.
2. Local mode retains its offline runtime, pinned-model verification, coding dictionary, and network-denied systemd sandbox.
3. Meta and OpenAI modes connect before recording, stream bounded PCM without writing audio files, finalize on stop, and return one bounded transcript.
4. The extension continues to see only bounded state, level, result, and error events over `/run/user/1000/pi-dictate.sock`.
5. API keys, provider wire events, partial hypotheses, and raw audio never cross into Pi.
6. Switching backend is fail-closed: configuration, installed dependency profile, and systemd network hardening must agree before the service starts.
## Non-goals
- Automatic failover between providers.
- Sending one recording to multiple providers.
- A runtime backend picker in Pi.
- Partial text in Pi's editor.
- Voice-activity-controlled submission; `Alt+M` remains the turn boundary.
- Diarized editor output, timestamps, confidence scores, or translation.
- Arbitrary local inference runtimes in the first slice. The local adapter remains NeMo-compatible; its model archive, checksum, terms file, and boost can become configuration.
- Reusing credentials from Pi's provider auth store. Dictate gets dedicated host credentials so the service does not need access to all of `~/.pi/agent/auth.json`.
## Current architecture and constraints
- `extensions/dictate/index.ts` is a strict protocol-v2 Unix-socket client. It owns no audio, model, or network capability and pastes only a current finalized result.
- `services/dictate/src/pi_dictate_service/server.py` serializes one generation globally and currently assumes a lazy `TranscriptionModel` with `load()` followed by `transcribe(samples)` after recording.
- `services/dictate/src/pi_dictate_service/recorder.py` launches `pw-record` at 16 kHz mono PCM16, drains stdout/stderr in threads, retains at most five minutes of PCM, and derives the local level meter.
- `services/dictate/src/pi_dictate_service/model.py` supports one explicit `.nemo` archive, verifies its SHA-256, applies the coding dictionary, and runs offline on CUDA.
- `systemd/user/pi-dictate.service` denies all IP networking and limits address families to `AF_UNIX`. That must remain true in local mode but cannot be shared unchanged by cloud mode.
- `services/dictate/install.sh` always installs the full NeMo/CUDA environment and pinned model. Cloud-only installation should not require either.
- `profiles/nono/pi-local.json` already grants the exact Dictate Unix socket and explicitly denies the installed local runtime. It is policy-controlled and must remain unchanged for this feature; no Pi-side network, credential, PipeWire, model, or device grant is required.
## External API contract baseline
The implementation must turn the current provider documentation into checked fixtures before production code is merged, because Muse Voice Transcribe was released on the date of this plan and its contract may still move.
### Meta Muse Voice Transcribe
Primary references:
- [Meta announcement](https://research.meta.ai/blog/introducing-muse-voice-transcribe)
- [Meta speech-to-text guide](https://ai.developer.meta.com/docs/speech-to-text)
- [Meta realtime voice API reference](https://ai.developer.meta.com/docs/api-reference/voice/realtime)
- [Meta voice event schemas](https://ai.developer.meta.com/docs/api-reference/voice/schemas)
- [Meta Model API authentication](https://ai.developer.meta.com/docs/getting-started/authentication)
- [Meta voice cookbook](https://github.com/meta-models/meta-model-cookbook/blob/main/06_muse_voice/01_voice_api_fundamentals/README.md)
Current documented shape:
- WebSocket endpoint: `wss://api.meta.ai/v1/asr/realtime`, with an optional `sessionId` query parameter.
- Model: `muse-voice-transcribe-1.0`; no omitted-model default is documented.
- The first text frame is a handshake requiring `authorization.accessToken`, `audioEncoding`, and `model`. The key value is `Bearer <MODEL_API_KEY>` inside that object; an HTTP authorization header or API-key query parameter is ignored.
- The public API accepts signed little-endian mono PCM16 as binary frames. Use native `PCM_24KHZ` (48,000 bytes/second); `PCM_16KHZ` is also accepted and server-resampled.
- `PUSH_TO_TALK` is the mode matching Dictate's one explicit turn. Defaults also include `partialMode: "CUMULATIVE"` and `emitAudioProgress: true`.
- Wait for the server's `{ "sessionId": "..." }` acknowledgement before sending audio. Frames carry no headers, timestamps, sequence numbers, or semantic boundaries; approximately 80 ms pacing is guidance rather than a required frame size.
- End input with the text frame `{ "type": "endStream" }` and require a `transcript` event with `final: true`. Treat that correlated final as completion and close the connection rather than waiting for Meta's idle close, while preserving any provider failure that races with finalization.
- Cumulative partials replace the prior hypothesis. Dictate ignores partials, turn boundaries, endpointing, and speaker events and retains only the explicit bounded final transcript.
- `keywords` and `languageBias` are handshake-only hints. The existing coding dictionary can feed `keywords`; no free-form prompt field is documented.
- Meta documents a 60-minute session, at most five seconds of audio backlog with approximately realtime pacing, eight concurrent streams, and 1,000 streams/hour. Dictate's own five-minute cap remains stricter.
- Treat close codes `1008` (invalid request/pacing), `1011` (backend or duration), and `1013` (rate limit), plus bounded `error` events, as provider failures.
The contract spike must freeze these fields as sanitized fixtures and confirm them against a live non-sensitive sample, especially because Muse Voice Transcribe was released on the date of this plan. Do not copy the undocumented `shortwave.facebook.com` Muse Code endpoint used by third-party extensions.
### OpenAI Realtime transcription
Primary references:
- [Realtime transcription guide](https://developers.openai.com/api/docs/guides/realtime-transcription)
- [Realtime WebSocket guide](https://developers.openai.com/api/docs/guides/realtime-websocket)
- [Speech-to-text guide](https://developers.openai.com/api/docs/guides/speech-to-text)
- [`gpt-live-transcribe` model](https://developers.openai.com/api/docs/models/gpt-live-transcribe)
Current documented shape:
- Use the GA server-side WebSocket connection `wss://api.openai.com/v1/realtime?intent=transcription` with `Authorization: Bearer <key>`. Do not mix this with the deprecated `/v1/realtime/transcription_sessions` schema or put the transcription model in the connection query.
- After `session.created`, send the nested GA `session.update` shape with `type: "transcription"`, the configured transcription model, 24 kHz mono PCM16, and `turn_detection: null`; wait for `session.updated` before capture.
- PCM is base64 encoded in bounded `input_audio_buffer.append` JSON events. The API's per-append limit is 15 MiB, but Dictate must use much smaller realtime chunks.
- Stop sends `input_audio_buffer.commit`; it starts transcription without creating an assistant response.
- Partial text arrives as `conversation.item.input_audio_transcription.delta`; final text arrives as `conversation.item.input_audio_transcription.completed` and is correlated by `item_id`. Transcription-specific failures use `conversation.item.input_audio_transcription.failed`; general protocol failures use `error`.
- `gpt-live-transcribe` is the recommended low-latency model and supports prompt, keywords, languages, and bounded delay settings. A credentialed live spike on 03SEP2026 confirmed `delay: "low"`.
- `gpt-transcribe` can use the same Realtime session but begins transcription after commit and can return detected languages.
- `whisper-1` remains a file-transcription model for older integrations, translation, and timestamp use cases; its model page says realtime transcription is unsupported.
- OpenAI documents a 60-minute Realtime session. Dictate's own five-minute cap remains stricter.
- The current guide recommends `delay` for `gpt-live-transcribe`, while one API reference associates it with `gpt-realtime-whisper`. The live GA session accepted `low`; Dictate permits `minimal`, `low`, `medium`, `high`, and `xhigh`, and otherwise omits the field.
## Proposed host configuration
Add `services/dictate/src/pi_dictate_service/config.py` with strict, versioned parsing. The installed authority is:
```text
~/.config/pi-dictate/
config.json
credentials/
meta-api-key
openai-api-key
```
Illustrative schema (field names may be refined during implementation):
```json
{
"version": 1,
"backend": "openai",
"idleSeconds": 900,
"local": {
"adapter": "nemo-parakeet",
"modelPath": "~/.local/share/pi-dictate/models/parakeet.nemo",
"modelSha256": "...",
"termsPath": "~/.local/share/pi-dictate/config/coding-terms.txt",
"boostAlpha": 1.0
},
"meta": {
"model": "muse-voice-transcribe-1.0",
"apiKeyFile": "~/.config/pi-dictate/credentials/meta-api-key",
"termsPath": "~/.local/share/pi-dictate/config/coding-terms.txt"
},
"openai": {
"model": "gpt-live-transcribe",
"apiKeyFile": "~/.config/pi-dictate/credentials/openai-api-key",
"delay": "low",
"prompt": "Coding dictation",
"termsPath": "~/.local/share/pi-dictate/config/coding-terms.txt",
"languages": ["en"]
}
}
```
Configuration rules:
- Reject unknown root and selected-backend keys, malformed JSON, unsupported versions, invalid numeric bounds, and incompatible options.
- Read only the selected backend's credential. Never require or expose credentials for inactive backends.
- Credential files must be regular files owned by the service user, not symlinks, with no group/other permission bits and a small byte limit. Trim one trailing newline; reject empty or multi-line values.
- Never accept literal keys on command lines, in systemd environment variables, in the Pi protocol, or in logs.
- Keep provider endpoints fixed in production. Inject transports in tests rather than adding a credential-exfiltration-prone endpoint override. If a localhost override is needed for an opt-in integration harness, gate it behind an explicit test-only setting.
- Keep the current pinned Parakeet revision as the generated local default. A custom local archive must supply an explicit checksum and use a supported NeMo adapter.
- Validate coding terms against the existing byte/count limits before mapping them to provider keyword limits. Provider-specific filtering must reject unsupported characters instead of silently altering terms.
- Configuration is read at process startup. Changes require the documented configure operation and service restart.
## Backend-neutral service design
Replace the model-only assumption with a backend that creates one prepared turn:
```text
TranscriptionBackend
descriptor # bounded backend/model diagnostics
start_turn() -> Turn # load/reuse local model or connect/auth cloud session
TranscriptionTurn
sample_rate_hz
retain_pcm
enqueue_pcm(chunk) # nonblocking, bounded
finish(capture) -> text # infer locally or commit/await cloud final
poll_error()
cancel()
close()
```
`start_turn()` and `finish()` run outside the socket loop. A local backend persists its loaded model across turns; each cloud turn owns its own WebSocket and provider workers.
Exact Python protocols can differ, but the following invariants are mandatory:
1. `start_turn()` completes before `pw-record` starts. The UI must never say `listening` before provider auth/handshake succeeds.
2. The backend declares the recorder rate: 16 kHz for the existing local adapter and 24 kHz for Meta/OpenAI.
3. Local mode retains bounded PCM for post-stop inference. Cloud mode streams through a bounded queue and does not retain the whole recording.
4. The recorder drain thread must never perform network I/O. It enqueues PCM quickly; a provider worker drains the queue.
5. Queue overflow or a stalled provider is a terminal error. Stop capture and discard the generation; never silently drop audio and return a plausible but incomplete transcript.
6. On stop, signal `pw-record`, drain stdout to EOF, wait for child close, then enqueue the provider end/commit marker after all PCM. This preserves the existing no-truncated-tail guarantee.
7. Ignore partial hypotheses at the service boundary. Only an explicit provider final event may produce the protocol `result`.
8. Every provider event, transcript, close reason, and error is byte-bounded before parsing or logging.
9. Cloud transports use normal certificate and hostname verification, fixed production endpoints, no credential-bearing redirects, and no ambient proxy environment unless a separately reviewed proxy feature is added.
10. Cancellation/disconnect tombstones delivery immediately, stops capture, closes the provider turn, joins worker threads with bounded timeouts, and suppresses every later final event.
11. The service remains globally single-generation. Idle exit is suspended during local load/inference and cloud connect/stream/finalization.
### Lifecycle
The existing extension states remain valid:
```text
idle -> connecting -> recording -> stopping -> transcribing -> idle
```
Service interpretation becomes backend-neutral:
- `loading`: preparing the selected backend (local model load or cloud connect/auth/session setup);
- `listening`: recorder and, for cloud mode, provider stream are both ready;
- `stopping`: recorder shutdown and complete PCM drain;
- `transcribing`: local inference or cloud commit/end-of-input plus final-result wait;
- `idle`: generation resources are closed and no result remains deliverable.
Change extension text from `loading model...` to `preparing transcription...`.
## Provider adapters
### Local NeMo/Parakeet
Refactor `NemoParakeetModel` behind `LocalNemoBackend` without changing its observable transcription behavior:
- lazy checksum-verified load;
- 16 kHz PCM16 capture and current five-minute/9.6 MB bound;
- offline environment and CUDA behavior;
- coding-term phrase boosting;
- post-stop conversion and inference;
- warm model reuse until idle service exit.
This slice must pass the existing service tests before either cloud adapter is introduced.
### Meta
Add `backends/meta.py` and a small bounded WebSocket transport:
- read and validate the dedicated Meta key;
- connect only to the public `api.meta.ai` realtime endpoint;
- send the documented first-frame bearer handshake for `muse-voice-transcribe-1.0`, `PCM_24KHZ`, `PUSH_TO_TALK`, and `CUMULATIVE` partials;
- wait for the `sessionId` acknowledgement before starting capture;
- send recorder bytes as ordered, approximately realtime-paced binary frames and enforce Meta's five-second backlog limit locally;
- consume but do not expose cumulative partial hypotheses;
- send `{ "type": "endStream" }` after recorder EOF;
- require a `transcript` event with `final: true`, then consume the normal `1000` close;
- map bounded provider error events and close codes `1008`, `1011`, and `1013`, plus auth, timeout, malformed-event, and premature-close failures, to stable `meta_*` service error codes without including the key or raw provider payload.
Diarization and endpointing events are parsed defensively and ignored for editor delivery in this feature.
### OpenAI
Add `backends/openai.py`:
- connect to the GA `intent=transcription` URL with the standard key in the `Authorization` header, never the URL;
- wait for `session.created`, send the nested GA `session.update` configuration with 24 kHz PCM and `turn_detection: null`, then wait for `session.updated` before capture;
- emit bounded `input_audio_buffer.append` events with base64 PCM, far below the provider's 15 MiB event limit;
- commit only after recorder EOF;
- correlate final completion to the committed item and ignore stale/unmatched events;
- accept configurable model, prompt, keywords, languages, and the live-verified bounded `delay` values; reject unknown values rather than silently omitting them.
- default to `gpt-live-transcribe`; document and test `gpt-transcribe`'s commit-oriented behavior. Omit `delay` when it is not configured so the provider default remains authoritative.
- map `conversation.item.input_audio_transcription.failed`, general `error` events, auth/rate limits, timeouts, malformed events, and premature closes to stable `openai_*` errors without logging request bodies or keys.
Do not build around the OpenAI SDK unless it materially simplifies Realtime event handling without hiding cancellation, queue bounds, or frame limits. A direct WebSocket adapter is likely smaller and easier to fixture-test.
## Pi protocol and extension work
Preserve protocol version 2 unless the implementation discovers a required breaking frame change. The existing state/result contract already represents all three backends.
Additive changes are sufficient:
- Allow `pong` to report bounded `backend` and `model` strings for diagnostics. Never include paths, endpoints with query data, or credential state.
- Define `ready` as whether the selected backend can begin immediately without preparation; the extension must not depend on this field for state transitions.
- Add `/dictate-status` to establish a short hello/ping when idle and report selected backend/model. It is read-only and does not expose host config or credentials.
- Keep `Alt+M`, `Alt+N`, `/dictate-cancel`, stale-generation rejection, transcript bounds, and `pasteToEditor()` behavior unchanged.
- Do not add a Pi-side configuration writer. Backend changes are privileged host operations outside the agent sandbox.
If additive diagnostic fields are not worth the extra command/client complexity, omit them rather than bumping the protocol solely for display.
## Installer, packaging, and systemd
### Dependency profiles
Split `services/dictate/pyproject.toml` into a small core plus locked optional profiles:
- core: protocol, recorder, configuration, and service code;
- local extra: NeMo, NumPy, PyTorch, and current CUDA stack;
- cloud extra: the selected maintained WebSocket client and its locked dependencies.
Cloud-only installation must not download the Parakeet archive or install the multi-gigabyte local CUDA stack. Local installation remains reproducible and checksum pinned.
### Configure operation
Extend `services/dictate/install.sh` (or add an installer-owned companion command) with explicit operations such as:
```text
install.sh install --backend local [--model-source PATH]
install.sh install --backend meta --api-key-file PATH
install.sh install --backend openai --api-key-file PATH
install.sh configure --backend ...
install.sh check
```
- `--api-key-file` copies from a file into the fixed credential directory with mode `0600`; never accept a literal secret argument.
- Write config and generated systemd hardening atomically, run validation, daemon-reload, and restart the service/socket as one operation.
- Preserve existing config, keys, coding dictionary, and valid model archives on reinstall unless an explicit replacement option is supplied.
- Make uninstall's treatment of `~/.config/pi-dictate` explicit; prefer preserving credentials by default with a separate `--purge-config` confirmation path.
- Resolve the existing installer model-reuse todo while this flow is being changed: checksum and reuse a valid installed archive before attempting a network download.
### Backend-specific network hardening
Do not weaken local mode to make cloud mode work.
- Keep common hardening (`NoNewPrivileges`, private temp, core/swap limits, system protection, a masked home with narrow read-only runtime/config/unit and PipeWire-socket binds, and socket ownership) in the base service unit. Bind only the selected cloud credential through its generated drop-in. Clear ambient `HTTP_PROXY`, `HTTPS_PROXY`, and `ALL_PROXY` variables unless proxy support is separately designed and reviewed.
- Generate/install an offline drop-in for local mode containing `RestrictAddressFamilies=AF_UNIX` and `IPAddressDeny=any`.
- Generate/install a cloud drop-in permitting only `AF_UNIX`, `AF_INET`, and `AF_INET6` and no inbound listener beyond the inherited Unix socket. DNS and outbound TLS must be tested under the final unit.
- Stamp the expected network mode in the generated unit environment. At startup, compare it with the selected backend and refuse to run on mismatch. This prevents a hand-edited local config from silently running under cloud-capable hardening, and a cloud backend from failing mysteriously under the offline profile.
- Restrict service reads to the installed runtime, selected model/dictionary paths, and `~/.config/pi-dictate` where practical.
- Document that cloud mode necessarily grants the host transcription process general outbound IP connectivity; systemd cannot safely express a stable hostname allowlist for changing provider addresses.
Preserve `profiles/nono/pi-local.json` byte-for-byte. The existing resolved policy must be reviewed separately; Pi must not read host config or keys.
## Implementation phases
### Phase 0: Freeze provider contracts and run opt-in spikes
1. Save sanitized request/event fixtures from the current public Meta and OpenAI docs.
2. With explicit user consent and test keys, run a short prerecorded, non-sensitive PCM fixture against each API outside Pi.
3. Confirm handshake, framing, finalization, close/error behavior, first/final latency, model availability, and documented billing/rate limits.
4. Record exact model IDs and API doc dates. Treat a changed or unavailable Meta contract as a planning gate, not a reason to use an undocumented endpoint.
Completion: one minimal client per provider produces an exact final transcript and no unresolved required schema field remains.
### Phase 1: Add strict host config and preserve local behavior
1. Add the versioned config and credential readers with permission, ownership, size, and redaction checks.
2. Move local model path/checksum/terms/boost/idle settings from environment assumptions into typed config.
3. Introduce backend/turn protocols and wrap the current NeMo path without changing recorder or protocol behavior.
4. Add config and local-regression tests.
Completion: local Dictate passes all existing tests and runs with generated local config under the same offline unit restrictions.
### Phase 2: Generalize capture for realtime sinks
1. Make recorder sample rate and byte-duration cap explicit and tested.
2. Add a nonblocking PCM sink plus fixed queue/backpressure limits while retaining local-buffer mode.
3. Add ordered finish, provider-error polling, cancellation, and bounded worker joins to the service core.
4. Test 16/24 kHz commands, exact five-minute limits, stalled consumers, queue overflow, final PCM ordering, cancellation in every phase, and no thread/resource leaks.
Completion: a fake realtime backend receives byte-for-byte ordered PCM through stop and returns one final result; local behavior remains unchanged.
### Phase 3: Implement Meta
1. Implement the public Muse handshake and event parser from Phase 0 fixtures.
2. Integrate 24 kHz binary streaming, end-of-input, final-only delivery, timeout, and cancellation.
3. Map the coding dictionary into validated Meta bias terms.
4. Add fake-server contract tests and an opt-in live smoke test.
Completion: Meta mode starts recording only after handshake, streams bounded audio, pastes exactly one final transcript, and leaves no deliverable result after cancel/disconnect.
### Phase 4: Implement OpenAI
1. Implement authenticated Realtime transcription session setup.
2. Integrate base64 appends, explicit commit, item correlation, model/context settings, final-only delivery, timeout, and cancellation.
3. Test `gpt-live-transcribe` as default and `gpt-transcribe` as the documented commit-oriented alternative.
4. Add fake-server contract tests and an opt-in live smoke test.
Completion: OpenAI mode satisfies the same user-visible and cleanup contract as Meta and local mode.
### Phase 5: Make installation and hardening backend-aware
1. Add locked local/cloud dependency profiles and backend-specific provisioning.
2. Add atomic install/configure/check operations and secure key copying.
3. Add local/cloud systemd hardening generation plus startup mismatch checks.
4. Preserve `profiles/nono/pi-local.json` byte-for-byte and make no Pi policy changes.
5. Review the existing resolved profile separately, then test fresh install, every switch direction, reinstall preservation, invalid config, missing/insecure key files, local network denial, cloud DNS/TLS, idle exit, reactivation, and uninstall.
Completion: selecting a backend installs only its required runtime, applies matching hardening, and reactivates through the same Unix socket after restart.
### Phase 6: Extension diagnostics and documentation
1. Use backend-neutral status wording and optionally expose bounded backend/model diagnostics.
2. Add `/dictate-status` only if it can remain a simple read-only protocol operation.
3. Update `docs/dictate/protocol.md`, `extensions/dictate/README.md`, root `README.md`, attribution, privacy, troubleshooting, and operations sections.
4. Document exactly when cloud audio begins leaving the host, credential storage, provider retention/terms links, billing, five-minute cap, no partial editor insertion, and local offline guarantees.
Completion: docs let a new user install any one backend, switch safely, understand data flow/cost, diagnose failures, and return to local offline mode.
### Phase 7: Validation and independent review
1. Run focused TypeScript/Python tests, typecheck, package tests, and the full repository test suite.
2. Manually test cold/warm local starts and fresh cloud sessions, stop/cancel/disconnect/shutdown in every state, empty speech, provider auth/rate-limit/outage, network loss, duration limit, two competing Pi clients, service idle exit, and socket reactivation.
3. Confirm the local unit cannot resolve/connect to the Internet and the cloud unit cannot expose a TCP listener.
4. Confirm Pi's resolved nono policy has no new grants and Pi cannot read Dictate config/keys or reach either provider directly.
5. Have an independent reviewer audit credential handling, audio retention, network hardening, provider event validation, cancellation, and installer migration.
## Test matrix
| Area | Local | Meta | OpenAI |
| --- | --- | --- | --- |
| Prepare before recorder | lazy model load | connect/auth/handshake | connect/auth/session ack |
| Capture | 16 kHz PCM16 retained | 24 kHz PCM16 queued/binary | 24 kHz PCM16 queued/base64 JSON |
| Stop | drain, convert, infer | drain, end marker, final | drain, commit, matching completion |
| Context terms | NeMo phrase boost | Meta bias terms | keywords/prompt/languages |
| Cancellation | tombstone load/infer | close stream, suppress final | close session, suppress final |
| Hardening | AF_UNIX, IP denied | outbound TLS enabled | outbound TLS enabled |
| Main result | bounded string | bounded explicit final | bounded matching final |
| Live smoke | GPU/model opt-in | key/cost opt-in | key/cost opt-in |
Every provider also needs tests for malformed/oversized events, stale results, partial-only close, empty final, invalid UTF-8/JSON where applicable, auth failure, rate limit, timeout, queue overflow, recorder failure, service shutdown, and redacted errors.
## Acceptance criteria
- One strict host configuration selects local, Meta, or OpenAI and takes effect only after a validated service restart.
- Existing users remain local until they explicitly opt into a cloud backend.
- The local model path, checksum, terms, and boost are configurable without accepting an unverified archive.
- Meta uses only the public `api.meta.ai` Muse Voice Transcribe endpoint and documented bearer contract.
- OpenAI defaults to `gpt-live-transcribe`; the model is configurable and `whisper-1` is not silently treated as Realtime-compatible.
- Provider connection/session readiness precedes recorder startup.
- Cloud PCM is streamed in order through a fixed bounded queue; no audio is silently dropped, retried to another provider, written to disk, or exposed to Pi.
- Stop drains the recorder completely before provider finalization.
- Only one explicit bounded final result for the active generation can change the editor.
- Partial events, cancellation, stale generations, disconnects, provider failures, and cleanup never insert text.
- API keys live only in dedicated host credential files, pass strict file checks, never cross the Unix protocol, and never appear in logs, errors, command lines, URLs, redirects, or ambient proxies.
- Local mode retains IP denial; cloud mode's broader outbound capability is explicit, tested, and cannot be selected under mismatched hardening.
- Cloud-only installation does not provision NeMo/CUDA/model artifacts.
- The Pi nono profile gains no new filesystem, socket, network, PipeWire, GPU, or credential capability.
- Focused and full tests, typecheck, host smoke tests, and independent security review pass.
+26
View File
@@ -29,3 +29,29 @@ The complete transitive dependency versions and artifact hashes are recorded in
- NeMo ASR documentation: <https://docs.nvidia.com/nemo-framework/user-guide/latest/nemotoolkit/asr/intro.html> - NeMo ASR documentation: <https://docs.nvidia.com/nemo-framework/user-guide/latest/nemotoolkit/asr/intro.html>
- CC BY 4.0: <https://creativecommons.org/licenses/by/4.0/> - CC BY 4.0: <https://creativecommons.org/licenses/by/4.0/>
- Apache License 2.0: <https://www.apache.org/licenses/LICENSE-2.0> - Apache License 2.0: <https://www.apache.org/licenses/LICENSE-2.0>
## Optional cloud providers
Cloud backends are opt-in and are not required for local mode. Pi Dictate does not redistribute Meta or OpenAI models and does not include either provider's SDK. The host service sends audio only after the selected provider session is authenticated and ready. Provider terms, retention, security, and billing policies apply to cloud audio and any keyword or prompt context sent with it.
### Meta Model API Muse Voice Transcribe
- Announcement: <https://research.meta.ai/blog/introducing-muse-voice-transcribe>
- Speech-to-text guide: <https://ai.developer.meta.com/docs/speech-to-text>
- Realtime Voice API: <https://ai.developer.meta.com/docs/api-reference/voice/realtime>
- Voice event schemas: <https://ai.developer.meta.com/docs/api-reference/voice/schemas>
- Authentication: <https://ai.developer.meta.com/docs/getting-started/authentication>
- Meta terms: <https://ai.meta.com/terms/>
Pi Dictate uses only the documented `api.meta.ai` realtime endpoint and `muse-voice-transcribe-1.0` contract. It does not use the undocumented `shortwave.facebook.com` Muse Code endpoint.
### OpenAI Realtime transcription
- Realtime transcription guide: <https://developers.openai.com/api/docs/guides/realtime-transcription>
- Realtime WebSocket guide: <https://developers.openai.com/api/docs/guides/realtime-websocket>
- Speech-to-text guide: <https://developers.openai.com/api/docs/guides/speech-to-text>
- `gpt-live-transcribe` model: <https://developers.openai.com/api/docs/models/gpt-live-transcribe>
- OpenAI terms: <https://openai.com/policies/terms-of-use/>
- OpenAI privacy policy: <https://openai.com/policies/privacy-policy/>
The OpenAI default is `gpt-live-transcribe`; `gpt-transcribe` is a configurable commit-oriented alternative. `whisper-1` is not treated as a Realtime model.
+15
View File
@@ -46,3 +46,18 @@ A 37-term dictionary and boost strengths from 0.25 through 2.0 were tested again
The packaging, latency, memory, offline, and normal-dictation gates pass. Alpha `1.0` produced the same expected clean transcript as baseline, the same degraded transcript on seeded noisy input, and an empty transcript for silence. No dictionary hallucination was observed. The packaging, latency, memory, offline, and normal-dictation gates pass. Alpha `1.0` produced the same expected clean transcript as baseline, the same degraded transcript on seeded noisy input, and an empty transcript for silence. No dictionary hallucination was observed.
The coding-term remediation passes the final focused live check through the installed socket-activated service. The spoken phrase “Run npm test, then git status.” produced exactly `Run npm test, then git status.` with native phrase boosting at alpha `1.0`. The coding-term remediation passes the final focused live check through the installed socket-activated service. The spoken phrase “Run npm test, then git status.” produced exactly `Run npm test, then git status.` with native phrase boosting at alpha `1.0`.
The upgraded configurable service was revalidated on 03SEP2026. Immediate cancellation during preparation was delivery-neutral, a second generation started successfully, and a six-second end-to-end capture returned `Run npm test, then get status.` The wording variation does not replace the earlier accuracy fixture; it confirms the current installer, systemd sandbox, socket protocol, PipeWire capture, in-memory model path, and final-only delivery operate together. Local systemd policy exposed only the PipeWire runtime socket while retaining the masked home and IP-network denial.
## Configurable backend contract gate
Credentialed live contract checks passed on 03SEP2026 with a five-second, non-sensitive 24 kHz mono PCM microphone sample held only in memory. Dedicated mode-`0600` provider key files were read by the host-side adapters; no credential entered Pi, a command line, a URL, or captured output.
| Provider | Model | Result |
| --- | --- | --- |
| Meta | `muse-voice-transcribe-1.0` | The public `api.meta.ai` handshake accepted the session, streamed PCM, accepted `endStream`, and returned an explicit final transcript (`run npm test`). Total turn time was 30.5 seconds. |
| OpenAI | `gpt-live-transcribe` | The GA transcription session accepted 24 kHz PCM, explicit commit, and returned a correlated completion event (`Run npm test and`). Total turn time was 7.1 seconds. |
The first OpenAI spike exposed forward-compatible fields (`event_id` and `previous_item_id`) on the commit acknowledgement. The adapter and sanitized fixture now accept those bounded fields while still requiring a non-empty `item_id`. A separate credentialed session update confirmed that `gpt-live-transcribe` accepts transcription `delay: "low"`. Configuration therefore supports only the documented bounded values `minimal`, `low`, `medium`, `high`, and `xhigh`; omission keeps the provider default.
These checks validate the current wire contract and finalization path, not comparative transcription accuracy. The public contract references checked were the [Meta Realtime Voice API](https://ai.developer.meta.com/docs/api-reference/voice/realtime), [Meta event schemas](https://ai.developer.meta.com/docs/api-reference/voice/schemas), [OpenAI Realtime transcription guide](https://developers.openai.com/api/docs/guides/realtime-transcription), [OpenAI Realtime WebSocket guide](https://developers.openai.com/api/docs/guides/realtime-websocket), and [`gpt-live-transcribe` model page](https://developers.openai.com/api/docs/models/gpt-live-transcribe).
+72 -26
View File
@@ -1,28 +1,34 @@
# Pi Dictate local-service protocol # Pi Dictate host-service protocol
## Transport and version ## Transport and version
Protocol version `2` uses UTF-8, newline-delimited JSON over one persistent Unix stream socket. Each line contains one JSON object. The systemd socket unit owns the socket path and permissions. The service accepts the inherited listening file descriptor and does not bind or unlink the path. Protocol version `2` uses UTF-8, newline-delimited JSON over one persistent Unix stream socket. Each line contains one JSON object. The systemd socket unit owns the socket path and permissions. The service accepts the inherited listening file descriptor and does not bind or unlink the path.
Clients must send `hello` before any other request. A connection negotiates one version only. Coding terms are host-owned service configuration and never cross this protocol. Clients must send `hello` before any other request. A connection negotiates one version only. The selected backend, provider credentials, coding dictionary, model paths, and provider wire events are host-service configuration. They never cross this protocol.
## Limits The protocol is backend-neutral. The host service owns microphone capture and the selected transcription backend. In local mode it runs the pinned NeMo/Parakeet adapter offline. In Meta and OpenAI modes it owns the provider WebSocket, key, PCM stream, partial hypotheses, and finalization. The Pi extension sees only state, level, result, error, cancellation, and bounded diagnostics.
## Limits and validation
| Data | Limit | | Data | Limit |
| --- | ---: | | --- | ---: |
| JSON frame, excluding newline | 65,536 bytes | | JSON frame, excluding newline | 65,536 bytes |
| `requestId` or `generationId` | 128 UTF-8 bytes | | `requestId` or `generationId` | 128 UTF-8 bytes |
| `backend` or `model` diagnostic | 128 UTF-8 bytes |
| Final transcript | 32,768 UTF-8 bytes | | Final transcript | 32,768 UTF-8 bytes |
| Error detail | 2,048 UTF-8 bytes | | Error detail | 2,048 UTF-8 bytes |
| Simultaneous client connections | 16 | | Simultaneous client connections | 16 |
| Pending output per client | 65,536 bytes | | Pending output per client | 65,536 bytes |
| Recorder stderr tail | 65,536 bytes | | Recorder stderr tail | 65,536 bytes |
| Audio-level event rate | 10 per second | | Audio-level event rate | 10 per second |
| Mono 16 kHz signed-16 PCM | 9,600,000 bytes, five minutes | | Local retained PCM | 9,600,000 bytes, five minutes at 16 kHz |
| Cloud queued PCM | Fixed and bounded; the whole recording is never retained |
IDs are non-empty, opaque strings generated by the client. The service compares them only for request correlation, ownership, and stale-generation rejection. It does not interpret IDs as paths. Request objects reject unknown or missing fields. IDs are non-empty, opaque strings generated by the client. The service compares them only for request correlation, ownership, and stale-generation rejection. It does not interpret IDs as paths. Request objects reject unknown or missing fields.
Malformed JSON, a frame over the limit, or an unsupported version produces one bounded `error` frame and then closes the connection. The malformed-frame error uses an empty `requestId` when no client ID can be recovered. Schema, state, busy, and ownership errors leave the connection open. `backend` and `model` are optional diagnostic strings on `pong`. When present, they are non-empty, bounded, and safe for one-line display. They identify only the selected backend and model. They must not contain paths, query strings, key material, credential state, provider payloads, or endpoint details. Older services that omit these fields remain protocol-compatible; clients display `unknown` rather than inferring a backend.
Malformed JSON, invalid UTF-8, a frame over the limit, or an unsupported version produces one bounded `error` frame and then closes the connection. The malformed-frame error uses an empty `requestId` when no client ID can be recovered. Schema, state, busy, and ownership errors leave the connection open.
## Client requests ## Client requests
@@ -36,7 +42,7 @@ All examples are one line when transmitted.
Exact fields: `version`, `type`, `requestId`. Exact fields: `version`, `type`, `requestId`.
A successful response is `pong` with the same `requestId`. `ready` reports whether the model is warm. A successful response is `pong` with the same `requestId`. The response may include `service`, `ready`, `busy`, `backend`, and `model`. `ready` says whether the selected backend can begin immediately without preparation. The extension does not use `ready` for state transitions because a cold local model or a new cloud session may need preparation after a successful hello.
### `ping` ### `ping`
@@ -44,7 +50,9 @@ A successful response is `pong` with the same `requestId`. `ready` reports wheth
{"version":2,"type":"ping","requestId":"ping-1"} {"version":2,"type":"ping","requestId":"ping-1"}
``` ```
Exact fields: `version`, `type`, `requestId`. `hello` is required first. The response is `pong` and may include `ready` and `busy`. Exact fields: `version`, `type`, `requestId`. `hello` is required first. The response is `pong` and may include `ready`, `busy`, `backend`, and `model`.
The `/dictate-status` extension command uses a short-lived connection: it sends `hello`, waits for its `pong`, sends `ping`, reports the bounded backend/model fields from the ping response, and closes the connection. It never starts a generation or reveals configuration and credential state.
### `start` ### `start`
@@ -52,7 +60,13 @@ Exact fields: `version`, `type`, `requestId`. `hello` is required first. The res
{"version":2,"type":"start","requestId":"start-1","generationId":"generation-1"} {"version":2,"type":"start","requestId":"start-1","generationId":"generation-1"}
``` ```
Exact fields: `version`, `type`, `requestId`, `generationId`. The service accepts only one global generation. It first emits `state: loading` for a cold model or `state: listening` for a warm model. Recording starts only after model loading succeeds. Exact fields: `version`, `type`, `requestId`, `generationId`. The service accepts only one global generation. `start` prepares the selected backend before capture:
- local: load or reuse the checksum-verified NeMo/Parakeet model;
- Meta: connect to the fixed public endpoint, authenticate with the handshake frame, and wait for the session acknowledgement;
- OpenAI: connect to the fixed Realtime transcription endpoint, wait for `session.created`, send session configuration, and wait for `session.updated`.
Recording begins only after preparation succeeds. The service declares the recorder rate internally: 16 kHz for local mode and 24 kHz for either cloud mode. A cold start emits `state: loading` while the extension displays `preparing transcription...`; it never displays `listening` before the recorder and selected backend are ready.
### `stop` ### `stop`
@@ -60,7 +74,7 @@ Exact fields: `version`, `type`, `requestId`, `generationId`. The service accept
{"version":2,"type":"stop","requestId":"stop-1","generationId":"generation-1"} {"version":2,"type":"stop","requestId":"stop-1","generationId":"generation-1"}
``` ```
Exact fields: `version`, `type`, `requestId`, `generationId`. Only the owning connection may stop the active listening generation. Exact fields: `version`, `type`, `requestId`, `generationId`. Only the owning connection may stop the active listening generation. The service marks the generation stopping, signals `pw-record`, drains stdout to EOF, waits for child close, and then finalizes the selected backend. Local mode converts the complete PCM buffer and runs one inference. Cloud mode sends the provider end/commit marker only after every recorder chunk has been enqueued, then waits for one explicit provider final event.
### `cancel` ### `cancel`
@@ -68,7 +82,7 @@ Exact fields: `version`, `type`, `requestId`, `generationId`. Only the owning co
{"version":2,"type":"cancel","requestId":"cancel-1","generationId":"generation-1"} {"version":2,"type":"cancel","requestId":"cancel-1","generationId":"generation-1"}
``` ```
Exact fields: `version`, `type`, `requestId`, `generationId`. A valid cancellation immediately emits `cancelled`. Cancellation tombstones the generation, so it can never emit a later `result` or begin recording after a cancelled load. Exact fields: `version`, `type`, `requestId`, `generationId`. A valid cancellation immediately emits `cancelled`. Cancellation tombstones the generation, so it can never later emit a `result` or begin recording after cancelled preparation. The host closes a cloud provider turn, stops capture, joins workers with bounded timeouts, and discards retained or queued PCM as applicable.
## Service events ## Service events
@@ -77,9 +91,11 @@ All service events contain `version`, `type`, and a bounded client-generated `re
### `pong` ### `pong`
```json ```json
{"version":2,"type":"pong","requestId":"hello-1","service":"pi-dictate","ready":false} {"version":2,"type":"pong","requestId":"hello-1","service":"pi-dictate","ready":false,"busy":false,"backend":"local","model":"nemo-parakeet"}
``` ```
`service` is the fixed service identifier. `ready` and `busy` are bounded booleans. `backend` and `model` are optional bounded diagnostics. No event reports key presence, file paths, provider URLs, network state, or raw provider responses.
### `state` ### `state`
```json ```json
@@ -88,13 +104,15 @@ All service events contain `version`, `type`, and a bounded client-generated `re
`state` is one of `loading`, `listening`, `stopping`, `transcribing`, or `idle`. `state` is one of `loading`, `listening`, `stopping`, `transcribing`, or `idle`.
The wire name `loading` is retained for compatibility. It means backend preparation and includes local model load, cloud connection/authentication, or session setup. The extension renders it as `preparing transcription...`, not `loading model...`.
### `level` ### `level`
```json ```json
{"version":2,"type":"level","requestId":"start-1","generationId":"generation-1","level":0.625} {"version":2,"type":"level","requestId":"start-1","generationId":"generation-1","level":0.625}
``` ```
While listening, the service emits at most ten `level` events per second. `level` is a finite number from `0` through `1`, derived from the recorder's current 100 ms RMS window on a logarithmic -60 dBFS to 0 dBFS scale. The event carries no PCM samples and stops before recorder shutdown begins. While listening, the service emits at most ten `level` events per second. `level` is a finite number from `0` through `1`, derived from the recorder's current 100 ms RMS window on a logarithmic -60 dBFS to 0 dBFS scale. The event carries no PCM samples and stops before recorder shutdown begins. The recorder rate is selected by the host backend and is not exposed in the event.
### `result` ### `result`
@@ -102,7 +120,7 @@ While listening, the service emits at most ten `level` events per second. `level
{"version":2,"type":"result","requestId":"stop-1","generationId":"generation-1","transcript":"example text"} {"version":2,"type":"result","requestId":"stop-1","generationId":"generation-1","transcript":"example text"}
``` ```
There is at most one result for a generation. An empty transcript is a successful result with `"transcript":""`. There is at most one result for a generation. An empty transcript is a successful result with `"transcript":""`. Only an explicit final provider event may produce this event. Provider partials, cumulative hypotheses, endpointing, diarization, stale completions, or cleanup never produce a result.
### `cancelled` ### `cancelled`
@@ -110,7 +128,7 @@ There is at most one result for a generation. An empty transcript is a successfu
{"version":2,"type":"cancelled","requestId":"cancel-1","generationId":"generation-1"} {"version":2,"type":"cancelled","requestId":"cancel-1","generationId":"generation-1"}
``` ```
This is a terminal delivery guarantee: the generation cannot later produce a result. This is a terminal delivery guarantee: the generation cannot later produce a result. The service may remain busy while a non-interruptible local inference or provider cleanup worker finishes.
### `error` ### `error`
@@ -123,37 +141,65 @@ Stable codes include:
- framing and schema: `malformed_json`, `frame_too_large`, `unsupported_version`, `invalid_request`; - framing and schema: `malformed_json`, `frame_too_large`, `unsupported_version`, `invalid_request`;
- session: `hello_required`, `busy`, `not_owner`, `generation_mismatch`, `invalid_state`; - session: `hello_required`, `busy`, `not_owner`, `generation_mismatch`, `invalid_state`;
- recorder and audio: `recorder_spawn`, `recorder_exit`, `recorder_timeout`, `duration_limit`, `invalid_audio`; - recorder and audio: `recorder_spawn`, `recorder_exit`, `recorder_timeout`, `duration_limit`, `invalid_audio`;
- model and output: `model_failure`, `transcript_too_large`. - local preparation and output: `config_invalid`, `credentials_invalid`, `model_failure`, `transcript_too_large`;
- Meta provider failures: bounded `meta_auth`, `meta_rate_limit`, `meta_timeout`, `meta_protocol`, `meta_premature_close`, or another stable `meta_*` code;
- OpenAI provider failures: bounded `openai_auth`, `openai_rate_limit`, `openai_timeout`, `openai_protocol`, `openai_premature_close`, or another stable `openai_*` code.
Error details are bounded and redacted. They never include API keys, authorization headers, credential paths, request bodies, raw provider payloads, or raw audio. A provider's malformed, oversized, invalid-UTF-8, partial-only, premature-close, authentication, rate-limit, duration, and network failures are terminal for the active generation. The service discards the generation rather than returning a plausible incomplete transcript.
## Provider finalization contract
The extension does not receive provider frames, but the host service must enforce these rules before emitting a protocol result.
### Meta Model API Muse Voice Transcribe
The service uses `wss://api.meta.ai/v1/asr/realtime` with the documented first-frame bearer handshake for `muse-voice-transcribe-1.0`, `PCM_24KHZ`, `PUSH_TO_TALK`, and cumulative partials. It waits for the `sessionId` acknowledgement before capture, sends ordered binary PCM16 frames through a bounded queue, sends `{ "type": "endStream" }` after recorder EOF, and requires a `transcript` event with `final: true`. It consumes the normal `1000` close and maps close codes `1008`, `1011`, and `1013` to stable failures.
The service must use only the public endpoint. It must not use the undocumented `shortwave.facebook.com` Muse Code endpoint. See the [Meta API references](https://ai.developer.meta.com/docs/api-reference/voice/realtime), [event schemas](https://ai.developer.meta.com/docs/api-reference/voice/schemas), and [authentication guide](https://ai.developer.meta.com/docs/getting-started/authentication).
### OpenAI Realtime transcription
The service uses `wss://api.openai.com/v1/realtime?intent=transcription` with `Authorization: Bearer <key>`. It waits for `session.created`, sends the GA nested `session.update` with `type: "transcription"`, the configured model, 24 kHz mono PCM16, and `turn_detection: null`, then waits for `session.updated`. Audio uses bounded base64 `input_audio_buffer.append` events. Stop commits only after recorder EOF and correlates `conversation.item.input_audio_transcription.completed` to the committed item. Delta and unmatched events are ignored.
The default is `gpt-live-transcribe`. `gpt-transcribe` is the documented commit-oriented alternative. `whisper-1` is not silently used as a Realtime model. A credentialed live check on 03SEP2026 confirmed that `gpt-live-transcribe` accepts transcription `delay: "low"`; configuration permits `minimal`, `low`, `medium`, `high`, and `xhigh`, while omission keeps the provider default. See the [Realtime transcription guide](https://developers.openai.com/api/docs/guides/realtime-transcription), [WebSocket guide](https://developers.openai.com/api/docs/guides/realtime-websocket), and [`gpt-live-transcribe` model page](https://developers.openai.com/api/docs/models/gpt-live-transcribe).
## Lifecycle and ownership ## Lifecycle and ownership
| Current state | Request or event | Outcome | | Current state | Request or event | Outcome |
| --- | --- | --- | | --- | --- | --- |
| no generation | owner `start` | `loading`, then `listening`; warm models may emit `listening` directly | | no generation | owner `start` | `loading`, then `listening`; a prepared backend may emit `listening` directly |
| any active generation | another `start` | `busy`; connection stays open | | any active generation | another `start` | `busy`; connection stays open |
| `loading` | owner `cancel` | immediate `cancelled`; loading finishes in the worker but recording is suppressed | | `loading` | owner `cancel` | immediate `cancelled`; preparation finishes in the worker but recording is suppressed |
| `listening` | recorder audio | rate-limited `level` events for the owning client | | `listening` | recorder audio | rate-limited `level` events for the owning client |
| `listening` | owner `stop` | `stopping`; signal recorder, drain stdout to EOF, await child close, convert PCM, then `transcribing` | | `listening` | owner `stop` | `stopping`; stop and drain recorder fully, then `transcribing` and one local/provider final result |
| `listening` | owner `cancel` or disconnect | stop and drain recorder, discard audio, never infer | | `listening` | owner `cancel` or disconnect | stop and drain recorder, discard audio, never infer or finalize |
| `stopping` | owner `cancel` | immediate `cancelled`; completed audio is discarded | | `stopping` | owner `cancel` | immediate `cancelled`; completed audio is discarded |
| `transcribing` | owner `cancel` or disconnect | tombstone immediately; inference finishes but its result is discarded | | `transcribing` | owner `cancel` or disconnect | tombstone immediately; local inference/provider finalization finishes but its result is discarded |
| `transcribing` | successful model result | one `result`, then `idle` | | `transcribing` | successful explicit final | one `result`, then `idle` |
| any work state | terminal failure | one `error`, then `idle` | | any work state | terminal failure | one `error`, then `idle` |
| active generation | non-owner `stop` or `cancel` | `not_owner`; connection stays open | | active generation | non-owner `stop` or `cancel` | `not_owner`; connection stays open |
| active generation | stale generation ID | `generation_mismatch`; connection stays open | | active generation | stale generation ID | `generation_mismatch`; connection stays open |
Cancelled model loading and inference are non-interruptible. The global service remains busy until the worker finishes even though `cancelled` was already emitted. Disconnecting a generation owner cancels capture or tombstones loading/inference under the same rules. The service is globally single-generation. A cloud queue overflow, provider stall, or worker failure is terminal. PCM is never silently dropped, retried to another provider, written to disk, or exposed to Pi.
The recorder command is exactly: The recorder command is selected by the host service and is otherwise fixed:
```text ```text
pw-record --raw --format s16 --rate 16000 --channels 1 - pw-record --raw --format s16 --rate 16000 --channels 1 -
``` ```
The service launches `pw-record` in its own process group and marks the generation stopping before SIGTERM. It continuously drains bounded stdout and stderr, waits a bounded time for pipe EOF and child close, and only then converts or transcribes. A shutdown timeout escalates to a process-group kill and fails the generation. PipeWire's exit code `1` is accepted only after this service intentionally signalled a still-running recorder and stderr remained empty; an already-exited process or stderr-bearing exit remains a failure. Raw audio and recorder logs are held only in bounded memory and are not written to disk. The local adapter uses 16 kHz. Meta and OpenAI use 24 kHz. The service launches `pw-record` in its own process group, marks the generation stopping before SIGTERM, continuously drains bounded stdout and stderr, waits for pipe EOF and child close, and only then converts or finalizes. A shutdown timeout escalates to a process-group kill and fails the generation. The service creates no audio file.
Cancellation and disconnect are delivery tombstones. They stop capture, close a provider turn where present, release application-owned buffers, and suppress every later final event. Cleanup is best effort and cannot authorize editor delivery.
## Idle exit ## Idle exit
A successfully parsed protocol request resets the 15-minute monotonic idle activity timer, including requests that later produce state or ownership errors. Accepting a socket, receiving malformed data, and disconnecting do not reset activity. A successfully parsed protocol request resets the configured monotonic idle activity timer, including requests that later produce state or ownership errors. Accepting a socket, receiving malformed data, and disconnecting do not reset activity.
Idle exit is suspended while model loading, recording, recorder shutdown, PCM conversion, or inference is active. Once work becomes idle, the full timer starts again. Exiting closes accepted clients and the inherited listener but does not unlink the systemd-owned socket path. Idle exit is suspended while backend preparation, recording, recorder shutdown, PCM conversion, local inference, cloud streaming, provider finalization, or worker cleanup is active. Once work becomes idle, the configured timer starts again. It is 15 minutes by default. Exiting closes accepted clients and the inherited listener but does not unlink the systemd-owned socket path. The service's single owned worker is daemonized so bounded core shutdown permits normal interpreter exit even if native code is stuck. Explicit systemd shutdown also has a five-second stop deadline.
## Configuration and security boundary
The service reads strict versioned host configuration at startup from `~/.config/pi-dictate/config.json` and reads only the selected backend's regular, mode-`0600`, service-owned credential file. Keys never appear in command lines, systemd environment variables, URLs, redirects, protocol frames, diagnostics, or logs. Pi has no configuration writer and no access to this directory.
Local hardening denies IP networking and keeps the pinned runtime offline. Cloud hardening permits only `AF_UNIX`, `AF_INET`, and `AF_INET6` needed for provider DNS/TLS, while the base unit denies `bind()` and `listen()` so the service cannot create an inbound TCP listener. The service rejects a selected backend when the stamped network mode does not match the installed unit. Ambient `HTTP_PROXY`, `HTTPS_PROXY`, and `ALL_PROXY` variables are cleared unless separately reviewed. The exact host install and switch operations are documented in [`extensions/dictate/README.md`](../../extensions/dictate/README.md).
+6 -6
View File
@@ -1,10 +1,10 @@
# Dictate extension review and local Parakeet remediation plan # Dictate extension review and local Parakeet remediation plan
Status: implemented, with final host validation in progress. This document records the original review of `extensions/dictate/index.ts`, the agreed Linux implementation direction, and the acceptance criteria used for the replacement. Status: historical local-backend review; final host validation completed. This document records the original review of `extensions/dictate/index.ts`, the agreed Linux implementation direction, and the acceptance criteria used for the local replacement. The current configurable local, Meta, and OpenAI scope supersedes its local-only decisions; see [`configurable-asr-backends-plan.md`](configurable-asr-backends-plan.md) and [`protocol.md`](protocol.md).
## Decisions ## Decisions
The replacement will target this machine and workflow: The replacement will target this machine and workflow. The checked-in `profiles/nono/pi-local.json` is outside this change and remains byte-for-byte unchanged.
- **Platform:** Linux only, specifically CachyOS on x86-64 with niri and Ghostty. - **Platform:** Linux only, specifically CachyOS on x86-64 with niri and Ghostty.
- **Transcription:** local NVIDIA NeMo with `nvidia/parakeet-tdt-0.6b-v3` on the machine's RTX 4090. Deepgram and runtime cloud transcription are removed. - **Transcription:** local NVIDIA NeMo with `nvidia/parakeet-tdt-0.6b-v3` on the machine's RTX 4090. Deepgram and runtime cloud transcription are removed.
@@ -172,7 +172,7 @@ The service validates even-byte PCM length, converts little-endian signed 16-bit
Running capture and NeMo inside Pi would require PipeWire, model-directory, native-library, CUDA, and `/dev/nvidia*` access in the agent sandbox. Deepgram also requires cloud credentials and a network destination. Running capture and NeMo inside Pi would require PipeWire, model-directory, native-library, CUDA, and `/dev/nvidia*` access in the agent sandbox. Deepgram also requires cloud credentials and a network destination.
**Fix:** keep direct device and model capabilities in the host-side service. Add only `/run/user/1000/pi-dictate.sock` to `filesystem.unix_socket` in `profiles/nono/pi-local.json`. Do not add PipeWire configuration, `/run/user/1000/pipewire-0`, `/dev/snd`, `/dev/nvidia*`, model paths, Hugging Face cache paths, or `api.deepgram.com`. **Fix:** keep direct device and model capabilities in the host-side service. Preserve the existing `/run/user/1000/pi-dictate.sock` policy entry and make no profile edits. Do not add PipeWire configuration, `/run/user/1000/pipewire-0`, `/dev/snd`, `/dev/nvidia*`, model paths, Hugging Face cache paths, or `api.deepgram.com`.
The socket is deliberately a mediated microphone and transcription capability, not a harmless health endpoint. Any process running under `pi-local` or a descendant profile can request recording and receive the resulting text. The user accepts that broad profile scope; document it prominently and keep the service single-user with socket mode `0600`. The socket is deliberately a mediated microphone and transcription capability, not a harmless health endpoint. Any process running under `pi-local` or a descendant profile can request recording and receive the resulting text. The user accepts that broad profile scope; document it prominently and keep the service single-user with socket mode `0600`.
@@ -375,10 +375,10 @@ Build a throwaway prototype outside the production extension and nono sandbox:
### Phase 4: Add the minimum nono capability ### Phase 4: Add the minimum nono capability
1. Add `/run/user/1000/pi-dictate.sock` to `filesystem.unix_socket` in `profiles/nono/pi-local.json`. 1. Preserve the existing `/run/user/1000/pi-dictate.sock` policy entry and make no profile edits.
2. Do not add PipeWire, GPU, model, cache, Deepgram, or extra network permissions. 2. Do not add PipeWire, GPU, model, cache, Deepgram, or extra network permissions.
3. Validate the draft and resolved policy according to the nono-profile workflow. 3. Validate the existing resolved policy separately.
4. Compare the resolved policy before and after and confirm this feature adds only the exact service socket. 4. Confirm this feature does not change the resolved Pi policy.
5. Restart Pi because an active nono sandbox cannot gain socket access in place. 5. Restart Pi because an active nono sandbox cannot gain socket access in place.
6. Verify the extension can activate and use the service but still cannot connect directly to PipeWire or initialize NVML. 6. Verify the extension can activate and use the service but still cannot connect directly to PipeWire or initialize NVML.
+1 -1
View File
@@ -94,7 +94,7 @@ Below you will find my preferences so that we can be more aligned as we work tog
- Plan before we implement. Investigate the codebase, do research, and grill me with questions. I'd rather spend some time answering questions first and get a result I am happy with rather than something we'd have to do again. - Plan before we implement. Investigate the codebase, do research, and grill me with questions. I'd rather spend some time answering questions first and get a result I am happy with rather than something we'd have to do again.
- Prototype or create PoCs for large features first. If we can identify if something will or will not work early, that will save time in the long run. - Prototype or create PoCs for large features first. If we can identify if something will or will not work early, that will save time in the long run.
- Do not touch production or live databases unless explicitly instructed to do so. - Do not touch production or live databases unless explicitly instructed to do so.
- When presenting any single local file for my review, or a structured long-form report, plan, ADR, review, or summary, load the [`artifact-presenter`](../skills/document/artifact-presenter/SKILL.md) skill and open the complete file with mcat in its reusable Herdr pane instead of returning the full artifact inline. - When presenting any single local file for my review, or a structured long-form report, plan, ADR, review, or summary, load the [`artifact-presenter`](../skills/document/artifact-presenter/SKILL.md) skill and open the complete file through its environment-aware presenter instead of returning the full artifact inline.
- After reporting status, if there is no input required from me, continue work on the task. - After reporting status, if there is no input required from me, continue work on the task.
## Subagents and Parallel work ## Subagents and Parallel work
+1 -1
View File
@@ -49,7 +49,7 @@ Publish `@jbcode/pi-shared` before a new version of any extension that depends o
After configuring the same scope and token in the npm configuration visible to Pi, install the currently deployed release set with exact version pins: After configuring the same scope and token in the npm configuration visible to Pi, install the currently deployed release set with exact version pins:
```bash ```bash
pi install npm:@jbcode/pi-dictate@0.1.0 pi install npm:@jbcode/pi-dictate@0.2.0
pi install npm:@jbcode/pi-effort-controls@0.1.0 pi install npm:@jbcode/pi-effort-controls@0.1.0
pi install npm:@jbcode/pi-prompt-snippets@0.1.0 pi install npm:@jbcode/pi-prompt-snippets@0.1.0
pi install npm:@jbcode/pi-interactive-subagents@3.7.3 pi install npm:@jbcode/pi-interactive-subagents@3.7.3
+167 -93
View File
@@ -1,79 +1,152 @@
# Pi Dictate # Pi Dictate
Linux-only push-to-dictate for Pi. Audio capture and NVIDIA Parakeet run in a socket-activated host service. The Pi extension receives state and finalized text through a Unix socket, then inserts successful text with `pasteToEditor()`. Pi Dictate is a Linux-only push-to-dictate extension. The extension has no microphone, model, GPU, cloud, or credential access. It is a bounded JSON client for the host-owned `pi-dictate-service` Unix socket and pastes one finalized result into Pi's main editor.
Tested on CachyOS with niri, Ghostty, PipeWire, an RTX 4090, CUDA 13.0, NeMo 3.0.0, and Parakeet-TDT 0.6B v3. The configured backend is local by default. Cloud backends are opt-in because their audio leaves the host and may incur provider charges.
## Backends
| Backend | Capture | Runtime | Default model and endpoint |
| --- | --- | --- | --- |
| `local` | 16 kHz mono PCM16 | Offline NeMo/Parakeet on the host GPU | `nemo-parakeet` and a pinned local archive |
| `meta` | 24 kHz mono PCM16 | Meta Model API | `muse-voice-transcribe-1.0` at `api.meta.ai` |
| `openai` | 24 kHz mono PCM16 | OpenAI Realtime transcription | `gpt-live-transcribe` at `api.openai.com` |
Only one backend is selected in host configuration. There is no per-session or per-generation picker, automatic failover, or submission of one recording to multiple providers. Switching backend is a privileged host operation and takes effect after configuration validation and a service restart.
OpenAI's `gpt-transcribe` is a supported commit-oriented alternative. `whisper-1` is not silently treated as a Realtime model and is not the default.
## Requirements ## Requirements
- Linux with PipeWire and `pw-record` All backends require:
- systemd user services
- NVIDIA GPU with a working CUDA runtime
- `uv`
- Network access during initial dependency and model provisioning
Runtime is offline. The systemd service restricts networking and loads an explicit local model archive. - Linux with PipeWire and `pw-record`;
- a systemd user session;
- `uv` for installation;
- the Pi profile's exact `/run/user/1000/pi-dictate.sock` access, unless the socket path is changed for another UID.
## Install Local mode additionally requires an NVIDIA GPU with a working CUDA runtime and several gigabytes for the locked environment and model archive. Cloud-only mode does not install NeMo, CUDA, or the Parakeet archive, but it requires a provider key and outbound DNS/TLS access from the host service.
From the repository root: ## Install and configure
Run these commands from the repository root. The installer owns the installed runtime under `~/.local/share/pi-dictate/` and host configuration under `~/.config/pi-dictate/`.
Install the default offline backend:
```bash ```bash
./services/dictate/install.sh ./services/dictate/install.sh install --backend local
``` ```
The installer creates `~/.local/share/pi-dictate/`, installs the locked service as a non-editable wheel, downloads and verifies the pinned model snapshot, seeds the coding dictionary, installs host-owned copies of the systemd user units, and enables `pi-dictate.socket`. The installed runtime does not depend on the repository remaining at its installation path. Install a cloud backend by copying a key from a file. The key is never accepted as a command-line literal:
To reuse an already downloaded model archive:
```bash ```bash
./services/dictate/install.sh \ ./services/dictate/install.sh install --backend meta --api-key-file /path/to/meta-key
--model-source /path/to/parakeet-tdt-0.6b-v3.nemo ./services/dictate/install.sh install --backend openai --api-key-file /path/to/openai-key
``` ```
Preview without changing the host: Configure an installed service or inspect the effective setup:
```bash ```bash
./services/dictate/install.sh --dry-run --model-source /path/to/model.nemo ./services/dictate/install.sh configure --backend local
./services/dictate/install.sh configure --backend meta --api-key-file /path/to/meta-key
./services/dictate/install.sh configure --backend openai --api-key-file /path/to/openai-key
./services/dictate/install.sh check
``` ```
The pinned model revision and SHA-256 are recorded in [`../../docs/dictate/parakeet-attribution.md`](../../docs/dictate/parakeet-attribution.md). `--api-key-file` must identify a regular file. The installer copies it into a service-owned mode-`0600` file and does not put the secret in systemd environment variables, process arguments, URLs, protocol frames, diagnostics, or logs. At startup the service reads only the selected backend's key. Inactive backend keys are neither required nor read.
## Sandbox access The installed authority is:
`profiles/nono/pi-local.json` grants exactly: ```text
~/.config/pi-dictate/
config.json
credentials/
meta-api-key
openai-api-key
```
The service validates versioned JSON, selected-backend options, paths, numeric bounds, coding terms, and credential ownership and permissions before starting. Configuration changes require the configure operation and a validated service restart. Do not edit Pi extension source or try to write this directory from Pi.
Local model reuse remains checksum-gated. A custom archive must be accompanied by its explicit SHA-256 and a supported NeMo adapter:
```bash
MODEL=/path/to/parakeet-tdt-0.6b-v3.nemo
./services/dictate/install.sh install --backend local \
--model-source "$MODEL" \
--model-sha256 "$(sha256sum "$MODEL" | cut -d' ' -f1)"
```
The installer preserves valid existing model archives, dictionaries, configuration, and credentials on reinstall. Local and cloud modes use different systemd network hardening. Local mode denies IP networking; cloud mode permits IPv4/IPv6 for provider DNS and TLS while the base unit denies `bind()` and `listen()`, preventing a TCP listener. A backend/hardening mismatch fails closed. Cloud mode does not provide a stable hostname allowlist because provider addresses can change.
## Socket and sandbox boundary
The Pi profile grants access only to the exact service socket:
```text ```text
/run/user/1000/pi-dictate.sock /run/user/1000/pi-dictate.sock
``` ```
This is a mediated microphone and transcription capability. Any code running under `pi-local` or a descendant profile can ask the host service to record and can receive the transcript. Pi still receives no PipeWire, `/dev/snd`, CUDA device, model, cache, or dictionary access. An explicit filesystem deny protects `~/.local/share/pi-dictate` even if that directory is selected as `$WORKDIR`. This is a mediated microphone and transcription capability, not a health-only endpoint. Any code running under `pi-local` or a descendant profile can request a recording and receive its finalized transcript. Pi still has no PipeWire, `/dev/snd`, CUDA, model, dictionary, provider-key, or direct provider-network access. The checked-in profile policy is intentionally unchanged; review any local profile policy before exposing the service socket.
The active profile is repository-linked on the tested host. Start a new Pi sandbox after updating the profile because an existing nono process cannot gain the socket capability in place. Restart Pi after changing the nono profile or socket exposure. An existing nono process cannot gain a new socket capability in place.
## Controls ## Controls and delivery
| Control | Action | | Control | Action |
| --- | --- | | --- | --- |
| `Alt+M` | Start recording when idle; stop and transcribe while listening | | `Alt+M` | Start recording when idle; stop and transcribe while listening |
| `Alt+N` | Cancel the active generation | | `Alt+N` | Cancel the active generation |
| `/dictate-cancel` | Cancel the active generation | | `/dictate-cancel` | Cancel the active generation |
| `/dictate-status` | Open a short read-only hello/ping connection and show backend/model diagnostics |
Shortcuts are handled by Pi's main editor. They do not claim to work while a selector, confirmation, or custom modal owns input. If a modal opens after recording starts, a successful result still goes only to the main editor. Shortcuts are registered through Pi's public main-editor APIs. They work while the main editor has focus, not while a selector, confirmation, or custom modal owns input. If a modal opens after recording starts, a successful result still goes only to the main editor.
The footer shows service state. While recording, a red circle and twelve-cell scrolling waveform make active capture visible and react to the microphone level: The footer shows backend-neutral service states:
```text ```text
preparing transcription...
● listening ▁▂▄▇█▆▃▂▅▇▄▂ ● listening ▁▂▄▇█▆▃▂▅▇▄▂
stopping...
transcribing...
``` ```
The circle uses Pi's semantic error color and the waveform uses its accent color. The other states remain `loading model...`, `stopping...`, and `transcribing...`. Audio levels are bounded scalar protocol events; raw audio remains inside the host service. The waveform contains only bounded scalar level events. Raw audio, provider events, partial hypotheses, and credentials never cross the socket. Preparation includes local model load or cloud connection and authentication, so recording does not begin until the selected backend is ready. `/dictate-status` reports only bounded backend and model names. It never reports paths, endpoints with query data, key state, or provider payloads.
This indicator requires protocol version 2. Run the installer again after updating the repository so the installed host service and repository-linked extension upgrade together. Only one explicit, bounded final transcript for the active generation can change the editor. Partial hypotheses are discarded at the service boundary. Empty speech reports `No speech recognized.` without changing the editor. Cancellation, stale generations, disconnects, provider failures, and cleanup never paste text.
Only a non-empty finalized result for the current generation changes the editor. Empty speech, cancellation, stale results, disconnects, and failures do not insert text. ## Configuration and coding terms
## Coding dictionary The service reads host-owned configuration at process startup. A representative configuration is:
```json
{
"version": 1,
"backend": "openai",
"idleSeconds": 900,
"local": {
"adapter": "nemo-parakeet",
"modelPath": "~/.local/share/pi-dictate/models/parakeet.nemo",
"modelSha256": "...",
"termsPath": "~/.local/share/pi-dictate/config/coding-terms.txt",
"boostAlpha": 1.0
},
"meta": {
"model": "muse-voice-transcribe-1.0",
"apiKeyFile": "~/.config/pi-dictate/credentials/meta-api-key",
"termsPath": "~/.local/share/pi-dictate/config/coding-terms.txt"
},
"openai": {
"model": "gpt-live-transcribe",
"apiKeyFile": "~/.config/pi-dictate/credentials/openai-api-key",
"delay": "low",
"prompt": "Coding dictation",
"termsPath": "~/.local/share/pi-dictate/config/coding-terms.txt",
"languages": ["en"]
}
}
```
The exact defaults are installer-generated. Unknown keys, malformed JSON, unsupported versions, invalid bounds, incompatible options, missing selected keys, symlinks, multi-line credentials, and insecure credential permissions are rejected. OpenAI `delay` is optional; a credentialed live check confirmed `low`, and accepted values are `minimal`, `low`, `medium`, `high`, and `xhigh`. Omitting it keeps the provider default. Provider endpoints are fixed in production and are not configurable through Pi. The service clears ambient proxy variables unless an explicitly reviewed proxy feature is added.
The installer seeds: The installer seeds:
@@ -81,58 +154,94 @@ The installer seeds:
~/.local/share/pi-dictate/config/coding-terms.txt ~/.local/share/pi-dictate/config/coding-terms.txt
``` ```
The host service reads this file at model load and applies NeMo's native phrase boosting. Pi cannot read or modify it. Add one term or phrase per line, use `#` for comments, and restart the service after editing: Use one term or phrase per line and `#` for comments. In local mode, terms feed NeMo phrase boosting. In cloud mode, validated terms may be sent as provider keyword or language-bias context, so do not place secrets or sensitive phrases in the dictionary if they must remain on the host. The dictionary is bounded to 64 KiB, 512 distinct terms, and 128 UTF-8 bytes per term. Restart after an edit:
```bash ```bash
systemctl --user restart pi-dictate.service systemctl --user restart pi-dictate.service
``` ```
Reinstalling preserves an existing dictionary. The file is bounded to 64 KiB, 512 distinct terms, and 128 UTF-8 bytes per term. A conservative boost of `1.0` is configured in `pi-dictate.service`; aggressive values can create false positives. ## Privacy, network, and provider policies
## Model lifecycle and measured behavior Local mode captures and transcribes entirely on the host after installation. The runtime uses the pinned local model with offline settings and the systemd unit denies IP networking. No audio file is created. Application-owned buffers are bounded and released after transcription, cancellation, or failure. `LimitCORE=0` and `MemorySwapMax=0` reduce additional persistence where supported. This is best-effort non-persistence, not forensic zeroization: Python, NumPy, PyTorch, CUDA, drivers, host memory management, crash handling, and swap can retain copies outside the service's direct control.
The socket remains active while the Python service exits after 15 minutes of true inactivity. A later connection activates a fresh service. Loading, recording, recorder shutdown, and inference suspend idle exit. In Meta or OpenAI mode, the service connects and authenticates before recording. It then streams bounded 24 kHz PCM directly to the selected provider while recording. Stop drains `pw-record` completely before sending the provider end/commit marker, waits for one explicit final event, and returns only that bounded text. Audio is not written to disk and is not sent to Pi. Provider partials and wire events remain in the host service. The provider's current retention, terms, security, and billing policies apply to cloud audio and any context sent with the session. Review the links below before opting in, and monitor the provider account for usage charges.
Measured on the tested RTX 4090 using the pinned offline model: Provider references:
- Model archive: about 2.4 GB - Meta announcement: <https://research.meta.ai/blog/introducing-muse-voice-transcribe>
- Locked Python environment: about 5.6 GB in the Phase 0 prototype - Meta speech-to-text guide: <https://ai.developer.meta.com/docs/speech-to-text>
- Cold base-model load: about 6.4 seconds - Meta Realtime Voice API: <https://ai.developer.meta.com/docs/api-reference/voice/realtime>
- Production-adapter load with phrase-boost setup: about 10 seconds, including the private Triton cache build - Meta voice event schemas: <https://ai.developer.meta.com/docs/api-reference/voice/schemas>
- Ten-second microphone inference: about 0.29 seconds, 0.029 real-time factor - Meta authentication: <https://ai.developer.meta.com/docs/getting-started/authentication>
- Sixty-second repeated-speech inference: about 0.43 seconds, 0.0072 real-time factor - Meta terms: <https://ai.meta.com/terms/>
- Five-minute repeated-speech inference: about 0.82 seconds, 0.0027 real-time factor - OpenAI Realtime transcription: <https://developers.openai.com/api/docs/guides/realtime-transcription>
- Peak CUDA allocation: about 5.1 GB for short clips and 6.15 GB for five minutes - OpenAI Realtime WebSocket: <https://developers.openai.com/api/docs/guides/realtime-websocket>
- Five-minute raw PCM cap: 9.6 MB - OpenAI speech-to-text: <https://developers.openai.com/api/docs/guides/speech-to-text>
- OpenAI `gpt-live-transcribe`: <https://developers.openai.com/api/docs/models/gpt-live-transcribe>
- OpenAI terms: <https://openai.com/policies/terms-of-use/>
- OpenAI privacy policy: <https://openai.com/policies/privacy-policy/>
The 60-second and five-minute stress inputs repeat a real speech clip in memory. They validate bounded duration, latency, and GPU allocation, not long-form accuracy. The service uses only the documented public endpoints. It does not reuse Pi's provider auth store, put keys in URLs, follow credential-bearing redirects, or use the undocumented `shortwave.facebook.com` Muse Code endpoint.
## Privacy ## Local model, attribution, and measured behavior
The service passes this exact recorder command through pipes: Local mode uses NVIDIA's `parakeet-tdt-0.6b-v3` through NeMo. The model archive is about 2.4 GB, and the locked Python environment was about 5.6 GB in the Phase 0 prototype. On the tested RTX 4090, cold preparation took about 10 seconds with phrase-boost setup, warm inference was comfortably faster than real time, and peak CUDA allocation was about 5.1 GB for short clips and 6.15 GB for five minutes. The service caps a local recording at five minutes, approximately 9.6 MB of 16 kHz mono PCM16. Cloud sessions use the same five-minute Dictate cap even though provider sessions can be longer.
```text See [`../../docs/dictate/parakeet-attribution.md`](../../docs/dictate/parakeet-attribution.md) for the pinned revision, SHA-256, CC BY 4.0 model terms, NeMo Apache 2.0 terms, and source links. The repository does not redistribute model weights.
pw-record --raw --format s16 --rate 16000 --channels 1 -
```
It intentionally creates no audio files and releases application-owned buffers after transcription, cancellation, or failure. `LimitCORE=0` and `MemorySwapMax=0` reduce additional persistence. This is best-effort non-persistence, not forensic zeroization. Python, NumPy, PyTorch, CUDA, drivers, host memory management, and crash handling can retain copies outside the service's direct control. The service remains warm for the configured idle period, 15 minutes by default, while no recording, inference, preparation worker, or protocol activity is active. Idle exit releases local model/GPU resources. A later socket connection activates a fresh service.
The service does not log raw audio or transcripts by default. Recorder stderr and protocol output are bounded. The server accepts at most 16 clients and bounds each client's pending output to 64 KiB. ## Service operations and troubleshooting
## Service operations
```bash ```bash
systemctl --user status pi-dictate.socket pi-dictate.service systemctl --user status pi-dictate.socket pi-dictate.service
journalctl --user -u pi-dictate.service journalctl --user -u pi-dictate.service -n 100
systemctl --user restart pi-dictate.service systemctl --user restart pi-dictate.service
systemctl --user stop pi-dictate.service systemctl --user stop pi-dictate.service
./services/dictate/install.sh check
``` ```
Stopping only the service leaves the socket active for later reactivation. Stopping `pi-dictate.socket` removes the socket path. Stopping the service leaves the socket unit able to reactivate it. Stopping `pi-dictate.socket` removes the socket path. Service output is bounded and does not log keys, raw audio, transcripts, provider request bodies, or partial hypotheses by default.
### Socket or activation failure
```bash
ls -l "$XDG_RUNTIME_DIR/pi-dictate.sock"
systemctl --user status pi-dictate.socket pi-dictate.service
journalctl --user -u pi-dictate.service -n 100
```
The socket should be mode `0600` and owned by the current user. Run `/dictate-status` after a restart to confirm the selected backend and model. Restart Pi after a nono policy change.
### Local PipeWire, CUDA, or model failure
```bash
wpctl status
pw-record --raw --format s16 --rate 16000 --channels 1 - >/dev/null
nvidia-smi
```
Press `Ctrl+C` after checking capture. Confirm the model archive checksum and that the local systemd unit still denies IP networking. A local preparation failure must not start the recorder.
### Cloud key, DNS, TLS, or provider failure
Run `./services/dictate/install.sh check` and verify the selected credential file is a regular mode-`0600` file owned by the service user. Confirm the cloud drop-in is active, DNS works for the fixed provider endpoint, and the host can establish outbound TLS. Do not add a key to a command line or systemd environment. `meta_*` and `openai_*` service errors are intentionally bounded and redacted; provider request bodies are not a debugging log.
For an explicit live contract check, first select the cloud backend, then provide a non-sensitive raw 24 kHz mono PCM16 fixture of at most five seconds:
```bash
~/.local/share/pi-dictate/venv/bin/python -m pi_dictate_service.live_smoke /path/to/sample.pcm
```
This command immediately sends the sample to the configured provider and may incur charges. It reads only the managed selected credential, does not accept a literal key, rejects symlinked or oversized samples, and does not write audio.
### Busy, timeout, or duration errors
The service allows one generation globally. A cancelled preparation or inference worker can remain busy until its non-interruptible operation finishes. A stalled cloud stream, queue overflow, provider timeout, recorder failure, or five-minute limit discards the generation rather than returning a plausible incomplete transcript. Wait for it to settle or restart the service if the owning client crashed and the worker is stuck.
## niri and Ghostty ## niri and Ghostty
No compositor-global binding is required. Confirm the live niri config does not claim `Alt+M` or `Alt+N`, then validate it outside the Pi sandbox: No compositor-global binding is required. Confirm the live niri configuration does not claim `Alt+M` or `Alt+N`, then validate it outside the Pi sandbox:
```bash ```bash
niri validate niri validate
@@ -140,45 +249,10 @@ niri validate
The shortcuts should reach Pi only while its main editor has focus. No `wl-copy`, `wtype`, `ydotool`, X11 injection, or compositor spawn binding is used. The shortcuts should reach Pi only while its main editor has focus. No `wl-copy`, `wtype`, `ydotool`, X11 injection, or compositor spawn binding is used.
## Troubleshooting
### Socket or activation failure
```bash
systemctl --user status pi-dictate.socket pi-dictate.service
ls -l "$XDG_RUNTIME_DIR/pi-dictate.sock"
journalctl --user -u pi-dictate.service -n 100
```
The socket should be mode `0600` and owned by the current user. Restart Pi after changing nono policy.
### PipeWire or microphone failure
```bash
wpctl status
pw-record --raw --format s16 --rate 16000 --channels 1 - >/dev/null
```
Press `Ctrl+C` after confirming capture. The service uses the WirePlumber default input. Selectable devices are not implemented.
### CUDA or model failure
```bash
nvidia-smi
systemctl --user restart pi-dictate.service
journalctl --user -u pi-dictate.service -n 100
```
Confirm the installed model checksum matches the attribution document and that runtime network access is not needed.
### Service reports busy
Only one recording, load worker, or inference worker is allowed globally. A cancelled load or inference remains busy until its non-interruptible worker finishes. Wait for it to settle or restart the service if the owning client crashed and the worker is stuck.
## Uninstall ## Uninstall
```bash ```bash
./services/dictate/install.sh uninstall ./services/dictate/install.sh uninstall
``` ```
The uninstaller disables the socket, removes only installer-managed unit copies, reloads systemd, and removes the fixed `~/.local/share/pi-dictate/` directory. It ignores install-root environment overrides and refuses to replace or remove unmanaged unit files. Uninstall disables the socket, removes installer-managed unit copies, reloads systemd, and removes the installed runtime. It preserves `~/.config/pi-dictate/` and credentials by default. Use the installer's explicit, confirmed `--purge-config` path only when those credentials and settings should be removed. Uninstall does not remove unmanaged unit files.
+58 -2
View File
@@ -195,10 +195,66 @@ function serviceFrame(
test("registers main-editor shortcuts, cancel command, and shutdown cleanup", () => { test("registers main-editor shortcuts, cancel command, and shutdown cleanup", () => {
const harness = createHarness(); const harness = createHarness();
assert.deepEqual([...harness.shortcuts.keys()], ["alt+m", "alt+n"]); assert.deepEqual([...harness.shortcuts.keys()], ["alt+m", "alt+n"]);
assert.deepEqual([...harness.commands.keys()], ["dictate-cancel"]); assert.deepEqual([...harness.commands.keys()], ["dictate-cancel", "dictate-status"]);
assert.ok(harness.events.has("session_shutdown")); assert.ok(harness.events.has("session_shutdown"));
}); });
test("status command negotiates a short read-only connection and reports backend diagnostics", async () => {
const harness = createHarness();
await harness.command("dictate-status")(
"",
harness.context as ExtensionCommandContext,
);
const socket = harness.sockets[0];
assert.ok(socket);
socket.connect();
const hello = harness.sentFrames(socket)[0];
assert.equal(hello?.type, "hello");
socket.frame({
version: PROTOCOL_VERSION,
type: "pong",
requestId: hello?.requestId,
backend: "openai",
model: "gpt-live-transcribe",
});
const ping = harness.sentFrames(socket)[1];
assert.equal(ping?.type, "ping");
socket.frame({
version: PROTOCOL_VERSION,
type: "pong",
requestId: ping?.requestId,
});
assert.deepEqual(harness.notifications, [
{
message: "Dictation backend: openai; model: gpt-live-transcribe",
type: "info",
},
]);
assert.equal(socket.endCount, 1);
});
test("status diagnostics reject control characters without exposing them", async () => {
const harness = createHarness();
await harness.command("dictate-status")(
"",
harness.context as ExtensionCommandContext,
);
const socket = harness.sockets[0];
assert.ok(socket);
socket.connect();
const hello = harness.sentFrames(socket)[0];
socket.frame({
version: PROTOCOL_VERSION,
type: "pong",
requestId: hello?.requestId,
backend: "openai\nsecret",
});
assert.match(harness.notifications[0]?.message ?? "", /diagnostic backend is invalid/i);
assert.equal(socket.destroyCount, 1);
});
test("connection errors return to idle and the next start uses a fresh socket", async () => { test("connection errors return to idle and the next start uses a fresh socket", async () => {
const harness = createHarness(); const harness = createHarness();
await harness.shortcut("alt+m")(harness.context); await harness.shortcut("alt+m")(harness.context);
@@ -277,7 +333,7 @@ test("renders authoritative service states and sends a bounded stop command", as
serviceFrame(socket, generationId, "state", { state: "transcribing" }); serviceFrame(socket, generationId, "state", { state: "transcribing" });
assert.deepEqual(harness.statuses, [ assert.deepEqual(harness.statuses, [
"loading model...", "preparing transcription...",
"● listening ▁▁▁▁▁▁▁▁▁▁▁▁", "● listening ▁▁▁▁▁▁▁▁▁▁▁▁",
"stopping...", "stopping...",
"transcribing...", "transcribing...",
+228 -19
View File
@@ -9,6 +9,7 @@ export const MAX_ID_BYTES = 128;
const STATUS_KEY = "dictate"; const STATUS_KEY = "dictate";
const MAX_ERROR_DETAIL_BYTES = 2 * 1024; const MAX_ERROR_DETAIL_BYTES = 2 * 1024;
const MAX_DIAGNOSTIC_BYTES = 128;
const UTF8_DECODER = new TextDecoder("utf-8", { fatal: true }); const UTF8_DECODER = new TextDecoder("utf-8", { fatal: true });
const WAVEFORM_BARS = ["▁", "▂", "▃", "▄", "▅", "▆", "▇", "█"] as const; const WAVEFORM_BARS = ["▁", "▂", "▃", "▄", "▅", "▆", "▇", "█"] as const;
const WAVEFORM_SAMPLES = 12; const WAVEFORM_SAMPLES = 12;
@@ -76,6 +77,15 @@ function isBoundedId(value: unknown): value is string {
); );
} }
function isBoundedDiagnostic(value: unknown): value is string {
return (
typeof value === "string" &&
value.length > 0 &&
utf8Bytes(value) <= MAX_DIAGNOSTIC_BYTES &&
!/[\u0000-\u001f\u007f]/u.test(value)
);
}
function boundedText(value: string, maxBytes: number): string { function boundedText(value: string, maxBytes: number): string {
if (utf8Bytes(value) <= maxBytes) return value; if (utf8Bytes(value) <= maxBytes) return value;
let text = value; let text = value;
@@ -92,14 +102,21 @@ export function createDictateExtension(options: DictateExtensionOptions = {}) {
return function registerDictate(pi: ExtensionAPI) { return function registerDictate(pi: ExtensionAPI) {
let state: DictateState = "idle"; let state: DictateState = "idle";
let socket: DictateSocket | null = null; let socket: DictateSocket | null = null;
let statusSocket: DictateSocket | null = null;
let activeContext: ExtensionContext | null = null; let activeContext: ExtensionContext | null = null;
let statusContext: ExtensionContext | null = null;
let activeGenerationId: string | null = null; let activeGenerationId: string | null = null;
let helloRequestId: string | null = null; let helloRequestId: string | null = null;
let negotiated = false; let negotiated = false;
let receiveBuffer = Buffer.alloc(0); let receiveBuffer: Buffer<ArrayBufferLike> = Buffer.alloc(0);
let nextId = 0; let nextId = 0;
let resultDelivered = false; let resultDelivered = false;
let audioLevels: number[] = []; let audioLevels: number[] = [];
let statusRequest: "hello" | "ping" | null = null;
let statusRequestId: string | null = null;
let statusBackend: string | null = null;
let statusModel: string | null = null;
let statusReceiveBuffer: Buffer<ArrayBufferLike> = Buffer.alloc(0);
const newId = (kind: string) => `${kind}-${++nextId}`; const newId = (kind: string) => `${kind}-${++nextId}`;
@@ -135,7 +152,20 @@ export function createDictateExtension(options: DictateExtensionOptions = {}) {
} }
}; };
const closeStatusQuery = (forceClose = false) => {
const current = statusSocket;
statusSocket = null;
statusContext = null;
statusRequest = null;
statusRequestId = null;
statusBackend = null;
statusModel = null;
statusReceiveBuffer = Buffer.alloc(0);
if (current) closeSocket(current, forceClose);
};
const reset = (forceClose = false) => { const reset = (forceClose = false) => {
closeStatusQuery(true);
const current = socket; const current = socket;
socket = null; socket = null;
activeGenerationId = null; activeGenerationId = null;
@@ -162,6 +192,35 @@ export function createDictateExtension(options: DictateExtensionOptions = {}) {
reset(true); reset(true);
}; };
const failStatus = (message: string) => {
const context = statusContext;
closeStatusQuery(true);
context?.ui.notify(
boundedText(message, MAX_ERROR_DETAIL_BYTES),
"error",
);
};
const sendStatus = (
current: DictateSocket,
frame: Record<string, string | number>,
): boolean => {
if (statusSocket !== current) return false;
const encoded = `${JSON.stringify(frame)}\n`;
if (utf8Bytes(encoded) - 1 > MAX_FRAME_BYTES) {
failStatus("Dictation protocol error: outgoing frame is too large.");
return false;
}
try {
current.write(encoded);
return true;
} catch (error) {
const detail = error instanceof Error ? error.message : "write failed";
failStatus(`Dictation status request failed: ${detail}`);
return false;
}
};
const send = (frame: Record<string, string | number>): boolean => { const send = (frame: Record<string, string | number>): boolean => {
if (!socket) return false; if (!socket) return false;
const encoded = `${JSON.stringify(frame)}\n`; const encoded = `${JSON.stringify(frame)}\n`;
@@ -201,7 +260,7 @@ export function createDictateExtension(options: DictateExtensionOptions = {}) {
return true; return true;
} }
if (serviceState === "loading" && state === "connecting") { if (serviceState === "loading" && state === "connecting") {
setStatus("loading model..."); setStatus("preparing transcription...");
return true; return true;
} }
if (serviceState === "listening" && state === "connecting") { if (serviceState === "listening" && state === "connecting") {
@@ -246,6 +305,12 @@ export function createDictateExtension(options: DictateExtensionOptions = {}) {
} }
if (value.type === "pong") { if (value.type === "pong") {
for (const field of ["backend", "model"] as const) {
if (field in value && !isBoundedDiagnostic(value[field])) {
protocolError(`diagnostic ${field} is invalid.`);
return;
}
}
if ( if (
negotiated || negotiated ||
state !== "connecting" || state !== "connecting" ||
@@ -344,45 +409,184 @@ export function createDictateExtension(options: DictateExtensionOptions = {}) {
protocolError("unknown frame type."); protocolError("unknown frame type.");
}; };
const consumeData = (chunk: Uint8Array, current: DictateSocket) => { const handleStatusFrame = (value: unknown, current: DictateSocket) => {
if (socket !== current) return; if (statusSocket !== current) return;
receiveBuffer = Buffer.concat([receiveBuffer, Buffer.from(chunk)]); if (!isRecord(value)) {
while (socket === current) { failStatus("Dictation status protocol error: frame must be an object.");
const newline = receiveBuffer.indexOf(0x0a); return;
}
if (value.version !== PROTOCOL_VERSION) {
failStatus("Dictation status protocol version mismatch.");
return;
}
if (typeof value.type !== "string" || !isBoundedId(value.requestId)) {
failStatus("Dictation status protocol error: frame is invalid.");
return;
}
if (value.type === "error") {
const code = typeof value.code === "string" ? value.code : "service_error";
const detail = typeof value.detail === "string" ? value.detail : "Dictation status failed";
failStatus(
`Dictation service error (${boundedText(code, 64)}): ${boundedText(detail, 384)}`,
);
return;
}
if (value.type !== "pong" || !statusRequestId || value.requestId !== statusRequestId) {
failStatus("Dictation status protocol error: unexpected response.");
return;
}
for (const field of ["backend", "model"] as const) {
if (field in value && !isBoundedDiagnostic(value[field])) {
failStatus(`Dictation status protocol error: diagnostic ${field} is invalid.`);
return;
}
}
if (isBoundedDiagnostic(value.backend)) statusBackend = value.backend;
if (isBoundedDiagnostic(value.model)) statusModel = value.model;
if (statusRequest === "hello") {
statusRequest = "ping";
statusRequestId = newId("status-ping");
sendStatus(current, {
version: PROTOCOL_VERSION,
type: "ping",
requestId: statusRequestId,
});
return;
}
if (statusRequest !== "ping") {
failStatus("Dictation status protocol error: status request is invalid.");
return;
}
const backend = statusBackend ?? "unknown";
const model = statusModel ?? "unknown";
const context = statusContext;
closeStatusQuery();
context?.ui.notify(
`Dictation backend: ${backend}; model: ${model}`,
"info",
);
};
const consumeFramedData = (
chunk: Uint8Array,
current: DictateSocket,
isCurrent: () => boolean,
getBuffer: () => Buffer<ArrayBufferLike>,
setBuffer: (value: Buffer<ArrayBufferLike>) => void,
onFrame: (value: unknown, socket: DictateSocket) => void,
onProtocolError: (detail: string) => void,
) => {
if (!isCurrent()) return;
let buffer = Buffer.concat([getBuffer(), Buffer.from(chunk)]);
while (isCurrent()) {
const newline = buffer.indexOf(0x0a);
if (newline < 0) { if (newline < 0) {
if (receiveBuffer.length > MAX_FRAME_BYTES) { setBuffer(buffer);
protocolError("frame is too large."); if (buffer.length > MAX_FRAME_BYTES) {
onProtocolError("frame is too large.");
} }
return; return;
} }
if (newline > MAX_FRAME_BYTES) { if (newline > MAX_FRAME_BYTES) {
protocolError("frame is too large."); setBuffer(buffer);
onProtocolError("frame is too large.");
return; return;
} }
const line = receiveBuffer.subarray(0, newline); const line = buffer.subarray(0, newline);
receiveBuffer = receiveBuffer.subarray(newline + 1); buffer = buffer.subarray(newline + 1);
setBuffer(buffer);
if (line.length === 0) { if (line.length === 0) {
protocolError("empty frame."); onProtocolError("empty frame.");
return; return;
} }
let decoded: string; let decoded: string;
try { try {
decoded = UTF8_DECODER.decode(line); decoded = UTF8_DECODER.decode(line);
} catch { } catch {
protocolError("frame contains invalid UTF-8."); onProtocolError("frame contains invalid UTF-8.");
return; return;
} }
try { try {
handleFrame(JSON.parse(decoded), current); onFrame(JSON.parse(decoded), current);
} catch { } catch {
protocolError("malformed JSON frame."); onProtocolError("malformed JSON frame.");
return; return;
} }
} }
}; };
const consumeStatusData = (chunk: Uint8Array, current: DictateSocket) =>
consumeFramedData(
chunk,
current,
() => statusSocket === current,
() => statusReceiveBuffer,
(value) => {
statusReceiveBuffer = value;
},
handleStatusFrame,
(detail) => failStatus(`Dictation status protocol error: ${detail}`),
);
const consumeData = (chunk: Uint8Array, current: DictateSocket) =>
consumeFramedData(
chunk,
current,
() => socket === current,
() => receiveBuffer,
(value) => {
receiveBuffer = value;
},
handleFrame,
protocolError,
);
const statusDictation = (ctx: ExtensionContext) => {
if (state !== "idle") {
ctx.ui.notify("Dictation is active; status is unavailable.", "info");
return;
}
if (statusSocket) return;
statusContext = ctx;
statusRequest = "hello";
statusRequestId = newId("status-hello");
statusReceiveBuffer = Buffer.alloc(0);
let current: DictateSocket;
try {
current = socketFactory(socketPath);
} catch (error) {
const detail = error instanceof Error ? error.message : "connection failed";
failStatus(`Unable to query dictation service: ${detail}`);
return;
}
statusSocket = current;
current.onConnect(() => {
if (statusSocket !== current || !statusRequestId) return;
sendStatus(current, {
version: PROTOCOL_VERSION,
type: "hello",
requestId: statusRequestId,
});
});
current.onData((chunk) => consumeStatusData(chunk, current));
current.onError((error) => {
if (statusSocket !== current) return;
failStatus(`Dictation status connection failed: ${error.message}`);
});
current.onClose(() => {
if (statusSocket !== current) return;
failStatus("Dictation status service disconnected.");
});
};
const startDictation = (ctx: ExtensionContext) => { const startDictation = (ctx: ExtensionContext) => {
if (state !== "idle") return; if (state !== "idle") return;
closeStatusQuery(true);
state = "connecting"; state = "connecting";
activeContext = ctx; activeContext = ctx;
activeGenerationId = newId("generation"); activeGenerationId = newId("generation");
@@ -438,18 +642,23 @@ export function createDictateExtension(options: DictateExtensionOptions = {}) {
}; };
pi.registerShortcut("alt+m", { pi.registerShortcut("alt+m", {
description: "Start or stop local voice dictation", description: "Start or stop voice dictation",
handler: async (ctx) => toggleDictation(ctx), handler: async (ctx) => toggleDictation(ctx),
}); });
pi.registerShortcut("alt+n", { pi.registerShortcut("alt+n", {
description: "Cancel local voice dictation", description: "Cancel voice dictation",
handler: async () => cancelDictation(), handler: async () => cancelDictation(),
}); });
pi.registerCommand("dictate-cancel", { pi.registerCommand("dictate-cancel", {
description: "Cancel local voice dictation", description: "Cancel voice dictation",
handler: async () => cancelDictation(), handler: async () => cancelDictation(),
}); });
pi.registerCommand("dictate-status", {
description: "Show the selected dictation backend",
handler: async (_args, ctx) => statusDictation(ctx),
});
pi.on("session_shutdown", async () => { pi.on("session_shutdown", async () => {
closeStatusQuery(true);
if (state === "idle") return; if (state === "idle") return;
if (negotiated) sendGenerationCommand("cancel"); if (negotiated) sendGenerationCommand("cancel");
reset(); reset();
+1 -1
View File
@@ -1,6 +1,6 @@
{ {
"name": "@jbcode/pi-dictate", "name": "@jbcode/pi-dictate",
"version": "0.1.0", "version": "0.2.0",
"description": "Pi extension for local Linux voice dictation through a Unix socket service.", "description": "Pi extension for local Linux voice dictation through a Unix socket service.",
"type": "module", "type": "module",
"license": "MIT", "license": "MIT",
File diff suppressed because it is too large Load Diff
+8 -2
View File
@@ -1,13 +1,19 @@
[project] [project]
name = "pi-dictate-service" name = "pi-dictate-service"
version = "0.1.0" version = "0.1.0"
description = "Socket-activated local transcription service for the Pi dictate extension" description = "Socket-activated transcription service for the Pi dictate extension"
requires-python = ">=3.12,<3.13" requires-python = ">=3.12,<3.13"
dependencies = [ dependencies = []
[project.optional-dependencies]
local = [
"nemo-toolkit[asr]==3.0.0", "nemo-toolkit[asr]==3.0.0",
"numpy==2.5.2", "numpy==2.5.2",
"torch==2.13.0", "torch==2.13.0",
] ]
cloud = [
"websockets==15.0.1",
]
[project.scripts] [project.scripts]
pi-dictate-service = "pi_dictate_service.server:main" pi-dictate-service = "pi_dictate_service.server:main"
@@ -0,0 +1,533 @@
"""Backend-neutral transcription and realtime turn abstractions."""
from __future__ import annotations
from dataclasses import dataclass
import json
from queue import Empty, Full, Queue
import threading
import time
from typing import Any, Protocol, runtime_checkable
from .recorder import Capture
MAX_PCM_QUEUE_BYTES = 240_000 # five seconds of 24 kHz mono PCM16
MAX_PCM_QUEUE_CHUNKS = 64
MAX_PROVIDER_EVENT_BYTES = 64 * 1024
MAX_PROVIDER_EVENT_QUEUE_BYTES = 512 * 1024
MAX_PROVIDER_EVENT_QUEUE_ITEMS = 128
MAX_PROVIDER_TEXT_BYTES = 32 * 1024
RECEIVE_TIMEOUT = object()
class ProviderError(RuntimeError):
"""A bounded, redacted provider failure suitable for the service boundary."""
def __init__(self, code: str, detail: str = "transcription provider failed") -> None:
self.code = code
try:
self.detail = detail.encode("utf-8", errors="replace")[:2048].decode("utf-8", errors="ignore")
except (AttributeError, UnicodeError):
self.detail = "transcription provider failed"
super().__init__(self.detail)
@dataclass(frozen=True)
class BackendDescriptor:
backend: str
model: str
@runtime_checkable
class TranscriptionTurn(Protocol):
sample_rate_hz: int
retain_pcm: bool
def enqueue_pcm(self, chunk: bytes) -> None: ...
def finish(self, capture: Capture | None = None) -> str: ...
def poll_error(self) -> ProviderError | None: ...
def cancel(self) -> None: ...
def close(self) -> None: ...
@property
def cleanup_complete(self) -> bool: ...
@runtime_checkable
class TranscriptionBackend(Protocol):
descriptor: BackendDescriptor
sample_rate_hz: int
def start_turn(self) -> TranscriptionTurn: ...
def cancel_prepare(self) -> None: ...
class PcmQueue:
"""A byte- and item-bounded nonblocking queue for recorder drain threads."""
def __init__(
self,
*,
max_bytes: int = MAX_PCM_QUEUE_BYTES,
max_chunks: int = MAX_PCM_QUEUE_CHUNKS,
) -> None:
if max_bytes <= 0 or max_chunks <= 0:
raise ValueError("PCM queue bounds must be positive")
self._queue: Queue[bytes | None] = Queue(maxsize=max_chunks)
self._max_bytes = max_bytes
self._queued_bytes = 0
self._lock = threading.Lock()
self._close_lock = threading.Lock()
self._closed = False
self._end_queued = False
@property
def queued_bytes(self) -> int:
with self._lock:
return self._queued_bytes
def put(self, chunk: bytes) -> None:
if not isinstance(chunk, bytes) or not chunk:
raise ProviderError("queue_invalid", "provider received invalid audio")
with self._lock:
if self._closed:
raise ProviderError("queue_closed", "provider audio queue is closed")
if len(chunk) > self._max_bytes or self._queued_bytes + len(chunk) > self._max_bytes:
raise ProviderError("queue_overflow", "provider audio queue is full")
self._queued_bytes += len(chunk)
try:
self._queue.put_nowait(chunk)
except Full as error:
with self._lock:
self._queued_bytes -= len(chunk)
raise ProviderError("queue_overflow", "provider audio queue is full") from error
def close(self, *, timeout: float = 15.0) -> None:
if not self._close_lock.acquire(timeout=max(0.0, timeout)):
raise ProviderError("queue_overflow", "provider audio queue could not be finalized")
try:
with self._lock:
if self._end_queued:
return
self._closed = True
# A full queue still has valid audio that must be drained. The
# closed flag is a side-channel EOF marker when the sentinel cannot
# be inserted yet, rather than dropping the queued tail.
try:
self._queue.put_nowait(None)
except Full:
pass
with self._lock:
self._end_queued = True
finally:
self._close_lock.release()
def get(self, timeout: float | None = None) -> bytes | None:
deadline = None if timeout is None else time.monotonic() + timeout
while True:
poll_timeout = 0.1 if deadline is None else max(0.0, min(0.1, deadline - time.monotonic()))
try:
chunk = self._queue.get(timeout=poll_timeout)
except Empty:
with self._lock:
closed = self._closed
if closed:
return None
if deadline is not None and time.monotonic() >= deadline:
raise
continue
if chunk is not None:
with self._lock:
self._queued_bytes -= len(chunk)
return chunk
class LocalNemoTurn:
"""Post-stop local inference turn retaining audio only in the recorder."""
sample_rate_hz = 16_000
retain_pcm = True
def __init__(self, model: Any) -> None:
self._model = model
self._cancelled = False
self._closed = False
self._finished = False
@property
def cleanup_complete(self) -> bool:
return self._closed
def enqueue_pcm(self, chunk: bytes) -> None:
# Local turns use Capture.pcm so no second copy is made here.
if not self.retain_pcm:
raise ProviderError("local_invalid", "local turn cannot stream audio")
def finish(self, capture: Capture | None = None) -> str:
if self._cancelled:
raise ProviderError("local_cancelled", "local turn was cancelled")
if self._closed or self._finished:
raise ProviderError("local_invalid", "local turn was already finalized")
self._finished = True
if capture is None:
raise ProviderError("local_invalid", "local capture is missing")
if capture.sample_rate_hz != self.sample_rate_hz:
raise ProviderError("local_invalid", "local capture has an unsupported sample rate")
from .audio import pcm_s16le_to_floats
max_bytes = getattr(self._model, "max_pcm_bytes", 5 * 60 * 16_000 * 2)
samples = pcm_s16le_to_floats(capture.pcm, max_bytes=max_bytes)
result = self._model.transcribe(samples)
if not isinstance(result, str):
raise ProviderError("local_failure", "local model returned an invalid transcript")
return result
def poll_error(self) -> ProviderError | None:
return None
def cancel(self) -> None:
self._cancelled = True
def close(self) -> None:
self._closed = True
class LocalNemoBackend:
"""Backend wrapper preserving the lazy, warm NeMo model behavior."""
sample_rate_hz = 16_000
def __init__(self, model: Any) -> None:
self._model = model
self.descriptor = BackendDescriptor("local", "nemo-parakeet")
@property
def ready(self) -> bool:
return bool(getattr(self._model, "loaded", False))
def start_turn(self) -> LocalNemoTurn:
self._model.load()
return LocalNemoTurn(self._model)
def cancel_prepare(self) -> None:
# NeMo model loading is not interruptible, but the service tombstone
# prevents a cancelled preparation from starting capture.
return None
class QueuedCloudTurn:
"""Common ordered sender/receiver for cloud adapters.
Provider handshakes happen before this object is created. The recorder only
calls ``enqueue_pcm``; it never performs network I/O. A sender thread drains
the bounded queue and sends the end marker after every queued PCM chunk. A
receiver thread continuously consumes provider events during capture so a
websocket's receive buffer cannot stall the sender.
"""
sample_rate_hz = 24_000
retain_pcm = False
def __init__(
self,
transport: Any,
*,
timeout: float = 15.0,
max_queue_bytes: int = MAX_PCM_QUEUE_BYTES,
max_queue_chunks: int = MAX_PCM_QUEUE_CHUNKS,
) -> None:
if max_queue_bytes <= 0 or max_queue_bytes > MAX_PCM_QUEUE_BYTES:
raise ValueError("cloud PCM backlog must not exceed five seconds")
self._transport = transport
self._timeout = timeout
self._queue = PcmQueue(max_bytes=max_queue_bytes, max_chunks=max_queue_chunks)
self._provider_events: Queue[tuple[dict[str, Any], int]] = Queue(maxsize=MAX_PROVIDER_EVENT_QUEUE_ITEMS)
self._provider_event_bytes = 0
self._provider_closed = False
self._sender_error: ProviderError | None = None
self._receiver_error: ProviderError | None = None
self._event_lock = threading.Lock()
self._cancel_event = threading.Event()
self._transport_close_lock = threading.Lock()
self._transport_close_thread: threading.Thread | None = None
self._cancelled = False
self._finished = False
self._closed = False
self._sent_bytes = 0
self._stream_started_at = time.monotonic()
self._sender = threading.Thread(target=self._send_loop, daemon=True, name="dictate-provider-send")
self._receiver = threading.Thread(target=self._receive_loop, daemon=True, name="dictate-provider-recv")
self._sender.start()
self._receiver.start()
@property
def cleanup_complete(self) -> bool:
close_thread = self._transport_close_thread
return (
not self._sender.is_alive()
and not self._receiver.is_alive()
and (close_thread is None or not close_thread.is_alive())
)
def enqueue_pcm(self, chunk: bytes) -> None:
if self._cancelled or self._finished:
raise ProviderError(f"{self._provider_prefix}_queue_closed", "provider turn is no longer accepting audio")
try:
self._queue.put(chunk)
except ProviderError as error:
raise ProviderError(f"{self._provider_prefix}_{error.code}", error.detail) from error
def close_input(self) -> None:
"""Signal EOF after the recorder has drained stdout to EOF."""
self._queue.close()
def poll_error(self) -> ProviderError | None:
error = self._sender_error or self._receiver_error
if error is not None:
return error
with self._event_lock:
provider_closed = self._provider_closed
if provider_closed and not self._finished and not self._cancelled:
return ProviderError(
self._provider_prefix + "_premature_close",
"provider closed before recording was finalized",
)
return None
def finish(self, capture: Capture | None = None) -> str:
if self._cancelled:
raise ProviderError("cancelled", "provider turn was cancelled")
if self._finished:
raise ProviderError("invalid_state", "provider turn was already finalized")
self._finished = True
try:
self._queue.close(timeout=self._timeout)
except ProviderError:
self.cancel()
raise
self._sender.join(timeout=self._timeout)
if self._sender.is_alive():
self.cancel()
raise ProviderError(self._provider_prefix + "_timeout", "provider audio sender timed out")
if self._sender_error is not None:
error = self._sender_error
self.close()
raise error
try:
transcript = self._finish_provider()
# A correlated final isn't enough to publish a result while the
# receiver can still report a queued provider failure. Stop and
# join it first, then make the receiver error check authoritative.
self._cancel_event.set()
self._request_transport_close()
self._receiver.join(timeout=self._timeout)
if self._receiver.is_alive():
raise ProviderError(
self._provider_prefix + "_timeout",
"provider receiver did not stop after the final transcript",
)
if self._receiver_error is not None:
raise self._receiver_error
return transcript
finally:
self.close()
def cancel(self) -> None:
if self._cancelled and self.cleanup_complete:
return
self._cancelled = True
self._cancel_event.set()
self._request_transport_close()
try:
self._queue.close(timeout=0.1)
except ProviderError:
pass
# Closing the transport is the cancellation signal. Do not block the
# protocol loop waiting for provider threads; ``cleanup_complete``
# keeps the tombstoned generation busy until they actually exit.
def close(self) -> None:
if self._closed and self.cleanup_complete:
return
self._cancel_event.set()
self._request_transport_close()
# Keep close bounded and nonblocking from the service loop. A later
# poll retries the joins while cleanup_complete prevents a new turn.
for thread in (self._sender, self._receiver, self._transport_close_thread):
if thread is not None:
thread.join(timeout=min(self._timeout, 0.05))
self._closed = self.cleanup_complete
def _send_loop(self) -> None:
try:
while True:
chunk = self._queue.get(timeout=self._timeout)
if chunk is None:
# Queue closure means commit only for a successful recorder
# EOF. Cancellation closes the same bounded queue merely to
# wake this worker and must never finalize partial audio.
if not self._cancel_event.is_set():
self._send_end()
return
# Pace the stream against the declared PCM rate. This prevents
# a fast recorder from building more than five seconds of
# provider-side backlog while preserving ordered sends.
target_elapsed = self._sent_bytes / (self.sample_rate_hz * 2)
elapsed = time.monotonic() - self._stream_started_at
if target_elapsed > elapsed and self._cancel_event.wait(target_elapsed - elapsed):
return
if self._cancel_event.is_set():
return
self._send_audio(chunk)
self._sent_bytes += len(chunk)
except ProviderError as error:
self._sender_error = error
self._abort_transport()
except Empty:
self._sender_error = ProviderError(
self._provider_prefix + "_timeout", "provider audio sender timed out"
)
self._abort_transport()
except Exception:
self._sender_error = ProviderError(
self._provider_prefix + "_send", "provider audio upload failed"
)
self._abort_transport()
def _receive_loop(self) -> None:
try:
while not self._cancel_event.is_set():
event = self._receive_provider_event(timeout=0.1)
if event is RECEIVE_TIMEOUT:
continue
if event is None:
with self._event_lock:
self._provider_closed = True
return
if not isinstance(event, dict):
raise ProviderError(self._provider_prefix + "_malformed_event", "provider event is invalid")
try:
size = len(json.dumps(event, ensure_ascii=False, separators=(",", ":")).encode("utf-8"))
except (TypeError, UnicodeError, ValueError, RecursionError, OverflowError) as error:
raise ProviderError(self._provider_prefix + "_malformed_event", "provider event is invalid") from error
with self._event_lock:
if self._provider_event_bytes + size > MAX_PROVIDER_EVENT_QUEUE_BYTES:
raise ProviderError(self._provider_prefix + "_event_overflow", "provider event queue is full")
self._provider_event_bytes += size
try:
self._provider_events.put_nowait((event, size))
except Full as error:
with self._event_lock:
self._provider_event_bytes -= size
raise ProviderError(self._provider_prefix + "_event_overflow", "provider event queue is full") from error
except ProviderError as error:
# Preserve a provider failure that races with successful
# finalization. Cancellation does not consume this error, but it
# avoids recursively closing a transport we are already closing.
self._receiver_error = error
if not self._cancel_event.is_set():
self._abort_transport()
except Exception:
if not self._cancel_event.is_set():
self._receiver_error = ProviderError(
self._provider_prefix + "_receive", "provider event stream failed"
)
self._abort_transport()
def _next_provider_event(self, timeout: float) -> dict[str, Any] | None:
try:
event, size = self._provider_events.get(timeout=max(0.0, timeout))
except Empty as error:
error_from_receiver = self._receiver_error
if error_from_receiver is not None:
raise error_from_receiver
with self._event_lock:
closed = self._provider_closed
if closed:
return None
raise ProviderError(self._provider_prefix + "_timeout", "timed out waiting for provider event") from error
with self._event_lock:
self._provider_event_bytes -= size
return event
@property
def _provider_prefix(self) -> str:
return "provider"
def _send_audio(self, chunk: bytes) -> None:
raise NotImplementedError
def _send_end(self) -> None:
raise NotImplementedError
def _finish_provider(self) -> str:
raise NotImplementedError
def _receive_provider_event(self, *, timeout: float) -> dict[str, Any] | None | object:
raise NotImplementedError
def _request_transport_close(self) -> None:
with self._transport_close_lock:
if self._transport_close_thread is not None:
return
self._transport_close_thread = threading.Thread(
target=self._abort_transport,
daemon=True,
name="dictate-provider-close",
)
self._transport_close_thread.start()
def _abort_transport(self) -> None:
try:
self._transport.close()
except Exception:
pass
def bounded_provider_text(value: Any, *, field: str = "transcript") -> str:
if not isinstance(value, str):
raise ProviderError("provider_malformed", f"provider {field} is invalid")
try:
if len(value.encode("utf-8")) > MAX_PROVIDER_TEXT_BYTES:
raise ProviderError("provider_malformed", f"provider {field} is too large")
except (UnicodeEncodeError, AttributeError) as error:
raise ProviderError("provider_malformed", f"provider {field} is invalid") from error
return value
def provider_event(value: Any) -> dict[str, Any]:
"""Validate one already decoded event and apply a serialized-size bound."""
if not isinstance(value, dict) or any(not isinstance(key, str) for key in value):
raise ProviderError("provider_malformed", "provider event is not an object")
try:
encoded = json.dumps(value, ensure_ascii=False, separators=(",", ":")).encode("utf-8")
except (TypeError, UnicodeError, ValueError, RecursionError, OverflowError) as error:
raise ProviderError("provider_malformed", "provider event is invalid") from error
if len(encoded) > MAX_PROVIDER_EVENT_BYTES:
raise ProviderError("provider_malformed", "provider event is too large")
return value
def backend_from_config(config: Any) -> TranscriptionBackend:
"""Construct only the selected backend, keeping cloud imports lazy."""
if config.backend == "local":
from .backends.local import LocalNemoBackend
return LocalNemoBackend(config.local)
if config.backend == "meta":
from .backends.meta import MetaBackend
return MetaBackend(config.meta)
if config.backend == "openai":
from .backends.openai import OpenAIBackend
return OpenAIBackend(config.openai)
raise ProviderError("configuration", "unsupported transcription backend")
__all__ = [
"BackendDescriptor", "LocalNemoBackend", "LocalNemoTurn", "MAX_PCM_QUEUE_BYTES",
"MAX_PROVIDER_EVENT_BYTES", "MAX_PROVIDER_EVENT_QUEUE_BYTES", "MAX_PROVIDER_TEXT_BYTES",
"PcmQueue", "ProviderError", "QueuedCloudTurn", "RECEIVE_TIMEOUT",
"TranscriptionBackend", "TranscriptionTurn", "backend_from_config",
"bounded_provider_text", "provider_event",
]
@@ -0,0 +1,8 @@
"""Optional cloud and local Dictate backend adapters."""
from ..backend import backend_from_config
from .local import LocalNemoBackend
from .meta import MetaBackend, MetaTurn
from .openai import OpenAIBackend, OpenAITurn
__all__ = ["LocalNemoBackend", "MetaBackend", "MetaTurn", "OpenAIBackend", "OpenAITurn", "backend_from_config"]
@@ -0,0 +1,32 @@
"""Local NeMo backend factory."""
from __future__ import annotations
from typing import Any
from ..backend import LocalNemoBackend as _LocalNemoBackend
from ..config import LocalConfig
from ..model import NemoParakeetModel
class LocalNemoBackend(_LocalNemoBackend):
"""Build the existing lazy NeMo adapter from typed host configuration."""
def __init__(self, config: LocalConfig | Any) -> None:
if isinstance(config, LocalConfig):
model = NemoParakeetModel(
config.model_path,
terms_path=config.terms_path,
boost_alpha=config.boost_alpha,
expected_sha256=config.model_sha256,
)
else:
model = config
super().__init__(model)
@property
def model(self) -> Any:
return self._model
__all__ = ["LocalNemoBackend"]
@@ -0,0 +1,400 @@
"""Meta Muse Voice Transcribe realtime adapter.
The websocket client is imported only when cloud mode is selected. Tests inject
an object implementing ``send``, ``recv`` and ``close``; no network dependency is
needed to import or test the service in local mode.
"""
from __future__ import annotations
import json
import logging
from queue import Empty
import threading
import time
from typing import Any, Callable, Mapping
from ..backend import (
BackendDescriptor,
MAX_PROVIDER_EVENT_BYTES,
ProviderError,
QueuedCloudTurn,
RECEIVE_TIMEOUT,
bounded_provider_text,
provider_event,
)
from ..config import CredentialError, MetaConfig, provider_keywords, read_credential
META_ENDPOINT = "wss://api.meta.ai/v1/asr/realtime"
META_MODEL = "muse-voice-transcribe-1.0"
META_SAMPLE_RATE_HZ = 24_000
META_MAX_BACKLOG_BYTES = 240_000 # provider's documented five-second limit
META_HANDSHAKE_TIMEOUT = 10.0
META_EVENT_TIMEOUT = 45.0
META_SOCKET_IO_TIMEOUT = 10.0
META_URL = META_ENDPOINT
TransportFactory = Callable[..., Any]
def _default_transport(endpoint: str, **_kwargs: Any) -> Any:
try:
from websockets.sync.client import connect
except ImportError as error:
raise ProviderError("meta_dependency", "Meta backend websocket dependency is unavailable") from error
try:
logger = logging.getLogger("pi_dictate_service.provider.meta")
logger.handlers = [logging.NullHandler()]
logger.propagate = False
transport = connect(
endpoint,
open_timeout=META_HANDSHAKE_TIMEOUT,
close_timeout=5,
max_size=MAX_PROVIDER_EVENT_BYTES,
proxy=None,
logger=logger,
)
transport.socket.settimeout(META_SOCKET_IO_TIMEOUT)
return transport
except Exception as error:
response = getattr(error, "response", None)
status = getattr(response, "status_code", getattr(error, "status_code", getattr(error, "status", None)))
if status in {401, 403}:
raise ProviderError("meta_auth", "Meta authentication failed") from error
if status == 429:
raise ProviderError("meta_rate_limit", "Meta transcription rate limit reached") from error
raise ProviderError("meta_connect", "could not connect to Meta transcription service") from error
def _send_json(transport: Any, event: Mapping[str, Any]) -> None:
try:
payload = json.dumps(dict(event), separators=(",", ":"), ensure_ascii=False)
if len(payload.encode("utf-8")) > MAX_PROVIDER_EVENT_BYTES:
raise ProviderError("meta_event_too_large", "Meta transcription event is too large")
transport.send(payload)
except ProviderError:
raise
except Exception as error:
raise ProviderError("meta_send", "could not send Meta transcription event") from error
def _receive_event(
transport: Any, *, timeout: float, tolerate_timeout: bool = False
) -> dict[str, Any] | None | object:
try:
try:
raw = transport.recv(timeout=timeout)
except TypeError:
raw = transport.recv()
except (TimeoutError, Empty):
if tolerate_timeout:
return RECEIVE_TIMEOUT
raise ProviderError("meta_timeout", "timed out waiting for Meta transcription service")
except Exception as error:
code = _close_code(error)
if code == 1000:
return None
if code in {1008, 1011, 1013}:
raise ProviderError(f"meta_close_{code}", "Meta transcription service closed the session") from error
raise ProviderError("meta_premature_close", "Meta transcription service closed unexpectedly") from error
if raw is None:
return None
if isinstance(raw, dict):
try:
return provider_event(raw)
except ProviderError as error:
raise ProviderError("meta_malformed_event", "Meta sent an invalid event") from error
if isinstance(raw, bytes):
if len(raw) > MAX_PROVIDER_EVENT_BYTES:
raise ProviderError("meta_malformed_event", "Meta sent an oversized event")
try:
raw = raw.decode("utf-8")
except UnicodeDecodeError as error:
raise ProviderError("meta_malformed_event", "Meta sent an invalid event") from error
if not isinstance(raw, str):
raise ProviderError("meta_malformed_event", "Meta sent an invalid event")
try:
if len(raw.encode("utf-8")) > MAX_PROVIDER_EVENT_BYTES:
raise ProviderError("meta_malformed_event", "Meta sent an oversized event")
except UnicodeEncodeError as error:
raise ProviderError("meta_malformed_event", "Meta sent an invalid event") from error
try:
value = json.loads(raw)
except (UnicodeDecodeError, ValueError, RecursionError) as error:
raise ProviderError("meta_malformed_event", "Meta sent malformed event JSON") from error
try:
return provider_event(value)
except ProviderError as error:
raise ProviderError("meta_malformed_event", "Meta sent an invalid event") from error
def _close_transport(transport: Any) -> None:
try:
transport.close()
except Exception:
pass
def _close_code(error: BaseException) -> int | None:
code = getattr(error, "code", None)
if isinstance(code, int):
return code
received = getattr(error, "rcvd", None)
code = getattr(received, "code", None)
return code if isinstance(code, int) else None
class MetaTurn(QueuedCloudTurn):
sample_rate_hz = META_SAMPLE_RATE_HZ
retain_pcm = False
def _receive_provider_event(self, *, timeout: float) -> dict[str, Any] | None | object:
event = _receive_event(self._transport, timeout=timeout, tolerate_timeout=True)
if event is RECEIVE_TIMEOUT or event is None:
return event
_validate_meta_event(event)
if event.get("type") == "error" or "error" in event:
raise _meta_event_error(event)
if event.get("type") == "transcript" and event.get("final") is True:
return event
# Partials, progress, endpointing, and diarization may be frequent.
# Validate and discard them during capture instead of retaining an
# unbounded session history for finish().
return RECEIVE_TIMEOUT
@property
def _provider_prefix(self) -> str:
return "meta"
def _send_audio(self, chunk: bytes) -> None:
try:
self._transport.send(chunk)
except Exception as error:
raise ProviderError("meta_send", "could not send Meta audio") from error
def _send_end(self) -> None:
_send_json(self._transport, {"type": "endStream"})
def _finish_provider(self) -> str:
deadline = time.monotonic() + self._timeout
while time.monotonic() < deadline:
event = self._next_provider_event(max(0.01, deadline - time.monotonic()))
if event is None:
raise ProviderError("meta_premature_close", "Meta closed without a final transcript")
_validate_meta_event(event)
event_type = event["type"]
if event_type == "error":
raise _meta_event_error(event)
if event_type != "transcript":
# Partial, endpoint, and diarization events are intentionally
# consumed but never exposed to the Pi protocol.
continue
if event.get("final") is not True:
continue
value = event.get("text", event.get("transcript"))
if isinstance(value, Mapping):
value = value.get("text")
try:
return bounded_provider_text(value)
except ProviderError as error:
raise ProviderError("meta_malformed_event", "Meta sent an invalid transcript") from error
raise ProviderError("meta_timeout", "timed out waiting for Meta final transcript")
def _validate_meta_event(event: Mapping[str, Any]) -> None:
event_type = event.get("type")
if not isinstance(event_type, str) or not event_type or len(event_type.encode("utf-8")) > 128:
raise ProviderError("meta_malformed_event", "Meta event type is invalid")
if event_type == "transcript":
if not isinstance(event.get("final"), bool):
raise ProviderError("meta_malformed_event", "Meta transcript final flag is invalid")
if "text" not in event and "transcript" not in event:
raise ProviderError("meta_malformed_event", "Meta transcript text is missing")
if event_type == "error" or "error" in event:
nested = event.get("error")
if nested is not None and not isinstance(nested, Mapping):
raise ProviderError("meta_malformed_event", "Meta error event is invalid")
value: Any = event.get("code", event.get("errorCode"))
if isinstance(nested, Mapping):
value = nested.get("code", nested.get("type", value))
if not isinstance(value, (str, int)):
raise ProviderError("meta_malformed_event", "Meta error code is invalid")
def _meta_event_error(event: Mapping[str, Any]) -> ProviderError:
value: Any = event.get("code", event.get("errorCode"))
nested = event.get("error")
if isinstance(nested, Mapping):
value = nested.get("code", nested.get("type", value))
normalized = str(value).lower() if isinstance(value, (str, int)) else ""
if "auth" in normalized or "token" in normalized or normalized in {"401", "403"}:
code = "meta_auth"
elif "rate" in normalized or normalized in {"429", "1013"}:
code = "meta_rate_limit"
else:
code = "meta_error"
return ProviderError(code, "Meta transcription service reported an error")
class MetaBackend:
sample_rate_hz = META_SAMPLE_RATE_HZ
descriptor = BackendDescriptor("meta", META_MODEL)
def __init__(
self,
config: MetaConfig,
*,
transport_factory: TransportFactory = _default_transport,
timeout: float = META_EVENT_TIMEOUT,
max_queue_bytes: int = META_MAX_BACKLOG_BYTES,
max_queue_chunks: int = 64,
) -> None:
self.config = config
self._transport_factory = transport_factory
self._timeout = timeout
if max_queue_bytes > META_MAX_BACKLOG_BYTES:
raise ValueError("Meta backlog must not exceed five seconds (240000 bytes)")
self._max_queue_bytes = max_queue_bytes
self._max_queue_chunks = max_queue_chunks
self._prepare_lock = threading.Lock()
self._preparing_transport: Any | None = None
self._prepare_token = 0
self._reserved_prepare_token: int | None = None
def begin_prepare(self) -> None:
with self._prepare_lock:
self._prepare_token += 1
self._reserved_prepare_token = self._prepare_token
def cancel_prepare(self) -> None:
with self._prepare_lock:
self._prepare_token += 1
transport = self._preparing_transport
self._preparing_transport = None
if transport is not None:
threading.Thread(
target=_close_transport,
args=(transport,),
daemon=True,
name="dictate-meta-prepare-close",
).start()
def start_turn(self) -> MetaTurn:
transport: Any | None = None
with self._prepare_lock:
if self._reserved_prepare_token is None:
self._prepare_token += 1
prepare_token = self._prepare_token
else:
prepare_token = self._reserved_prepare_token
self._reserved_prepare_token = None
cancelled_before_start = prepare_token != self._prepare_token
if cancelled_before_start:
raise ProviderError("cancelled", "Meta preparation was cancelled")
try:
key = read_credential(self.config.api_key_file)
keywords = provider_keywords(self.config.terms_path)
transport = self._connect()
with self._prepare_lock:
if prepare_token != self._prepare_token:
cancelled = True
else:
cancelled = False
self._preparing_transport = transport
if cancelled:
threading.Thread(
target=_close_transport,
args=(transport,),
daemon=True,
name="dictate-meta-cancelled-connect-close",
).start()
raise ProviderError("cancelled", "Meta preparation was cancelled")
handshake: dict[str, Any] = {
"authorization": {"accessToken": f"Bearer {key}"},
"audioEncoding": "PCM_24KHZ",
"model": self.config.model,
"mode": "PUSH_TO_TALK",
"partialMode": "CUMULATIVE",
"emitAudioProgress": False,
}
if keywords:
handshake["keywords"] = keywords
language_bias = getattr(self.config, "language_bias", ())
if language_bias:
handshake["languageBias"] = list(language_bias)
_send_json(transport, handshake)
deadline = time.monotonic() + self._timeout
while time.monotonic() < deadline:
event = _receive_event(transport, timeout=max(0.01, deadline - time.monotonic()))
if event is None:
raise ProviderError("meta_premature_close", "Meta closed before session acknowledgement")
if not isinstance(event, dict):
raise ProviderError("meta_malformed_event", "Meta session acknowledgement is invalid")
if event.get("type") == "error" or "error" in event:
raise _meta_event_error(event)
if set(event) != {"sessionId"} or not isinstance(event.get("sessionId"), str) or not event["sessionId"]:
raise ProviderError("meta_malformed_event", "Meta session acknowledgement is invalid")
if len(event["sessionId"].encode("utf-8")) > 128:
raise ProviderError("meta_malformed_event", "Meta session acknowledgement is too large")
if isinstance(event.get("sessionId"), str) and event["sessionId"]:
with self._prepare_lock:
if prepare_token != self._prepare_token:
cancelled = True
else:
cancelled = False
self._preparing_transport = None
if cancelled:
raise ProviderError("cancelled", "Meta preparation was cancelled")
return MetaTurn(
transport,
timeout=self._timeout,
max_queue_bytes=self._max_queue_bytes,
max_queue_chunks=self._max_queue_chunks,
)
raise ProviderError("meta_timeout", "timed out waiting for Meta session acknowledgement")
except CredentialError as error:
raise ProviderError("meta_credentials", "Meta credential is unavailable") from error
except ProviderError:
with self._prepare_lock:
if prepare_token == self._prepare_token and self._preparing_transport is transport:
self._preparing_transport = None
if transport is not None:
threading.Thread(
target=_close_transport,
args=(transport,),
daemon=True,
name="dictate-meta-prepare-error-close",
).start()
raise
except Exception as error:
with self._prepare_lock:
if prepare_token == self._prepare_token and self._preparing_transport is transport:
self._preparing_transport = None
if transport is not None:
threading.Thread(
target=_close_transport,
args=(transport,),
daemon=True,
name="dictate-meta-prepare-failure-close",
).start()
raise ProviderError("meta_connect", "could not prepare Meta transcription session") from error
def _connect(self) -> Any:
try:
return self._transport_factory(META_ENDPOINT)
except TypeError:
# Small test transports often accept no keyword arguments.
return self._transport_factory(META_ENDPOINT, headers={})
except ProviderError:
raise
except Exception as error:
response = getattr(error, "response", None)
status = getattr(response, "status_code", getattr(error, "status_code", getattr(error, "status", None)))
if status in {401, 403}:
raise ProviderError("meta_auth", "Meta authentication failed") from error
if status == 429:
raise ProviderError("meta_rate_limit", "Meta transcription rate limit reached") from error
raise ProviderError("meta_connect", "could not connect to Meta transcription service") from error
__all__ = ["META_ENDPOINT", "META_URL", "META_MODEL", "META_MAX_BACKLOG_BYTES", "META_HANDSHAKE_TIMEOUT", "META_EVENT_TIMEOUT", "META_SOCKET_IO_TIMEOUT", "MetaBackend", "MetaTurn"]
@@ -0,0 +1,461 @@
"""OpenAI GA Realtime transcription adapter."""
from __future__ import annotations
import base64
import json
import logging
from queue import Empty
import threading
import time
from typing import Any, Callable, Mapping
from ..backend import (
BackendDescriptor,
MAX_PROVIDER_EVENT_BYTES,
ProviderError,
QueuedCloudTurn,
RECEIVE_TIMEOUT,
bounded_provider_text,
provider_event,
)
from ..config import OPENAI_DELAYS, CredentialError, OpenAIConfig, provider_keywords, read_credential
OPENAI_ENDPOINT = "wss://api.openai.com/v1/realtime?intent=transcription"
OPENAI_SAMPLE_RATE_HZ = 24_000
OPENAI_DEFAULT_MODEL = "gpt-live-transcribe"
OPENAI_URL = OPENAI_ENDPOINT
# Keep serialized append events comfortably below the provider limit and our
# own provider-event bound. Recorder chunks are 4.8 KiB at 24 kHz.
MAX_APPEND_BYTES = 32 * 1024
OPENAI_SOCKET_IO_TIMEOUT = 10.0
TransportFactory = Callable[..., Any]
def _default_transport(endpoint: str, *, headers: Mapping[str, str] | None = None, **_kwargs: Any) -> Any:
try:
from websockets.sync.client import connect
except ImportError as error:
raise ProviderError("openai_dependency", "OpenAI backend websocket dependency is unavailable") from error
try:
logger = logging.getLogger("pi_dictate_service.provider.openai")
logger.handlers = [logging.NullHandler()]
logger.propagate = False
transport = connect(
endpoint,
additional_headers=dict(headers or {}),
open_timeout=15,
close_timeout=5,
max_size=MAX_PROVIDER_EVENT_BYTES,
proxy=None,
logger=logger,
)
transport.socket.settimeout(OPENAI_SOCKET_IO_TIMEOUT)
return transport
except Exception as error:
response = getattr(error, "response", None)
status = getattr(response, "status_code", getattr(error, "status_code", getattr(error, "status", None)))
if status in {401, 403}:
raise ProviderError("openai_auth", "OpenAI authentication failed") from error
if status == 429:
raise ProviderError("openai_rate_limit", "OpenAI transcription rate limit reached") from error
raise ProviderError("openai_connect", "could not connect to OpenAI transcription service") from error
def _send_json(transport: Any, event: Mapping[str, Any]) -> None:
try:
payload = json.dumps(dict(event), separators=(",", ":"), ensure_ascii=False)
if len(payload.encode("utf-8")) > MAX_PROVIDER_EVENT_BYTES:
raise ProviderError("openai_event_too_large", "OpenAI transcription event is too large")
transport.send(payload)
except ProviderError:
raise
except Exception as error:
raise ProviderError("openai_send", "could not send OpenAI transcription event") from error
def _receive_event(
transport: Any, *, timeout: float, tolerate_timeout: bool = False
) -> dict[str, Any] | None | object:
try:
try:
raw = transport.recv(timeout=timeout)
except TypeError:
raw = transport.recv()
except (TimeoutError, Empty) as error:
if tolerate_timeout:
return RECEIVE_TIMEOUT
raise ProviderError("openai_timeout", "timed out waiting for OpenAI transcription service") from error
except Exception as error:
code = _close_code(error)
if code == 1000:
return None
if code in {1008, 1011, 1013}:
raise ProviderError(f"openai_close_{code}", "OpenAI transcription service closed the session") from error
raise ProviderError("openai_premature_close", "OpenAI transcription service closed unexpectedly") from error
if raw is None:
return None
if isinstance(raw, dict):
try:
return provider_event(raw)
except ProviderError as error:
raise ProviderError("openai_malformed_event", "OpenAI sent an invalid event") from error
if isinstance(raw, bytes):
if len(raw) > MAX_PROVIDER_EVENT_BYTES:
raise ProviderError("openai_malformed_event", "OpenAI sent an oversized event")
try:
raw = raw.decode("utf-8")
except UnicodeDecodeError as error:
raise ProviderError("openai_malformed_event", "OpenAI sent an invalid event") from error
if not isinstance(raw, str):
raise ProviderError("openai_malformed_event", "OpenAI sent an invalid event")
try:
if len(raw.encode("utf-8")) > MAX_PROVIDER_EVENT_BYTES:
raise ProviderError("openai_malformed_event", "OpenAI sent an oversized event")
except UnicodeEncodeError as error:
raise ProviderError("openai_malformed_event", "OpenAI sent an invalid event") from error
try:
value = json.loads(raw)
except (UnicodeDecodeError, ValueError, RecursionError) as error:
raise ProviderError("openai_malformed_event", "OpenAI sent malformed event JSON") from error
try:
return provider_event(value)
except ProviderError as error:
raise ProviderError("openai_malformed_event", "OpenAI sent an invalid event") from error
def _close_transport(transport: Any) -> None:
try:
transport.close()
except Exception:
pass
def _close_code(error: BaseException) -> int | None:
code = getattr(error, "code", None)
if isinstance(code, int):
return code
received = getattr(error, "rcvd", None)
code = getattr(received, "code", None)
return code if isinstance(code, int) else None
class OpenAITurn(QueuedCloudTurn):
sample_rate_hz = OPENAI_SAMPLE_RATE_HZ
retain_pcm = False
def __init__(self, transport: Any, *, timeout: float = 15.0, **kwargs: Any) -> None:
self._committed_item_id: str | None = None
super().__init__(transport, timeout=timeout, **kwargs)
def _receive_provider_event(self, *, timeout: float) -> dict[str, Any] | None | object:
event = _receive_event(self._transport, timeout=timeout, tolerate_timeout=True)
if event is RECEIVE_TIMEOUT or event is None:
return event
_validate_openai_event(event)
if event.get("type") == "error" or event.get("type") == "conversation.item.input_audio_transcription.failed":
raise _openai_event_error(event)
if event.get("type") in {
"input_audio_buffer.committed",
"conversation.item.input_audio_transcription.completed",
}:
return event
# Deltas and unrelated conversation events can be frequent. Validate
# and discard them during capture rather than queueing session history.
return RECEIVE_TIMEOUT
@property
def _provider_prefix(self) -> str:
return "openai"
def _send_audio(self, chunk: bytes) -> None:
if len(chunk) > MAX_APPEND_BYTES:
raise ProviderError("openai_audio_chunk", "OpenAI audio chunk is too large")
event = {
"type": "input_audio_buffer.append",
"audio": base64.b64encode(chunk).decode("ascii"),
}
_send_json(self._transport, event)
def _send_end(self) -> None:
# Commit starts transcription and deliberately does not create an
# assistant response. Never send response.create here.
_send_json(self._transport, {"type": "input_audio_buffer.commit"})
def _finish_provider(self) -> str:
deadline = time.monotonic() + self._timeout
transcript: str | None = None
while time.monotonic() < deadline:
event = self._next_provider_event(max(0.01, deadline - time.monotonic()))
if event is None:
if transcript is None:
raise ProviderError("openai_premature_close", "OpenAI closed without a final transcript")
return transcript
_validate_openai_event(event)
event_type = event["type"]
if event_type == "error" or event_type == "conversation.item.input_audio_transcription.failed":
raise _openai_event_error(event)
if event_type == "input_audio_buffer.committed":
item_id = event.get("item_id")
if not isinstance(item_id, str) or not item_id:
raise ProviderError("openai_malformed_event", "OpenAI commit acknowledgement is invalid")
if len(item_id.encode("utf-8")) > 128:
raise ProviderError("openai_malformed_event", "OpenAI commit acknowledgement is too large")
if self._committed_item_id is not None and self._committed_item_id != item_id:
raise ProviderError("openai_malformed_event", "OpenAI returned conflicting commit acknowledgements")
self._committed_item_id = item_id
continue
if event_type == "conversation.item.input_audio_transcription.completed":
item_id = event.get("item_id")
if not isinstance(item_id, str) or not item_id:
raise ProviderError("openai_malformed_event", "OpenAI completion correlation is invalid")
if "transcript" not in event:
raise ProviderError("openai_malformed_event", "OpenAI completion transcript is missing")
try:
text_value = bounded_provider_text(event["transcript"])
except ProviderError as error:
raise ProviderError("openai_malformed_event", "OpenAI sent an invalid transcript") from error
if self._committed_item_id == item_id:
# OpenAI keeps the realtime connection reusable after a
# completed transcription. The turn is done as soon as
# its correlated final arrives; finish() closes it.
return text_value
# Unmatched completions are stale events. They are validated
# but never allowed to authorize an active generation.
continue
# Deltas and unrelated conversation events are intentionally ignored.
raise ProviderError("openai_timeout", "timed out waiting for OpenAI final transcript")
def _validate_openai_event(event: Mapping[str, Any]) -> None:
event_type = event.get("type")
if not isinstance(event_type, str) or not event_type or len(event_type.encode("utf-8")) > 128:
raise ProviderError("openai_malformed_event", "OpenAI event type is invalid")
if event_type == "error" or event_type == "conversation.item.input_audio_transcription.failed":
error = event.get("error")
if not isinstance(error, Mapping):
raise ProviderError("openai_malformed_event", "OpenAI error event is invalid")
code = error.get("code", error.get("type"))
if not isinstance(code, (str, int)):
raise ProviderError("openai_malformed_event", "OpenAI error event has an invalid code")
if isinstance(code, str) and len(code.encode("utf-8")) > 128:
raise ProviderError("openai_malformed_event", "OpenAI error event code is too large")
def _openai_event_error(event: Mapping[str, Any]) -> ProviderError:
error = event.get("error")
if not isinstance(error, Mapping):
raise ProviderError("openai_malformed_event", "OpenAI error event is invalid")
code_value: Any = error.get("code", error.get("type", event.get("code")))
normalized = str(code_value).lower() if isinstance(code_value, (str, int)) else ""
if "auth" in normalized or "unauthor" in normalized or normalized in {"401", "403"}:
code = "openai_auth"
elif "rate" in normalized or normalized in {"429"}:
code = "openai_rate_limit"
else:
code = "openai_error"
return ProviderError(code, "OpenAI transcription service reported an error")
class OpenAIBackend:
sample_rate_hz = OPENAI_SAMPLE_RATE_HZ
def __init__(
self,
config: OpenAIConfig,
*,
transport_factory: TransportFactory = _default_transport,
timeout: float = 15.0,
max_queue_bytes: int = 240_000,
max_queue_chunks: int = 64,
) -> None:
if config.delay is not None and config.delay not in OPENAI_DELAYS:
raise ValueError("unsupported OpenAI delay")
self.config = config
self.descriptor = BackendDescriptor("openai", config.model)
self._transport_factory = transport_factory
self._timeout = timeout
if max_queue_bytes > 240_000:
raise ValueError("OpenAI PCM backlog must not exceed five seconds (240000 bytes)")
self._max_queue_bytes = max_queue_bytes
self._max_queue_chunks = max_queue_chunks
self._prepare_lock = threading.Lock()
self._preparing_transport: Any | None = None
self._prepare_token = 0
self._reserved_prepare_token: int | None = None
def begin_prepare(self) -> None:
with self._prepare_lock:
self._prepare_token += 1
self._reserved_prepare_token = self._prepare_token
def cancel_prepare(self) -> None:
with self._prepare_lock:
self._prepare_token += 1
transport = self._preparing_transport
self._preparing_transport = None
if transport is not None:
threading.Thread(
target=_close_transport,
args=(transport,),
daemon=True,
name="dictate-openai-prepare-close",
).start()
def start_turn(self) -> OpenAITurn:
transport: Any | None = None
with self._prepare_lock:
if self._reserved_prepare_token is None:
self._prepare_token += 1
prepare_token = self._prepare_token
else:
prepare_token = self._reserved_prepare_token
self._reserved_prepare_token = None
cancelled_before_start = prepare_token != self._prepare_token
if cancelled_before_start:
raise ProviderError("cancelled", "OpenAI preparation was cancelled")
try:
key = read_credential(self.config.api_key_file)
keywords = provider_keywords(self.config.terms_path)
transport = self._connect(key)
with self._prepare_lock:
if prepare_token != self._prepare_token:
cancelled = True
else:
cancelled = False
self._preparing_transport = transport
if cancelled:
threading.Thread(
target=_close_transport,
args=(transport,),
daemon=True,
name="dictate-openai-cancelled-connect-close",
).start()
raise ProviderError("cancelled", "OpenAI preparation was cancelled")
self._wait_for_session_created(transport)
_send_json(transport, self._session_update(keywords))
self._wait_for_session_updated(transport)
with self._prepare_lock:
if prepare_token != self._prepare_token:
cancelled = True
else:
cancelled = False
self._preparing_transport = None
if cancelled:
raise ProviderError("cancelled", "OpenAI preparation was cancelled")
return OpenAITurn(
transport,
timeout=self._timeout,
max_queue_bytes=self._max_queue_bytes,
max_queue_chunks=self._max_queue_chunks,
)
except CredentialError as error:
raise ProviderError("openai_credentials", "OpenAI credential is unavailable") from error
except ProviderError:
with self._prepare_lock:
if prepare_token == self._prepare_token and self._preparing_transport is transport:
self._preparing_transport = None
if transport is not None:
threading.Thread(
target=_close_transport,
args=(transport,),
daemon=True,
name="dictate-openai-prepare-error-close",
).start()
raise
except Exception as error:
with self._prepare_lock:
if prepare_token == self._prepare_token and self._preparing_transport is transport:
self._preparing_transport = None
if transport is not None:
threading.Thread(
target=_close_transport,
args=(transport,),
daemon=True,
name="dictate-openai-prepare-failure-close",
).start()
raise ProviderError("openai_connect", "could not prepare OpenAI transcription session") from error
def _connect(self, key: str) -> Any:
try:
return self._transport_factory(
OPENAI_ENDPOINT,
headers={"Authorization": f"Bearer {key}"},
)
except TypeError as error:
# Test transports may accept positional headers, but never fall
# back to an unauthenticated connection.
try:
return self._transport_factory(OPENAI_ENDPOINT, {"Authorization": f"Bearer {key}"})
except TypeError as positional_error:
raise ProviderError("openai_connect", "OpenAI transport does not accept authenticated headers") from positional_error
except ProviderError:
raise
except Exception as error:
response = getattr(error, "response", None)
status = getattr(response, "status_code", getattr(error, "status_code", getattr(error, "status", None)))
if status in {401, 403}:
raise ProviderError("openai_auth", "OpenAI authentication failed") from error
if status == 429:
raise ProviderError("openai_rate_limit", "OpenAI transcription rate limit reached") from error
raise ProviderError("openai_connect", "could not connect to OpenAI transcription service") from error
def _wait_for_session_created(self, transport: Any) -> None:
deadline = time.monotonic() + self._timeout
while time.monotonic() < deadline:
event = _receive_event(transport, timeout=max(0.01, deadline - time.monotonic()))
if event is None:
raise ProviderError("openai_premature_close", "OpenAI closed before session creation")
_validate_openai_event(event)
event_type = event["type"]
if event_type == "error" or "error" in event:
raise _openai_event_error(event)
if event_type == "session.created":
return
raise ProviderError("openai_timeout", "timed out waiting for OpenAI session creation")
def _wait_for_session_updated(self, transport: Any) -> None:
deadline = time.monotonic() + self._timeout
while time.monotonic() < deadline:
event = _receive_event(transport, timeout=max(0.01, deadline - time.monotonic()))
if event is None:
raise ProviderError("openai_premature_close", "OpenAI closed before session update")
_validate_openai_event(event)
event_type = event["type"]
if event_type == "error" or "error" in event:
raise _openai_event_error(event)
if event_type == "session.updated":
return
raise ProviderError("openai_timeout", "timed out waiting for OpenAI session update")
def _session_update(self, keywords: list[str]) -> dict[str, Any]:
transcription: dict[str, Any] = {"model": self.config.model}
prompt = getattr(self.config, "prompt", None)
languages = getattr(self.config, "languages", ())
if prompt:
transcription["prompt"] = prompt
if keywords:
transcription["keywords"] = keywords
if languages:
transcription["languages"] = list(languages)
if self.config.delay:
transcription["delay"] = self.config.delay
return {
"type": "session.update",
"session": {
"type": "transcription",
"audio": {
"input": {
"format": {"type": "audio/pcm", "rate": OPENAI_SAMPLE_RATE_HZ},
"transcription": transcription,
"turn_detection": None,
}
},
},
}
__all__ = [
"OPENAI_DEFAULT_MODEL", "OPENAI_ENDPOINT", "OPENAI_URL", "OPENAI_SAMPLE_RATE_HZ", "OPENAI_SOCKET_IO_TIMEOUT", "OpenAIBackend", "OpenAITurn",
]
@@ -0,0 +1,388 @@
"""Strict host-side configuration and credential handling for Dictate.
This module intentionally has no dependency on the provider SDKs. It is loaded by
the socket service, outside Pi's sandbox, and never serializes credentials onto
the protocol.
"""
from __future__ import annotations
from dataclasses import dataclass
import json
import math
import os
from pathlib import Path
import re
import stat
from typing import Any, Mapping
import unicodedata
from .model import (
MAX_CODING_TERMS,
MAX_TERMS_FILE_BYTES,
parse_coding_terms,
)
CONFIG_VERSION = 1
DEFAULT_IDLE_SECONDS = 15 * 60
MIN_IDLE_SECONDS = 60
MAX_IDLE_SECONDS = 24 * 60 * 60
MAX_CONFIG_BYTES = 64 * 1024
MAX_CREDENTIAL_BYTES = 4096
MAX_PROMPT_BYTES = 4096
MAX_LANGUAGE_BYTES = 32
MAX_LANGUAGES = 16
MAX_KEYWORDS = MAX_CODING_TERMS
MAX_PROVIDER_KEYWORDS_BYTES = 16 * 1024
MAX_MODEL_BYTES = 128
_CONFIG_FIELDS = frozenset({"version", "backend", "idleSeconds", "local", "meta", "openai"})
_LOCAL_FIELDS = frozenset({"adapter", "modelPath", "modelSha256", "termsPath", "boostAlpha"})
_META_FIELDS = frozenset({"model", "apiKeyFile", "termsPath", "languageBias"})
# ``delay`` is intentionally not accepted until the conflicting OpenAI
# documentation is resolved by a credentialed live spike. It must not be
# silently accepted and then omitted from the wire request.
_OPENAI_FIELDS = frozenset({"model", "apiKeyFile", "prompt", "termsPath", "languages", "delay"})
OPENAI_DELAYS = frozenset({"minimal", "low", "medium", "high", "xhigh"})
_HEX_SHA256 = re.compile(r"^[0-9a-fA-F]{64}$")
class ConfigError(ValueError):
"""A safe, user-facing configuration failure (never includes a key)."""
class CredentialError(ConfigError):
pass
@dataclass(frozen=True)
class LocalConfig:
adapter: str
model_path: Path
model_sha256: str
terms_path: Path
boost_alpha: float
@dataclass(frozen=True)
class MetaConfig:
model: str
api_key_file: Path
terms_path: Path
language_bias: tuple[str, ...] = ()
@dataclass(frozen=True)
class OpenAIConfig:
model: str
api_key_file: Path
terms_path: Path
delay: str | None = None
prompt: str | None = None
languages: tuple[str, ...] = ()
@dataclass(frozen=True)
class DictateConfig:
backend: str
idle_seconds: float
local: LocalConfig | None = None
meta: MetaConfig | None = None
openai: OpenAIConfig | None = None
version: int = CONFIG_VERSION
@property
def selected(self) -> LocalConfig | MetaConfig | OpenAIConfig:
selected = getattr(self, self.backend)
if selected is None:
raise ConfigError("selected backend configuration is missing")
return selected
def _string(value: Any, field: str, *, nonempty: bool = True) -> str:
if not isinstance(value, str) or (nonempty and not value):
raise ConfigError(f"{field} must be a non-empty string")
try:
value.encode("utf-8")
except UnicodeEncodeError as error:
raise ConfigError(f"{field} must contain valid UTF-8") from error
if "\x00" in value or "\r" in value or "\n" in value:
raise ConfigError(f"{field} contains unsupported control characters")
return value
def _path(value: Any, field: str) -> Path:
path = Path(_string(value, field)).expanduser()
if not path.is_absolute():
raise ConfigError(f"{field} must be an absolute path")
return path
def _section(value: Any, field: str, fields: frozenset[str]) -> Mapping[str, Any]:
if not isinstance(value, Mapping):
raise ConfigError(f"{field} configuration is required")
unknown = set(value) - fields
if unknown:
raise ConfigError(f"unknown {field} configuration key")
return value
def _optional_string(section: Mapping[str, Any], field: str, default: str | None = None) -> str | None:
value = section.get(field, default)
if value is None:
return None
return _string(value, field)
def parse_config(value: Any, *, config_path: str | os.PathLike[str] | None = None) -> DictateConfig:
"""Parse a strict versioned configuration mapping without reading credentials."""
if not isinstance(value, Mapping):
raise ConfigError("configuration root must be an object")
unknown = set(value) - _CONFIG_FIELDS
if unknown:
raise ConfigError("unknown configuration key")
version = value.get("version")
if version != CONFIG_VERSION or isinstance(version, bool):
raise ConfigError(f"unsupported configuration version (expected {CONFIG_VERSION})")
backend = value.get("backend")
if not isinstance(backend, str) or backend not in {"local", "meta", "openai"}:
raise ConfigError("backend must be local, meta, or openai")
idle = value.get("idleSeconds", DEFAULT_IDLE_SECONDS)
if isinstance(idle, bool) or not isinstance(idle, int):
raise ConfigError("idleSeconds must be an integer")
if not MIN_IDLE_SECONDS <= float(idle) <= MAX_IDLE_SECONDS:
raise ConfigError("idleSeconds is out of bounds")
local: LocalConfig | None = None
meta: MetaConfig | None = None
openai: OpenAIConfig | None = None
if backend == "local":
section = _section(value.get("local"), "local", _LOCAL_FIELDS)
adapter = _string(section.get("adapter"), "local.adapter")
if adapter != "nemo-parakeet":
raise ConfigError("unsupported local adapter")
model_path = _path(section.get("modelPath"), "local.modelPath")
if model_path.suffix != ".nemo":
raise ConfigError("local.modelPath must name a .nemo archive")
model_sha = _string(section.get("modelSha256"), "local.modelSha256")
if not _HEX_SHA256.fullmatch(model_sha):
raise ConfigError("local.modelSha256 must be a SHA-256 digest")
terms_path = _path(section.get("termsPath"), "local.termsPath")
boost = section.get("boostAlpha")
if isinstance(boost, bool) or not isinstance(boost, (int, float)) or not math.isfinite(float(boost)):
raise ConfigError("local.boostAlpha must be a finite number")
if not 0 < float(boost) <= 10:
raise ConfigError("local.boostAlpha is out of bounds")
local = LocalConfig(adapter, model_path, model_sha.lower(), terms_path, float(boost))
elif backend == "meta":
section = _section(value.get("meta"), "meta", _META_FIELDS)
model = _string(section.get("model"), "meta.model")
if model != "muse-voice-transcribe-1.0":
raise ConfigError("unsupported Meta model")
api_key_file = _path(section.get("apiKeyFile"), "meta.apiKeyFile")
terms_path = _path(section.get("termsPath"), "meta.termsPath")
language_bias_value = section.get("languageBias", ())
if not isinstance(language_bias_value, (list, tuple)) or len(language_bias_value) > MAX_LANGUAGES:
raise ConfigError("meta.languageBias must be a bounded array")
language_bias: list[str] = []
for language in language_bias_value:
language_value = _string(language, "meta.languageBias entry")
if len(language_value.encode("utf-8")) > MAX_LANGUAGE_BYTES:
raise ConfigError("Meta language bias is too long")
language_bias.append(language_value)
meta = MetaConfig(model, api_key_file, terms_path, tuple(language_bias))
else:
section = _section(value.get("openai"), "openai", _OPENAI_FIELDS)
model = _string(section.get("model"), "openai.model")
if len(model.encode("utf-8")) > MAX_MODEL_BYTES:
raise ConfigError("openai.model is too long")
if model == "whisper-1":
raise ConfigError("whisper-1 is not supported by the Realtime transcription backend")
delay = _optional_string(section, "delay")
if delay is not None and delay not in OPENAI_DELAYS:
raise ConfigError("openai.delay is unsupported")
prompt = _optional_string(section, "prompt")
if prompt is not None and len(prompt.encode("utf-8")) > MAX_PROMPT_BYTES:
raise ConfigError("openai.prompt is too large")
languages_value = section.get("languages", ())
if not isinstance(languages_value, (list, tuple)) or ("languages" in section and not languages_value):
raise ConfigError("openai.languages must be a non-empty array when provided")
if len(languages_value) > MAX_LANGUAGES:
raise ConfigError("openai.languages has too many entries")
languages: list[str] = []
for language in languages_value:
language_value = _string(language, "openai.languages entry")
if len(language_value.encode("utf-8")) > MAX_LANGUAGE_BYTES:
raise ConfigError("openai language is too long")
languages.append(language_value)
openai = OpenAIConfig(
model=model,
api_key_file=_path(section.get("apiKeyFile"), "openai.apiKeyFile"),
terms_path=_path(section.get("termsPath"), "openai.termsPath"),
delay=delay,
prompt=prompt,
languages=tuple(languages),
)
config = DictateConfig(backend, float(idle), local, meta, openai)
_validate_managed_credential_path(config)
return config
def _validate_managed_credential_path(config: DictateConfig) -> None:
selected = config.selected
if isinstance(selected, LocalConfig):
return
expected = Path("~/.config/pi-dictate/credentials").expanduser()
# A test or service may use a different HOME. The fixed directory is
# therefore checked after expansion, not by string matching the JSON.
path = selected.api_key_file
if path.parent != expected or path.name != f"{config.backend}-api-key":
raise ConfigError("credential path must use the managed credential directory")
def load_config(path: str | os.PathLike[str] | None = None) -> DictateConfig:
"""Read and parse the service configuration, failing closed on file issues."""
config_path = Path(path or os.environ.get("PI_DICTATE_CONFIG_PATH", "~/.config/pi-dictate/config.json")).expanduser().absolute()
try:
info = config_path.lstat()
if stat.S_ISLNK(info.st_mode) or not stat.S_ISREG(info.st_mode):
raise ConfigError("configuration is not a regular file")
if info.st_uid != os.getuid() or info.st_mode & 0o077:
raise ConfigError("configuration ownership or permissions are insecure")
if info.st_size > 64 * 1024:
raise ConfigError("configuration is too large")
raw = _read_bounded_descriptor(config_path, MAX_CONFIG_BYTES, require_private=True)
value = json.loads(raw.decode("utf-8"))
except ConfigError:
raise
except (OSError, UnicodeError, json.JSONDecodeError, ValueError) as error:
raise ConfigError("could not read valid configuration") from error
return parse_config(value, config_path=config_path)
def _read_bounded_descriptor(
path: Path,
limit: int,
*,
require_private: bool = False,
require_mode: int | None = None,
) -> bytes:
"""Read one regular file through an O_NOFOLLOW descriptor.
The descriptor is checked before reading, so a path replacement after the
check cannot redirect the data consumed by the service.
"""
descriptor: int | None = None
try:
flags = os.O_RDONLY | getattr(os, "O_CLOEXEC", 0) | getattr(os, "O_NOFOLLOW", 0)
descriptor = os.open(path, flags)
info = os.fstat(descriptor)
if not stat.S_ISREG(info.st_mode):
raise ConfigError("file is not a regular file")
if info.st_uid != os.getuid():
raise ConfigError("file ownership is insecure")
if require_private and info.st_mode & 0o077:
raise ConfigError("file permissions are insecure")
if require_mode is not None and stat.S_IMODE(info.st_mode) != require_mode:
raise ConfigError("file permissions are insecure")
if info.st_size > limit:
raise ConfigError("file is too large")
chunks: list[bytes] = []
remaining = limit + 1
while remaining:
chunk = os.read(descriptor, min(4096, remaining))
if not chunk:
break
chunks.append(chunk)
remaining -= len(chunk)
raw = b"".join(chunks)
if len(raw) > limit:
raise ConfigError("file is too large")
return raw
finally:
if descriptor is not None:
try:
os.close(descriptor)
except OSError:
pass
def _reject_symlink_ancestry(path: Path) -> None:
current = path.absolute()
while current != current.parent:
try:
if current.is_symlink():
raise CredentialError("credential path ancestry contains a symlink")
except OSError as error:
raise CredentialError("credential path ancestry could not be checked") from error
current = current.parent
def read_credential(path: str | os.PathLike[str]) -> str:
"""Read one selected, mode-0600 regular credential without logging its value."""
credential_path = Path(path).expanduser()
_reject_symlink_ancestry(credential_path.parent)
try:
raw = _read_bounded_descriptor(
credential_path, MAX_CREDENTIAL_BYTES, require_mode=0o600
)
value = raw.decode("utf-8")
except CredentialError:
raise
except (OSError, UnicodeError, ConfigError) as error:
if isinstance(error, ConfigError):
raise CredentialError(str(error)) from error
raise CredentialError("could not read credential") from error
if value.endswith("\n"):
value = value[:-1]
if not value or "\r" in value or "\n" in value or "\x00" in value:
raise CredentialError("credential must be one non-empty line")
return value
def provider_keywords(path: str | os.PathLike[str]) -> list[str]:
"""Load terms once through a bounded no-follow descriptor."""
try:
raw = _read_bounded_descriptor(Path(path).expanduser(), MAX_TERMS_FILE_BYTES)
except (OSError, ConfigError) as error:
raise ConfigError("coding dictionary could not be read") from error
# LF separates dictionary entries; CR is never accepted because it is
# normalized by splitlines() and would otherwise silently alter a keyword.
if b"\r" in raw:
raise ConfigError("coding terms contain unsupported provider keyword characters")
try:
terms = parse_coding_terms(raw)
except (OSError, UnicodeError, ValueError) as error:
raise ConfigError("coding dictionary is invalid") from error
if len(terms) > MAX_KEYWORDS:
raise ConfigError("coding dictionary has too many provider keywords")
if sum(len(term.encode("utf-8")) for term in terms) > MAX_PROVIDER_KEYWORDS_BYTES:
raise ConfigError("coding dictionary provider keywords are too large")
for term in terms:
if (
"<" in term
or ">" in term
or any(unicodedata.category(character) in {"Cc", "Cf"} for character in term)
):
raise ConfigError("coding terms contain unsupported provider keyword characters")
return terms
# Short aliases make the host-facing API convenient without exposing any raw
# configuration dictionary to the service core.
Config = DictateConfig
read_api_key = read_credential
__all__ = [
"CONFIG_VERSION", "Config", "ConfigError", "CredentialError", "DictateConfig",
"MAX_CONFIG_BYTES",
"LocalConfig", "MetaConfig", "OpenAIConfig", "parse_config", "load_config",
"read_credential", "read_api_key", "provider_keywords", "MAX_PROVIDER_KEYWORDS_BYTES",
"DEFAULT_IDLE_SECONDS",
]
@@ -0,0 +1,69 @@
"""Opt-in live cloud contract smoke test using a short raw PCM fixture."""
from __future__ import annotations
import argparse
import os
from pathlib import Path
import stat
from .backend import backend_from_config
from .config import load_config
MAX_SAMPLE_BYTES = 240_000 # five seconds of 24 kHz mono PCM16
CHUNK_BYTES = 4_800
def _read_sample(path: Path) -> bytes:
descriptor = os.open(
path,
os.O_RDONLY | getattr(os, "O_CLOEXEC", 0) | getattr(os, "O_NOFOLLOW", 0),
)
try:
info = os.fstat(descriptor)
if not stat.S_ISREG(info.st_mode):
raise ValueError("sample must be a regular file")
if not 0 < info.st_size <= MAX_SAMPLE_BYTES or info.st_size % 2:
raise ValueError("sample must contain at most five seconds of even-length PCM16")
sample = bytearray()
while len(sample) <= MAX_SAMPLE_BYTES:
chunk = os.read(descriptor, min(64 * 1024, MAX_SAMPLE_BYTES + 1 - len(sample)))
if not chunk:
break
sample.extend(chunk)
if len(sample) != info.st_size:
raise ValueError("sample changed while it was being read")
return bytes(sample)
finally:
os.close(descriptor)
def main() -> None:
parser = argparse.ArgumentParser(
description="Send one non-sensitive 24 kHz mono PCM16 sample to the configured cloud backend."
)
parser.add_argument("sample", type=Path, help="raw 24 kHz mono signed-16 little-endian PCM")
parser.add_argument(
"--config",
type=Path,
default=Path("~/.config/pi-dictate/config.json").expanduser(),
)
arguments = parser.parse_args()
config = load_config(arguments.config)
if config.backend not in {"meta", "openai"}:
parser.error("the configured backend must be meta or openai")
backend = backend_from_config(config)
sample = _read_sample(arguments.sample)
turn = backend.start_turn()
try:
for offset in range(0, len(sample), CHUNK_BYTES):
turn.enqueue_pcm(sample[offset : offset + CHUNK_BYTES])
transcript = turn.finish()
finally:
turn.close()
print(f"{backend.descriptor.backend}/{backend.descriptor.model}: {transcript!r}")
if __name__ == "__main__":
main()
+113 -49
View File
@@ -6,6 +6,7 @@ from copy import deepcopy
import hashlib import hashlib
import os import os
from pathlib import Path from pathlib import Path
import stat
from typing import Protocol, Sequence from typing import Protocol, Sequence
MAX_TERMS_FILE_BYTES = 64 * 1024 MAX_TERMS_FILE_BYTES = 64 * 1024
@@ -19,18 +20,19 @@ class TranscriptionModel(Protocol):
def transcribe(self, samples: Sequence[float]) -> str: ... def transcribe(self, samples: Sequence[float]) -> str: ...
def load_coding_terms(path: str | os.PathLike[str]) -> list[str]: def parse_coding_terms(raw: bytes) -> list[str]:
"""Load a bounded host-owned phrase list, ignoring comments and duplicates.""" """Parse one bounded, already-read host-owned phrase-list descriptor."""
terms_path = Path(path).expanduser() if len(raw) > MAX_TERMS_FILE_BYTES:
if not terms_path.is_file():
raise ValueError("PI_DICTATE_TERMS_PATH must name an existing local dictionary")
if terms_path.stat().st_size > MAX_TERMS_FILE_BYTES:
raise ValueError(f"coding dictionary exceeds {MAX_TERMS_FILE_BYTES} bytes") raise ValueError(f"coding dictionary exceeds {MAX_TERMS_FILE_BYTES} bytes")
try:
text = raw.decode("utf-8")
except UnicodeDecodeError as error:
raise ValueError("coding dictionary is not valid UTF-8") from error
terms: list[str] = [] terms: list[str] = []
seen: set[str] = set() seen: set[str] = set()
for raw_line in terms_path.read_text(encoding="utf-8").splitlines(): for raw_line in text.splitlines():
term = raw_line.strip() term = raw_line.strip()
if not term or term.startswith("#"): if not term or term.startswith("#"):
continue continue
@@ -44,9 +46,47 @@ def load_coding_terms(path: str | os.PathLike[str]) -> list[str]:
return terms return terms
def load_coding_terms(path: str | os.PathLike[str]) -> list[str]:
"""Load a bounded dictionary from a no-follow descriptor, avoiding TOCTOU."""
terms_path = Path(path).expanduser()
descriptor: int | None = None
try:
flags = os.O_RDONLY | getattr(os, "O_CLOEXEC", 0) | getattr(os, "O_NOFOLLOW", 0)
descriptor = os.open(terms_path, flags)
info = os.fstat(descriptor)
if not stat.S_ISREG(info.st_mode):
raise ValueError("PI_DICTATE_TERMS_PATH must name a regular dictionary")
if info.st_size > MAX_TERMS_FILE_BYTES:
raise ValueError(f"coding dictionary exceeds {MAX_TERMS_FILE_BYTES} bytes")
chunks: list[bytes] = []
remaining = MAX_TERMS_FILE_BYTES + 1
while remaining:
chunk = os.read(descriptor, min(4096, remaining))
if not chunk:
break
chunks.append(chunk)
remaining -= len(chunk)
raw = b"".join(chunks)
if len(raw) > MAX_TERMS_FILE_BYTES:
raise ValueError(f"coding dictionary exceeds {MAX_TERMS_FILE_BYTES} bytes")
return parse_coding_terms(raw)
except OSError as error:
raise ValueError("PI_DICTATE_TERMS_PATH must name an existing local dictionary") from error
finally:
if descriptor is not None:
try:
os.close(descriptor)
except OSError:
pass
class NemoParakeetModel: class NemoParakeetModel:
"""Load NeMo only when requested so core tests require neither CUDA nor NeMo.""" """Load NeMo only when requested so core tests require neither CUDA nor NeMo."""
sample_rate_hz = 16_000
max_pcm_bytes = 5 * 60 * sample_rate_hz * 2
def __init__( def __init__(
self, self,
model_path: str | os.PathLike[str], model_path: str | os.PathLike[str],
@@ -56,7 +96,11 @@ class NemoParakeetModel:
expected_sha256: str = PINNED_MODEL_SHA256, expected_sha256: str = PINNED_MODEL_SHA256,
) -> None: ) -> None:
path = Path(model_path).expanduser() path = Path(model_path).expanduser()
if path.suffix != ".nemo" or not path.is_file(): try:
info = path.lstat()
except OSError as error:
raise ValueError("PI_DICTATE_MODEL_PATH must name an existing local .nemo file") from error
if path.suffix != ".nemo" or stat.S_ISLNK(info.st_mode) or not stat.S_ISREG(info.st_mode):
raise ValueError("PI_DICTATE_MODEL_PATH must name an existing local .nemo file") raise ValueError("PI_DICTATE_MODEL_PATH must name an existing local .nemo file")
if boost_alpha <= 0: if boost_alpha <= 0:
raise ValueError("PI_DICTATE_BOOST_ALPHA must be positive") raise ValueError("PI_DICTATE_BOOST_ALPHA must be positive")
@@ -74,48 +118,68 @@ class NemoParakeetModel:
def load(self) -> None: def load(self) -> None:
if self._model is not None: if self._model is not None:
return return
with self._model_path.open("rb") as model_file: descriptor: int | None = None
actual_sha256 = hashlib.file_digest(model_file, "sha256").hexdigest() try:
if actual_sha256 != self._expected_sha256: flags = os.O_RDONLY | getattr(os, "O_CLOEXEC", 0) | getattr(os, "O_NOFOLLOW", 0)
raise ValueError( descriptor = os.open(self._model_path, flags)
"model checksum mismatch: expected the pinned Parakeet revision" info = os.fstat(descriptor)
if not stat.S_ISREG(info.st_mode):
raise ValueError("model archive is not a regular file")
with os.fdopen(descriptor, "rb", closefd=False) as model_file:
actual_sha256 = hashlib.file_digest(model_file, "sha256").hexdigest()
if actual_sha256 != self._expected_sha256:
raise ValueError(
"model checksum mismatch: expected the pinned Parakeet revision"
)
os.lseek(descriptor, 0, os.SEEK_SET)
os.environ["HF_HUB_OFFLINE"] = "1"
os.environ["TRANSFORMERS_OFFLINE"] = "1"
os.environ["HF_DATASETS_OFFLINE"] = "1"
os.environ["NEMO_OFFLINE"] = "1"
import numpy
import torch
from nemo.collections.asr.models import EncDecRNNTBPEModel
from omegaconf import open_dict
# Keep the verified inode open while NeMo restores it. Reopening
# the configured pathname here would allow a replacement between
# checksum verification and model loading.
model = EncDecRNNTBPEModel.restore_from(
restore_path=f"/proc/self/fd/{descriptor}",
map_location=torch.device("cuda"),
) )
os.environ["HF_HUB_OFFLINE"] = "1" model.to("cuda")
os.environ["TRANSFORMERS_OFFLINE"] = "1" model.eval()
os.environ["HF_DATASETS_OFFLINE"] = "1" if self._terms:
os.environ["NEMO_OFFLINE"] = "1" decoding = deepcopy(model.cfg.decoding)
with open_dict(decoding):
import numpy greedy = decoding["greedy"]
import torch greedy["boosting_tree"] = {
from nemo.collections.asr.models import EncDecRNNTBPEModel "key_phrases_list": list(self._terms),
from omegaconf import open_dict "context_score": 1.0,
"depth_scaling": 2.0,
model = EncDecRNNTBPEModel.restore_from( "unk_score": 0.0,
restore_path=str(self._model_path), "final_eos_score": 1.0,
map_location=torch.device("cuda"), "score_per_phrase": 0.0,
) "source_lang": "en",
model.to("cuda") "use_triton": True,
model.eval() "uniform_weights": False,
if self._terms: "use_bpe_dropout": False,
decoding = deepcopy(model.cfg.decoding) }
with open_dict(decoding): greedy["boosting_tree_alpha"] = self._boost_alpha
greedy = decoding["greedy"] model.change_decoding_strategy(decoding, verbose=False)
greedy["boosting_tree"] = { self._numpy = numpy
"key_phrases_list": list(self._terms), self._model = model
"context_score": 1.0, except OSError as error:
"depth_scaling": 2.0, raise ValueError("model archive could not be read") from error
"unk_score": 0.0, finally:
"final_eos_score": 1.0, if descriptor is not None:
"score_per_phrase": 0.0, try:
"source_lang": "en", os.close(descriptor)
"use_triton": True, except OSError:
"uniform_weights": False, pass
"use_bpe_dropout": False,
}
greedy["boosting_tree_alpha"] = self._boost_alpha
model.change_decoding_strategy(decoding, verbose=False)
self._numpy = numpy
self._model = model
def transcribe(self, samples: Sequence[float]) -> str: def transcribe(self, samples: Sequence[float]) -> str:
if self._model is None or self._numpy is None: if self._model is None or self._numpy is None:
@@ -51,9 +51,9 @@ _REQUEST_FIELDS = {
def _bounded_utf8(value: str, limit: int) -> str: def _bounded_utf8(value: str, limit: int) -> str:
# Normalize replacement characters even when the original string contains
# lone surrogates. This keeps error output serializable and bounded.
raw = value.encode("utf-8", errors="replace") raw = value.encode("utf-8", errors="replace")
if len(raw) <= limit:
return value
return raw[:limit].decode("utf-8", errors="ignore") return raw[:limit].decode("utf-8", errors="ignore")
@@ -100,7 +100,11 @@ def parse_request(frame: Mapping[str, Any]) -> Request:
raise ProtocolError("invalid_request", "unknown request type", request_id=request_id) raise ProtocolError("invalid_request", "unknown request type", request_id=request_id)
expected_fields = _REQUEST_FIELDS[request_type] expected_fields = _REQUEST_FIELDS[request_type]
if frozenset(frame) != expected_fields: try:
actual_fields = frozenset(frame)
except TypeError as error:
raise ProtocolError("invalid_request", "request contains invalid field names", request_id=request_id) from error
if actual_fields != expected_fields:
raise ProtocolError( raise ProtocolError(
"invalid_request", "invalid_request",
f"{request_type} requires exactly: {', '.join(sorted(expected_fields))}", f"{request_type} requires exactly: {', '.join(sorted(expected_fields))}",
@@ -125,28 +129,38 @@ class FrameDecoder:
def feed(self, data: bytes) -> list[dict[str, Any]]: def feed(self, data: bytes) -> list[dict[str, Any]]:
if self._failed: if self._failed:
raise ProtocolError("connection_closed", "decoder is closed", close_connection=True) raise ProtocolError("connection_closed", "decoder is closed", close_connection=True)
self._buffer.extend(data) if not isinstance(data, bytes):
self._fail("malformed_json", "frame data is invalid")
frames: list[dict[str, Any]] = [] frames: list[dict[str, Any]] = []
remaining = data
while True: # Consume each newline-delimited frame before considering the next one.
newline = self._buffer.find(b"\n") # The size limit applies to an individual frame, not to a recv batch:
# clients may pipeline any number of bounded frames in one feed call.
while remaining:
newline = remaining.find(b"\n")
if newline < 0: if newline < 0:
if len(self._buffer) > MAX_FRAME_BYTES: if len(self._buffer) + len(remaining) > MAX_FRAME_BYTES:
self._fail("frame_too_large", f"frame exceeds {MAX_FRAME_BYTES} bytes") self._fail("frame_too_large", f"frame exceeds {MAX_FRAME_BYTES} bytes")
self._buffer.extend(remaining)
return frames return frames
if newline > MAX_FRAME_BYTES: if len(self._buffer) + newline > MAX_FRAME_BYTES:
self._fail("frame_too_large", f"frame exceeds {MAX_FRAME_BYTES} bytes") self._fail("frame_too_large", f"frame exceeds {MAX_FRAME_BYTES} bytes")
raw = bytes(self._buffer[:newline]) self._buffer.extend(remaining[:newline])
del self._buffer[: newline + 1] raw = bytes(self._buffer)
self._buffer.clear()
remaining = remaining[newline + 1 :]
try: try:
value = json.loads(raw.decode("utf-8")) value = json.loads(raw.decode("utf-8"))
except (UnicodeDecodeError, ValueError): except (UnicodeDecodeError, ValueError, RecursionError):
self._fail("malformed_json", "frame contains malformed JSON") self._fail("malformed_json", "frame contains malformed JSON")
if not isinstance(value, dict): if not isinstance(value, dict):
self._fail("malformed_json", "frame must contain a JSON object") self._fail("malformed_json", "frame must contain a JSON object")
frames.append(value) frames.append(value)
return frames
def _fail(self, code: str, detail: str) -> None: def _fail(self, code: str, detail: str) -> None:
self._failed = True self._failed = True
self._buffer.clear() self._buffer.clear()
@@ -161,8 +175,11 @@ def encode_frame(frame: Mapping[str, Any]) -> bytes:
transcript = payload.get("transcript") transcript = payload.get("transcript")
if not isinstance(transcript, str): if not isinstance(transcript, str):
raise ProtocolError("invalid_event", "result transcript must be a string") raise ProtocolError("invalid_event", "result transcript must be a string")
if len(transcript.encode("utf-8")) > MAX_TRANSCRIPT_BYTES: try:
raise ProtocolError("transcript_too_large", "transcript exceeds output limit") if len(transcript.encode("utf-8")) > MAX_TRANSCRIPT_BYTES:
raise ProtocolError("transcript_too_large", "transcript exceeds output limit")
except UnicodeEncodeError as error:
raise ProtocolError("invalid_event", "result transcript contains invalid Unicode") from error
if payload.get("type") == "error": if payload.get("type") == "error":
detail = payload.get("detail", "") detail = payload.get("detail", "")
if not isinstance(detail, str): if not isinstance(detail, str):
@@ -175,7 +192,10 @@ def encode_frame(frame: Mapping[str, Any]) -> bytes:
else: else:
payload.pop("generationId", None) payload.pop("generationId", None)
encoded = json.dumps(payload, ensure_ascii=False, separators=(",", ":")).encode("utf-8") try:
encoded = json.dumps(payload, ensure_ascii=False, separators=(",", ":")).encode("utf-8")
except (TypeError, ValueError, UnicodeError, RecursionError, OverflowError) as error:
raise ProtocolError("invalid_event", "event contains unsupported values") from error
if len(encoded) > MAX_FRAME_BYTES: if len(encoded) > MAX_FRAME_BYTES:
raise ProtocolError("frame_too_large", "encoded frame exceeds output limit") raise ProtocolError("frame_too_large", "encoded frame exceeds output limit")
return encoded + b"\n" return encoded + b"\n"
@@ -186,8 +206,8 @@ def error_event(error: ProtocolError) -> dict[str, Any]:
"version": PROTOCOL_VERSION, "version": PROTOCOL_VERSION,
"type": "error", "type": "error",
"requestId": _safe_id(error.request_id), "requestId": _safe_id(error.request_id),
"code": error.code, "code": _bounded_utf8(error.code if isinstance(error.code, str) else "service_error", MAX_ID_BYTES),
"detail": _bounded_utf8(error.detail, MAX_ERROR_DETAIL_BYTES), "detail": _bounded_utf8(error.detail if isinstance(error.detail, str) else str(error.detail), MAX_ERROR_DETAIL_BYTES),
} }
generation_id = _safe_id(error.generation_id) generation_id = _safe_id(error.generation_id)
if generation_id: if generation_id:
@@ -1,4 +1,4 @@
"""PipeWire recorder adapter with bounded in-memory output.""" """PipeWire recorder adapter with bounded buffering and optional streaming."""
from __future__ import annotations from __future__ import annotations
@@ -13,29 +13,42 @@ import threading
import time import time
from typing import BinaryIO, Callable, Protocol from typing import BinaryIO, Callable, Protocol
from .audio import MAX_PCM_BYTES
PW_RECORD_COMMAND = [ PW_RECORD_COMMAND = [
"pw-record", "pw-record", "--raw", "--format", "s16", "--rate", "16000",
"--raw", "--channels", "1", "-",
"--format",
"s16",
"--rate",
"16000",
"--channels",
"1",
"-",
] ]
MAX_STDERR_BYTES = 64 * 1024 MAX_STDERR_BYTES = 64 * 1024
AUDIO_LEVEL_WINDOW_BYTES = 3_200 # 100 ms of mono 16 kHz signed-16 PCM AUDIO_LEVEL_WINDOW_BYTES = 3_200 # 100 ms of mono 16 kHz signed-16 PCM
def audio_level_window_bytes(sample_rate_hz: int) -> int:
"""Return one 100 ms mono PCM16 meter window for a supported rate."""
if sample_rate_hz not in {16_000, 24_000}:
raise ValueError("unsupported recorder sample rate")
return sample_rate_hz * 2 // 10
def pw_record_command(sample_rate_hz: int) -> list[str]:
"""Build the explicit mono PCM16 PipeWire command for a backend rate."""
if sample_rate_hz not in {16_000, 24_000}:
raise ValueError("unsupported recorder sample rate")
command = list(PW_RECORD_COMMAND)
command[command.index("16000")] = str(sample_rate_hz)
return command
class PcmSink(Protocol):
"""Nonblocking sink used by realtime turns from the recorder drain thread."""
def enqueue_pcm(self, chunk: bytes) -> None: ...
def close_input(self) -> None: ...
def cancel(self) -> None: ...
class Process(Protocol): class Process(Protocol):
pid: int pid: int
stdout: BinaryIO stdout: BinaryIO
stderr: BinaryIO stderr: BinaryIO
returncode: int | None returncode: int | None
def poll(self) -> int | None: ... def poll(self) -> int | None: ...
def terminate(self) -> None: ... def terminate(self) -> None: ...
def kill(self) -> None: ... def kill(self) -> None: ...
@@ -52,17 +65,11 @@ class Capture:
returncode: int returncode: int
signal: int | None signal: int | None
stderr_tail: bytes stderr_tail: bytes
sample_rate_hz: int = 16_000
class RecorderError(RuntimeError): class RecorderError(RuntimeError):
def __init__( def __init__(self, code: str, detail: str, *, returncode: int | None = None, signal_number: int | None = None) -> None:
self,
code: str,
detail: str,
*,
returncode: int | None = None,
signal_number: int | None = None,
) -> None:
super().__init__(detail) super().__init__(detail)
self.code = code self.code = code
self.detail = detail self.detail = detail
@@ -71,20 +78,40 @@ class RecorderError(RuntimeError):
class PwRecorder: class PwRecorder:
"""Capture raw PCM while continuously draining both child pipes.""" """Capture raw PCM while continuously draining both child pipes.
``pcm_sink`` receives each stdout chunk using a nonblocking call. The sink
is closed only after stdout reaches EOF and the child has exited, providing
the no-truncated-tail ordering required by realtime providers.
"""
def __init__( def __init__(
self, self,
*, *,
sample_rate_hz: int = 16_000,
pcm_sink: PcmSink | None = None,
sample_rate: int | None = None,
retain_pcm: bool = True,
process_factory: ProcessFactory = subprocess.Popen, process_factory: ProcessFactory = subprocess.Popen,
max_bytes: int = MAX_PCM_BYTES, max_bytes: int | None = None,
stderr_limit: int = MAX_STDERR_BYTES, stderr_limit: int = MAX_STDERR_BYTES,
stop_timeout: float = 3.0, stop_timeout: float = 3.0,
clock: Callable[[], float] = time.monotonic, clock: Callable[[], float] = time.monotonic,
process_group_signal: ProcessGroupSignal = os.killpg, process_group_signal: ProcessGroupSignal = os.killpg,
) -> None: ) -> None:
if sample_rate is not None:
if sample_rate_hz != 16_000 and sample_rate_hz != sample_rate:
raise ValueError("conflicting recorder sample rates")
sample_rate_hz = sample_rate
if sample_rate_hz not in {16_000, 24_000}:
raise ValueError("unsupported recorder sample rate")
self.sample_rate_hz = sample_rate_hz
self._pcm_sink = pcm_sink
self._retain_pcm = retain_pcm
self._process_factory = process_factory self._process_factory = process_factory
self._max_bytes = max_bytes self._max_bytes = max_bytes if max_bytes is not None else 5 * 60 * sample_rate_hz * 2
if self._max_bytes <= 0:
raise ValueError("recorder byte limit must be positive")
self._stderr_limit = stderr_limit self._stderr_limit = stderr_limit
self._stop_timeout = stop_timeout self._stop_timeout = stop_timeout
self._clock = clock self._clock = clock
@@ -100,6 +127,20 @@ class PwRecorder:
self._duration_stop_at: float | None = None self._duration_stop_at: float | None = None
self._reported = False self._reported = False
self._audio_level = 0.0 self._audio_level = 0.0
self._sink_error: RecorderError | None = None
self._drainer_error: RecorderError | None = None
self._sink_closed = False
self._audio_level_window_bytes = audio_level_window_bytes(sample_rate_hz)
self._captured_bytes = 0
self._cleanup_pending = False
self._cleanup_complete = False
self._cleanup_error: RecorderError | None = None
@property
def cleanup_complete(self) -> bool:
"""Whether the child process and both pipe drainers have been reaped."""
with self._lock:
return self._cleanup_complete
@property @property
def buffered_bytes(self) -> int: def buffered_bytes(self) -> int:
@@ -116,17 +157,22 @@ class PwRecorder:
raise RecorderError("invalid_state", "recorder is already started") raise RecorderError("invalid_state", "recorder is already started")
try: try:
process = self._process_factory( process = self._process_factory(
list(PW_RECORD_COMMAND), pw_record_command(self.sample_rate_hz), stdin=subprocess.DEVNULL,
stdin=subprocess.DEVNULL, stdout=subprocess.PIPE, stderr=subprocess.PIPE, start_new_session=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
start_new_session=True,
) )
except OSError as error: except OSError as error:
self._abort_sink()
self._mark_cleanup_complete()
raise RecorderError("recorder_spawn", f"could not start pw-record: {error}") from error raise RecorderError("recorder_spawn", f"could not start pw-record: {error}") from error
if process.stdout is None or process.stderr is None:
raise RecorderError("recorder_spawn", "pw-record pipes were not created")
self._process = process self._process = process
if process.stdout is None or process.stderr is None:
try:
self.cancel()
except RecorderError:
# A timed-out child remains cleanup-pending and is retried by
# the service before another generation can begin.
pass
raise RecorderError("recorder_spawn", "pw-record pipes were not created")
self._stdout_thread = threading.Thread(target=self._drain_stdout, daemon=True) self._stdout_thread = threading.Thread(target=self._drain_stdout, daemon=True)
self._stderr_thread = threading.Thread(target=self._drain_stderr, daemon=True) self._stderr_thread = threading.Thread(target=self._drain_stderr, daemon=True)
self._stdout_thread.start() self._stdout_thread.start()
@@ -141,119 +187,257 @@ class PwRecorder:
finally: finally:
with self._lock: with self._lock:
self._pcm.clear() self._pcm.clear()
self._abort_sink()
def poll_unexpected(self) -> RecorderError | None: def poll_unexpected(self) -> RecorderError | None:
process = self._require_process() process = self._require_process()
if self._cleanup_pending:
try:
self._retry_pending_cleanup(process)
except RecorderError:
# The pending terminal error was already reported. Keep the
# recorder tracked so a later poll can retry the reap.
return None
return None
with self._lock:
sink_error = self._sink_error
drainer_error = self._drainer_error
if sink_error is not None and not self._reported:
self._reported = True
if process.poll() is None:
self._signal_group(process, signal.SIGTERM)
return sink_error
if drainer_error is not None and not self._reported:
self._reported = True
if process.poll() is None:
self._signal_group(process, signal.SIGTERM)
return drainer_error
returncode = process.poll() returncode = process.poll()
if returncode is None: if returncode is None:
with self._lock: with self._lock:
duration_stop_at = self._duration_stop_at duration_stop_at = self._duration_stop_at
if ( if (
not self._intentional not self._intentional and not self._reported and duration_stop_at is not None
and not self._reported
and duration_stop_at is not None
and self._clock() - duration_stop_at >= self._stop_timeout and self._clock() - duration_stop_at >= self._stop_timeout
): ):
self._reported = True self._reported = True
self._signal_group(process, signal.SIGKILL) self._signal_group(process, signal.SIGKILL)
process.wait(timeout=self._stop_timeout) try:
self._join_drainers() process.wait(timeout=self._stop_timeout)
except subprocess.TimeoutExpired as kill_error:
error = RecorderError(
"recorder_timeout",
"pw-record did not exit after the duration-limit kill",
returncode=process.returncode,
signal_number=signal.SIGKILL,
)
self._mark_cleanup_pending(error)
return error
try:
self._join_and_close_sink()
except RecorderError as error:
self._mark_cleanup_pending(error)
raise
self._abort_sink()
self._mark_cleanup_complete()
with self._lock: with self._lock:
self._pcm.clear() self._pcm.clear()
return RecorderError( return RecorderError(
"duration_limit", "duration_limit",
"recording exceeded the five-minute duration limit; pw-record was killed after ignoring SIGTERM", "recording exceeded the five-minute duration limit; pw-record was killed after ignoring SIGTERM",
returncode=process.returncode, returncode=process.returncode, signal_number=signal.SIGKILL,
signal_number=signal.SIGKILL,
) )
return None return None
if self._intentional or self._reported: if self._intentional or self._reported:
return None return None
self._reported = True self._reported = True
self._join_drainers() try:
self._join_and_close_sink()
except RecorderError as error:
self._mark_cleanup_pending(error)
raise
self._abort_sink()
self._mark_cleanup_complete()
with self._lock: with self._lock:
self._pcm.clear() self._pcm.clear()
if self._duration_limited: if self._duration_limited:
return RecorderError( return RecorderError(
"duration_limit", "duration_limit", "recording exceeded the five-minute duration limit",
"recording exceeded the five-minute duration limit", returncode=returncode, signal_number=-returncode if returncode < 0 else None,
returncode=returncode,
signal_number=-returncode if returncode < 0 else None,
) )
return self._exit_error(returncode, "pw-record exited unexpectedly") return self._exit_error(returncode, "pw-record exited unexpectedly")
def _finish(self, *, discard: bool) -> Capture: def _finish(self, *, discard: bool) -> Capture:
process = self._require_process() process = self._require_process()
self._intentional = True if self._cleanup_pending:
self._retry_pending_cleanup(process)
if discard:
return self._discarded_capture(process)
with self._lock:
cleanup_error = self._cleanup_error
if cleanup_error is not None:
raise cleanup_error
raise RecorderError("recorder_timeout", "pw-record cleanup did not complete")
# Check the child before claiming the exit was caused by this stop.
# A clean exit observed here is still an unexpected recorder exit: it
# may have left only a partial clip in the bounded buffer.
terminated_running_process = process.poll() is None terminated_running_process = process.poll() is None
if terminated_running_process: if not terminated_running_process:
self._signal_group(process, signal.SIGTERM) self._reported = True
self._intentional = terminated_running_process
if terminated_running_process and not self._signal_group(process, signal.SIGTERM):
if process.poll() is not None:
terminated_running_process = False
self._intentional = False
self._reported = True
try: try:
returncode = process.wait(timeout=self._stop_timeout) returncode = process.wait(timeout=self._stop_timeout)
except subprocess.TimeoutExpired as error: except subprocess.TimeoutExpired as error:
self._signal_group(process, signal.SIGKILL) self._signal_group(process, signal.SIGKILL)
process.wait(timeout=self._stop_timeout) try:
self._join_drainers() process.wait(timeout=self._stop_timeout)
except subprocess.TimeoutExpired as kill_error:
timeout_error = RecorderError(
"recorder_timeout",
"pw-record did not exit after SIGKILL",
returncode=process.returncode,
signal_number=signal.SIGKILL,
)
self._mark_cleanup_pending(timeout_error)
raise timeout_error from kill_error
try:
self._join_and_close_sink()
except RecorderError as join_error:
self._mark_cleanup_pending(join_error)
raise
self._abort_sink()
self._mark_cleanup_complete()
with self._lock: with self._lock:
self._pcm.clear() self._pcm.clear()
raise RecorderError( raise RecorderError(
"recorder_timeout", "recorder_timeout", "pw-record did not stop after SIGTERM and was killed",
"pw-record did not stop after SIGTERM and was killed", returncode=process.returncode, signal_number=signal.SIGKILL,
returncode=process.returncode,
signal_number=signal.SIGKILL,
) from error ) from error
self._join_drainers() # Joining both drainers before closing the sink is the critical
# tail-ordering guarantee. Providers cannot be finalized before the
# last PCM chunk has been enqueued.
try:
self._join_and_close_sink()
except RecorderError as error:
self._mark_cleanup_pending(error)
raise
self._mark_cleanup_complete()
stderr_tail = self._stderr_bytes() stderr_tail = self._stderr_bytes()
with self._lock:
sink_error = self._sink_error
if sink_error is not None:
with self._lock:
self._pcm.clear()
self._abort_sink()
raise sink_error
if self._duration_limited: if self._duration_limited:
with self._lock: with self._lock:
self._pcm.clear() self._pcm.clear()
self._abort_sink()
raise RecorderError( raise RecorderError(
"duration_limit", "duration_limit", "recording exceeded the five-minute duration limit",
"recording exceeded the five-minute duration limit", returncode=returncode, signal_number=-returncode if returncode < 0 else None,
returncode=returncode,
signal_number=-returncode if returncode < 0 else None,
) )
clean_pipewire_sigterm = ( if self._captured_bytes % 2:
terminated_running_process and returncode == 1 and not stderr_tail with self._lock:
) self._pcm.clear()
self._abort_sink()
raise RecorderError("invalid_audio", "PCM byte length must be even", returncode=returncode)
if not terminated_running_process:
with self._lock:
self._pcm.clear()
self._abort_sink()
if discard:
return self._discarded_capture(process)
raise self._exit_error(returncode, "pw-record exited unexpectedly")
clean_pipewire_sigterm = returncode == 1 and not stderr_tail
if returncode not in (0, -signal.SIGTERM) and not clean_pipewire_sigterm: if returncode not in (0, -signal.SIGTERM) and not clean_pipewire_sigterm:
with self._lock: with self._lock:
self._pcm.clear() self._pcm.clear()
self._abort_sink()
raise self._exit_error(returncode, "pw-record failed while stopping") raise self._exit_error(returncode, "pw-record failed while stopping")
if discard:
self._abort_sink()
else:
self._close_sink()
with self._lock: with self._lock:
pcm = b"" if discard else bytes(self._pcm) pcm = b"" if discard or not self._retain_pcm else bytes(self._pcm)
self._pcm.clear() self._pcm.clear()
return Capture( return Capture(
pcm=pcm, pcm=pcm, returncode=returncode,
returncode=returncode,
signal=-returncode if returncode < 0 else None, signal=-returncode if returncode < 0 else None,
stderr_tail=stderr_tail, stderr_tail=stderr_tail, sample_rate_hz=self.sample_rate_hz,
) )
def _drain_stdout(self) -> None: def _drain_stdout(self) -> None:
try:
self._drain_stdout_inner()
except Exception as error:
self._record_drainer_error("stdout", error)
def _drain_stdout_inner(self) -> None:
process = self._require_process() process = self._require_process()
level_window = bytearray() level_window = bytearray()
remainder = b""
while True: while True:
chunk = process.stdout.read(AUDIO_LEVEL_WINDOW_BYTES) raw_chunk = process.stdout.read(self._audio_level_window_bytes)
if not chunk: if not raw_chunk:
if remainder:
self._record_drainer_error("stdout", ValueError("PCM byte length must be even"))
return return
chunk = remainder + raw_chunk
remainder = b""
if len(chunk) % 2:
remainder, chunk = chunk[-1:], chunk[:-1]
if not chunk:
continue
level_window.extend(chunk) level_window.extend(chunk)
while len(level_window) >= AUDIO_LEVEL_WINDOW_BYTES: while len(level_window) >= self._audio_level_window_bytes:
self._update_audio_level(level_window[:AUDIO_LEVEL_WINDOW_BYTES]) self._update_audio_level(level_window[: self._audio_level_window_bytes])
del level_window[:AUDIO_LEVEL_WINDOW_BYTES] del level_window[: self._audio_level_window_bytes]
with self._lock: with self._lock:
if self._duration_limited: if self._duration_limited or self._sink_error is not None:
continue continue
if len(self._pcm) + len(chunk) > self._max_bytes: if self._captured_bytes + len(chunk) > self._max_bytes:
self._duration_limited = True self._duration_limited = True
self._duration_stop_at = self._clock() self._duration_stop_at = self._clock()
self._pcm.clear() self._pcm.clear()
over_duration = True
else: else:
self._pcm.extend(chunk) self._captured_bytes += len(chunk)
over_duration = False
if over_duration:
if process.poll() is None:
self._signal_group(process, signal.SIGTERM)
continue
try:
if self._pcm_sink is not None:
self._pcm_sink.enqueue_pcm(chunk)
except Exception as error:
with self._lock:
if self._sink_error is None:
code = getattr(error, "code", "stream_failure")
detail = getattr(error, "detail", "provider audio queue is full")
if not isinstance(code, str) or not code.startswith(("meta_", "openai_")):
code = f"provider_{code}"
self._sink_error = RecorderError(code, str(detail)[:2048])
self._pcm.clear()
if process.poll() is None:
self._signal_group(process, signal.SIGTERM)
# Continue draining so a failed provider cannot deadlock pw-record.
continue
with self._lock:
if self._duration_limited or self._sink_error is not None:
continue continue
if process.poll() is None: if self._retain_pcm:
self._signal_group(process, signal.SIGTERM) self._pcm.extend(chunk)
def _update_audio_level(self, chunk: bytes) -> None: def _update_audio_level(self, chunk: bytes) -> None:
sample_bytes = len(chunk) - (len(chunk) % 2) sample_bytes = len(chunk) - (len(chunk) % 2)
@@ -269,6 +453,12 @@ class PwRecorder:
self._audio_level = max(level, self._audio_level * 0.65) self._audio_level = max(level, self._audio_level * 0.65)
def _drain_stderr(self) -> None: def _drain_stderr(self) -> None:
try:
self._drain_stderr_inner()
except Exception as error:
self._record_drainer_error("stderr", error)
def _drain_stderr_inner(self) -> None:
process = self._require_process() process = self._require_process()
while True: while True:
chunk = process.stderr.read(16 * 1024) chunk = process.stderr.read(16 * 1024)
@@ -280,22 +470,130 @@ class PwRecorder:
if overflow > 0: if overflow > 0:
del self._stderr[:overflow] del self._stderr[:overflow]
def _join_drainers(self) -> None: def _retry_pending_cleanup(self, process: Process) -> None:
"""Make another bounded attempt to reap a timed-out child and pipes."""
if process.poll() is None:
self._signal_group(process, signal.SIGKILL)
try:
process.wait(timeout=self._stop_timeout)
except subprocess.TimeoutExpired as error:
self._abort_sink()
with self._lock:
cleanup_error = self._cleanup_error
self._pcm.clear()
if cleanup_error is None:
cleanup_error = RecorderError(
"recorder_timeout", "pw-record cleanup did not complete",
returncode=process.returncode, signal_number=signal.SIGKILL,
)
raise cleanup_error from error
try:
# A drainer error is terminal for the recording, but a completed
# thread is no longer a resource leak and can be marked reaped.
self._join_drainers(check_errors=False)
except RecorderError:
self._abort_sink()
with self._lock:
self._pcm.clear()
raise
self._abort_sink()
self._mark_cleanup_complete()
def _mark_cleanup_pending(self, error: RecorderError) -> None:
with self._lock:
self._cleanup_pending = True
self._cleanup_complete = False
self._cleanup_error = error
# Abort provider input before a later retry. A failed recording must
# never send the normal end/commit marker.
self._abort_sink()
with self._lock:
self._pcm.clear()
def _mark_cleanup_complete(self) -> None:
with self._lock:
self._cleanup_pending = False
self._cleanup_complete = True
def _discarded_capture(self, process: Process) -> Capture:
returncode = process.returncode
if returncode is None:
returncode = -signal.SIGKILL
return Capture(
pcm=b"", returncode=returncode,
signal=-returncode if returncode < 0 else None,
stderr_tail=self._stderr_bytes(), sample_rate_hz=self.sample_rate_hz,
)
def _join_and_close_sink(self) -> None:
try:
self._join_drainers()
except RecorderError:
# A dead or stuck drainer must abort the streaming sink so its
# worker cannot wait indefinitely or finalize partial audio.
self._abort_sink()
with self._lock:
self._pcm.clear()
raise
def _record_drainer_error(self, stream: str, _error: BaseException) -> None:
with self._lock:
if self._drainer_error is None:
self._drainer_error = RecorderError(
"recorder_pipe", f"pw-record {stream} pipe could not be drained"
)
process = self._process
if process is not None and process.poll() is None:
self._signal_group(process, signal.SIGTERM)
def _join_drainers(self, *, check_errors: bool = True) -> None:
for thread in (self._stdout_thread, self._stderr_thread): for thread in (self._stdout_thread, self._stderr_thread):
if thread is None: if thread is None:
continue continue
thread.join(timeout=self._stop_timeout) thread.join(timeout=self._stop_timeout)
if thread.is_alive(): if thread.is_alive():
raise RecorderError( raise RecorderError("recorder_timeout", "pw-record pipe did not close after the process group stopped")
"recorder_timeout", if check_errors:
"pw-record pipe did not close after the process group stopped", with self._lock:
) drainer_error = self._drainer_error
if drainer_error is not None:
raise drainer_error
def _signal_group(self, process: Process, signal_number: int) -> None: def _abort_sink(self) -> None:
if self._pcm_sink is None or self._sink_closed:
return
self._sink_closed = True
cancel = getattr(self._pcm_sink, "cancel", None)
if callable(cancel):
try:
cancel()
except Exception:
pass
def _close_sink(self) -> None:
if self._pcm_sink is None or self._sink_closed:
return
self._sink_closed = True
close_input = getattr(self._pcm_sink, "close_input", None)
if not callable(close_input):
return
try:
close_input()
except Exception as error:
with self._lock:
if self._sink_error is None:
code = getattr(error, "code", "stream_failure")
detail = getattr(error, "detail", "provider audio queue could not be finalized")
if not isinstance(code, str) or not code.startswith(("meta_", "openai_")):
code = f"provider_{code}"
self._sink_error = RecorderError(code, str(detail)[:2048])
def _signal_group(self, process: Process, signal_number: int) -> bool:
try: try:
self._process_group_signal(process.pid, signal_number) self._process_group_signal(process.pid, signal_number)
except ProcessLookupError: except ProcessLookupError:
pass return False
return True
def _stderr_bytes(self) -> bytes: def _stderr_bytes(self) -> bytes:
with self._lock: with self._lock:
@@ -303,21 +601,14 @@ class PwRecorder:
def _exit_error(self, returncode: int, prefix: str) -> RecorderError: def _exit_error(self, returncode: int, prefix: str) -> RecorderError:
if returncode < 0: if returncode < 0:
classification = f"signal {-returncode}" classification, signal_number = f"signal {-returncode}", -returncode
signal_number = -returncode
else: else:
classification = f"code {returncode}" classification, signal_number = f"code {returncode}", None
signal_number = None
stderr = self._stderr_bytes().decode("utf-8", errors="replace").strip() stderr = self._stderr_bytes().decode("utf-8", errors="replace").strip()
detail = f"{prefix} with {classification}" detail = f"{prefix} with {classification}"
if stderr: if stderr:
detail = f"{detail}: {stderr}" detail = f"{detail}: {stderr}"
return RecorderError( return RecorderError("recorder_exit", detail, returncode=returncode, signal_number=signal_number)
"recorder_exit",
detail,
returncode=returncode,
signal_number=signal_number,
)
def _require_process(self) -> Process: def _require_process(self) -> Process:
if self._process is None: if self._process is None:
+457 -88
View File
@@ -3,17 +3,20 @@
from __future__ import annotations from __future__ import annotations
from collections.abc import Sequence from collections.abc import Sequence
from concurrent.futures import Executor, Future, ThreadPoolExecutor from concurrent.futures import Executor, Future
from dataclasses import dataclass from dataclasses import dataclass
import math import math
import os import os
from queue import Empty, SimpleQueue from queue import Empty, Queue, SimpleQueue
import selectors import selectors
import socket import socket
import threading
import time import time
from typing import Callable, Protocol from typing import Callable, Protocol
from .audio import AudioError, pcm_s16le_to_floats from .audio import AudioError, pcm_s16le_to_floats
from .backend import ProviderError, TranscriptionBackend, TranscriptionTurn, backend_from_config
from .config import ConfigError, load_config
from .model import NemoParakeetModel, TranscriptionModel from .model import NemoParakeetModel, TranscriptionModel
from .protocol import ( from .protocol import (
MAX_FRAME_BYTES, MAX_FRAME_BYTES,
@@ -34,6 +37,19 @@ MAX_CONNECTION_OUTPUT_BYTES = MAX_FRAME_BYTES
LEVEL_EVENT_INTERVAL_SECONDS = 0.1 LEVEL_EVENT_INTERVAL_SECONDS = 0.1
def _looks_like_turn(value: object) -> bool:
"""Keep injected test backends source-compatible while validating turns."""
return all(callable(getattr(value, name, None)) for name in ("enqueue_pcm", "finish", "poll_error", "cancel", "close"))
def _turn_cleanup_complete(turn: object) -> bool:
return bool(getattr(turn, "cleanup_complete", True))
def _recorder_cleanup_complete(recorder: object) -> bool:
return bool(getattr(recorder, "cleanup_complete", True))
class Recorder(Protocol): class Recorder(Protocol):
@property @property
def audio_level(self) -> float: ... def audio_level(self) -> float: ...
@@ -57,7 +73,12 @@ class _Generation:
request_id: str request_id: str
state: str state: str
recorder: Recorder | None = None recorder: Recorder | None = None
recorder_cleanup_started: bool = False
turn: TranscriptionTurn | None = None
tombstoned: bool = False tombstoned: bool = False
failure_reported: bool = False
cleanup_pending: bool = False
cleanup_notify_idle: bool = False
last_level_event_at: float | None = None last_level_event_at: float | None = None
@@ -68,28 +89,84 @@ class _Completion:
future: Future[object] future: Future[object]
class _DaemonExecutor(Executor):
"""Single-worker executor that cannot hold interpreter shutdown hostage."""
def __init__(self) -> None:
self._queue: Queue[tuple[Future[object], Callable[..., object], tuple[object, ...], dict[str, object]] | None] = Queue()
self._lock = threading.Lock()
self._shutdown = False
self._worker = threading.Thread(target=self._run, daemon=True, name="dictate-worker")
self._worker.start()
def submit(self, fn: Callable[..., object], /, *args: object, **kwargs: object) -> Future[object]:
future: Future[object] = Future()
with self._lock:
if self._shutdown:
raise RuntimeError("cannot schedule new futures after shutdown")
self._queue.put((future, fn, args, kwargs))
return future
def shutdown(self, wait: bool = True, *, cancel_futures: bool = False) -> None:
with self._lock:
if not self._shutdown:
self._shutdown = True
if cancel_futures:
while True:
try:
pending = self._queue.get_nowait()
except Empty:
break
if pending is not None:
pending[0].cancel()
self._queue.put(None)
if wait:
self._worker.join()
def _run(self) -> None:
while True:
work = self._queue.get()
if work is None:
return
future, fn, args, kwargs = work
if not future.set_running_or_notify_cancel():
continue
try:
result = fn(*args, **kwargs)
except BaseException as error:
future.set_exception(error)
else:
future.set_result(result)
class DictateServiceCore: class DictateServiceCore:
"""Single-generation protocol state machine with injected host boundaries.""" """Single-generation protocol state machine with injected host boundaries."""
def __init__( def __init__(
self, self,
*, *,
model: TranscriptionModel, model: TranscriptionModel | None = None,
recorder_factory: Callable[[], Recorder] = PwRecorder, backend: TranscriptionBackend | None = None,
recorder_factory: Callable[..., Recorder] = PwRecorder,
executor: Executor | None = None, executor: Executor | None = None,
clock: Callable[[], float] = time.monotonic, clock: Callable[[], float] = time.monotonic,
idle_seconds: float = DEFAULT_IDLE_SECONDS, idle_seconds: float = DEFAULT_IDLE_SECONDS,
) -> None: ) -> None:
if model is None and backend is None:
raise ValueError("either model or backend is required")
if model is not None and backend is not None:
raise ValueError("model and backend are mutually exclusive")
self._model = model self._model = model
self._backend = backend
self._recorder_factory = recorder_factory self._recorder_factory = recorder_factory
self._executor = executor or ThreadPoolExecutor(max_workers=1, thread_name_prefix="dictate") self._executor = executor or _DaemonExecutor()
self._owns_executor = executor is None self._owns_executor = executor is None
self._clock = clock self._clock = clock
self._idle_seconds = idle_seconds self._idle_seconds = idle_seconds
self._last_activity = clock() self._last_activity = clock()
self._clients: dict[str, bool] = {} self._clients: dict[str, bool] = {}
self._active: _Generation | None = None self._active: _Generation | None = None
self._model_loaded = bool(getattr(model, "loaded", False)) self._model_loaded = bool(getattr(model, "loaded", False)) if model is not None else False
self._worker_kind: str | None = None self._worker_kind: str | None = None
self._completions: SimpleQueue[_Completion] = SimpleQueue() self._completions: SimpleQueue[_Completion] = SimpleQueue()
self._events: list[Outbound] = [] self._events: list[Outbound] = []
@@ -98,6 +175,19 @@ class DictateServiceCore:
def active_generation_id(self) -> str | None: def active_generation_id(self) -> str | None:
return self._active.generation_id if self._active else None return self._active.generation_id if self._active else None
def _ready(self) -> bool:
if self._backend is None:
return self._model_loaded
return bool(getattr(self._backend, "ready", False))
def _diagnostics(self) -> dict[str, str]:
descriptor = getattr(self._backend, "descriptor", None)
if descriptor is None:
return {}
# Descriptors are fixed, bounded values supplied by the host backend,
# never paths, endpoint URLs, or credential state.
return {"backend": str(descriptor.backend)[:64], "model": str(descriptor.model)[:128]}
def accepted(self, client_id: str) -> None: def accepted(self, client_id: str) -> None:
"""Track a socket without treating acceptance as protocol activity.""" """Track a socket without treating acceptance as protocol activity."""
self._clients.setdefault(client_id, False) self._clients.setdefault(client_id, False)
@@ -116,7 +206,8 @@ class DictateServiceCore:
"type": "pong", "type": "pong",
"requestId": request.request_id, "requestId": request.request_id,
"service": "pi-dictate", "service": "pi-dictate",
"ready": self._model_loaded, "ready": self._ready(),
**self._diagnostics(),
}, },
) )
return return
@@ -133,8 +224,9 @@ class DictateServiceCore:
"type": "pong", "type": "pong",
"requestId": request.request_id, "requestId": request.request_id,
"service": "pi-dictate", "service": "pi-dictate",
"ready": self._model_loaded, "ready": self._ready(),
"busy": self._active is not None, "busy": self._active is not None,
**self._diagnostics(),
}, },
) )
elif request.type == "start": elif request.type == "start":
@@ -154,14 +246,38 @@ class DictateServiceCore:
self._apply_completion(completion) self._apply_completion(completion)
active = self._active active = self._active
if active and active.cleanup_pending and self._worker_kind is None:
if active.recorder is not None and not _recorder_cleanup_complete(active.recorder):
self._submit("cleanup", active, active.recorder.poll_unexpected)
return
self._finish_generation(active, notify_idle=active.cleanup_notify_idle)
if self._active is active:
return
active = self._active
if active and active.tombstoned and self._worker_kind is None:
if active.recorder is not None and not active.recorder_cleanup_started:
active.recorder_cleanup_started = True
active.state = "stopping"
self._submit("cancel", active, active.recorder.cancel)
else:
self._finish_generation(active, notify_idle=False)
return
if active and active.state == "listening" and active.recorder and self._worker_kind is None: if active and active.state == "listening" and active.recorder and self._worker_kind is None:
if active.turn is not None:
failure_from_turn = active.turn.poll_error()
if failure_from_turn is not None:
active.failure_reported = True
self._async_error(active, failure_from_turn.code, failure_from_turn.detail)
self._abort_active(active)
return
try: try:
failure = active.recorder.poll_unexpected() failure = active.recorder.poll_unexpected()
except RecorderError as error: except RecorderError as error:
failure = error failure = error
if failure is not None: if failure is not None:
active.failure_reported = True
self._async_error(active, failure.code, failure.detail) self._async_error(active, failure.code, failure.detail)
self._finish_generation(active, notify_idle=True) self._abort_active(active)
return return
self._emit_audio_level(active) self._emit_audio_level(active)
@@ -175,9 +291,15 @@ class DictateServiceCore:
if active is None or active.owner != client_id: if active is None or active.owner != client_id:
return return
active.tombstoned = True active.tombstoned = True
if active.state == "listening" and active.recorder is not None: self._cancel_preparation(active)
if active.turn is not None:
self._cancel_turn(active)
if active.recorder is not None and self._worker_kind is None and not active.recorder_cleanup_started:
active.recorder_cleanup_started = True
active.state = "stopping" active.state = "stopping"
self._submit("cancel", active, active.recorder.cancel) self._submit("cancel", active, active.recorder.cancel)
elif self._worker_kind is None:
self._finish_generation(active, notify_idle=False)
def should_exit(self) -> bool: def should_exit(self) -> bool:
if self._active is not None or self._worker_kind is not None: if self._active is not None or self._worker_kind is not None:
@@ -186,14 +308,30 @@ class DictateServiceCore:
def close(self) -> None: def close(self) -> None:
active = self._active active = self._active
if active and active.recorder and active.state == "listening": if active:
try: self._cancel_preparation(active)
active.recorder.cancel() if active.turn is not None:
except RecorderError: self._cancel_turn(active)
pass if active.recorder is not None:
try:
active.recorder.cancel()
except Exception:
pass
if active.turn is not None:
deadline = time.monotonic() + 2.0
while not _turn_cleanup_complete(active.turn) and time.monotonic() < deadline:
try:
active.turn.close()
except Exception:
break
time.sleep(0.01)
self._active = None self._active = None
if self._owns_executor: if self._owns_executor:
self._executor.shutdown(wait=True, cancel_futures=False) # A recorder/provider worker may be stuck in native or network I/O.
# The owned worker is daemonized and isn't registered with Python's
# ThreadPoolExecutor exit hook, so this bounded shutdown also permits
# normal interpreter exit.
self._executor.shutdown(wait=False, cancel_futures=True)
def _start(self, client_id: str, request: Request) -> None: def _start(self, client_id: str, request: Request) -> None:
assert request.generation_id is not None assert request.generation_id is not None
@@ -202,28 +340,70 @@ class DictateServiceCore:
return return
active = _Generation(client_id, request.generation_id, request.request_id, "loading") active = _Generation(client_id, request.generation_id, request.request_id, "loading")
self._active = active self._active = active
if self._backend is not None:
begin_prepare = getattr(self._backend, "begin_prepare", None)
if callable(begin_prepare):
begin_prepare()
self._state(active, "loading")
self._submit("prepare", active, self._backend.start_turn)
return
assert self._model is not None
if not self._model_loaded: if not self._model_loaded:
self._state(active, "loading") self._state(active, "loading")
self._submit("load", active, self._model.load) self._submit("load", active, self._model.load)
return return
self._begin_recording(active) self._begin_recording(active)
def _begin_recording(self, active: _Generation) -> None: def _begin_recording(self, active: _Generation, turn: TranscriptionTurn | None = None) -> None:
if active.tombstoned: if active.tombstoned:
self._finish_generation(active, notify_idle=False) self._finish_generation(active, notify_idle=False)
return return
try: try:
recorder = self._recorder_factory() active.turn = turn
if turn is None:
recorder = self._recorder_factory()
else:
# PwRecorder accepts these options. Keep injected legacy test
# factories source-compatible while requiring a sink for cloud.
sink = None if turn.retain_pcm else turn
try:
recorder = self._recorder_factory(
sample_rate_hz=turn.sample_rate_hz,
pcm_sink=sink,
retain_pcm=turn.retain_pcm,
)
except TypeError:
try:
recorder = self._recorder_factory(
sample_rate_hz=turn.sample_rate_hz, sink=sink
)
except TypeError:
try:
recorder = self._recorder_factory(turn.sample_rate_hz, sink)
except TypeError:
if not turn.retain_pcm:
raise RecorderError(
"recorder_config",
"realtime recorder factory must accept a PCM sink",
)
recorder = self._recorder_factory()
active.recorder = recorder
recorder.start() recorder.start()
except RecorderError as error: except RecorderError as error:
self._async_error(active, error.code, error.detail) self._async_error(active, error.code, error.detail)
self._finish_generation(active, notify_idle=True) active.failure_reported = True
self._abort_active(active)
return return
except OSError as error: except OSError as error:
self._async_error(active, "recorder_spawn", str(error)) self._async_error(active, "recorder_spawn", str(error))
self._finish_generation(active, notify_idle=True) active.failure_reported = True
self._abort_active(active)
return
except Exception as error:
self._async_error(active, "recorder_failure", str(error))
active.failure_reported = True
self._abort_active(active)
return return
active.recorder = recorder
active.last_level_event_at = self._clock() active.last_level_event_at = self._clock()
self._state(active, "listening") self._state(active, "listening")
@@ -263,11 +443,15 @@ class DictateServiceCore:
failure = error failure = error
if failure is not None: if failure is not None:
self._async_error(active, failure.code, failure.detail) self._async_error(active, failure.code, failure.detail)
self._finish_generation(active, notify_idle=True) active.failure_reported = True
self._abort_active(active)
return return
active.state = "stopping" active.state = "stopping"
self._state(active, "stopping", assign=False) self._state(active, "stopping", assign=False)
self._submit("stop", active, self._stop_and_convert, active.recorder) if active.turn is None:
self._submit("stop", active, self._stop_and_convert, active.recorder)
else:
self._submit("stop", active, self._stop_capture, active.recorder)
def _cancel(self, client_id: str, request: Request) -> None: def _cancel(self, client_id: str, request: Request) -> None:
active = self._owned_generation(client_id, request) active = self._owned_generation(client_id, request)
@@ -278,6 +462,9 @@ class DictateServiceCore:
return return
active.tombstoned = True active.tombstoned = True
active.request_id = request.request_id active.request_id = request.request_id
# Emit the delivery tombstone before potentially blocking provider or
# recorder cleanup. The client must learn cancellation promptly while
# the host remains busy reaping the old generation.
self._emit( self._emit(
client_id, client_id,
{ {
@@ -287,9 +474,49 @@ class DictateServiceCore:
"generationId": active.generation_id, "generationId": active.generation_id,
}, },
) )
if active.state == "listening" and active.recorder is not None: self._cancel_preparation(active)
if active.turn is not None:
self._cancel_turn(active)
if active.recorder is not None and self._worker_kind is None and not active.recorder_cleanup_started:
active.recorder_cleanup_started = True
active.state = "stopping" active.state = "stopping"
self._submit("cancel", active, active.recorder.cancel) self._submit("cancel", active, active.recorder.cancel)
elif active.recorder is None and self._worker_kind is None:
self._finish_generation(active, notify_idle=False)
def _cancel_preparation(self, active: _Generation) -> None:
if self._worker_kind != "prepare" or self._backend is None:
return
cancel_prepare = getattr(self._backend, "cancel_prepare", None)
if callable(cancel_prepare):
try:
cancel_prepare()
except Exception:
pass
@staticmethod
def _cancel_turn(active: _Generation) -> None:
if active.turn is None:
return
try:
active.turn.cancel()
except Exception:
pass
def _abort_active(self, active: _Generation) -> None:
"""Tombstone a failed generation and always reap recorder/provider work."""
active.tombstoned = True
self._cancel_preparation(active)
if active.turn is not None:
self._cancel_turn(active)
if active.recorder is not None and getattr(active.recorder, "cleanup_complete", False) is True:
self._finish_generation(active, notify_idle=False)
elif active.recorder is not None and self._worker_kind is None and not active.recorder_cleanup_started:
active.recorder_cleanup_started = True
active.state = "stopping"
self._submit("cancel", active, active.recorder.cancel)
elif active.recorder is None and self._worker_kind is None:
self._finish_generation(active, notify_idle=False)
def _owned_generation(self, client_id: str, request: Request) -> _Generation | None: def _owned_generation(self, client_id: str, request: Request) -> _Generation | None:
assert request.generation_id is not None assert request.generation_id is not None
@@ -315,6 +542,10 @@ class DictateServiceCore:
capture = recorder.stop() capture = recorder.stop()
return pcm_s16le_to_floats(capture.pcm) return pcm_s16le_to_floats(capture.pcm)
@staticmethod
def _stop_capture(recorder: Recorder) -> Capture:
return recorder.stop()
def _submit( def _submit(
self, self,
kind: str, kind: str,
@@ -339,23 +570,47 @@ class DictateServiceCore:
try: try:
result = completion.future.result() result = completion.future.result()
except RecorderError as error: except RecorderError as error:
if not active.tombstoned: if not active.tombstoned and not active.failure_reported:
self._async_error(active, error.code, error.detail) self._async_error(active, error.code, error.detail)
self._finish_generation(active, notify_idle=not active.tombstoned) active.failure_reported = True
self._abort_active(active)
return return
except AudioError as error: except AudioError as error:
if not active.tombstoned: if not active.tombstoned and not active.failure_reported:
self._async_error(active, "invalid_audio", str(error)) self._async_error(active, "invalid_audio", str(error))
self._finish_generation(active, notify_idle=not active.tombstoned) active.failure_reported = True
self._abort_active(active)
return
except ProviderError as error:
if not active.tombstoned and not active.failure_reported:
self._async_error(active, error.code, error.detail)
active.failure_reported = True
self._abort_active(active)
return return
except BaseException as error: except BaseException as error:
if not active.tombstoned: if not active.tombstoned and not active.failure_reported:
code = "model_failure" if completion.kind in ("load", "infer") else "recorder_failure" code = "model_failure" if completion.kind in ("load", "prepare", "infer") else "recorder_failure"
self._async_error(active, code, str(error)) self._async_error(active, code, str(error))
self._finish_generation(active, notify_idle=not active.tombstoned) active.failure_reported = True
self._abort_active(active)
return return
if completion.kind == "load": if completion.kind == "prepare":
if active.tombstoned:
if _looks_like_turn(result):
active.turn = result
try:
result.cancel()
except Exception:
pass
self._finish_generation(active, notify_idle=False)
elif _looks_like_turn(result):
self._begin_recording(active, result)
else:
self._async_error(active, "backend_failure", "backend returned an invalid transcription turn")
active.failure_reported = True
self._abort_active(active)
elif completion.kind == "load":
self._model_loaded = True self._model_loaded = True
if active.tombstoned: if active.tombstoned:
self._finish_generation(active, notify_idle=False) self._finish_generation(active, notify_idle=False)
@@ -363,14 +618,25 @@ class DictateServiceCore:
self._begin_recording(active) self._begin_recording(active)
elif completion.kind == "stop": elif completion.kind == "stop":
if active.tombstoned: if active.tombstoned:
self._finish_generation(active, notify_idle=False) self._abort_active(active)
return
if active.turn is not None:
if not isinstance(result, Capture):
self._async_error(active, "invalid_audio", "recorder returned invalid capture")
active.failure_reported = True
self._abort_active(active)
return
self._state(active, "transcribing")
self._submit("infer", active, active.turn.finish, result)
return return
samples = result samples = result
if not isinstance(samples, Sequence) or isinstance(samples, (str, bytes, bytearray)): if not isinstance(samples, Sequence) or isinstance(samples, (str, bytes, bytearray)):
self._async_error(active, "invalid_audio", "audio conversion returned invalid samples") self._async_error(active, "invalid_audio", "audio conversion returned invalid samples")
self._finish_generation(active, notify_idle=True) active.failure_reported = True
self._abort_active(active)
return return
self._state(active, "transcribing") self._state(active, "transcribing")
assert self._model is not None
self._submit("infer", active, self._model.transcribe, samples) self._submit("infer", active, self._model.transcribe, samples)
elif completion.kind == "infer": elif completion.kind == "infer":
if active.tombstoned: if active.tombstoned:
@@ -378,29 +644,55 @@ class DictateServiceCore:
return return
if not isinstance(result, str): if not isinstance(result, str):
self._async_error(active, "model_failure", "model returned a non-string transcript") self._async_error(active, "model_failure", "model returned a non-string transcript")
elif len(result.encode("utf-8")) > MAX_TRANSCRIPT_BYTES:
self._async_error(active, "transcript_too_large", "transcript exceeds output limit")
else: else:
self._emit( try:
active.owner, transcript_bytes = len(result.encode("utf-8"))
{ except UnicodeEncodeError:
"version": PROTOCOL_VERSION, self._async_error(active, "model_failure", "model returned invalid Unicode")
"type": "result", self._finish_generation(active, notify_idle=True)
"requestId": active.request_id, return
"generationId": active.generation_id, if transcript_bytes > MAX_TRANSCRIPT_BYTES:
"transcript": result, self._async_error(active, "transcript_too_large", "transcript exceeds output limit")
}, else:
) self._emit(
active.owner,
{
"version": PROTOCOL_VERSION,
"type": "result",
"requestId": active.request_id,
"generationId": active.generation_id,
"transcript": result,
},
)
self._finish_generation(active, notify_idle=True) self._finish_generation(active, notify_idle=True)
elif completion.kind == "cancel": elif completion.kind == "cancel":
self._finish_generation(active, notify_idle=False) self._finish_generation(active, notify_idle=not active.tombstoned)
elif completion.kind == "cleanup":
self._finish_generation(active, notify_idle=active.cleanup_notify_idle)
def _finish_generation(self, active: _Generation, *, notify_idle: bool) -> None: def _finish_generation(self, active: _Generation, *, notify_idle: bool) -> None:
if self._active is not active: if self._active is not active:
return return
active.cleanup_pending = True
active.cleanup_notify_idle = active.cleanup_notify_idle or notify_idle
if active.recorder is not None and not _recorder_cleanup_complete(active.recorder):
# Cleanup retries run through the executor so a stubborn child never
# blocks the socket loop. The pending generation remains exclusive.
return
if active.turn is not None:
try:
active.turn.close()
except Exception:
pass
# A provider close can be blocked by a wedged transport. Keep the
# tombstoned generation busy until both provider workers have
# actually exited, so no thread or transport crosses generations.
if not _turn_cleanup_complete(active.turn):
return
active.cleanup_pending = False
self._active = None self._active = None
self._last_activity = self._clock() self._last_activity = self._clock()
if notify_idle and active.owner in self._clients: if (notify_idle or active.failure_reported) and active.owner in self._clients:
self._state(active, "idle", assign=False) self._state(active, "idle", assign=False)
def _state(self, active: _Generation, state: str, *, assign: bool = True) -> None: def _state(self, active: _Generation, state: str, *, assign: bool = True) -> None:
@@ -448,8 +740,12 @@ class _Connection:
close_after_write: bool = False close_after_write: bool = False
def inherited_listener(environ: dict[str, str] | None = None) -> socket.socket: def inherited_listener(
"""Return systemd's single inherited listener at fd 3.""" environ: dict[str, str] | None = None,
*,
expected_path: str | os.PathLike[str] | None = None,
) -> socket.socket:
"""Return systemd's fd 3 only when it is the expected Unix stream socket."""
environment = os.environ if environ is None else environ environment = os.environ if environ is None else environ
try: try:
listen_pid = int(environment.get("LISTEN_PID", "0")) listen_pid = int(environment.get("LISTEN_PID", "0"))
@@ -458,9 +754,65 @@ def inherited_listener(environ: dict[str, str] | None = None) -> socket.socket:
raise RuntimeError("invalid systemd socket activation environment") from error raise RuntimeError("invalid systemd socket activation environment") from error
if listen_pid != os.getpid() or listen_fds != 1: if listen_pid != os.getpid() or listen_fds != 1:
raise RuntimeError("expected exactly one systemd socket-activated listener") raise RuntimeError("expected exactly one systemd socket-activated listener")
listener = socket.socket(fileno=3) listener: socket.socket | None = None
listener.setblocking(False) try:
return listener listener = socket.socket(fileno=3)
if listener.family != socket.AF_UNIX:
raise RuntimeError("inherited listener must be AF_UNIX")
socket_type = listener.type & getattr(socket, "SOCK_TYPE_MASK", 0xF)
if socket_type != socket.SOCK_STREAM:
raise RuntimeError("inherited listener must be SOCK_STREAM")
try:
accepting = listener.getsockopt(socket.SOL_SOCKET, socket.SO_ACCEPTCONN)
except OSError as error:
raise RuntimeError("could not validate inherited listener state") from error
if accepting != 1:
raise RuntimeError("inherited listener is not listening")
actual = listener.getsockname()
if isinstance(actual, bytes):
actual = actual.decode("utf-8", errors="strict")
if not isinstance(actual, str):
raise RuntimeError("inherited listener has an invalid socket path")
expected = expected_path
if expected is None:
runtime_dir = environment.get("XDG_RUNTIME_DIR", f"/run/user/{os.getuid()}")
expected = os.path.join(runtime_dir, "pi-dictate.sock")
if os.path.abspath(actual) != os.path.abspath(os.fspath(expected)):
raise RuntimeError("inherited listener is not the expected pi-dictate socket")
listener.setblocking(False)
return listener
except Exception:
if listener is not None:
listener.close()
raise
def _queue_connection_frame(
selector: selectors.BaseSelector,
connections: dict[str, _Connection],
core: DictateServiceCore,
client_id: str,
frame: dict[str, object],
) -> bool:
"""Encode and append one frame without exceeding the per-client bound."""
connection = connections.get(client_id)
if connection is None:
return False
try:
encoded = encode_frame(frame)
except ProtocolError:
_close_connection(selector, connections, core, client_id)
return False
if len(connection.output) + len(encoded) > MAX_CONNECTION_OUTPUT_BYTES:
_close_connection(selector, connections, core, client_id)
return False
connection.output.extend(encoded)
selector.modify(
connection.socket,
selectors.EVENT_READ | selectors.EVENT_WRITE,
client_id,
)
return True
def serve(listener: socket.socket, core: DictateServiceCore) -> None: def serve(listener: socket.socket, core: DictateServiceCore) -> None:
@@ -477,19 +829,8 @@ def serve(listener: socket.socket, core: DictateServiceCore) -> None:
connection = connections.get(outbound.client_id) connection = connections.get(outbound.client_id)
if connection is None: if connection is None:
continue continue
try: _queue_connection_frame(
encoded = encode_frame(outbound.frame) selector, connections, core, outbound.client_id, outbound.frame
except ProtocolError:
_close_connection(selector, connections, core, outbound.client_id)
continue
if len(connection.output) + len(encoded) > MAX_CONNECTION_OUTPUT_BYTES:
_close_connection(selector, connections, core, outbound.client_id)
continue
connection.output.extend(encoded)
selector.modify(
connection.socket,
selectors.EVENT_READ | selectors.EVENT_WRITE,
outbound.client_id,
) )
for key, mask in selector.select(timeout=0.1): for key, mask in selector.select(timeout=0.1):
if key.data is None: if key.data is None:
@@ -527,9 +868,19 @@ def serve(listener: socket.socket, core: DictateServiceCore) -> None:
for frame in connection.decoder.feed(data): for frame in connection.decoder.feed(data):
core.handle_request(client_id, parse_request(frame)) core.handle_request(client_id, parse_request(frame))
except ProtocolError as error: except ProtocolError as error:
connection.output.extend(encode_frame(error_event(error))) if _queue_connection_frame(
if error.close_connection: selector,
connections,
core,
client_id,
error_event(error),
) and error.close_connection:
connection.close_after_write = True connection.close_after_write = True
# Queueing a protocol error can close this connection when its
# output bound is exceeded. The selector event was created for
# the old socket, so do not send to or modify that stale socket.
if connections.get(client_id) is not connection:
continue
if mask & selectors.EVENT_WRITE and connection.output: if mask & selectors.EVENT_WRITE and connection.output:
try: try:
sent = connection.socket.send(connection.output) sent = connection.socket.send(connection.output)
@@ -573,26 +924,44 @@ def _close_connection(
def main() -> None: def main() -> None:
model_path = os.environ.get("PI_DICTATE_MODEL_PATH") # Installed services use the strict host config. The environment fallback
if not model_path: # keeps direct development invocations and existing local deployments
raise SystemExit("PI_DICTATE_MODEL_PATH must name an explicit local .nemo file") # compatible until install.sh has written config.json.
terms_path = os.environ.get("PI_DICTATE_TERMS_PATH") config_path = os.environ.get("PI_DICTATE_CONFIG_PATH")
if not terms_path: default_config = os.path.expanduser("~/.config/pi-dictate/config.json")
raise SystemExit("PI_DICTATE_TERMS_PATH must name the host coding dictionary") if config_path or os.path.isfile(default_config):
idle_value = os.environ.get("PI_DICTATE_IDLE_SECONDS", str(DEFAULT_IDLE_SECONDS)) try:
boost_value = os.environ.get("PI_DICTATE_BOOST_ALPHA", "1.0") config = load_config(config_path)
try: network_mode = os.environ.get("PI_DICTATE_NETWORK_MODE")
idle_seconds = float(idle_value) expected_network_mode = "offline" if config.backend == "local" else "cloud"
boost_alpha = float(boost_value) if network_mode is not None and network_mode != expected_network_mode:
except ValueError as error: raise ConfigError("configured backend does not match service network mode")
raise SystemExit("PI_DICTATE_IDLE_SECONDS and PI_DICTATE_BOOST_ALPHA must be numeric") from error backend = backend_from_config(config)
if idle_seconds <= 0: except (ConfigError, ValueError) as error:
raise SystemExit("PI_DICTATE_IDLE_SECONDS must be positive") raise SystemExit(f"Invalid pi-dictate configuration: {error}") from error
if boost_alpha <= 0: core = DictateServiceCore(backend=backend, idle_seconds=config.idle_seconds)
raise SystemExit("PI_DICTATE_BOOST_ALPHA must be positive") else:
model_path = os.environ.get("PI_DICTATE_MODEL_PATH")
model = NemoParakeetModel(model_path, terms_path=terms_path, boost_alpha=boost_alpha) if not model_path:
core = DictateServiceCore(model=model, idle_seconds=idle_seconds) raise SystemExit("PI_DICTATE_MODEL_PATH must name an explicit local .nemo file")
terms_path = os.environ.get("PI_DICTATE_TERMS_PATH")
if not terms_path:
raise SystemExit("PI_DICTATE_TERMS_PATH must name the host coding dictionary")
idle_value = os.environ.get("PI_DICTATE_IDLE_SECONDS", str(DEFAULT_IDLE_SECONDS))
boost_value = os.environ.get("PI_DICTATE_BOOST_ALPHA", "1.0")
try:
idle_seconds = int(idle_value)
if str(idle_seconds) != idle_value:
raise ValueError
boost_alpha = float(boost_value)
except ValueError as error:
raise SystemExit("PI_DICTATE_IDLE_SECONDS must be an integer; boost alpha must be numeric") from error
if not 60 <= idle_seconds <= 86_400:
raise SystemExit("PI_DICTATE_IDLE_SECONDS must be between 60 and 86400")
if boost_alpha <= 0:
raise SystemExit("PI_DICTATE_BOOST_ALPHA must be positive")
model = NemoParakeetModel(model_path, terms_path=terms_path, boost_alpha=boost_alpha)
core = DictateServiceCore(model=model, idle_seconds=idle_seconds)
listener = inherited_listener() listener = inherited_listener()
serve(listener, core) serve(listener, core)
@@ -0,0 +1,170 @@
"""Fail-closed checks for the installed systemd hardening contract."""
from __future__ import annotations
import hashlib
import json
import os
from pathlib import Path
import stat
from .config import ConfigError, DictateConfig, load_config
MAX_STARTUP_FILE_BYTES = 64 * 1024
EXPECTED_BASE_UNIT_SHA256 = "345c79165029e52b446087e780c982961e9478da74b5ac5680ac27f5231ff4ce"
EXPECTED_SOCKET_UNIT_SHA256 = "4868f54e2ec25fad132df858d5dfb9bf8337e9d5930998e7af83e58e57f737d0"
def _read_regular(path: Path, limit: int, *, mode: int | None = None) -> bytes:
descriptor: int | None = None
try:
descriptor = os.open(
path,
os.O_RDONLY | getattr(os, "O_CLOEXEC", 0) | getattr(os, "O_NOFOLLOW", 0),
)
info = os.fstat(descriptor)
if not stat.S_ISREG(info.st_mode) or info.st_uid != os.getuid():
raise ConfigError("managed systemd file is not a service-owned regular file")
if mode is not None and stat.S_IMODE(info.st_mode) != mode:
raise ConfigError("managed systemd file permissions are insecure")
if info.st_size > limit:
raise ConfigError("managed systemd file is too large")
chunks: list[bytes] = []
remaining = limit + 1
while remaining:
chunk = os.read(descriptor, min(4096, remaining))
if not chunk:
break
chunks.append(chunk)
remaining -= len(chunk)
raw = b"".join(chunks)
if len(raw) > limit:
raise ConfigError("managed systemd file is too large")
return raw
finally:
if descriptor is not None:
try:
os.close(descriptor)
except OSError:
pass
def _reject_symlink_ancestry(path: Path) -> None:
current = path.absolute()
while True:
try:
info = current.lstat()
except FileNotFoundError:
current = current.parent
if current == current.parent:
return
continue
except OSError as error:
raise ConfigError("systemd drop-in ancestry could not be checked") from error
if stat.S_ISLNK(info.st_mode):
raise ConfigError("systemd drop-in ancestry contains a symlink")
if current == current.parent:
return
current = current.parent
def _bind_line(path: Path) -> str:
expanded = str(path.expanduser()).replace("%", "%%")
return "BindReadOnlyPaths=" + json.dumps(expanded, ensure_ascii=False)
def _expected_dropin(network_mode: str, config: DictateConfig) -> str:
if network_mode == "offline":
return "\n".join(
[
"# Managed by pi-dictate install.sh",
"[Service]",
"Environment=PI_DICTATE_NETWORK_MODE=offline",
"Environment=HF_HUB_OFFLINE=1",
"Environment=TRANSFORMERS_OFFLINE=1",
"Environment=NEMO_HOME=/tmp/pi-dictate-nemo",
"Environment=HF_HOME=/tmp/pi-dictate-huggingface",
"Environment=TRITON_CACHE_DIR=/tmp/pi-dictate-triton",
"Environment=NUMBA_CACHE_DIR=/tmp/pi-dictate-numba",
"RestrictAddressFamilies=AF_UNIX",
"IPAddressDeny=any",
_bind_line(config.selected.model_path),
_bind_line(config.selected.terms_path),
]
) + "\n"
if network_mode == "cloud":
return "\n".join(
[
"# Managed by pi-dictate install.sh",
"[Service]",
"Environment=PI_DICTATE_NETWORK_MODE=cloud",
"# Cloud backends need outbound DNS and TLS, but never a listening IP socket.",
"Environment=HF_HUB_OFFLINE=",
"Environment=TRANSFORMERS_OFFLINE=",
"Environment=NEMO_HOME=",
"Environment=HF_HOME=",
"Environment=TRITON_CACHE_DIR=",
"Environment=NUMBA_CACHE_DIR=",
"Environment=PI_DICTATE_MODEL_PATH=",
"Environment=PI_DICTATE_BOOST_ALPHA=",
"RestrictAddressFamilies=",
"RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6",
"IPAddressDeny=",
_bind_line(config.selected.api_key_file),
_bind_line(config.selected.terms_path),
]
) + "\n"
raise ConfigError("invalid systemd network mode")
def validate_startup() -> None:
"""Validate config, marker, base unit, and effective generated drop-in."""
config_path = Path(os.environ.get("PI_DICTATE_CONFIG_PATH", "~/.config/pi-dictate/config.json")).expanduser()
config = load_config(config_path)
network_mode = os.environ.get("PI_DICTATE_NETWORK_MODE", "")
expected_mode = "offline" if config.backend == "local" else "cloud"
if network_mode != expected_mode:
raise ConfigError("configured backend does not match systemd network mode")
marker = Path(os.environ.get("PI_DICTATE_DEPENDENCY_PROFILE_PATH", "~/.local/share/pi-dictate/.dependency-profile")).expanduser()
marker_raw = _read_regular(marker, 32, mode=0o600)
if marker_raw not in {b"local\n", b"cloud\n"} or marker_raw.rstrip(b"\n").decode("ascii") != expected_mode.replace("offline", "local"):
raise ConfigError("installed dependency profile does not match configured backend")
unit_root = Path("~/.config/systemd/user").expanduser()
_reject_symlink_ancestry(unit_root)
dropin_dir = unit_root / "pi-dictate.service.d"
dropin_path = dropin_dir / "10-pi-dictate-network.conf"
_reject_symlink_ancestry(dropin_dir)
if not dropin_dir.is_dir():
raise ConfigError("managed systemd drop-in directory is missing")
try:
entries = list(dropin_dir.iterdir())
except OSError as error:
raise ConfigError("managed systemd drop-in directory cannot be read") from error
if any(entry.name != dropin_path.name for entry in entries):
raise ConfigError("unexpected systemd drop-in is present")
if _read_regular(dropin_path, MAX_STARTUP_FILE_BYTES).decode("utf-8") != _expected_dropin(network_mode, config):
raise ConfigError("effective systemd network hardening is not the expected managed policy")
unit_path = unit_root / "pi-dictate.service"
unit_raw = _read_regular(unit_path, MAX_STARTUP_FILE_BYTES)
if hashlib.sha256(unit_raw).hexdigest() != EXPECTED_BASE_UNIT_SHA256:
raise ConfigError("base systemd hardening is incomplete or tampered")
socket_unit_path = unit_root / "pi-dictate.socket"
socket_unit_raw = _read_regular(socket_unit_path, MAX_STARTUP_FILE_BYTES)
if hashlib.sha256(socket_unit_raw).hexdigest() != EXPECTED_SOCKET_UNIT_SHA256:
raise ConfigError("socket systemd policy is incomplete or tampered")
def main() -> None:
try:
validate_startup()
except (ConfigError, OSError, UnicodeError, ValueError) as error:
raise SystemExit(f"Invalid pi-dictate startup policy: {error}") from error
if __name__ == "__main__":
main()
+2
View File
@@ -0,0 +1,2 @@
{"type":"transcript","final":false,"text":"partial"}
{"type":"transcript","final":true,"text":"final fixture transcript"}
+1
View File
@@ -0,0 +1 @@
{"sessionId":"fixture-session-1"}
+4
View File
@@ -0,0 +1,4 @@
{"type":"session.created"}
{"type":"session.updated"}
{"type":"input_audio_buffer.committed","event_id":"event-fixture-1","previous_item_id":null,"item_id":"fixture-item-1"}
{"type":"conversation.item.input_audio_transcription.completed","item_id":"fixture-item-1","transcript":"final fixture transcript"}
+586
View File
@@ -0,0 +1,586 @@
from __future__ import annotations
import base64
import json
from pathlib import Path
import tempfile
import threading
import time
from types import SimpleNamespace
import unittest
from unittest.mock import Mock, patch
from pi_dictate_service.backend import PcmQueue, ProviderError, provider_event
from pi_dictate_service.backends.meta import (
META_ENDPOINT,
META_MAX_BACKLOG_BYTES,
META_SOCKET_IO_TIMEOUT,
MetaBackend,
_default_transport as default_meta_transport,
)
from pi_dictate_service.backends.openai import (
OPENAI_ENDPOINT,
OPENAI_SOCKET_IO_TIMEOUT,
OpenAIBackend,
_default_transport as default_openai_transport,
)
from pi_dictate_service.config import MetaConfig, OpenAIConfig
FIXTURE_ROOT = Path(__file__).parent / "fixtures"
def fixture_events(provider: str) -> list[str]:
return (FIXTURE_ROOT / provider / "events.jsonl").read_text(encoding="utf-8").splitlines()
def _capture_start_error(backend: object, errors: list[BaseException]) -> None:
try:
backend.start_turn()
except BaseException as error:
errors.append(error)
class FakeTransport:
def __init__(self, events: list[str | bytes | None]) -> None:
self.events = list(events)
self.sent: list[str | bytes] = []
self.closed = False
def send(self, value: str | bytes) -> None:
self.sent.append(value)
def recv(self, timeout: float | None = None) -> str | bytes | None:
if not self.events:
return None
return self.events.pop(0)
def close(self) -> None:
self.closed = True
class PersistentTransport(FakeTransport):
"""Keep the provider session open after all queued events are delivered."""
def __init__(self, events: list[str | bytes | None]) -> None:
super().__init__(events)
self.close_requested = threading.Event()
def recv(self, timeout: float | None = None) -> str | bytes | None:
if self.events:
return self.events.pop(0)
if self.close_requested.wait(timeout):
return None
raise TimeoutError
def close(self) -> None:
self.close_requested.set()
super().close()
class ErrorOnCloseTransport(PersistentTransport):
"""Release a provider error concurrently with client-initiated closure."""
def __init__(self, events: list[str | bytes | None], error_event: str) -> None:
super().__init__(events)
self.error_event = error_event
self.error_delivered = False
def recv(self, timeout: float | None = None) -> str | bytes | None:
if self.events:
return self.events.pop(0)
if not self.close_requested.wait(timeout):
raise TimeoutError
if not self.error_delivered:
self.error_delivered = True
return self.error_event
return None
class BlockingCloseTransport(FakeTransport):
def __init__(self, events: list[str | bytes | None]) -> None:
super().__init__(events)
self.close_started = threading.Event()
self.allow_close = threading.Event()
def close(self) -> None:
self.close_started.set()
self.allow_close.wait(1)
super().close()
class HttpFailure(RuntimeError):
def __init__(self, status_code: int) -> None:
super().__init__(f"HTTP {status_code}")
self.response = SimpleNamespace(status_code=status_code)
class BackendTests(unittest.TestCase):
def test_production_transports_apply_a_socket_io_deadline(self) -> None:
for factory, timeout, arguments in (
(default_meta_transport, META_SOCKET_IO_TIMEOUT, (META_ENDPOINT,)),
(
default_openai_transport,
OPENAI_SOCKET_IO_TIMEOUT,
(OPENAI_ENDPOINT,),
),
):
with self.subTest(factory=factory.__module__):
transport = SimpleNamespace(socket=Mock())
with patch("websockets.sync.client.connect", return_value=transport):
if factory is default_openai_transport:
factory(*arguments, headers={"Authorization": "Bearer test"})
else:
factory(*arguments)
transport.socket.settimeout.assert_called_once_with(timeout)
def make_files(self) -> tuple[Path, Path, tempfile.TemporaryDirectory[str]]:
temporary = tempfile.TemporaryDirectory()
root = Path(temporary.name)
key = root / "key"
key.write_text("secret", encoding="utf-8")
key.chmod(0o600)
terms = root / "terms"
terms.write_text("JSON\n", encoding="utf-8")
return key, terms, temporary
def test_meta_waits_for_ack_sends_documented_handshake_and_finalizes_after_tail(self) -> None:
key, terms, temporary = self.make_files()
self.addCleanup(temporary.cleanup)
acknowledgement = (FIXTURE_ROOT / "meta" / "session_ack.json").read_text(encoding="utf-8").strip()
transport = FakeTransport([acknowledgement, *fixture_events("meta"), None])
connections: list[str] = []
def connect(endpoint: str) -> FakeTransport:
connections.append(endpoint)
return transport
backend = MetaBackend(
MetaConfig("muse-voice-transcribe-1.0", key, terms, language_bias=("en",)),
transport_factory=connect,
timeout=1,
)
turn = backend.start_turn()
turn.enqueue_pcm(b"pcm")
self.assertEqual(turn.finish(), "final fixture transcript")
self.assertEqual(connections, [META_ENDPOINT])
handshake = json.loads(transport.sent[0])
self.assertEqual(handshake["authorization"]["accessToken"], "Bearer secret")
self.assertEqual(handshake["audioEncoding"], "PCM_24KHZ")
self.assertEqual(handshake["mode"], "PUSH_TO_TALK")
self.assertFalse(handshake["emitAudioProgress"])
self.assertEqual(handshake["languageBias"], ["en"])
self.assertEqual(handshake["keywords"], ["JSON"])
self.assertEqual(transport.sent[1], b"pcm")
self.assertEqual(json.loads(transport.sent[2]), {"type": "endStream"})
def test_meta_final_transcript_does_not_wait_for_provider_close(self) -> None:
key, terms, temporary = self.make_files()
self.addCleanup(temporary.cleanup)
acknowledgement = (FIXTURE_ROOT / "meta" / "session_ack.json").read_text(encoding="utf-8").strip()
transport = PersistentTransport([acknowledgement, *fixture_events("meta")])
backend = MetaBackend(
MetaConfig("muse-voice-transcribe-1.0", key, terms),
transport_factory=lambda _endpoint: transport,
timeout=1,
)
turn = backend.start_turn()
turn.enqueue_pcm(b"pcm")
started_at = time.monotonic()
self.assertEqual(turn.finish(), "final fixture transcript")
self.assertLess(time.monotonic() - started_at, 0.5)
self.assertTrue(transport.closed)
def test_meta_provider_error_racing_with_finalization_is_authoritative(self) -> None:
key, terms, temporary = self.make_files()
self.addCleanup(temporary.cleanup)
acknowledgement = (FIXTURE_ROOT / "meta" / "session_ack.json").read_text(encoding="utf-8").strip()
transport = ErrorOnCloseTransport(
[acknowledgement, *fixture_events("meta")],
json.dumps({"type": "error", "code": "provider_failure"}),
)
backend = MetaBackend(
MetaConfig("muse-voice-transcribe-1.0", key, terms),
transport_factory=lambda _endpoint: transport,
timeout=1,
)
turn = backend.start_turn()
turn.enqueue_pcm(b"pcm")
with self.assertRaises(ProviderError) as error:
turn.finish()
self.assertEqual(error.exception.code, "meta_error")
self.assertTrue(transport.error_delivered)
def test_openai_uses_ga_session_shape_and_correlates_committed_item(self) -> None:
key, terms, temporary = self.make_files()
self.addCleanup(temporary.cleanup)
transport = FakeTransport(fixture_events("openai"))
connections: list[tuple[str, dict[str, str]]] = []
def connect(endpoint: str, *, headers: dict[str, str]) -> FakeTransport:
connections.append((endpoint, headers))
return transport
backend = OpenAIBackend(
OpenAIConfig(
"gpt-live-transcribe",
key,
terms,
delay="low",
prompt="Coding",
languages=("en",),
),
transport_factory=connect,
timeout=1,
)
turn = backend.start_turn()
turn.enqueue_pcm(b"pcm")
self.assertEqual(turn.finish(), "final fixture transcript")
self.assertEqual(
connections,
[(OPENAI_ENDPOINT, {"Authorization": "Bearer secret"})],
)
update = json.loads(transport.sent[0])
session = update["session"]
self.assertEqual(session["type"], "transcription")
input_audio = session["audio"]["input"]
self.assertEqual(input_audio["format"], {"type": "audio/pcm", "rate": 24000})
self.assertEqual(input_audio["transcription"]["languages"], ["en"])
self.assertEqual(input_audio["transcription"]["delay"], "low")
self.assertEqual(input_audio["transcription"]["keywords"], ["JSON"])
self.assertIsNone(input_audio["turn_detection"])
self.assertEqual(json.loads(transport.sent[-1]), {"type": "input_audio_buffer.commit"})
self.assertFalse(any(isinstance(item, str) and "response.create" in item for item in transport.sent))
def test_meta_rejects_malformed_ack_and_auth_error(self) -> None:
key, terms, temporary = self.make_files()
self.addCleanup(temporary.cleanup)
malformed = MetaBackend(
MetaConfig("muse-voice-transcribe-1.0", key, terms),
transport_factory=lambda _endpoint: FakeTransport(['{"sessionId":"s","unexpected":1}']),
timeout=0.1,
)
with self.assertRaisesRegex(ProviderError, "acknowledgement"):
malformed.start_turn()
auth = MetaBackend(
MetaConfig("muse-voice-transcribe-1.0", key, terms),
transport_factory=lambda _endpoint: FakeTransport(['{"type":"error","code":"401"}']),
timeout=0.1,
)
with self.assertRaisesRegex(ProviderError, "Meta transcription service reported") as error:
auth.start_turn()
self.assertEqual(error.exception.code, "meta_auth")
with self.assertRaises(ValueError):
MetaBackend(MetaConfig("muse-voice-transcribe-1.0", key, terms), max_queue_bytes=META_MAX_BACKLOG_BYTES + 1)
def test_cloud_turn_rejects_partial_only_close_and_queue_overflow(self) -> None:
key, terms, temporary = self.make_files()
self.addCleanup(temporary.cleanup)
transport = FakeTransport(['{"sessionId":"s"}', '{"type":"transcript","final":false,"text":"partial"}', None])
backend = MetaBackend(
MetaConfig("muse-voice-transcribe-1.0", key, terms),
transport_factory=lambda _endpoint: transport,
timeout=0.1,
)
turn = backend.start_turn()
with self.assertRaisesRegex(ProviderError, "final transcript"):
turn.finish()
queue = PcmQueue(max_bytes=2, max_chunks=1)
queue.put(b"12")
with self.assertRaisesRegex(ProviderError, "full"):
queue.put(b"3")
def test_cancel_before_prepare_worker_runs_prevents_connection(self) -> None:
key, terms, temporary = self.make_files()
self.addCleanup(temporary.cleanup)
for backend in (
MetaBackend(
MetaConfig("muse-voice-transcribe-1.0", key, terms),
transport_factory=lambda *_args, **_kwargs: self.fail("Meta connected"),
),
OpenAIBackend(
OpenAIConfig("gpt-live-transcribe", key, terms),
transport_factory=lambda *_args, **_kwargs: self.fail("OpenAI connected"),
),
):
with self.subTest(backend=backend.descriptor.backend):
backend.begin_prepare()
backend.cancel_prepare()
with self.assertRaises(ProviderError) as error:
backend.start_turn()
self.assertEqual(error.exception.code, "cancelled")
def test_cancel_during_connect_prevents_authenticated_handshake(self) -> None:
key, terms, temporary = self.make_files()
self.addCleanup(temporary.cleanup)
for backend_name in ("meta", "openai"):
with self.subTest(backend=backend_name):
entered = threading.Event()
release = threading.Event()
transport = FakeTransport([])
errors: list[BaseException] = []
def connect(*_args: object, **_kwargs: object) -> FakeTransport:
entered.set()
release.wait(1)
return transport
if backend_name == "meta":
backend = MetaBackend(
MetaConfig("muse-voice-transcribe-1.0", key, terms),
transport_factory=connect,
timeout=0.1,
)
else:
backend = OpenAIBackend(
OpenAIConfig("gpt-live-transcribe", key, terms),
transport_factory=connect,
timeout=0.1,
)
worker = threading.Thread(
target=lambda: _capture_start_error(backend, errors), daemon=True
)
worker.start()
self.assertTrue(entered.wait(1))
backend.cancel_prepare()
release.set()
worker.join(1)
self.assertFalse(worker.is_alive())
self.assertEqual(getattr(errors[0], "code", None), "cancelled")
self.assertEqual(transport.sent, [])
def test_meta_discards_high_volume_partials_instead_of_overflowing_event_queue(self) -> None:
key, terms, temporary = self.make_files()
self.addCleanup(temporary.cleanup)
partial = '{"type":"transcript","final":false,"text":"partial"}'
transport = FakeTransport([
'{"sessionId":"s"}',
*([partial] * 200),
'{"type":"transcript","final":true,"text":"final"}',
None,
])
turn = MetaBackend(
MetaConfig("muse-voice-transcribe-1.0", key, terms),
transport_factory=lambda _endpoint: transport,
timeout=1,
).start_turn()
self.assertEqual(turn.finish(), "final")
def test_cancel_never_sends_a_provider_end_marker(self) -> None:
key, terms, temporary = self.make_files()
self.addCleanup(temporary.cleanup)
transports_and_turns = []
meta_transport = FakeTransport(['{"sessionId":"s"}'])
transports_and_turns.append((
meta_transport,
MetaBackend(
MetaConfig("muse-voice-transcribe-1.0", key, terms),
transport_factory=lambda _endpoint: meta_transport,
timeout=1,
).start_turn(),
))
openai_transport = FakeTransport([
'{"type":"session.created"}', '{"type":"session.updated"}'
])
transports_and_turns.append((
openai_transport,
OpenAIBackend(
OpenAIConfig("gpt-live-transcribe", key, terms),
transport_factory=lambda _endpoint, **_kwargs: openai_transport,
timeout=1,
).start_turn(),
))
for transport, turn in transports_and_turns:
turn.cancel()
deadline = time.monotonic() + 1
while not turn.cleanup_complete and time.monotonic() < deadline:
turn.close()
sent_types = [
value.get("type")
for raw in transport.sent
if isinstance(raw, str) and isinstance((value := json.loads(raw)), dict)
]
self.assertNotIn("endStream", sent_types)
self.assertNotIn("input_audio_buffer.commit", sent_types)
def test_cancel_does_not_block_on_a_slow_transport_close(self) -> None:
key, terms, temporary = self.make_files()
self.addCleanup(temporary.cleanup)
transport = BlockingCloseTransport(['{"sessionId":"s"}'])
turn = MetaBackend(
MetaConfig("muse-voice-transcribe-1.0", key, terms),
transport_factory=lambda _endpoint: transport,
timeout=0.1,
).start_turn()
started = time.monotonic()
turn.cancel()
elapsed = time.monotonic() - started
self.assertLess(elapsed, 0.05)
self.assertTrue(transport.close_started.wait(1))
self.assertFalse(turn.cleanup_complete)
transport.allow_close.set()
deadline = time.monotonic() + 1
while not turn.cleanup_complete and time.monotonic() < deadline:
turn.close()
self.assertTrue(turn.cleanup_complete)
def test_http_failures_use_websockets_response_status(self) -> None:
key, terms, temporary = self.make_files()
self.addCleanup(temporary.cleanup)
meta = MetaBackend(
MetaConfig("muse-voice-transcribe-1.0", key, terms),
transport_factory=lambda _endpoint: (_ for _ in ()).throw(HttpFailure(401)),
)
with self.assertRaises(ProviderError) as meta_error:
meta.start_turn()
self.assertEqual(meta_error.exception.code, "meta_auth")
openai = OpenAIBackend(
OpenAIConfig("gpt-live-transcribe", key, terms),
transport_factory=lambda _endpoint, **_kwargs: (_ for _ in ()).throw(HttpFailure(429)),
)
with self.assertRaises(ProviderError) as openai_error:
openai.start_turn()
self.assertEqual(openai_error.exception.code, "openai_rate_limit")
def test_cloud_turn_reports_provider_close_during_recording(self) -> None:
key, terms, temporary = self.make_files()
self.addCleanup(temporary.cleanup)
backend = MetaBackend(
MetaConfig("muse-voice-transcribe-1.0", key, terms),
transport_factory=lambda _endpoint: FakeTransport([
'{"sessionId":"s"}',
None,
]),
timeout=0.1,
)
turn = backend.start_turn()
deadline = time.monotonic() + 1
failure = turn.poll_error()
while failure is None and time.monotonic() < deadline:
time.sleep(0.001)
failure = turn.poll_error()
self.assertIsNotNone(failure)
self.assertEqual(failure.code, "meta_premature_close")
def test_provider_event_rejects_recursive_values(self) -> None:
recursive: dict[str, object] = {}
recursive["self"] = recursive
with self.assertRaises(ProviderError):
provider_event(recursive)
def test_openai_rejects_malformed_error_and_unmatched_completion(self) -> None:
key, terms, temporary = self.make_files()
self.addCleanup(temporary.cleanup)
malformed = OpenAIBackend(
OpenAIConfig("gpt-live-transcribe", key, terms),
transport_factory=lambda _endpoint, **_kwargs: FakeTransport([
'{"type":"session.created"}',
'{"type":"error","error":{"message":"bad"}}',
]),
timeout=0.1,
)
with self.assertRaisesRegex(ProviderError, "error event"):
malformed.start_turn()
transport = FakeTransport([
'{"type":"session.created"}',
'{"type":"session.updated"}',
'{"type":"input_audio_buffer.committed","item_id":"new"}',
'{"type":"conversation.item.input_audio_transcription.completed","item_id":"old","transcript":"stale"}',
None,
])
backend = OpenAIBackend(
OpenAIConfig("gpt-live-transcribe", key, terms),
transport_factory=lambda _endpoint, **_kwargs: transport,
timeout=0.1,
)
turn = backend.start_turn()
with self.assertRaises(ProviderError) as error:
turn.finish()
self.assertIn(error.exception.code, {"openai_premature_close", "openai_timeout"})
def test_openai_commit_oriented_alternative_uses_explicit_commit(self) -> None:
key, terms, temporary = self.make_files()
self.addCleanup(temporary.cleanup)
transport = FakeTransport(fixture_events("openai"))
turn = OpenAIBackend(
OpenAIConfig("gpt-transcribe", key, terms),
transport_factory=lambda _endpoint, **_kwargs: transport,
timeout=1,
).start_turn()
turn.enqueue_pcm(b"pcm")
self.assertEqual(turn.finish(), "final fixture transcript")
update = json.loads(transport.sent[0])
self.assertEqual(
update["session"]["audio"]["input"]["transcription"]["model"],
"gpt-transcribe",
)
self.assertEqual(json.loads(transport.sent[-1]), {"type": "input_audio_buffer.commit"})
def test_openai_rejects_unknown_delay_from_direct_callers(self) -> None:
key, terms, temporary = self.make_files()
self.addCleanup(temporary.cleanup)
with self.assertRaisesRegex(ValueError, "delay"):
OpenAIBackend(OpenAIConfig("gpt-live-transcribe", key, terms, delay="fast"))
def test_openai_receiver_failure_overrides_an_already_queued_completion(self) -> None:
key, terms, temporary = self.make_files()
self.addCleanup(temporary.cleanup)
transport = FakeTransport([
'{"type":"session.created"}',
'{"type":"session.updated"}',
'{"type":"input_audio_buffer.committed","item_id":"i"}',
'{"type":"conversation.item.input_audio_transcription.completed","item_id":"i","transcript":"unsafe"}',
'{}',
])
turn = OpenAIBackend(
OpenAIConfig("gpt-live-transcribe", key, terms),
transport_factory=lambda _endpoint, **_kwargs: transport,
timeout=1,
).start_turn()
deadline = time.monotonic() + 1
while turn._receiver_error is None and time.monotonic() < deadline:
time.sleep(0.001)
with self.assertRaises(ProviderError) as error:
turn.finish()
self.assertEqual(error.exception.code, "openai_malformed_event")
def test_openai_audio_is_bounded_base64(self) -> None:
key, terms, temporary = self.make_files()
self.addCleanup(temporary.cleanup)
transport = FakeTransport(
['{"type":"session.created"}', '{"type":"session.updated"}',
'{"type":"input_audio_buffer.committed","item_id":"i"}',
'{"type":"conversation.item.input_audio_transcription.completed","item_id":"i","transcript":""}']
)
backend = OpenAIBackend(
OpenAIConfig("gpt-live-transcribe", key, terms),
transport_factory=lambda _endpoint, **_kwargs: transport,
timeout=1,
)
turn = backend.start_turn()
turn.enqueue_pcm(b"\x01\x02")
self.assertEqual(turn.finish(), "")
append = json.loads(transport.sent[1])
self.assertEqual(base64.b64decode(append["audio"]), b"\x01\x02")
if __name__ == "__main__":
unittest.main()
+120
View File
@@ -0,0 +1,120 @@
from __future__ import annotations
import json
import os
from pathlib import Path
import tempfile
import unittest
from unittest.mock import patch
from pi_dictate_service.config import (
ConfigError,
CredentialError,
load_config,
parse_config,
provider_keywords,
read_credential,
)
from pi_dictate_service.recorder import audio_level_window_bytes, pw_record_command
class ConfigTests(unittest.TestCase):
def test_recorder_supports_both_backend_sample_rates(self) -> None:
self.assertEqual(pw_record_command(16_000)[5], "16000")
self.assertEqual(pw_record_command(24_000)[5], "24000")
self.assertEqual(audio_level_window_bytes(24_000), 4_800)
def test_rejects_unknown_selected_fields_and_invalid_backend_values(self) -> None:
for invalid_backend in ([], {}, 1, True, None):
with self.subTest(backend=invalid_backend):
with self.assertRaisesRegex(ConfigError, "backend must be"):
parse_config({"version": 1, "backend": invalid_backend})
base = {
"version": 1,
"backend": "openai",
"openai": {
"model": "gpt-live-transcribe",
"apiKeyFile": "~/.config/pi-dictate/credentials/openai-api-key",
"termsPath": "/tmp/terms",
},
}
with patch.dict(os.environ, {"HOME": "/tmp/pi-dictate-test"}):
invalid = {**base, "openai": {**base["openai"], "endpoint": "https://bad"}}
with self.assertRaises(ConfigError):
parse_config(invalid)
with self.assertRaises(ConfigError):
parse_config({**base, "version": 2})
configured = parse_config(
{**base, "openai": {**base["openai"], "delay": "low"}}
)
self.assertEqual(configured.openai.delay, "low")
with self.assertRaises(ConfigError):
parse_config({**base, "openai": {**base["openai"], "delay": "fast"}})
with self.assertRaises(ConfigError):
parse_config({**base, "openai": {**base["openai"], "termsPath": "relative/terms"}})
def test_reads_only_one_trailing_newline_from_a_private_credential(self) -> None:
with tempfile.TemporaryDirectory() as directory:
path = Path(directory) / "key"
path.write_text("secret\n", encoding="utf-8")
path.chmod(0o600)
self.assertEqual(read_credential(path), "secret")
path.chmod(0o640)
with self.assertRaises(CredentialError):
read_credential(path)
path.chmod(0o600)
path.write_text("secret\nextra", encoding="utf-8")
with self.assertRaises(CredentialError):
read_credential(path)
def test_rejects_a_credential_reached_through_a_symlinked_directory(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
actual = root / "actual"
actual.mkdir()
key = actual / "key"
key.write_text("secret", encoding="utf-8")
key.chmod(0o600)
linked = root / "credentials"
linked.symlink_to(actual, target_is_directory=True)
with self.assertRaisesRegex(CredentialError, "ancestry"):
read_credential(linked / "key")
def test_config_file_is_regular_private_and_provider_terms_reject_cr(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
config_path = root / "config.json"
config_path.write_text(
json.dumps(
{
"version": 1,
"backend": "local",
"local": {
"adapter": "nemo-parakeet",
"modelPath": "/tmp/model.nemo",
"modelSha256": "0" * 64,
"termsPath": "/tmp/terms",
"boostAlpha": 1.0,
},
}
),
encoding="utf-8",
)
config_path.chmod(0o600)
self.assertEqual(load_config(config_path).backend, "local")
terms = root / "terms"
terms.write_bytes(b"bad\rterm")
with self.assertRaises(ConfigError):
provider_keywords(terms)
terms.write_bytes(b"safe\nbad\x00term\n")
with self.assertRaises(ConfigError):
provider_keywords(terms)
if __name__ == "__main__":
unittest.main()
+496
View File
@@ -0,0 +1,496 @@
from __future__ import annotations
import hashlib
import json
import os
from pathlib import Path
import shutil
import stat
import subprocess
import tempfile
import unittest
SCRIPT = Path(__file__).resolve().parents[1] / "install.sh"
SYSTEMD_ROOT = SCRIPT.parents[2] / "systemd/user"
LEGACY_SERVICE = """# Managed by pi-dictate install.sh
[Unit]
Description=Pi local Parakeet dictation service
Requires=pi-dictate.socket
After=pipewire.service
[Service]
Type=simple
ExecStart=%h/.local/share/pi-dictate/venv/bin/pi-dictate-service
Environment=PI_DICTATE_MODEL_PATH=%h/.local/share/pi-dictate/models/parakeet-tdt-0.6b-v3/parakeet-tdt-0.6b-v3.nemo
Environment=PI_DICTATE_TERMS_PATH=%h/.local/share/pi-dictate/config/coding-terms.txt
Environment=PI_DICTATE_BOOST_ALPHA=1.0
Environment=PI_DICTATE_IDLE_SECONDS=1800
Environment=HF_HUB_OFFLINE=1
Environment=TRANSFORMERS_OFFLINE=1
Environment=NEMO_HOME=/tmp/pi-dictate-nemo
Environment=HF_HOME=/tmp/pi-dictate-huggingface
Environment=TRITON_CACHE_DIR=/tmp/pi-dictate-triton
Environment=NUMBA_CACHE_DIR=/tmp/pi-dictate-numba
Environment=PYTHONDONTWRITEBYTECODE=1
UMask=0077
LimitCORE=0
MemorySwapMax=0
NoNewPrivileges=yes
PrivateTmp=yes
ProtectClock=yes
ProtectControlGroups=yes
ProtectKernelLogs=yes
ProtectKernelModules=yes
ProtectKernelTunables=yes
ProtectSystem=strict
ProtectHome=read-only
RestrictAddressFamilies=AF_UNIX
IPAddressDeny=any
LockPersonality=yes
# The socket unit owns %t/pi-dictate.sock. The service must not unlink it.
"""
class InstallFlowTests(unittest.TestCase):
def setUp(self) -> None:
self.directory = tempfile.TemporaryDirectory()
self.root = Path(self.directory.name)
self.home = self.root / "home"
self.home.mkdir()
self.bin = self.root / "bin"
self.bin.mkdir()
uv = self.bin / "uv"
# A real uv sync creates the requested environment. Keep the fake
# faithful to that contract so transactional environment swapping is
# exercised without installing dependencies.
uv.write_text("#!/bin/sh\nmkdir -p \"$UV_PROJECT_ENVIRONMENT\"\nexit 0\n", encoding="utf-8")
uv.chmod(0o755)
systemctl = self.bin / "systemctl"
systemctl.write_text("#!/bin/sh\nexit 0\n", encoding="utf-8")
systemctl.chmod(0o755)
def tearDown(self) -> None:
self.directory.cleanup()
def run_installer(self, *arguments: str) -> subprocess.CompletedProcess[str]:
environment = {
**os.environ,
"HOME": str(self.home),
"XDG_CONFIG_HOME": str(self.home / ".config"),
"PATH": f"{self.bin}:{os.environ['PATH']}",
}
return subprocess.run(
[str(SCRIPT), *arguments],
env=environment,
capture_output=True,
text=True,
check=False,
)
def test_real_staged_cloud_venv_python_runs_after_transactional_move(self) -> None:
real_uv = shutil.which("uv")
self.assertIsNotNone(real_uv)
(self.bin / "uv").unlink()
source = self.root / "openai-key"
source.write_text("test-key", encoding="utf-8")
result = self.run_installer(
"install", "--backend", "openai", "--api-key-file", str(source)
)
self.assertEqual(result.returncode, 0, result.stderr)
python = self.home / ".local/share/pi-dictate/venv/bin/python"
executed = subprocess.run(
[str(python), "-c", "import pi_dictate_service"],
capture_output=True,
text=True,
check=False,
)
self.assertEqual(executed.returncode, 0, executed.stderr)
def test_upgrades_the_previous_service_unit_and_preserves_its_idle_setting(self) -> None:
unit_dir = self.home / ".config/systemd/user"
unit_dir.mkdir(parents=True)
(unit_dir / "pi-dictate.service").write_text(LEGACY_SERVICE, encoding="utf-8")
(unit_dir / "pi-dictate.socket").write_bytes(
(SYSTEMD_ROOT / "pi-dictate.socket").read_bytes()
)
source = self.root / "custom.nemo"
source.write_bytes(b"custom archive")
digest = hashlib.sha256(source.read_bytes()).hexdigest()
result = self.run_installer(
"install",
"--backend",
"local",
"--model-source",
str(source),
"--model-sha256",
digest,
)
self.assertEqual(result.returncode, 0, result.stderr)
self.assertEqual(
(unit_dir / "pi-dictate.service").read_bytes(),
(SYSTEMD_ROOT / "pi-dictate.service").read_bytes(),
)
config = json.loads(
(self.home / ".config/pi-dictate/config.json").read_text(encoding="utf-8")
)
self.assertEqual(config["idleSeconds"], 1800)
def test_rejects_symlinked_install_ancestry(self) -> None:
outside = self.root / "outside"
outside.mkdir()
(self.home / ".local").symlink_to(outside, target_is_directory=True)
source = self.root / "custom.nemo"
source.write_bytes(b"custom archive")
digest = hashlib.sha256(source.read_bytes()).hexdigest()
result = self.run_installer(
"install",
"--backend",
"local",
"--model-source",
str(source),
"--model-sha256",
digest,
)
self.assertNotEqual(result.returncode, 0)
self.assertIn("symlinked path ancestry", result.stderr)
self.assertEqual(list(outside.iterdir()), [])
def test_cloud_install_uses_only_cloud_profile_and_managed_key(self) -> None:
source = self.root / "meta-key"
source.write_text("test-key\n", encoding="utf-8")
result = self.run_installer(
"install", "--backend", "meta", "--api-key-file", str(source)
)
self.assertEqual(result.returncode, 0, result.stderr)
self.assertNotIn("test-key", result.stdout + result.stderr)
self.assertNotIn("nvidia", result.stdout)
self.assertFalse(
(self.home / ".local/share/pi-dictate/models").exists()
)
config_path = self.home / ".config/pi-dictate/config.json"
config = json.loads(config_path.read_text(encoding="utf-8"))
self.assertEqual(config["backend"], "meta")
self.assertEqual(config["meta"]["model"], "muse-voice-transcribe-1.0")
self.assertEqual(config["meta"]["languageBias"], [])
self.assertEqual(
(self.home / ".config/pi-dictate/credentials/meta-api-key").read_text(
encoding="utf-8"
),
"test-key",
)
self.assertEqual(stat.S_IMODE(config_path.stat().st_mode), 0o600)
self.assertEqual(
stat.S_IMODE(
(self.home / ".config/pi-dictate/credentials/meta-api-key").stat().st_mode
),
0o600,
)
self.assertEqual(
(self.home / ".local/share/pi-dictate/.dependency-profile").read_text(
encoding="utf-8"
),
"cloud\n",
)
def test_check_rejects_values_that_runtime_configuration_rejects(self) -> None:
source = self.root / "openai-key"
source.write_text("test-key", encoding="utf-8")
self.assertEqual(
self.run_installer(
"install", "--backend", "openai", "--api-key-file", str(source)
).returncode,
0,
)
config_path = self.home / ".config/pi-dictate/config.json"
valid = json.loads(config_path.read_text(encoding="utf-8"))
invalid_configs = []
boolean_version = {**valid, "version": True}
invalid_configs.append(boolean_version)
unhashable_backend = {**valid, "backend": []}
invalid_configs.append(unhashable_backend)
long_model = json.loads(json.dumps(valid))
long_model["openai"]["model"] = "m" * 129
invalid_configs.append(long_model)
unhashable_delay = json.loads(json.dumps(valid))
unhashable_delay["openai"]["delay"] = []
invalid_configs.append(unhashable_delay)
relative_terms = json.loads(json.dumps(valid))
relative_terms["openai"]["termsPath"] = "relative/terms.txt"
invalid_configs.append(relative_terms)
for invalid in invalid_configs:
with self.subTest(invalid=invalid):
config_path.write_text(json.dumps(invalid), encoding="utf-8")
config_path.chmod(0o600)
result = self.run_installer("check")
self.assertNotEqual(result.returncode, 0)
self.assertNotIn("Traceback", result.stderr)
def test_check_rejects_extra_dependency_marker_newlines(self) -> None:
source = self.root / "openai-key"
source.write_text("test-key", encoding="utf-8")
self.assertEqual(
self.run_installer(
"install", "--backend", "openai", "--api-key-file", str(source)
).returncode,
0,
)
marker = self.home / ".local/share/pi-dictate/.dependency-profile"
marker.write_bytes(b"cloud\n\n")
marker.chmod(0o600)
result = self.run_installer("check")
self.assertNotEqual(result.returncode, 0)
self.assertIn("dependency profile", result.stderr)
def test_uninstall_removes_an_exact_openai_dropin(self) -> None:
source = self.root / "openai-key"
source.write_text("test-key", encoding="utf-8")
self.assertEqual(
self.run_installer(
"install", "--backend", "openai", "--api-key-file", str(source)
).returncode,
0,
)
result = self.run_installer("uninstall")
self.assertEqual(result.returncode, 0, result.stderr)
self.assertFalse(
(self.home / ".config/systemd/user/pi-dictate.service.d/10-pi-dictate-network.conf").exists()
)
def test_openai_configure_preserves_live_verified_delay(self) -> None:
source = self.root / "openai-key"
source.write_text("test-key", encoding="utf-8")
self.assertEqual(
self.run_installer(
"install", "--backend", "openai", "--api-key-file", str(source)
).returncode,
0,
)
config_path = self.home / ".config/pi-dictate/config.json"
config = json.loads(config_path.read_text(encoding="utf-8"))
config["openai"]["delay"] = "low"
config_path.write_text(json.dumps(config), encoding="utf-8")
config_path.chmod(0o600)
result = self.run_installer("configure", "--backend", "openai")
self.assertEqual(result.returncode, 0, result.stderr)
configured = json.loads(config_path.read_text(encoding="utf-8"))
self.assertEqual(configured["openai"]["delay"], "low")
def test_meta_configure_preserves_valid_language_bias(self) -> None:
source = self.root / "meta-key"
source.write_text("test-key", encoding="utf-8")
self.assertEqual(
self.run_installer(
"install", "--backend", "meta", "--api-key-file", str(source)
).returncode,
0,
)
config_path = self.home / ".config/pi-dictate/config.json"
config = json.loads(config_path.read_text(encoding="utf-8"))
config["meta"]["languageBias"] = ["en", "fr"]
config_path.write_text(json.dumps(config), encoding="utf-8")
config_path.chmod(0o600)
result = self.run_installer("configure", "--backend", "meta")
self.assertEqual(result.returncode, 0, result.stderr)
configured = json.loads(config_path.read_text(encoding="utf-8"))
self.assertEqual(configured["meta"]["languageBias"], ["en", "fr"])
def test_local_custom_archive_requires_and_records_explicit_digest(self) -> None:
source = self.root / "custom.nemo"
source.write_bytes(b"custom archive")
digest = hashlib.sha256(source.read_bytes()).hexdigest()
result = self.run_installer(
"install",
"--backend",
"local",
"--model-source",
str(source),
"--model-sha256",
digest,
)
self.assertEqual(result.returncode, 0, result.stderr)
config = json.loads(
(self.home / ".config/pi-dictate/config.json").read_text(encoding="utf-8")
)
self.assertEqual(config["local"]["modelSha256"], digest)
installed = self.home / (
".local/share/pi-dictate/models/parakeet-tdt-0.6b-v3/"
"parakeet-tdt-0.6b-v3.nemo"
)
self.assertEqual(installed.read_bytes(), source.read_bytes())
def test_replacing_a_custom_local_model_preserves_its_configured_path(self) -> None:
first = self.root / "first.nemo"
first.write_bytes(b"first archive")
first_digest = hashlib.sha256(first.read_bytes()).hexdigest()
self.assertEqual(
self.run_installer(
"install", "--backend", "local", "--model-source", str(first),
"--model-sha256", first_digest,
).returncode,
0,
)
config_path = self.home / ".config/pi-dictate/config.json"
config = json.loads(config_path.read_text(encoding="utf-8"))
custom_model = self.root / "custom-models/model.nemo"
custom_model.parent.mkdir()
custom_model.write_bytes(first.read_bytes())
config["local"]["modelPath"] = str(custom_model)
config_path.write_text(json.dumps(config), encoding="utf-8")
config_path.chmod(0o600)
second = self.root / "second.nemo"
second.write_bytes(b"second archive")
second_digest = hashlib.sha256(second.read_bytes()).hexdigest()
result = self.run_installer(
"configure", "--backend", "local", "--model-source", str(second),
"--model-sha256", second_digest,
)
self.assertEqual(result.returncode, 0, result.stderr)
configured = json.loads(config_path.read_text(encoding="utf-8"))
self.assertEqual(configured["local"]["modelPath"], str(custom_model))
self.assertEqual(custom_model.read_bytes(), second.read_bytes())
self.assertFalse(list(custom_model.parent.glob("*.pi-dictate-rollback.*")))
def test_switching_from_cloud_to_local_accepts_a_local_archive(self) -> None:
key = self.root / "meta-key"
key.write_text("test-key", encoding="utf-8")
self.assertEqual(
self.run_installer(
"install", "--backend", "meta", "--api-key-file", str(key)
).returncode,
0,
)
source = self.root / "custom.nemo"
source.write_bytes(b"custom archive")
digest = hashlib.sha256(source.read_bytes()).hexdigest()
result = self.run_installer(
"configure",
"--backend",
"local",
"--model-source",
str(source),
"--model-sha256",
digest,
)
self.assertEqual(result.returncode, 0, result.stderr)
config = json.loads(
(self.home / ".config/pi-dictate/config.json").read_text(encoding="utf-8")
)
self.assertEqual(config["backend"], "local")
self.assertEqual(config["local"]["modelSha256"], digest)
self.assertIn("meta", config)
def test_systemd_failure_rolls_back_staged_install(self) -> None:
old_venv = self.home / ".local/share/pi-dictate/venv"
old_venv.mkdir(parents=True)
(old_venv / "previous-runtime").write_text("old", encoding="utf-8")
source = self.root / "meta-key"
source.write_text("test-key", encoding="utf-8")
(self.bin / "systemctl").write_text(
"#!/bin/sh\necho \"$*\" >> \"$HOME/systemctl.log\"\n"
"case \"$*\" in *'is-enabled'*) exit 1;; "
"*'stop pi-dictate.service'*) exit 9;; esac\nexit 0\n",
encoding="utf-8",
)
(self.bin / "systemctl").chmod(0o755)
result = self.run_installer(
"install", "--backend", "meta", "--api-key-file", str(source)
)
self.assertNotEqual(result.returncode, 0)
self.assertFalse((self.home / ".config/pi-dictate/config.json").exists())
self.assertFalse((self.home / ".local/share/pi-dictate/.dependency-profile").exists())
self.assertFalse((self.home / ".local/share/pi-dictate/config/coding-terms.txt").exists())
self.assertEqual((old_venv / "previous-runtime").read_text(encoding="utf-8"), "old")
systemctl_calls = (self.home / "systemctl.log").read_text(encoding="utf-8")
self.assertGreaterEqual(systemctl_calls.count("daemon-reload"), 1)
self.assertIn("disable pi-dictate.socket", systemctl_calls)
def test_invalid_cloud_key_does_not_commit_a_dependency_marker(self) -> None:
source = self.root / "invalid-key"
source.write_text("two\nlines\n", encoding="utf-8")
result = self.run_installer(
"install", "--backend", "meta", "--api-key-file", str(source)
)
self.assertNotEqual(result.returncode, 0)
self.assertFalse(
(self.home / ".local/share/pi-dictate/.dependency-profile").exists()
)
def test_dictionary_symlink_is_rejected(self) -> None:
dictionary = self.home / ".local/share/pi-dictate/config/coding-terms.txt"
dictionary.parent.mkdir(parents=True)
target = self.root / "unexpected-terms.txt"
target.write_text("unexpected", encoding="utf-8")
dictionary.symlink_to(target)
source = self.root / "custom.nemo"
source.write_bytes(b"custom archive")
digest = hashlib.sha256(source.read_bytes()).hexdigest()
result = self.run_installer(
"install",
"--backend",
"local",
"--model-source",
str(source),
"--model-sha256",
digest,
)
self.assertNotEqual(result.returncode, 0)
self.assertIn("non-regular coding dictionary", result.stderr)
def test_uninstall_preserves_config_unless_purge_is_requested(self) -> None:
source = self.root / "meta-key"
source.write_text("test-key", encoding="utf-8")
self.assertEqual(
self.run_installer(
"install", "--backend", "meta", "--api-key-file", str(source)
).returncode,
0,
)
result = self.run_installer("uninstall")
self.assertEqual(result.returncode, 0, result.stderr)
self.assertTrue((self.home / ".config/pi-dictate/config.json").exists())
self.assertFalse((self.home / ".local/share/pi-dictate").exists())
# Reinstalling from the preserved key is possible without passing a
# literal secret or a second key source.
result = self.run_installer("install", "--backend", "meta")
self.assertEqual(result.returncode, 0, result.stderr)
self.assertEqual(
self.run_installer("uninstall", "--purge-config").returncode, 0
)
self.assertFalse((self.home / ".config/pi-dictate").exists())
if __name__ == "__main__":
unittest.main()
+26 -1
View File
@@ -1,3 +1,4 @@
import hashlib
from pathlib import Path from pathlib import Path
import os import os
import subprocess import subprocess
@@ -24,8 +25,11 @@ class InstallerTests(unittest.TestCase):
home / ".local/share/pi-dictate" home / ".local/share/pi-dictate"
), ),
} }
command = [str(SCRIPT), *arguments, "--dry-run"]
if not any(action in arguments for action in ("uninstall", "check")):
command.extend(("--model-source", str(model), "--model-sha256", hashlib.sha256(model.read_bytes()).hexdigest()))
return subprocess.run( return subprocess.run(
[str(SCRIPT), *arguments, "--dry-run", "--model-source", str(model)], command,
env=environment, env=environment,
capture_output=True, capture_output=True,
text=True, text=True,
@@ -47,7 +51,9 @@ class InstallerTests(unittest.TestCase):
self.assertNotIn("ln -s", result.stdout) self.assertNotIn("ln -s", result.stdout)
self.assertIn(".nemo.install.", result.stdout) self.assertIn(".nemo.install.", result.stdout)
self.assertIn("mv -f", result.stdout) self.assertIn("mv -f", result.stdout)
self.assertIn("uv venv --relocatable", result.stdout)
self.assertIn("uv sync --frozen --no-dev --no-editable", result.stdout) self.assertIn("uv sync --frozen --no-dev --no-editable", result.stdout)
self.assertIn("--reinstall-package pi-dictate-service", result.stdout)
self.assertIn("Would install pi-dictate", result.stdout) self.assertIn("Would install pi-dictate", result.stdout)
self.assertFalse((home / ".local/share/pi-dictate").exists()) self.assertFalse((home / ".local/share/pi-dictate").exists())
@@ -87,6 +93,25 @@ class InstallerTests(unittest.TestCase):
"[Socket]\nListenStream=/tmp/unrelated.sock\n", "[Socket]\nListenStream=/tmp/unrelated.sock\n",
) )
def test_uninstall_refuses_a_modified_managed_unit(self) -> None:
with tempfile.TemporaryDirectory() as directory:
home = Path(directory)
model = home / "model.nemo"
model.write_bytes(b"model")
unit = home / ".config/systemd/user/pi-dictate.socket"
unit.parent.mkdir(parents=True)
unit.write_text(
"# Managed by pi-dictate install.sh\n[Socket]\nListenStream=/tmp/tampered.sock\n",
encoding="utf-8",
)
result = self.run_installer(home, model, "uninstall")
self.assertEqual(result.returncode, 1)
self.assertIn("unmanaged systemd unit", result.stderr)
self.assertNotIn("systemctl --user disable", result.stdout)
self.assertTrue(unit.exists())
def test_reinstall_preserves_an_existing_host_dictionary(self) -> None: def test_reinstall_preserves_an_existing_host_dictionary(self) -> None:
with tempfile.TemporaryDirectory() as directory: with tempfile.TemporaryDirectory() as directory:
home = Path(directory) home = Path(directory)
+34
View File
@@ -0,0 +1,34 @@
from __future__ import annotations
from pathlib import Path
import tempfile
import unittest
from pi_dictate_service.live_smoke import MAX_SAMPLE_BYTES, _read_sample
class LiveSmokeTests(unittest.TestCase):
def test_reads_one_bounded_even_pcm_fixture(self) -> None:
with tempfile.TemporaryDirectory() as directory:
sample = Path(directory) / "sample.pcm"
sample.write_bytes(b"\x00\x01" * 100)
self.assertEqual(_read_sample(sample), sample.read_bytes())
def test_rejects_symlink_odd_and_oversized_fixtures(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
valid = root / "valid.pcm"
valid.write_bytes(b"\x00\x00")
link = root / "link.pcm"
link.symlink_to(valid)
odd = root / "odd.pcm"
odd.write_bytes(b"x")
oversized = root / "oversized.pcm"
oversized.write_bytes(b"x" * (MAX_SAMPLE_BYTES + 2))
for path in (link, odd, oversized):
with self.subTest(path=path), self.assertRaises((OSError, ValueError)):
_read_sample(path)
if __name__ == "__main__":
unittest.main()
+33 -1
View File
@@ -38,12 +38,20 @@ class FakeNemoModel:
class FakeConcreteParakeetModel: class FakeConcreteParakeetModel:
restored_path: str | None = None restored_path: str | None = None
restored_location: object | None = None restored_location: object | None = None
restored_bytes: bytes | None = None
replace_before_restore: tuple[Path, bytes] | None = None
restored: FakeNemoModel | None = None restored: FakeNemoModel | None = None
@classmethod @classmethod
def restore_from(cls, restore_path: str, map_location: object) -> FakeNemoModel: def restore_from(cls, restore_path: str, map_location: object) -> FakeNemoModel:
cls.restored_path = restore_path cls.restored_path = restore_path
cls.restored_location = map_location cls.restored_location = map_location
if cls.replace_before_restore is not None:
path, replacement = cls.replace_before_restore
replacement_path = path.with_suffix(".replacement")
replacement_path.write_bytes(replacement)
replacement_path.replace(path)
cls.restored_bytes = Path(restore_path).read_bytes()
self = FakeNemoModel() self = FakeNemoModel()
cls.restored = self cls.restored = self
return self return self
@@ -57,6 +65,13 @@ class FakeNumpy(ModuleType):
class NemoParakeetModelTests(unittest.TestCase): class NemoParakeetModelTests(unittest.TestCase):
def setUp(self) -> None:
FakeConcreteParakeetModel.restored_path = None
FakeConcreteParakeetModel.restored_location = None
FakeConcreteParakeetModel.restored_bytes = None
FakeConcreteParakeetModel.replace_before_restore = None
FakeConcreteParakeetModel.restored = None
def test_load_is_lazy_offline_applies_coding_terms_and_transcribes_in_memory_audio(self) -> None: def test_load_is_lazy_offline_applies_coding_terms_and_transcribes_in_memory_audio(self) -> None:
with tempfile.TemporaryDirectory() as directory: with tempfile.TemporaryDirectory() as directory:
model_path = Path(directory) / "parakeet.nemo" model_path = Path(directory) / "parakeet.nemo"
@@ -78,7 +93,8 @@ class NemoParakeetModelTests(unittest.TestCase):
self.assertEqual(os.environ["HF_HUB_OFFLINE"], "1") self.assertEqual(os.environ["HF_HUB_OFFLINE"], "1")
self.assertEqual(os.environ["TRANSFORMERS_OFFLINE"], "1") self.assertEqual(os.environ["TRANSFORMERS_OFFLINE"], "1")
self.assertEqual(FakeConcreteParakeetModel.restored_path, str(model_path)) self.assertRegex(FakeConcreteParakeetModel.restored_path or "", r"^/proc/self/fd/\d+$")
self.assertEqual(FakeConcreteParakeetModel.restored_bytes, b"model")
self.assertEqual(FakeConcreteParakeetModel.restored_location, "device:cuda") self.assertEqual(FakeConcreteParakeetModel.restored_location, "device:cuda")
self.assertTrue(FakeConcreteParakeetModel.restored.eval_called) self.assertTrue(FakeConcreteParakeetModel.restored.eval_called)
self.assertEqual(FakeConcreteParakeetModel.restored.to_device, "cuda") self.assertEqual(FakeConcreteParakeetModel.restored.to_device, "cuda")
@@ -107,6 +123,22 @@ class NemoParakeetModelTests(unittest.TestCase):
with self.assertRaisesRegex(ValueError, "128 UTF-8 bytes"): with self.assertRaisesRegex(ValueError, "128 UTF-8 bytes"):
load_coding_terms(terms_path) load_coding_terms(terms_path)
def test_load_uses_the_verified_inode_if_the_configured_path_is_replaced(self) -> None:
with tempfile.TemporaryDirectory() as directory:
model_path = Path(directory) / "parakeet.nemo"
model_path.write_bytes(b"verified model")
FakeConcreteParakeetModel.replace_before_restore = (model_path, b"replacement")
with patch.dict(sys.modules, self.fake_modules(), clear=False):
adapter = NemoParakeetModel(
model_path,
expected_sha256=hashlib.sha256(b"verified model").hexdigest(),
)
adapter.load()
self.assertEqual(model_path.read_bytes(), b"replacement")
self.assertEqual(FakeConcreteParakeetModel.restored_bytes, b"verified model")
def test_refuses_a_model_that_no_longer_matches_the_pinned_hash(self) -> None: def test_refuses_a_model_that_no_longer_matches_the_pinned_hash(self) -> None:
with tempfile.TemporaryDirectory() as directory: with tempfile.TemporaryDirectory() as directory:
model_path = Path(directory) / "parakeet.nemo" model_path = Path(directory) / "parakeet.nemo"
+20
View File
@@ -0,0 +1,20 @@
import unittest
from pi_dictate_service.protocol import MAX_FRAME_BYTES, FrameDecoder
class FrameDecoderRegressionTests(unittest.TestCase):
def test_accepts_pipelined_bounded_frames_larger_than_aggregate_limit(self) -> None:
decoder = FrameDecoder()
frame = b'{"version":2,"type":"ping","requestId":"request"}\n'
payload = frame * ((MAX_FRAME_BYTES // len(frame)) + 1)
self.assertGreater(len(payload), MAX_FRAME_BYTES + 1)
decoded = decoder.feed(payload)
self.assertEqual(len(decoded), payload.count(b"\n"))
self.assertEqual(decoded[0]["requestId"], "request")
if __name__ == "__main__":
unittest.main()
+156 -3
View File
@@ -36,6 +36,18 @@ class TrackingStream(io.BytesIO):
return value return value
class BlockingTrackingStream(TrackingStream):
def __init__(self, value: bytes, events: list[str], name: str) -> None:
super().__init__(value, events, name)
self.allow_eof = threading.Event()
def read(self, size: int = -1) -> bytes:
value = super().read(size)
if not value:
self.allow_eof.wait(1)
return value
class FakeProcess: class FakeProcess:
def __init__( def __init__(
self, self,
@@ -45,6 +57,7 @@ class FakeProcess:
returncode: int | None = None, returncode: int | None = None,
timeout: bool = False, timeout: bool = False,
terminate_returncode: int = -15, terminate_returncode: int = -15,
kill_stalls: bool = False,
) -> None: ) -> None:
self.pid = 1234 self.pid = 1234
self.stdout = stdout self.stdout = stdout
@@ -52,6 +65,7 @@ class FakeProcess:
self.returncode = returncode self.returncode = returncode
self._timeout = timeout self._timeout = timeout
self._terminate_returncode = terminate_returncode self._terminate_returncode = terminate_returncode
self._kill_stalls = kill_stalls
self.events = stdout._events self.events = stdout._events
def poll(self) -> int | None: def poll(self) -> int | None:
@@ -64,8 +78,9 @@ class FakeProcess:
def kill(self) -> None: def kill(self) -> None:
self.events.append("kill") self.events.append("kill")
self.returncode = -9 if not self._kill_stalls:
self._timeout = False self.returncode = -9
self._timeout = False
def wait(self, timeout: float | None = None) -> int: def wait(self, timeout: float | None = None) -> int:
self.events.append("wait") self.events.append("wait")
@@ -86,6 +101,22 @@ class ProcessFactory:
return self.process return self.process
class StreamingSink:
def __init__(self) -> None:
self.chunks: list[bytes] = []
self.finalized = False
self.cancelled = False
def enqueue_pcm(self, chunk: bytes) -> None:
self.chunks.append(chunk)
def close_input(self) -> None:
self.finalized = True
def cancel(self) -> None:
self.cancelled = True
class Clock: class Clock:
def __init__(self) -> None: def __init__(self) -> None:
self.now = 0.0 self.now = 0.0
@@ -95,6 +126,16 @@ class Clock:
class PwRecorderTests(unittest.TestCase): class PwRecorderTests(unittest.TestCase):
def test_spawn_failure_is_already_cleanup_complete(self) -> None:
def fail_spawn(*_args: object, **_kwargs: object) -> FakeProcess:
raise OSError("spawn failed")
recorder = PwRecorder(process_factory=fail_spawn)
with self.assertRaisesRegex(RecorderError, "could not start"):
recorder.start()
self.assertTrue(recorder.cleanup_complete)
def make_recorder( def make_recorder(
self, self,
pcm: bytes = b"\x00\x00\x01\x00", pcm: bytes = b"\x00\x00\x01\x00",
@@ -107,14 +148,23 @@ class PwRecorderTests(unittest.TestCase):
clock: Clock | None = None, clock: Clock | None = None,
terminate_returncode: int = -15, terminate_returncode: int = -15,
stdout_read_size: int | None = None, stdout_read_size: int | None = None,
kill_stalls: bool = False,
block_stdout_at_eof: bool = False,
pcm_sink: StreamingSink | None = None,
) -> tuple[PwRecorder, FakeProcess, ProcessFactory, list[str]]: ) -> tuple[PwRecorder, FakeProcess, ProcessFactory, list[str]]:
events: list[str] = [] events: list[str] = []
stdout = (
BlockingTrackingStream(pcm, events, "stdout")
if block_stdout_at_eof
else TrackingStream(pcm, events, "stdout", stdout_read_size)
)
process = FakeProcess( process = FakeProcess(
TrackingStream(pcm, events, "stdout", stdout_read_size), stdout,
TrackingStream(stderr, events, "stderr"), TrackingStream(stderr, events, "stderr"),
returncode=returncode, returncode=returncode,
timeout=timeout, timeout=timeout,
terminate_returncode=terminate_returncode, terminate_returncode=terminate_returncode,
kill_stalls=kill_stalls,
) )
factory = ProcessFactory(process) factory = ProcessFactory(process)
def signal_group(pid: int, signal_number: int) -> None: def signal_group(pid: int, signal_number: int) -> None:
@@ -133,9 +183,44 @@ class PwRecorderTests(unittest.TestCase):
stop_timeout=0.01, stop_timeout=0.01,
clock=clock or Clock(), clock=clock or Clock(),
process_group_signal=signal_group, process_group_signal=signal_group,
pcm_sink=pcm_sink,
) )
return recorder, process, factory, events return recorder, process, factory, events
def test_successful_stop_finalizes_streaming_sink(self) -> None:
sink = StreamingSink()
recorder, process, _factory, _events = self.make_recorder(pcm_sink=sink)
recorder.start()
self.assertTrue(process.stdout.eof.wait(1))
recorder.stop()
self.assertTrue(sink.finalized)
self.assertFalse(sink.cancelled)
def test_cancel_aborts_streaming_sink_without_finalizing(self) -> None:
sink = StreamingSink()
recorder, process, _factory, _events = self.make_recorder(pcm_sink=sink)
recorder.start()
self.assertTrue(process.stdout.eof.wait(1))
recorder.cancel()
self.assertTrue(sink.cancelled)
self.assertFalse(sink.finalized)
def test_unexpected_exit_aborts_streaming_sink_without_finalizing(self) -> None:
sink = StreamingSink()
recorder, process, _factory, _events = self.make_recorder(returncode=3, pcm_sink=sink)
recorder.start()
self.assertTrue(process.stdout.eof.wait(1))
error = recorder.poll_unexpected()
self.assertIsNotNone(error)
self.assertTrue(sink.cancelled)
self.assertFalse(sink.finalized)
def test_spawns_exact_pw_record_command_and_drains_bounded_streams(self) -> None: def test_spawns_exact_pw_record_command_and_drains_bounded_streams(self) -> None:
recorder, process, factory, events = self.make_recorder( recorder, process, factory, events = self.make_recorder(
pcm=b"\x00\x00\x01\x00", stderr=b"0123456789", stderr_limit=8 pcm=b"\x00\x00\x01\x00", stderr=b"0123456789", stderr_limit=8
@@ -230,6 +315,39 @@ class PwRecorderTests(unittest.TestCase):
with self.assertRaises(RecorderError): with self.assertRaises(RecorderError):
noisy.stop() noisy.stop()
def test_shutdown_reports_a_bounded_error_when_sigkill_does_not_reap(self) -> None:
recorder, process, _factory, _events = self.make_recorder(
timeout=True,
kill_stalls=True,
)
recorder.start()
with self.assertRaisesRegex(RecorderError, "after SIGKILL") as error:
recorder.stop()
self.assertEqual(error.exception.code, "recorder_timeout")
self.assertIn("kill", process.events)
def test_shutdown_retries_process_and_drainer_cleanup_after_double_timeout(self) -> None:
recorder, process, _factory, _events = self.make_recorder(
timeout=True,
kill_stalls=True,
block_stdout_at_eof=True,
)
recorder.start()
self.assertTrue(process.stdout.eof.wait(1))
with self.assertRaises(RecorderError):
recorder.stop()
self.assertFalse(recorder.cleanup_complete)
process._kill_stalls = False
process.stdout.allow_eof.set()
recorder.cancel()
self.assertTrue(recorder.cleanup_complete)
def test_shutdown_timeout_escalates_to_kill_and_fails(self) -> None: def test_shutdown_timeout_escalates_to_kill_and_fails(self) -> None:
recorder, _process, _factory, events = self.make_recorder(timeout=True) recorder, _process, _factory, events = self.make_recorder(timeout=True)
recorder.start() recorder.start()
@@ -240,6 +358,22 @@ class PwRecorderTests(unittest.TestCase):
self.assertEqual(failed.exception.code, "recorder_timeout") self.assertEqual(failed.exception.code, "recorder_timeout")
self.assertLess(events.index("terminate"), events.index("kill")) self.assertLess(events.index("terminate"), events.index("kill"))
def test_stop_rejects_an_exit_observed_after_the_last_listening_poll(self) -> None:
recorder, process, _factory, _events = self.make_recorder()
recorder.start()
self.assertTrue(process.stdout.eof.wait(1))
# The process was alive during the listening poll, then exited cleanly
# before stop could signal it. A partial buffer is not a valid capture.
self.assertIsNone(recorder.poll_unexpected())
process.returncode = 0
with self.assertRaises(RecorderError) as failed:
recorder.stop()
self.assertEqual(failed.exception.code, "recorder_exit")
self.assertEqual(recorder.buffered_bytes, 0)
def test_classifies_unexpected_exit_code_and_signal(self) -> None: def test_classifies_unexpected_exit_code_and_signal(self) -> None:
recorder, process, _factory, _events = self.make_recorder( recorder, process, _factory, _events = self.make_recorder(
stderr=b"pipewire failed", returncode=7 stderr=b"pipewire failed", returncode=7
@@ -288,6 +422,25 @@ class PwRecorderTests(unittest.TestCase):
self.assertEqual(failed.exception.code, "duration_limit") self.assertEqual(failed.exception.code, "duration_limit")
self.assertEqual(recorder.buffered_bytes, 0) self.assertEqual(recorder.buffered_bytes, 0)
def test_duration_limit_reports_a_bounded_error_when_sigkill_does_not_reap(self) -> None:
clock = Clock()
recorder, process, _factory, _events = self.make_recorder(
pcm=b"\x00\x00" * 3,
max_bytes=4,
timeout=True,
kill_stalls=True,
clock=clock,
)
recorder.start()
self.assertTrue(process.stdout.eof.wait(1))
clock.now = 1
failure = recorder.poll_unexpected()
self.assertIsNotNone(failure)
self.assertEqual(failure.code, "recorder_timeout")
self.assertIn("kill", process.events)
def test_duration_limit_kills_a_recorder_that_ignores_sigterm(self) -> None: def test_duration_limit_kills_a_recorder_that_ignores_sigterm(self) -> None:
clock = Clock() clock = Clock()
recorder, process, _factory, events = self.make_recorder( recorder, process, _factory, events = self.make_recorder(
+552 -2
View File
@@ -1,11 +1,22 @@
from concurrent.futures import Future from concurrent.futures import Future
from collections.abc import Sequence from collections.abc import Sequence
from dataclasses import dataclass from dataclasses import dataclass
import errno
import selectors
import subprocess
import sys
import unittest import unittest
from unittest.mock import patch
from pi_dictate_service.protocol import MAX_TRANSCRIPT_BYTES, PROTOCOL_VERSION, parse_request from pi_dictate_service.protocol import (
MAX_FRAME_BYTES,
MAX_TRANSCRIPT_BYTES,
PROTOCOL_VERSION,
encode_frame,
parse_request,
)
from pi_dictate_service.recorder import Capture, RecorderError from pi_dictate_service.recorder import Capture, RecorderError
from pi_dictate_service.server import DictateServiceCore, Outbound from pi_dictate_service.server import DictateServiceCore, Outbound, serve
class Clock: class Clock:
@@ -93,6 +104,154 @@ class FakeRecorder:
return failure return failure
class SpawnFailRecorder(FakeRecorder):
cleanup_complete = True
def start(self) -> None:
raise RecorderError("recorder_spawn", "could not start pw-record")
class SpawnFailRecorderFactory:
def __call__(self) -> SpawnFailRecorder:
return SpawnFailRecorder()
class DelayedCleanupRecorder(FakeRecorder):
def __init__(self) -> None:
super().__init__()
self.cleanup_complete = False
self.cleanup_attempts = 0
self.cleanup_started = False
def stop(self) -> Capture:
self.cleanup_started = True
raise RecorderError("recorder_timeout", "pw-record cleanup did not complete")
def poll_unexpected(self) -> RecorderError | None:
if not self.cleanup_started:
return None
self.cleanup_attempts += 1
if self.cleanup_attempts >= 3:
self.cleanup_complete = True
return None
class DelayedCleanupRecorderFactory:
def __init__(self) -> None:
self.instance = DelayedCleanupRecorder()
def __call__(self) -> DelayedCleanupRecorder:
return self.instance
class DelayedCloseTurn:
sample_rate_hz = 16_000
retain_pcm = True
def __init__(self) -> None:
self.close_calls = 0
@property
def cleanup_complete(self) -> bool:
return self.close_calls >= 3
def enqueue_pcm(self, chunk: bytes) -> None:
return None
def finish(self, capture: Capture | None = None) -> str:
return "cloud result"
def poll_error(self) -> None:
return None
def cancel(self) -> None:
return None
def close(self) -> None:
self.close_calls += 1
class StreamingTurn:
sample_rate_hz = 24_000
retain_pcm = False
def __init__(self, events: list[str]) -> None:
self.events = events
self.closed = False
@property
def cleanup_complete(self) -> bool:
return self.closed
def enqueue_pcm(self, chunk: bytes) -> None:
self.events.append(f"audio-{chunk.decode('ascii')}")
def finish(self, capture: Capture | None = None) -> str:
self.events.append("finish")
self.assert_capture = capture
return "cloud result"
def poll_error(self) -> None:
return None
def cancel(self) -> None:
self.events.append("cancel")
def close(self) -> None:
self.closed = True
class StreamingRecorder(FakeRecorder):
def __init__(self, sink: StreamingTurn, events: list[str]) -> None:
super().__init__()
self.sink = sink
self.events = events
self.capture = Capture(b"", -15, 15, b"")
def start(self) -> None:
super().start()
self.sink.enqueue_pcm(b"a")
self.sink.enqueue_pcm(b"b")
def stop(self) -> Capture:
self.events.append("recorder-stop")
return super().stop()
class StreamingRecorderFactory:
def __init__(self, events: list[str]) -> None:
self.events = events
self.sample_rate_hz: int | None = None
self.instance: StreamingRecorder | None = None
def __call__(
self,
*,
sample_rate_hz: int,
pcm_sink: StreamingTurn,
retain_pcm: bool,
) -> StreamingRecorder:
self.sample_rate_hz = sample_rate_hz
if retain_pcm:
raise AssertionError("cloud capture must not retain PCM")
self.instance = StreamingRecorder(pcm_sink, self.events)
return self.instance
class FakeBackend:
sample_rate_hz = 16_000
descriptor = type("Descriptor", (), {"backend": "fake", "model": "fake-model"})()
def __init__(self, turn: DelayedCloseTurn | StreamingTurn) -> None:
self.turn = turn
def start_turn(self) -> DelayedCloseTurn:
return self.turn
def cancel_prepare(self) -> None:
return None
class RecorderFactory: class RecorderFactory:
def __init__(self) -> None: def __init__(self) -> None:
self.instances: list[FakeRecorder] = [] self.instances: list[FakeRecorder] = []
@@ -271,6 +430,121 @@ class StopAndInferenceTests(CoreTestCase):
self.assertEqual(completed[0]["transcript"], "dictated text") self.assertEqual(completed[0]["transcript"], "dictated text")
self.assertEqual(completed[1]["state"], "idle") self.assertEqual(completed[1]["state"], "idle")
def test_realtime_backend_streams_ordered_pcm_and_finalizes_after_recorder_stop(self) -> None:
events: list[str] = []
turn = StreamingTurn(events)
recorders = StreamingRecorderFactory(events)
core = DictateServiceCore(
backend=FakeBackend(turn),
recorder_factory=recorders,
executor=self.executor,
clock=self.clock,
idle_seconds=900,
)
core.accepted("a")
core.handle_request("a", parse_request({"version": 2, "type": "hello", "requestId": "hello"}))
core.drain_events()
core.handle_request("a", parse_request({
"version": 2,
"type": "start",
"requestId": "start",
"generationId": "g1",
}))
self.executor.run_next()
core.poll()
self.assertEqual(recorders.sample_rate_hz, 24_000)
self.assertEqual(events, ["audio-a", "audio-b"])
core.drain_events()
core.handle_request("a", parse_request({
"version": 2,
"type": "stop",
"requestId": "stop",
"generationId": "g1",
}))
self.executor.run_next()
core.poll()
self.executor.run_next()
core.poll()
self.assertEqual(events, ["audio-a", "audio-b", "recorder-stop", "finish"])
result = [event.frame for event in core.drain_events() if event.frame["type"] == "result"]
self.assertEqual(result[0]["transcript"], "cloud result")
self.assertIsNone(core.active_generation_id)
def test_cancel_after_prepare_completion_tracks_turn_cleanup_before_release(self) -> None:
events: list[str] = []
turn = StreamingTurn(events)
core = DictateServiceCore(
backend=FakeBackend(turn),
recorder_factory=StreamingRecorderFactory(events),
executor=self.executor,
clock=self.clock,
idle_seconds=900,
)
core.accepted("a")
core.handle_request("a", parse_request({"version": 2, "type": "hello", "requestId": "hello"}))
core.drain_events()
core.handle_request("a", parse_request({
"version": 2,
"type": "start",
"requestId": "start",
"generationId": "g1",
}))
self.executor.run_next()
# The future has completed, but the socket loop has not yet applied it.
core.handle_request("a", parse_request({
"version": 2,
"type": "cancel",
"requestId": "cancel",
"generationId": "g1",
}))
core.poll()
self.assertIn("cancel", events)
self.assertTrue(turn.cleanup_complete)
self.assertIsNone(core.active_generation_id)
def test_completed_turn_retries_bounded_cleanup_before_releasing_generation(self) -> None:
turn = DelayedCloseTurn()
core = DictateServiceCore(
backend=FakeBackend(turn),
recorder_factory=self.recorders,
executor=self.executor,
clock=self.clock,
idle_seconds=900,
)
core.accepted("a")
core.handle_request("a", parse_request({"version": 2, "type": "hello", "requestId": "hello"}))
core.drain_events()
core.handle_request("a", parse_request({
"version": 2,
"type": "start",
"requestId": "start",
"generationId": "g1",
}))
self.executor.run_next()
core.poll()
core.drain_events()
core.handle_request("a", parse_request({
"version": 2,
"type": "stop",
"requestId": "stop",
"generationId": "g1",
}))
self.executor.run_next()
core.poll()
self.executor.run_next()
core.poll()
self.assertEqual(core.active_generation_id, "g1")
self.assertEqual(turn.close_calls, 2)
core.poll()
self.assertIsNone(core.active_generation_id)
self.assertEqual(turn.close_calls, 3)
def test_empty_transcript_is_a_result(self) -> None: def test_empty_transcript_is_a_result(self) -> None:
self.model.transcript = "" self.model.transcript = ""
self.load_to_listening() self.load_to_listening()
@@ -321,6 +595,8 @@ class StopAndInferenceTests(CoreTestCase):
recorder = self.load_to_listening() recorder = self.load_to_listening()
recorder.unexpected = RecorderError("recorder_exit", "pw-record exited with code 3") recorder.unexpected = RecorderError("recorder_exit", "pw-record exited with code 3")
self.core.poll()
self.executor.run_next()
self.core.poll() self.core.poll()
failed = self.frames(self.core.drain_events(), "a") failed = self.frames(self.core.drain_events(), "a")
@@ -332,12 +608,286 @@ class StopAndInferenceTests(CoreTestCase):
recorder.unexpected = RecorderError("recorder_exit", "pw-record exited with code 0") recorder.unexpected = RecorderError("recorder_exit", "pw-record exited with code 0")
failed = self.request("a", "stop", "stop", "g1") failed = self.request("a", "stop", "stop", "g1")
self.executor.run_next()
self.core.poll()
failed.extend(self.frames(self.core.drain_events(), "a"))
self.assertEqual(failed[0]["code"], "recorder_exit") self.assertEqual(failed[0]["code"], "recorder_exit")
self.assertEqual(failed[-1]["state"], "idle") self.assertEqual(failed[-1]["state"], "idle")
self.assertEqual(self.executor.pending, []) self.assertEqual(self.executor.pending, [])
def test_recorder_spawn_failure_releases_generation_without_cleanup_loop(self) -> None:
core = DictateServiceCore(
model=self.model,
recorder_factory=SpawnFailRecorderFactory(),
executor=self.executor,
clock=self.clock,
idle_seconds=900,
)
core.accepted("a")
core.handle_request("a", parse_request({
"version": 2, "type": "hello", "requestId": "hello"
}))
core.drain_events()
core.handle_request("a", parse_request({
"version": 2,
"type": "start",
"requestId": "start",
"generationId": "g1",
}))
self.executor.run_next()
core.poll()
events = self.frames(core.drain_events(), "a")
self.assertEqual(events[-2]["code"], "recorder_spawn")
self.assertEqual(events[-1]["state"], "idle")
self.assertIsNone(core.active_generation_id)
self.assertEqual(self.executor.pending, [])
def test_unreaped_recorder_keeps_generation_busy_until_cleanup_completes(self) -> None:
recorders = DelayedCleanupRecorderFactory()
core = DictateServiceCore(
model=self.model,
recorder_factory=recorders,
executor=self.executor,
clock=self.clock,
idle_seconds=900,
)
core.accepted("a")
core.handle_request("a", parse_request({
"version": 2, "type": "hello", "requestId": "hello"
}))
core.drain_events()
core.handle_request("a", parse_request({
"version": 2,
"type": "start",
"requestId": "start",
"generationId": "g1",
}))
self.executor.run_next()
core.poll()
core.drain_events()
core.handle_request("a", parse_request({
"version": 2,
"type": "stop",
"requestId": "stop",
"generationId": "g1",
}))
self.executor.run_next()
core.poll()
self.assertEqual(core.active_generation_id, "g1")
# First reap the cancellation worker scheduled by the terminal stop.
self.executor.run_next()
core.poll()
for attempt in range(1, 4):
self.executor.run_next()
core.poll()
if attempt < 3:
self.assertEqual(core.active_generation_id, "g1")
self.assertIsNone(core.active_generation_id)
self.assertEqual(recorders.instance.cleanup_attempts, 3)
class SocketLoopRegressionTests(unittest.TestCase):
def test_output_overflow_does_not_use_the_closed_socket_from_same_selector_event(self) -> None:
frame = {"padding": "x" * (MAX_FRAME_BYTES - len(encode_frame({"padding": ""})))}
self.assertEqual(len(encode_frame(frame)), MAX_FRAME_BYTES)
client = _FakeClientSocket()
listener = _FakeListener(client)
selector = _FakeSelector(client)
core = _FakeLoopCore(
Outbound("client-1", frame),
)
with patch("pi_dictate_service.server.selectors.DefaultSelector", return_value=selector):
serve(listener, core)
self.assertEqual(client.send_calls, 0)
self.assertTrue(client.closed)
self.assertTrue(listener.closed)
def test_close_does_not_wait_for_owned_executor_workers(self) -> None:
executor = _ShutdownRecordingExecutor()
with patch("pi_dictate_service.server._DaemonExecutor", return_value=executor):
core = DictateServiceCore(model=FakeModel())
core.accepted("a")
core.handle_request(
"a", parse_request({"version": PROTOCOL_VERSION, "type": "hello", "requestId": "h"})
)
core.drain_events()
core.handle_request(
"a",
parse_request(
{
"version": PROTOCOL_VERSION,
"type": "start",
"requestId": "s",
"generationId": "g",
}
),
)
core.close()
self.assertEqual(executor.shutdown_args, (False, True))
def test_real_owned_executor_does_not_block_interpreter_exit(self) -> None:
program = """
import threading
import time
from pi_dictate_service.protocol import parse_request
from pi_dictate_service.server import DictateServiceCore
class StuckModel:
loaded = False
def load(self):
threading.Event().wait()
def transcribe(self, _samples):
return ''
core = DictateServiceCore(model=StuckModel())
core.accepted('a')
core.handle_request('a', parse_request({'version': 2, 'type': 'hello', 'requestId': 'h'}))
core.drain_events()
core.handle_request('a', parse_request({
'version': 2, 'type': 'start', 'requestId': 's', 'generationId': 'g'
}))
time.sleep(0.05)
core.close()
print('closed')
"""
result = subprocess.run(
[sys.executable, "-c", program],
text=True,
capture_output=True,
timeout=2,
check=False,
)
self.assertEqual(result.returncode, 0, result.stderr)
self.assertEqual(result.stdout.strip(), "closed")
@dataclass
class _FakeSelectorKey:
data: str | None
class _FakeSelector:
def __init__(self, client: "_FakeClientSocket") -> None:
self._events = [
[(_FakeSelectorKey(None), selectors.EVENT_READ)],
[(_FakeSelectorKey("client-1"), selectors.EVENT_READ | selectors.EVENT_WRITE)],
]
self.client = client
self.unregistered: list[object] = []
self.closed = False
def register(self, fileobj: object, _events: int, _data: object = None) -> None:
return None
def unregister(self, fileobj: object) -> None:
self.unregistered.append(fileobj)
def modify(self, fileobj: object, _events: int, _data: object = None) -> None:
if fileobj is self.client and self.client.closed:
raise OSError(errno.EBADF, "bad file descriptor")
def select(self, timeout: float | None = None) -> list[tuple[_FakeSelectorKey, int]]:
return self._events.pop(0)
def close(self) -> None:
self.closed = True
class _FakeClientSocket:
def __init__(self) -> None:
self.closed = False
self.send_calls = 0
self._read = True
def setblocking(self, _value: bool) -> None:
return None
def recv(self, _size: int) -> bytes:
if self._read:
self._read = False
return b"{broken}\n"
return b""
def send(self, _data: bytearray) -> int:
self.send_calls += 1
raise OSError(errno.EBADF, "bad file descriptor")
def close(self) -> None:
self.closed = True
class _FakeListener:
def __init__(self, client: _FakeClientSocket) -> None:
self.client = client
self.closed = False
self._accepted = False
def setblocking(self, _value: bool) -> None:
return None
def accept(self) -> tuple[_FakeClientSocket, object]:
if self._accepted:
raise BlockingIOError
self._accepted = True
return self.client, None
def close(self) -> None:
self.closed = True
class _FakeLoopCore:
def __init__(self, outbound: Outbound) -> None:
self.outbound = outbound
self.accepted_client: str | None = None
self._outbound_sent = False
self._exit_checks = 0
def should_exit(self) -> bool:
self._exit_checks += 1
return self._exit_checks >= 3
def poll(self) -> None:
return None
def drain_events(self) -> list[Outbound]:
if self.accepted_client is not None and not self._outbound_sent:
self._outbound_sent = True
return [self.outbound]
return []
def accepted(self, client_id: str) -> None:
self.accepted_client = client_id
def disconnect(self, _client_id: str) -> None:
return None
def close(self) -> None:
return None
class _ShutdownRecordingExecutor:
def __init__(self) -> None:
self.shutdown_args: tuple[bool, bool] | None = None
def submit(self, _function: object, *_args: object) -> Future[object]:
return Future()
def shutdown(self, *, wait: bool = True, cancel_futures: bool = False) -> None:
self.shutdown_args = (wait, cancel_futures)
if wait:
raise AssertionError("shutdown waited for a hung worker")
class IdleTests(CoreTestCase): class IdleTests(CoreTestCase):
def test_only_valid_requests_reset_idle_and_workers_suspend_then_restart_timer(self) -> None: def test_only_valid_requests_reset_idle_and_workers_suspend_then_restart_timer(self) -> None:
self.core.accepted("a") self.core.accepted("a")
+263
View File
@@ -0,0 +1,263 @@
from __future__ import annotations
import hashlib
import json
import os
from pathlib import Path
import subprocess
import tempfile
import unittest
from unittest.mock import patch
from pi_dictate_service.config import ConfigError
from pi_dictate_service.startup import validate_startup
REPOSITORY_ROOT = Path(__file__).resolve().parents[3]
SCRIPT = REPOSITORY_ROOT / "services/dictate/install.sh"
class SystemdProfileTests(unittest.TestCase):
def run_install(self, home: Path, fake_bin: Path, *arguments: str) -> subprocess.CompletedProcess[str]:
environment = {
**os.environ,
"HOME": str(home),
"XDG_CONFIG_HOME": str(home / ".config"),
"PATH": f"{fake_bin}:{os.environ['PATH']}",
}
return subprocess.run(
[str(SCRIPT), *arguments],
env=environment,
capture_output=True,
text=True,
check=False,
)
def make_fake_bin(self, root: Path) -> Path:
fake_bin = root / "bin"
fake_bin.mkdir()
uv = fake_bin / "uv"
# Match uv sync's contract so transactional environment swapping is
# exercised without installing dependencies.
uv.write_text(
"#!/bin/sh\nmkdir -p \"$UV_PROJECT_ENVIRONMENT\"\nexit 0\n",
encoding="utf-8",
)
uv.chmod(0o755)
systemctl = fake_bin / "systemctl"
systemctl.write_text("#!/bin/sh\nexit 0\n", encoding="utf-8")
systemctl.chmod(0o755)
return fake_bin
def test_generated_cloud_dropin_replaces_offline_network_restrictions(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
home = root / "home"
home.mkdir()
fake_bin = self.make_fake_bin(root)
key = root / "key"
key.write_text("test-key", encoding="utf-8")
result = self.run_install(
home, fake_bin, "install", "--backend", "openai", "--api-key-file", str(key)
)
self.assertEqual(result.returncode, 0, result.stderr)
dropin = (
home / ".config/systemd/user/pi-dictate.service.d/"
"10-pi-dictate-network.conf"
).read_text(encoding="utf-8")
self.assertIn("Environment=PI_DICTATE_NETWORK_MODE=cloud", dropin)
self.assertIn(
"RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6", dropin
)
self.assertIn("IPAddressDeny=", dropin)
self.assertNotIn("IPAddressDeny=any", dropin)
self.assertNotIn("HF_HUB_OFFLINE=1", dropin)
self.assertIn(
f'BindReadOnlyPaths="{home}/.config/pi-dictate/credentials/openai-api-key"',
dropin,
)
self.assertIn(
f'BindReadOnlyPaths="{home}/.local/share/pi-dictate/config/coding-terms.txt"',
dropin,
)
def test_check_rejects_tampered_network_dropin(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
home = root / "home"
home.mkdir()
fake_bin = self.make_fake_bin(root)
key = root / "key"
key.write_text("test-key", encoding="utf-8")
result = self.run_install(
home, fake_bin, "install", "--backend", "meta", "--api-key-file", str(key)
)
self.assertEqual(result.returncode, 0, result.stderr)
dropin = (
home / ".config/systemd/user/pi-dictate.service.d/"
"10-pi-dictate-network.conf"
)
with dropin.open("a", encoding="utf-8") as stream:
stream.write("IPAddressAllow=any\\n")
result = self.run_install(home, fake_bin, "check")
self.assertNotEqual(result.returncode, 0)
self.assertIn("expected managed policy", result.stderr)
def test_startup_requires_the_exact_validation_command(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
home = root / "home"
home.mkdir()
fake_bin = self.make_fake_bin(root)
key = root / "key"
key.write_text("test-key", encoding="utf-8")
result = self.run_install(
home, fake_bin, "install", "--backend", "meta", "--api-key-file", str(key)
)
self.assertEqual(result.returncode, 0, result.stderr)
environment = {
"HOME": str(home),
"XDG_CONFIG_HOME": str(home / ".config"),
"PI_DICTATE_CONFIG_PATH": str(home / ".config/pi-dictate/config.json"),
"PI_DICTATE_DEPENDENCY_PROFILE_PATH": str(
home / ".local/share/pi-dictate/.dependency-profile"
),
"PI_DICTATE_NETWORK_MODE": "cloud",
}
with patch.dict(os.environ, environment, clear=False):
validate_startup()
service = home / ".config/systemd/user/pi-dictate.service"
text = service.read_text(encoding="utf-8")
service.write_text(
"\n".join(
"ExecStartPre=/bin/true" if line.startswith("ExecStartPre=") else line
for line in text.splitlines()
) + "\n",
encoding="utf-8",
)
with self.assertRaisesRegex(ConfigError, "hardening"):
validate_startup()
def test_startup_rejects_a_world_writable_socket_unit(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
home = root / "home"
home.mkdir()
fake_bin = self.make_fake_bin(root)
key = root / "key"
key.write_text("test-key", encoding="utf-8")
result = self.run_install(
home, fake_bin, "install", "--backend", "meta", "--api-key-file", str(key)
)
self.assertEqual(result.returncode, 0, result.stderr)
environment = {
"HOME": str(home),
"XDG_CONFIG_HOME": str(home / ".config"),
"PI_DICTATE_CONFIG_PATH": str(home / ".config/pi-dictate/config.json"),
"PI_DICTATE_DEPENDENCY_PROFILE_PATH": str(
home / ".local/share/pi-dictate/.dependency-profile"
),
"PI_DICTATE_NETWORK_MODE": "cloud",
}
socket_unit = home / ".config/systemd/user/pi-dictate.socket"
socket_unit.write_text(
socket_unit.read_text(encoding="utf-8").replace(
"SocketMode=0600", "SocketMode=0666"
),
encoding="utf-8",
)
with patch.dict(os.environ, environment, clear=False):
with self.assertRaisesRegex(ConfigError, "socket systemd policy"):
validate_startup()
def test_local_dropin_binds_custom_model_and_dictionary_paths(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
home = root / "home"
home.mkdir()
fake_bin = self.make_fake_bin(root)
source = root / "source.nemo"
source.write_bytes(b"model")
digest = hashlib.sha256(source.read_bytes()).hexdigest()
result = self.run_install(
home,
fake_bin,
"install",
"--backend",
"local",
"--model-source",
str(source),
"--model-sha256",
digest,
)
self.assertEqual(result.returncode, 0, result.stderr)
custom_model = home / "custom model.nemo"
custom_terms = home / "custom terms.txt"
custom_model.write_bytes(source.read_bytes())
custom_terms.write_text("JSON\n", encoding="utf-8")
config_path = home / ".config/pi-dictate/config.json"
config = json.loads(config_path.read_text(encoding="utf-8"))
config["local"]["modelPath"] = str(custom_model)
config["local"]["termsPath"] = str(custom_terms)
config_path.write_text(json.dumps(config), encoding="utf-8")
config_path.chmod(0o600)
result = self.run_install(home, fake_bin, "configure", "--backend", "local")
self.assertEqual(result.returncode, 0, result.stderr)
dropin = (home / ".config/systemd/user/pi-dictate.service.d/10-pi-dictate-network.conf").read_text(encoding="utf-8")
self.assertIn(f'BindReadOnlyPaths="{custom_model}"', dropin)
self.assertIn(f'BindReadOnlyPaths="{custom_terms}"', dropin)
environment = {
"HOME": str(home),
"XDG_CONFIG_HOME": str(home / ".config"),
"PI_DICTATE_CONFIG_PATH": str(config_path),
"PI_DICTATE_DEPENDENCY_PROFILE_PATH": str(
home / ".local/share/pi-dictate/.dependency-profile"
),
"PI_DICTATE_NETWORK_MODE": "offline",
}
with patch.dict(os.environ, environment, clear=False):
validate_startup()
def test_local_dropin_and_profile_deny_are_restrictive(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
home = root / "home"
home.mkdir()
fake_bin = self.make_fake_bin(root)
model = root / "model"
model.write_bytes(b"model")
# Dry-run keeps this test independent of the pinned model bytes.
result = self.run_install(
home,
fake_bin,
"install",
"--backend",
"local",
"--model-source",
str(model),
"--model-sha256",
hashlib.sha256(model.read_bytes()).hexdigest(),
"--dry-run",
)
self.assertEqual(result.returncode, 0, result.stderr)
self.assertIn("offline network hardening", result.stdout)
profile = json.loads(
(REPOSITORY_ROOT / "profiles/nono/pi-local.json").read_text(
encoding="utf-8"
)
)
self.assertIn("$HOME/.local/share/pi-dictate", profile["filesystem"]["deny"])
# User decision: the profile is intentionally unchanged from HEAD.
self.assertNotIn("$HOME/.config/pi-dictate", profile["filesystem"]["deny"])
if __name__ == "__main__":
unittest.main()
+19 -1
View File
@@ -1,12 +1,21 @@
import hashlib
from pathlib import Path from pathlib import Path
import unittest import unittest
from pi_dictate_service.startup import EXPECTED_BASE_UNIT_SHA256
REPOSITORY_ROOT = Path(__file__).resolve().parents[3] REPOSITORY_ROOT = Path(__file__).resolve().parents[3]
UNIT_DIR = REPOSITORY_ROOT / "systemd/user" UNIT_DIR = REPOSITORY_ROOT / "systemd/user"
class SystemdUnitTests(unittest.TestCase): class SystemdUnitTests(unittest.TestCase):
def test_startup_hash_matches_the_checked_in_service_unit(self) -> None:
self.assertEqual(
hashlib.sha256((UNIT_DIR / "pi-dictate.service").read_bytes()).hexdigest(),
EXPECTED_BASE_UNIT_SHA256,
)
def test_socket_unit_owns_the_exact_private_runtime_path(self) -> None: def test_socket_unit_owns_the_exact_private_runtime_path(self) -> None:
socket_unit = (UNIT_DIR / "pi-dictate.socket").read_text(encoding="utf-8") socket_unit = (UNIT_DIR / "pi-dictate.socket").read_text(encoding="utf-8")
self.assertIn("ListenStream=%t/pi-dictate.sock", socket_unit) self.assertIn("ListenStream=%t/pi-dictate.sock", socket_unit)
@@ -24,11 +33,20 @@ class SystemdUnitTests(unittest.TestCase):
"TRANSFORMERS_OFFLINE=1", "TRANSFORMERS_OFFLINE=1",
"TRITON_CACHE_DIR=/tmp/pi-dictate-triton", "TRITON_CACHE_DIR=/tmp/pi-dictate-triton",
"NUMBA_CACHE_DIR=/tmp/pi-dictate-numba", "NUMBA_CACHE_DIR=/tmp/pi-dictate-numba",
"TimeoutStopSec=5s",
"LimitCORE=0", "LimitCORE=0",
"MemorySwapMax=0", "MemorySwapMax=0",
"ProtectSystem=strict", "ProtectSystem=strict",
"ProtectHome=read-only", "ProtectHome=tmpfs",
"BindReadOnlyPaths=%h/.local/share/pi-dictate",
"BindReadOnlyPaths=%h/.local/share/uv/python",
"BindReadOnlyPaths=%h/.config/pi-dictate/config.json",
"BindReadOnlyPaths=%h/.config/systemd/user/pi-dictate.service",
"BindReadOnlyPaths=%h/.config/systemd/user/pi-dictate.service.d",
"BindReadOnlyPaths=%h/.config/systemd/user/pi-dictate.socket",
"BindReadOnlyPaths=%t/pipewire-0",
"RestrictAddressFamilies=AF_UNIX", "RestrictAddressFamilies=AF_UNIX",
"SystemCallFilter=~bind listen",
"IPAddressDeny=any", "IPAddressDeny=any",
): ):
self.assertIn(expected, service_unit) self.assertIn(expected, service_unit)
+12 -4
View File
@@ -173,6 +173,10 @@ class UnixLoopTests(unittest.TestCase):
class ActivationTests(unittest.TestCase): class ActivationTests(unittest.TestCase):
def test_inherited_listener_requires_one_fd_for_this_process_and_wraps_fd_three(self) -> None: def test_inherited_listener_requires_one_fd_for_this_process_and_wraps_fd_three(self) -> None:
listener = Mock() listener = Mock()
listener.family = socket.AF_UNIX
listener.type = socket.SOCK_STREAM
listener.getsockopt.return_value = 1
listener.getsockname.return_value = "/run/user/1000/pi-dictate.sock"
with patch("pi_dictate_service.server.os.getpid", return_value=42), patch( with patch("pi_dictate_service.server.os.getpid", return_value=42), patch(
"pi_dictate_service.server.socket.socket", return_value=listener "pi_dictate_service.server.socket.socket", return_value=listener
) as socket_factory: ) as socket_factory:
@@ -189,7 +193,9 @@ class ActivationTests(unittest.TestCase):
inherited_listener({"LISTEN_PID": "41", "LISTEN_FDS": "1"}) inherited_listener({"LISTEN_PID": "41", "LISTEN_FDS": "1"})
def test_main_requires_local_model_path_and_reads_idle_override(self) -> None: def test_main_requires_local_model_path_and_reads_idle_override(self) -> None:
with patch.dict(os.environ, {}, clear=True): with patch.dict(os.environ, {}, clear=True), patch(
"pi_dictate_service.server.os.path.isfile", return_value=False
):
with self.assertRaisesRegex(SystemExit, "PI_DICTATE_MODEL_PATH"): with self.assertRaisesRegex(SystemExit, "PI_DICTATE_MODEL_PATH"):
main() main()
@@ -201,10 +207,12 @@ class ActivationTests(unittest.TestCase):
"PI_DICTATE_MODEL_PATH": "/models/parakeet.nemo", "PI_DICTATE_MODEL_PATH": "/models/parakeet.nemo",
"PI_DICTATE_TERMS_PATH": "/config/coding-terms.txt", "PI_DICTATE_TERMS_PATH": "/config/coding-terms.txt",
"PI_DICTATE_BOOST_ALPHA": "0.5", "PI_DICTATE_BOOST_ALPHA": "0.5",
"PI_DICTATE_IDLE_SECONDS": "12.5", "PI_DICTATE_IDLE_SECONDS": "120",
}, },
clear=True, clear=True,
), patch("pi_dictate_service.server.NemoParakeetModel", return_value=model) as adapter, patch( ), patch("pi_dictate_service.server.os.path.isfile", return_value=False), patch(
"pi_dictate_service.server.NemoParakeetModel", return_value=model
) as adapter, patch(
"pi_dictate_service.server.DictateServiceCore" "pi_dictate_service.server.DictateServiceCore"
) as core_factory, patch( ) as core_factory, patch(
"pi_dictate_service.server.inherited_listener", return_value=listener "pi_dictate_service.server.inherited_listener", return_value=listener
@@ -217,7 +225,7 @@ class ActivationTests(unittest.TestCase):
terms_path="/config/coding-terms.txt", terms_path="/config/coding-terms.txt",
boost_alpha=0.5, boost_alpha=0.5,
) )
core_factory.assert_called_once_with(model=model, idle_seconds=12.5) core_factory.assert_called_once_with(model=model, idle_seconds=120)
serve_loop.assert_called_once_with(listener, core) serve_loop.assert_called_once_with(listener, core)
+31 -4
View File
@@ -1328,7 +1328,12 @@ wheels = [
name = "pi-dictate-service" name = "pi-dictate-service"
version = "0.1.0" version = "0.1.0"
source = { editable = "." } source = { editable = "." }
dependencies = [
[package.optional-dependencies]
cloud = [
{ name = "websockets" },
]
local = [
{ name = "nemo-toolkit", extra = ["asr"] }, { name = "nemo-toolkit", extra = ["asr"] },
{ name = "numpy" }, { name = "numpy" },
{ name = "torch" }, { name = "torch" },
@@ -1336,10 +1341,12 @@ dependencies = [
[package.metadata] [package.metadata]
requires-dist = [ requires-dist = [
{ name = "nemo-toolkit", extras = ["asr"], specifier = "==3.0.0" }, { name = "nemo-toolkit", extras = ["asr"], marker = "extra == 'local'", specifier = "==3.0.0" },
{ name = "numpy", specifier = "==2.5.2" }, { name = "numpy", marker = "extra == 'local'", specifier = "==2.5.2" },
{ name = "torch", specifier = "==2.13.0" }, { name = "torch", marker = "extra == 'local'", specifier = "==2.13.0" },
{ name = "websockets", marker = "extra == 'cloud'", specifier = "==15.0.1" },
] ]
provides-extras = ["local", "cloud"]
[[package]] [[package]]
name = "pillow" name = "pillow"
@@ -2136,6 +2143,26 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/d9/00/aca6beb3658dab4ed3dbb41a78e6e7f31342e0b41d28088f205525751601/webdataset-1.0.2-py3-none-any.whl", hash = "sha256:3dbfced32b25c0d199c6b9787937b6f85742bc3c84f652c846893075c1c082d9", size = 74956, upload-time = "2025-06-19T23:26:20.354Z" }, { url = "https://files.pythonhosted.org/packages/d9/00/aca6beb3658dab4ed3dbb41a78e6e7f31342e0b41d28088f205525751601/webdataset-1.0.2-py3-none-any.whl", hash = "sha256:3dbfced32b25c0d199c6b9787937b6f85742bc3c84f652c846893075c1c082d9", size = 74956, upload-time = "2025-06-19T23:26:20.354Z" },
] ]
[[package]]
name = "websockets"
version = "15.0.1"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/21/e6/26d09fab466b7ca9c7737474c52be4f76a40301b08362eb2dbc19dcc16c1/websockets-15.0.1.tar.gz", hash = "sha256:82544de02076bafba038ce055ee6412d68da13ab47f0c60cab827346de828dee", size = 177016, upload-time = "2025-03-05T20:03:41.606Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/51/6b/4545a0d843594f5d0771e86463606a3988b5a09ca5123136f8a76580dd63/websockets-15.0.1-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:3e90baa811a5d73f3ca0bcbf32064d663ed81318ab225ee4f427ad4e26e5aff3", size = 175437, upload-time = "2025-03-05T20:02:16.706Z" },
{ url = "https://files.pythonhosted.org/packages/f4/71/809a0f5f6a06522af902e0f2ea2757f71ead94610010cf570ab5c98e99ed/websockets-15.0.1-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:592f1a9fe869c778694f0aa806ba0374e97648ab57936f092fd9d87f8bc03665", size = 173096, upload-time = "2025-03-05T20:02:18.832Z" },
{ url = "https://files.pythonhosted.org/packages/3d/69/1a681dd6f02180916f116894181eab8b2e25b31e484c5d0eae637ec01f7c/websockets-15.0.1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:0701bc3cfcb9164d04a14b149fd74be7347a530ad3bbf15ab2c678a2cd3dd9a2", size = 173332, upload-time = "2025-03-05T20:02:20.187Z" },
{ url = "https://files.pythonhosted.org/packages/a6/02/0073b3952f5bce97eafbb35757f8d0d54812b6174ed8dd952aa08429bcc3/websockets-15.0.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:e8b56bdcdb4505c8078cb6c7157d9811a85790f2f2b3632c7d1462ab5783d215", size = 183152, upload-time = "2025-03-05T20:02:22.286Z" },
{ url = "https://files.pythonhosted.org/packages/74/45/c205c8480eafd114b428284840da0b1be9ffd0e4f87338dc95dc6ff961a1/websockets-15.0.1-cp312-cp312-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:0af68c55afbd5f07986df82831c7bff04846928ea8d1fd7f30052638788bc9b5", size = 182096, upload-time = "2025-03-05T20:02:24.368Z" },
{ url = "https://files.pythonhosted.org/packages/14/8f/aa61f528fba38578ec553c145857a181384c72b98156f858ca5c8e82d9d3/websockets-15.0.1-cp312-cp312-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:64dee438fed052b52e4f98f76c5790513235efaa1ef7f3f2192c392cd7c91b65", size = 182523, upload-time = "2025-03-05T20:02:25.669Z" },
{ url = "https://files.pythonhosted.org/packages/ec/6d/0267396610add5bc0d0d3e77f546d4cd287200804fe02323797de77dbce9/websockets-15.0.1-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:d5f6b181bb38171a8ad1d6aa58a67a6aa9d4b38d0f8c5f496b9e42561dfc62fe", size = 182790, upload-time = "2025-03-05T20:02:26.99Z" },
{ url = "https://files.pythonhosted.org/packages/02/05/c68c5adbf679cf610ae2f74a9b871ae84564462955d991178f95a1ddb7dd/websockets-15.0.1-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:5d54b09eba2bada6011aea5375542a157637b91029687eb4fdb2dab11059c1b4", size = 182165, upload-time = "2025-03-05T20:02:30.291Z" },
{ url = "https://files.pythonhosted.org/packages/29/93/bb672df7b2f5faac89761cb5fa34f5cec45a4026c383a4b5761c6cea5c16/websockets-15.0.1-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:3be571a8b5afed347da347bfcf27ba12b069d9d7f42cb8c7028b5e98bbb12597", size = 182160, upload-time = "2025-03-05T20:02:31.634Z" },
{ url = "https://files.pythonhosted.org/packages/ff/83/de1f7709376dc3ca9b7eeb4b9a07b4526b14876b6d372a4dc62312bebee0/websockets-15.0.1-cp312-cp312-win32.whl", hash = "sha256:c338ffa0520bdb12fbc527265235639fb76e7bc7faafbb93f6ba80d9c06578a9", size = 176395, upload-time = "2025-03-05T20:02:33.017Z" },
{ url = "https://files.pythonhosted.org/packages/7d/71/abf2ebc3bbfa40f391ce1428c7168fb20582d0ff57019b69ea20fa698043/websockets-15.0.1-cp312-cp312-win_amd64.whl", hash = "sha256:fcd5cf9e305d7b8338754470cf69cf81f420459dbae8a3b40cee57417f4614a7", size = 176841, upload-time = "2025-03-05T20:02:34.498Z" },
{ url = "https://files.pythonhosted.org/packages/fa/a8/5b41e0da817d64113292ab1f8247140aac61cbf6cfd085d6a0fa77f4984f/websockets-15.0.1-py3-none-any.whl", hash = "sha256:f7a866fbc1e97b5c617ee4116daaa09b722101d4a3c170c787450ba409f9736f", size = 169743, upload-time = "2025-03-05T20:03:39.41Z" },
]
[[package]] [[package]]
name = "werkzeug" name = "werkzeug"
version = "3.1.8" version = "3.1.8"
+8 -6
View File
@@ -1,6 +1,6 @@
--- ---
name: artifact-presenter name: artifact-presenter
description: Present local artifacts for review: use Browser Harness for interactive web artifacts, especially HTML, and mcat for non-web files, Markdown, and static source. description: Present local artifacts for review: use Browser Harness for interactive web artifacts, Orca for editor-friendly text in Orca-managed sessions, and mcat for other non-web files.
--- ---
# Artifact presenter # Artifact presenter
@@ -26,7 +26,7 @@ Keep ordinary explanations, code diffs, logs, multi-file comparisons, and user-r
Choose the presentation path: Choose the presentation path:
- **Interactive web artifacts, especially `.html`:** use the Browser Harness/Browser Use-backed workflow. - **Interactive web artifacts, especially `.html`:** use the Browser Harness/Browser Use-backed workflow.
- **Non-web files, Markdown, and static source:** use mcat in the reusable Herdr pane. - **Non-web artifacts:** run this skill's presenter helper. It opens editor-friendly UTF-8 text directly in Orca when the harness is running in an Orca-managed terminal; otherwise it uses mcat in the reusable Herdr pane. Binary and media files stay on the mcat path.
Completion criterion: select exactly one existing or generated artifact path before presentation. Completion criterion: select exactly one existing or generated artifact path before presentation.
@@ -46,7 +46,7 @@ Load and follow the [`browser-harness`](../browser-harness/SKILL.md) skill. Open
Completion criterion: the active browser target reports the local artifact URL and page title. Completion criterion: the active browser target reports the local artifact URL and page title.
## Present non-web artifacts with mcat ## Present non-web artifacts
Run the helper using its absolute path resolved from this skill directory: Run the helper using its absolute path resolved from this skill directory:
@@ -54,9 +54,11 @@ Run the helper using its absolute path resolved from this skill directory:
python3 <skill-directory>/scripts/present.py <artifact-path> python3 <skill-directory>/scripts/present.py <artifact-path>
``` ```
The helper reuses the pane labeled `artifact presenter` in the current Herdr workspace. It serializes pane discovery, creation, and replacement per workspace, with an additional per-pane lock during viewer handoff. When that pane is already displaying an artifact, it closes the active `mcat` viewer and waits for the pane shell before launching the replacement through a shell-neutral helper command with encoded path arguments. It confirms the helper started the viewer rather than trusting terminal input delivery alone. It leaves unrelated foreground programs untouched and creates a new viewer pane instead. When no reusable pane exists, it creates a right split without taking focus. It launches `mcat <artifact-path>` so mcat can infer and display documents, source, archives, PDFs, images, and videos. It falls back to artifact-only delivery when Herdr or mcat is unavailable. The helper detects an Orca-managed harness from Orca's terminal environment markers (`TERM_PROGRAM=Orca` or scoped `ORCA_*` workspace, worktree, pane, or terminal IDs). For editor-friendly UTF-8 text, it resolves the session's version-appropriate CLI and runs `orca file open <artifact-path> --json`. Orca only opens paths inside its selected worktree, so a rejected path falls through to the normal non-web fallback.
Completion criterion: capture whether the helper opened, reused, or skipped the viewer pane. The fallback reuses the pane labeled `artifact presenter` in the current Herdr workspace. It serializes pane discovery, creation, and replacement per workspace, with an additional per-pane lock during viewer handoff. When that pane is already displaying an artifact, it closes the active `mcat` viewer and waits for the pane shell before launching the replacement through a shell-neutral helper command with encoded path arguments. It confirms the helper started the viewer rather than trusting terminal input delivery alone. It leaves unrelated foreground programs untouched and creates a new viewer pane instead. When no reusable pane exists, it creates a right split without taking focus. It launches `mcat <artifact-path>` so mcat can infer and display documents, source, archives, PDFs, images, and videos. It falls back to artifact-only delivery when neither Orca nor the Herdr/mcat path can present the file.
Completion criterion: capture whether Orca opened the file, the helper opened or reused an mcat pane, or presentation fell back to the artifact path only.
## Convert only when requested ## Convert only when requested
@@ -77,6 +79,6 @@ Return:
- a one- to three-sentence result summary; - a one- to three-sentence result summary;
- the artifact path; - the artifact path;
- for interactive web artifacts, that Browser Harness presented the artifact and verified its URL/title; - for interactive web artifacts, that Browser Harness presented the artifact and verified its URL/title;
- for other artifacts, whether the reusable mcat pane was opened or reused, or why presentation fell back to the file only. - for other artifacts, whether Orca opened the file, the reusable mcat pane was opened or reused, or why presentation fell back to the file only.
Do not repeat the complete artifact in chat. Honor a later request to paste any section inline. Do not repeat the complete artifact in chat. Honor a later request to paste any section inline.
@@ -1,5 +1,5 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
"""Open one review artifact in a reusable right-side Herdr mcat pane.""" """Present one review artifact through Orca or a reusable Herdr mcat pane."""
from __future__ import annotations from __future__ import annotations
@@ -22,6 +22,13 @@ from typing import Callable, Mapping, Sequence
PANE_LABEL = "artifact presenter" PANE_LABEL = "artifact presenter"
VIEWER_PROCESS_NAME = "mcat" VIEWER_PROCESS_NAME = "mcat"
TEXT_SAMPLE_BYTES = 8192
ORCA_SESSION_ENVIRONMENT_KEYS = (
"ORCA_WORKSPACE_ID",
"ORCA_WORKTREE_ID",
"ORCA_PANE_KEY",
"ORCA_TERMINAL_HANDLE",
)
PANE_READY_ATTEMPTS = 40 PANE_READY_ATTEMPTS = 40
PANE_READY_DELAY_SECONDS = 0.05 PANE_READY_DELAY_SECONDS = 0.05
PANE_SHELL_STABLE_CHECKS = 2 PANE_SHELL_STABLE_CHECKS = 2
@@ -57,9 +64,66 @@ class PresentationResult:
def message(self) -> str: def message(self) -> str:
if self.status == "artifact-only": if self.status == "artifact-only":
return f"artifact-only: {self.reason}; {self.artifact}" return f"artifact-only: {self.reason}; {self.artifact}"
if self.status == "orca-opened":
return f"orca-opened: {self.artifact}"
return f"{self.status}: pane {self.pane_id}; {self.artifact}" return f"{self.status}: pane {self.pane_id}; {self.artifact}"
def is_orca_managed_session(env: Mapping[str, str]) -> bool:
"""Return whether this process belongs to an Orca-managed terminal session."""
if env.get("TERM_PROGRAM", "").casefold() == "orca":
return True
return any(env.get(key) for key in ORCA_SESSION_ENVIRONMENT_KEYS)
def is_editor_friendly_text(artifact: Path) -> bool:
"""Recognize UTF-8 text without maintaining a source-extension allowlist."""
with artifact.open("rb") as handle:
sample = handle.read(TEXT_SAMPLE_BYTES)
if b"\0" in sample:
return False
try:
sample.decode("utf-8")
except UnicodeDecodeError:
return False
return True
def orca_executable(
env: Mapping[str, str],
which: FindExecutable,
) -> str | None:
configured = env.get("ORCA_CLI_COMMAND")
if configured:
return which(configured)
if env.get("ORCA_DEV_REPO_ROOT"):
return which("orca-dev")
return which("orca")
def open_in_orca(
orca: str,
artifact: Path,
run: RunCommand,
) -> PresentationResult:
completed = run(
[orca, "file", "open", str(artifact), "--json"],
check=True,
capture_output=True,
text=True,
)
response = json.loads(completed.stdout)
result = response.get("result") if isinstance(response, dict) else None
if (
not isinstance(response, dict)
or response.get("ok") is not True
or not isinstance(result, dict)
or result.get("opened") is not True
):
raise ValueError("Orca did not confirm that it opened the file")
return PresentationResult("orca-opened", artifact)
def run_json( def run_json(
executable: str, executable: str,
args: Sequence[str], args: Sequence[str],
@@ -345,6 +409,14 @@ def present_document(
if not artifact.is_file(): if not artifact.is_file():
raise ValueError(f"artifact is not a file: {artifact}") raise ValueError(f"artifact is not a file: {artifact}")
if is_orca_managed_session(env) and is_editor_friendly_text(artifact):
orca = orca_executable(env, which)
if orca:
try:
return open_in_orca(orca, artifact, run)
except (json.JSONDecodeError, OSError, subprocess.SubprocessError, ValueError):
pass
herdr = which("herdr") herdr = which("herdr")
mcat = which("mcat") mcat = which("mcat")
workspace_id = env.get("HERDR_WORKSPACE_ID") workspace_id = env.get("HERDR_WORKSPACE_ID")
@@ -120,6 +120,22 @@ class FakeRunner:
return subprocess.CompletedProcess(argv, 0, json.dumps(output), "") return subprocess.CompletedProcess(argv, 0, json.dumps(output), "")
class OrcaAwareRunner(FakeRunner):
def __init__(self, *args, fail_orca_open: bool = False, **kwargs) -> None:
super().__init__(*args, **kwargs)
self.fail_orca_open = fail_orca_open
def __call__(self, command, **kwargs):
argv = [str(value) for value in command]
if argv[1:3] == ["file", "open"]:
self.calls.append(argv)
if self.fail_orca_open:
raise subprocess.CalledProcessError(1, argv)
output = {"ok": True, "result": {"opened": True}}
return subprocess.CompletedProcess(argv, 0, json.dumps(output), "")
return super().__call__(command, **kwargs)
class ArtifactPresenterTests(unittest.TestCase): class ArtifactPresenterTests(unittest.TestCase):
def setUp(self) -> None: def setUp(self) -> None:
self.tempdir = tempfile.TemporaryDirectory() self.tempdir = tempfile.TemporaryDirectory()
@@ -132,6 +148,7 @@ class ArtifactPresenterTests(unittest.TestCase):
self.executables = { self.executables = {
"herdr": "/usr/bin/herdr", "herdr": "/usr/bin/herdr",
"mcat": "/usr/bin/mcat", "mcat": "/usr/bin/mcat",
"orca": "/usr/bin/orca",
} }
def tearDown(self) -> None: def tearDown(self) -> None:
@@ -381,21 +398,71 @@ class ArtifactPresenterTests(unittest.TestCase):
) )
self.assertFalse(any(call[1:3] == ["pane", "split"] for call in runner.calls)) self.assertFalse(any(call[1:3] == ["pane", "split"] for call in runner.calls))
def test_presents_non_markdown_file_directly_with_mcat(self) -> None: def test_opens_editor_friendly_file_in_orca_managed_session(self) -> None:
video = self.artifact.with_name("review.mp4") source = self.artifact.with_name("module.ts")
video.write_bytes(b"video fixture") source.write_text("export const answer = 42;\n", encoding="utf-8")
runner = FakeRunner( runner = OrcaAwareRunner()
result = MODULE.present_document(
source,
env={**self.env, "ORCA_WORKSPACE_ID": "workspace"},
which=self.which,
run=runner,
)
self.assertEqual(result.status, "orca-opened")
self.assertEqual(
runner.calls,
[["/usr/bin/orca", "file", "open", str(source.resolve()), "--json"]],
)
def test_uses_configured_orca_cli_command(self) -> None:
self.executables["orca-wsl"] = "/opt/orca/orca-wsl"
runner = OrcaAwareRunner()
result = MODULE.present_document(
self.artifact,
env={"TERM_PROGRAM": "Orca", "ORCA_CLI_COMMAND": "orca-wsl"},
which=self.which,
run=runner,
)
self.assertEqual(result.status, "orca-opened")
self.assertEqual(runner.calls[0][0], "/opt/orca/orca-wsl")
def test_falls_back_to_mcat_when_orca_cannot_open_text_file(self) -> None:
runner = OrcaAwareRunner(
[{"pane_id": "workspace:p-viewer", "label": MODULE.PANE_LABEL}], [{"pane_id": "workspace:p-viewer", "label": MODULE.PANE_LABEL}],
fail_orca_open=True,
) )
result = MODULE.present_document( result = MODULE.present_document(
video, self.artifact,
env=self.env, env={**self.env, "ORCA_TERMINAL_HANDLE": "term-1"},
which=self.which, which=self.which,
run=runner, run=runner,
) )
self.assertEqual(result.status, "reused") self.assertEqual(result.status, "reused")
self.assertEqual(runner.calls[0][1:3], ["file", "open"])
self.mcat_run_call(runner, "workspace:p-viewer")
def test_presents_binary_file_directly_with_mcat_in_orca_session(self) -> None:
video = self.artifact.with_name("review.mp4")
video.write_bytes(b"video\0fixture")
runner = OrcaAwareRunner(
[{"pane_id": "workspace:p-viewer", "label": MODULE.PANE_LABEL}],
)
result = MODULE.present_document(
video,
env={**self.env, "TERM_PROGRAM": "Orca"},
which=self.which,
run=runner,
)
self.assertEqual(result.status, "reused")
self.assertFalse(any(call[1:3] == ["file", "open"] for call in runner.calls))
self.mcat_run_call(runner, "workspace:p-viewer", video) self.mcat_run_call(runner, "workspace:p-viewer", video)
def test_encodes_artifact_paths_before_terminal_delivery(self) -> None: def test_encodes_artifact_paths_before_terminal_delivery(self) -> None:
+26 -2
View File
@@ -1,12 +1,19 @@
# Managed by pi-dictate install.sh # Managed by pi-dictate install.sh
[Unit] [Unit]
Description=Pi local Parakeet dictation service Description=Pi dictation transcription service
Requires=pi-dictate.socket Requires=pi-dictate.socket
After=pipewire.service After=pipewire.service
[Service] [Service]
Type=simple Type=simple
TimeoutStopSec=5s
ExecStart=%h/.local/share/pi-dictate/venv/bin/pi-dictate-service ExecStart=%h/.local/share/pi-dictate/venv/bin/pi-dictate-service
Environment=PI_DICTATE_CONFIG_PATH=%h/.config/pi-dictate/config.json
Environment=PI_DICTATE_DEPENDENCY_PROFILE_PATH=%h/.local/share/pi-dictate/.dependency-profile
# local/offline is the fail-closed base. The installer-owned cloud drop-in
# replaces these values only after a cloud configuration and dependency profile
# have been installed.
Environment=PI_DICTATE_NETWORK_MODE=offline
Environment=PI_DICTATE_MODEL_PATH=%h/.local/share/pi-dictate/models/parakeet-tdt-0.6b-v3/parakeet-tdt-0.6b-v3.nemo Environment=PI_DICTATE_MODEL_PATH=%h/.local/share/pi-dictate/models/parakeet-tdt-0.6b-v3/parakeet-tdt-0.6b-v3.nemo
Environment=PI_DICTATE_TERMS_PATH=%h/.local/share/pi-dictate/config/coding-terms.txt Environment=PI_DICTATE_TERMS_PATH=%h/.local/share/pi-dictate/config/coding-terms.txt
Environment=PI_DICTATE_BOOST_ALPHA=1.0 Environment=PI_DICTATE_BOOST_ALPHA=1.0
@@ -18,6 +25,13 @@ Environment=HF_HOME=/tmp/pi-dictate-huggingface
Environment=TRITON_CACHE_DIR=/tmp/pi-dictate-triton Environment=TRITON_CACHE_DIR=/tmp/pi-dictate-triton
Environment=NUMBA_CACHE_DIR=/tmp/pi-dictate-numba Environment=NUMBA_CACHE_DIR=/tmp/pi-dictate-numba
Environment=PYTHONDONTWRITEBYTECODE=1 Environment=PYTHONDONTWRITEBYTECODE=1
# Never inherit ambient proxy credentials or route audio through an unreviewed proxy.
Environment=HTTP_PROXY=
Environment=HTTPS_PROXY=
Environment=ALL_PROXY=
Environment=http_proxy=
Environment=https_proxy=
Environment=all_proxy=
UMask=0077 UMask=0077
LimitCORE=0 LimitCORE=0
MemorySwapMax=0 MemorySwapMax=0
@@ -29,9 +43,19 @@ ProtectKernelLogs=yes
ProtectKernelModules=yes ProtectKernelModules=yes
ProtectKernelTunables=yes ProtectKernelTunables=yes
ProtectSystem=strict ProtectSystem=strict
ProtectHome=read-only ProtectHome=tmpfs
BindReadOnlyPaths=%h/.local/share/pi-dictate
BindReadOnlyPaths=%h/.local/share/uv/python
BindReadOnlyPaths=%h/.config/pi-dictate/config.json
BindReadOnlyPaths=%h/.config/systemd/user/pi-dictate.service
BindReadOnlyPaths=%h/.config/systemd/user/pi-dictate.service.d
BindReadOnlyPaths=%h/.config/systemd/user/pi-dictate.socket
BindReadOnlyPaths=%t/pipewire-0
RestrictAddressFamilies=AF_UNIX RestrictAddressFamilies=AF_UNIX
SystemCallFilter=~bind listen
IPAddressDeny=any IPAddressDeny=any
LockPersonality=yes LockPersonality=yes
# Refuse a service/config/dependency/network-mode or hardening mismatch before runtime code starts.
ExecStartPre=%h/.local/share/pi-dictate/venv/bin/python -c 'from pi_dictate_service.startup import validate_startup; validate_startup()'
# The socket unit owns %t/pi-dictate.sock. The service must not unlink it. # The socket unit owns %t/pi-dictate.sock. The service must not unlink it.