feat(dictate): support configurable ASR backends
This commit is contained in:
@@ -11,7 +11,7 @@ Personal [Pi](https://github.com/earendil-works/pi-mono) extensions, skills, glo
|
||||
- the `manage-pi-links` and `nono-profiles` project-only skills under `.pi/skills/`;
|
||||
- no global Pi extension symlinks. Active extensions are installed separately as pinned Gitea npm packages.
|
||||
|
||||
The repository contains `copy-all`, `dictate`, `effort-controls`, `file-search`, `interactive-subagents`, `lm-studio`, `native-hooks`, `prompt-snippets`, `subagents`, `todo-herdr`, `update`, and the private `workflow` extension. The link manifest does not install any of them globally. The `pptx`, `stop-slop`, `subagents`, and `xlsx` skills are also intentionally excluded. Archived resources under `__archived/` are never installed. Update `config/pi-links.json` deliberately if that policy changes.
|
||||
The repository contains `copy-all`, `dictate` (Linux host-service dictation), `effort-controls`, `file-search`, `interactive-subagents`, `lm-studio`, `native-hooks`, `prompt-snippets`, `subagents`, `todo-herdr`, `update`, and the private `workflow` extension. The link manifest does not install any of them globally. The `pptx`, `stop-slop`, `subagents`, and `xlsx` skills are also intentionally excluded. Archived resources under `__archived/` are never installed. Update `config/pi-links.json` deliberately if that policy changes.
|
||||
|
||||
## Fresh-machine setup
|
||||
|
||||
@@ -435,31 +435,36 @@ Do not place real provider keys in files readable by sandboxed Pi if the goal is
|
||||
|
||||
## Optional Dictate service
|
||||
|
||||
Dictate is Linux-only and requires:
|
||||
Dictate is a Linux-only host service. Every backend requires:
|
||||
|
||||
- PipeWire with `pw-record`;
|
||||
- systemd user services;
|
||||
- an NVIDIA GPU with a working CUDA runtime;
|
||||
- `uv`;
|
||||
- network access for the initial locked dependency and model download;
|
||||
- UID 1000 unless the hardcoded socket path is changed first.
|
||||
- access to the exact `/run/user/1000/pi-dictate.sock`, unless the socket path is changed for another UID.
|
||||
|
||||
Install the host service from the repository root:
|
||||
The default backend is local NeMo/Parakeet. Local installation additionally requires an NVIDIA GPU, a working CUDA runtime, network access for initial provisioning, and several gigabytes for the locked environment and pinned model. Meta and OpenAI are explicit cloud opt-ins. Cloud-only installation does not install NeMo, CUDA, or model weights, but it requires a dedicated host credential and outbound DNS/TLS access from the service.
|
||||
|
||||
Install one backend from the repository root:
|
||||
|
||||
```bash
|
||||
cd /srv/data/code/harness-tools
|
||||
./services/dictate/install.sh
|
||||
./services/dictate/install.sh install --backend local
|
||||
# or:
|
||||
./services/dictate/install.sh install --backend meta --api-key-file /path/to/meta-key
|
||||
./services/dictate/install.sh install --backend openai --api-key-file /path/to/openai-key
|
||||
```
|
||||
|
||||
`--api-key-file` copies a regular source file into the host-owned mode-`0600` credential directory. Never pass a literal key. Configure an installed backend or validate it with:
|
||||
|
||||
```bash
|
||||
./services/dictate/install.sh configure --backend local
|
||||
./services/dictate/install.sh check
|
||||
systemctl --user status pi-dictate.socket pi-dictate.service
|
||||
```
|
||||
|
||||
The installer provisions the locked Python environment, verifies the pinned model, installs host-owned systemd units, and enables socket activation. Initial storage is several gigabytes. To reuse an existing model archive or preview the operation:
|
||||
Local mode remains offline after provisioning and uses the checksum-pinned Parakeet archive. Cloud mode streams bounded PCM to the selected provider only after its session is authenticated and ready. Backend changes are privileged host operations, require validation and a service restart, and fail closed if systemd network hardening does not match the selection. The service does not write audio files or expose provider credentials to Pi. Cloud provider retention, terms, and billing policies apply to audio and configured context sent with a cloud session.
|
||||
|
||||
```bash
|
||||
./services/dictate/install.sh --model-source /path/to/parakeet-tdt-0.6b-v3.nemo
|
||||
./services/dictate/install.sh --dry-run --model-source /path/to/model.nemo
|
||||
```
|
||||
|
||||
Restart a sandboxed Pi after profile or socket changes. In Pi, press `Alt+M` to start or stop dictation and `Alt+N` to cancel. See [`extensions/dictate/README.md`](extensions/dictate/README.md) for privacy, dictionary, model, and troubleshooting details.
|
||||
Restart a sandboxed Pi after profile or socket changes. In Pi, press `Alt+M` to start or stop dictation, `Alt+N` or `/dictate-cancel` to cancel, and `/dictate-status` to display bounded backend/model diagnostics. See [`extensions/dictate/README.md`](extensions/dictate/README.md) for configuration, provider links, privacy, dictionary, model, and troubleshooting details.
|
||||
|
||||
## Maintenance
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Configurable Dictate ASR backends
|
||||
|
||||
Status: proposed plan, 01SEP2026
|
||||
Status: implemented and validated, 03SEP2026
|
||||
|
||||
## Decisions
|
||||
|
||||
@@ -43,7 +43,7 @@ Status: proposed plan, 01SEP2026
|
||||
- `services/dictate/src/pi_dictate_service/model.py` supports one explicit `.nemo` archive, verifies its SHA-256, applies the coding dictionary, and runs offline on CUDA.
|
||||
- `systemd/user/pi-dictate.service` denies all IP networking and limits address families to `AF_UNIX`. That must remain true in local mode but cannot be shared unchanged by cloud mode.
|
||||
- `services/dictate/install.sh` always installs the full NeMo/CUDA environment and pinned model. Cloud-only installation should not require either.
|
||||
- `profiles/nono/pi-local.json` already grants only the exact Dictate Unix socket and explicitly denies the installed local runtime. Add a deny for `~/.config/pi-dictate` so choosing that directory as `$WORKDIR` cannot expose provider keys; no new Pi-side network, credential, PipeWire, model, or device grant is required.
|
||||
- `profiles/nono/pi-local.json` already grants the exact Dictate Unix socket and explicitly denies the installed local runtime. It is policy-controlled and must remain unchanged for this feature; no Pi-side network, credential, PipeWire, model, or device grant is required.
|
||||
|
||||
## External API contract baseline
|
||||
|
||||
@@ -92,11 +92,11 @@ Current documented shape:
|
||||
- PCM is base64 encoded in bounded `input_audio_buffer.append` JSON events. The API's per-append limit is 15 MiB, but Dictate must use much smaller realtime chunks.
|
||||
- Stop sends `input_audio_buffer.commit`; it starts transcription without creating an assistant response.
|
||||
- Partial text arrives as `conversation.item.input_audio_transcription.delta`; final text arrives as `conversation.item.input_audio_transcription.completed` and is correlated by `item_id`. Transcription-specific failures use `conversation.item.input_audio_transcription.failed`; general protocol failures use `error`.
|
||||
- `gpt-live-transcribe` is the recommended low-latency model and supports prompt, keywords, languages, and delay tuning.
|
||||
- `gpt-live-transcribe` is the recommended low-latency model and supports prompt, keywords, languages, and bounded delay settings. A credentialed live spike on 03SEP2026 confirmed `delay: "low"`.
|
||||
- `gpt-transcribe` can use the same Realtime session but begins transcription after commit and can return detected languages.
|
||||
- `whisper-1` remains a file-transcription model for older integrations, translation, and timestamp use cases; its model page says realtime transcription is unsupported.
|
||||
- OpenAI documents a 60-minute Realtime session. Dictate's own five-minute cap remains stricter.
|
||||
- The current guide recommends `delay` for `gpt-live-transcribe`, while one API reference says it is supported only by `gpt-realtime-whisper`; Phase 0 must verify actual GA behavior and omit `delay` if the selected model rejects it.
|
||||
- The current guide recommends `delay` for `gpt-live-transcribe`, while one API reference associates it with `gpt-realtime-whisper`. The live GA session accepted `low`; Dictate permits `minimal`, `low`, `medium`, `high`, and `xhigh`, and otherwise omits the field.
|
||||
|
||||
## Proposed host configuration
|
||||
|
||||
@@ -132,7 +132,7 @@ Illustrative schema (field names may be refined during implementation):
|
||||
"openai": {
|
||||
"model": "gpt-live-transcribe",
|
||||
"apiKeyFile": "~/.config/pi-dictate/credentials/openai-api-key",
|
||||
"delay": "medium",
|
||||
"delay": "low",
|
||||
"prompt": "Coding dictation",
|
||||
"termsPath": "~/.local/share/pi-dictate/config/coding-terms.txt",
|
||||
"languages": ["en"]
|
||||
@@ -244,8 +244,8 @@ Add `backends/openai.py`:
|
||||
- emit bounded `input_audio_buffer.append` events with base64 PCM, far below the provider's 15 MiB event limit;
|
||||
- commit only after recorder EOF;
|
||||
- correlate final completion to the committed item and ignore stale/unmatched events;
|
||||
- accept configurable model, prompt, keywords, languages, and provider-supported delay values;
|
||||
- default to `gpt-live-transcribe`; document and test `gpt-transcribe`'s commit-oriented behavior, and contract-test whether `delay` is accepted;
|
||||
- accept configurable model, prompt, keywords, languages, and the live-verified bounded `delay` values; reject unknown values rather than silently omitting them.
|
||||
- default to `gpt-live-transcribe`; document and test `gpt-transcribe`'s commit-oriented behavior. Omit `delay` when it is not configured so the provider default remains authoritative.
|
||||
- map `conversation.item.input_audio_transcription.failed`, general `error` events, auth/rate limits, timeouts, malformed events, and premature closes to stable `openai_*` errors without logging request bodies or keys.
|
||||
|
||||
Do not build around the OpenAI SDK unless it materially simplifies Realtime event handling without hiding cancellation, queue bounds, or frame limits. A direct WebSocket adapter is likely smaller and easier to fixture-test.
|
||||
@@ -298,14 +298,14 @@ install.sh check
|
||||
|
||||
Do not weaken local mode to make cloud mode work.
|
||||
|
||||
- Keep common hardening (`NoNewPrivileges`, private temp, core/swap limits, system protection, read-only home, and socket ownership) in the base service unit. Clear ambient `HTTP_PROXY`, `HTTPS_PROXY`, and `ALL_PROXY` variables unless proxy support is separately designed and reviewed.
|
||||
- Keep common hardening (`NoNewPrivileges`, private temp, core/swap limits, system protection, a masked home with narrow read-only runtime/config/unit and PipeWire-socket binds, and socket ownership) in the base service unit. Bind only the selected cloud credential through its generated drop-in. Clear ambient `HTTP_PROXY`, `HTTPS_PROXY`, and `ALL_PROXY` variables unless proxy support is separately designed and reviewed.
|
||||
- Generate/install an offline drop-in for local mode containing `RestrictAddressFamilies=AF_UNIX` and `IPAddressDeny=any`.
|
||||
- Generate/install a cloud drop-in permitting only `AF_UNIX`, `AF_INET`, and `AF_INET6` and no inbound listener beyond the inherited Unix socket. DNS and outbound TLS must be tested under the final unit.
|
||||
- Stamp the expected network mode in the generated unit environment. At startup, compare it with the selected backend and refuse to run on mismatch. This prevents a hand-edited local config from silently running under cloud-capable hardening, and a cloud backend from failing mysteriously under the offline profile.
|
||||
- Restrict service reads to the installed runtime, selected model/dictionary paths, and `~/.config/pi-dictate` where practical.
|
||||
- Document that cloud mode necessarily grants the host transcription process general outbound IP connectivity; systemd cannot safely express a stable hostname allowlist for changing provider addresses.
|
||||
|
||||
Update `profiles/nono/pi-local.json` only to deny `~/.config/pi-dictate`, mirroring the existing runtime deny. The resolved policy must add no capability: Pi still reaches only `/run/user/1000/pi-dictate.sock` and cannot read the host config or keys even if launched with that directory as `$WORKDIR`.
|
||||
Preserve `profiles/nono/pi-local.json` byte-for-byte. The existing resolved policy must be reviewed separately; Pi must not read host config or keys.
|
||||
|
||||
## Implementation phases
|
||||
|
||||
@@ -359,8 +359,8 @@ Completion: OpenAI mode satisfies the same user-visible and cleanup contract as
|
||||
1. Add locked local/cloud dependency profiles and backend-specific provisioning.
|
||||
2. Add atomic install/configure/check operations and secure key copying.
|
||||
3. Add local/cloud systemd hardening generation plus startup mismatch checks.
|
||||
4. Following the repository's nono-profile workflow, draft and strictly validate the explicit deny for `~/.config/pi-dictate`, review the resolved policy, and obtain approval for the exact profile diff before copying it into `profiles/nono/pi-local.json`.
|
||||
5. Verify the resolved profile adds no capability, then test fresh install, every switch direction, reinstall preservation, invalid config, missing/insecure key files, local network denial, cloud DNS/TLS, idle exit, reactivation, and uninstall.
|
||||
4. Preserve `profiles/nono/pi-local.json` byte-for-byte and make no Pi policy changes.
|
||||
5. Review the existing resolved profile separately, then test fresh install, every switch direction, reinstall preservation, invalid config, missing/insecure key files, local network denial, cloud DNS/TLS, idle exit, reactivation, and uninstall.
|
||||
|
||||
Completion: selecting a backend installs only its required runtime, applies matching hardening, and reactivates through the same Unix socket after restart.
|
||||
|
||||
|
||||
@@ -29,3 +29,29 @@ The complete transitive dependency versions and artifact hashes are recorded in
|
||||
- NeMo ASR documentation: <https://docs.nvidia.com/nemo-framework/user-guide/latest/nemotoolkit/asr/intro.html>
|
||||
- CC BY 4.0: <https://creativecommons.org/licenses/by/4.0/>
|
||||
- Apache License 2.0: <https://www.apache.org/licenses/LICENSE-2.0>
|
||||
|
||||
## Optional cloud providers
|
||||
|
||||
Cloud backends are opt-in and are not required for local mode. Pi Dictate does not redistribute Meta or OpenAI models and does not include either provider's SDK. The host service sends audio only after the selected provider session is authenticated and ready. Provider terms, retention, security, and billing policies apply to cloud audio and any keyword or prompt context sent with it.
|
||||
|
||||
### Meta Model API Muse Voice Transcribe
|
||||
|
||||
- Announcement: <https://research.meta.ai/blog/introducing-muse-voice-transcribe>
|
||||
- Speech-to-text guide: <https://ai.developer.meta.com/docs/speech-to-text>
|
||||
- Realtime Voice API: <https://ai.developer.meta.com/docs/api-reference/voice/realtime>
|
||||
- Voice event schemas: <https://ai.developer.meta.com/docs/api-reference/voice/schemas>
|
||||
- Authentication: <https://ai.developer.meta.com/docs/getting-started/authentication>
|
||||
- Meta terms: <https://ai.meta.com/terms/>
|
||||
|
||||
Pi Dictate uses only the documented `api.meta.ai` realtime endpoint and `muse-voice-transcribe-1.0` contract. It does not use the undocumented `shortwave.facebook.com` Muse Code endpoint.
|
||||
|
||||
### OpenAI Realtime transcription
|
||||
|
||||
- Realtime transcription guide: <https://developers.openai.com/api/docs/guides/realtime-transcription>
|
||||
- Realtime WebSocket guide: <https://developers.openai.com/api/docs/guides/realtime-websocket>
|
||||
- Speech-to-text guide: <https://developers.openai.com/api/docs/guides/speech-to-text>
|
||||
- `gpt-live-transcribe` model: <https://developers.openai.com/api/docs/models/gpt-live-transcribe>
|
||||
- OpenAI terms: <https://openai.com/policies/terms-of-use/>
|
||||
- OpenAI privacy policy: <https://openai.com/policies/privacy-policy/>
|
||||
|
||||
The OpenAI default is `gpt-live-transcribe`; `gpt-transcribe` is a configurable commit-oriented alternative. `whisper-1` is not treated as a Realtime model.
|
||||
|
||||
@@ -46,3 +46,18 @@ A 37-term dictionary and boost strengths from 0.25 through 2.0 were tested again
|
||||
The packaging, latency, memory, offline, and normal-dictation gates pass. Alpha `1.0` produced the same expected clean transcript as baseline, the same degraded transcript on seeded noisy input, and an empty transcript for silence. No dictionary hallucination was observed.
|
||||
|
||||
The coding-term remediation passes the final focused live check through the installed socket-activated service. The spoken phrase “Run npm test, then git status.” produced exactly `Run npm test, then git status.` with native phrase boosting at alpha `1.0`.
|
||||
|
||||
The upgraded configurable service was revalidated on 03SEP2026. Immediate cancellation during preparation was delivery-neutral, a second generation started successfully, and a six-second end-to-end capture returned `Run npm test, then get status.` The wording variation does not replace the earlier accuracy fixture; it confirms the current installer, systemd sandbox, socket protocol, PipeWire capture, in-memory model path, and final-only delivery operate together. Local systemd policy exposed only the PipeWire runtime socket while retaining the masked home and IP-network denial.
|
||||
|
||||
## Configurable backend contract gate
|
||||
|
||||
Credentialed live contract checks passed on 03SEP2026 with a five-second, non-sensitive 24 kHz mono PCM microphone sample held only in memory. Dedicated mode-`0600` provider key files were read by the host-side adapters; no credential entered Pi, a command line, a URL, or captured output.
|
||||
|
||||
| Provider | Model | Result |
|
||||
| --- | --- | --- |
|
||||
| Meta | `muse-voice-transcribe-1.0` | The public `api.meta.ai` handshake accepted the session, streamed PCM, accepted `endStream`, and returned an explicit final transcript (`run npm test`). Total turn time was 30.5 seconds. |
|
||||
| OpenAI | `gpt-live-transcribe` | The GA transcription session accepted 24 kHz PCM, explicit commit, and returned a correlated completion event (`Run npm test and`). Total turn time was 7.1 seconds. |
|
||||
|
||||
The first OpenAI spike exposed forward-compatible fields (`event_id` and `previous_item_id`) on the commit acknowledgement. The adapter and sanitized fixture now accept those bounded fields while still requiring a non-empty `item_id`. A separate credentialed session update confirmed that `gpt-live-transcribe` accepts transcription `delay: "low"`. Configuration therefore supports only the documented bounded values `minimal`, `low`, `medium`, `high`, and `xhigh`; omission keeps the provider default.
|
||||
|
||||
These checks validate the current wire contract and finalization path, not comparative transcription accuracy. The public contract references checked were the [Meta Realtime Voice API](https://ai.developer.meta.com/docs/api-reference/voice/realtime), [Meta event schemas](https://ai.developer.meta.com/docs/api-reference/voice/schemas), [OpenAI Realtime transcription guide](https://developers.openai.com/api/docs/guides/realtime-transcription), [OpenAI Realtime WebSocket guide](https://developers.openai.com/api/docs/guides/realtime-websocket), and [`gpt-live-transcribe` model page](https://developers.openai.com/api/docs/models/gpt-live-transcribe).
|
||||
|
||||
+72
-26
@@ -1,28 +1,34 @@
|
||||
# Pi Dictate local-service protocol
|
||||
# Pi Dictate host-service protocol
|
||||
|
||||
## Transport and version
|
||||
|
||||
Protocol version `2` uses UTF-8, newline-delimited JSON over one persistent Unix stream socket. Each line contains one JSON object. The systemd socket unit owns the socket path and permissions. The service accepts the inherited listening file descriptor and does not bind or unlink the path.
|
||||
|
||||
Clients must send `hello` before any other request. A connection negotiates one version only. Coding terms are host-owned service configuration and never cross this protocol.
|
||||
Clients must send `hello` before any other request. A connection negotiates one version only. The selected backend, provider credentials, coding dictionary, model paths, and provider wire events are host-service configuration. They never cross this protocol.
|
||||
|
||||
## Limits
|
||||
The protocol is backend-neutral. The host service owns microphone capture and the selected transcription backend. In local mode it runs the pinned NeMo/Parakeet adapter offline. In Meta and OpenAI modes it owns the provider WebSocket, key, PCM stream, partial hypotheses, and finalization. The Pi extension sees only state, level, result, error, cancellation, and bounded diagnostics.
|
||||
|
||||
## Limits and validation
|
||||
|
||||
| Data | Limit |
|
||||
| --- | ---: |
|
||||
| JSON frame, excluding newline | 65,536 bytes |
|
||||
| `requestId` or `generationId` | 128 UTF-8 bytes |
|
||||
| `backend` or `model` diagnostic | 128 UTF-8 bytes |
|
||||
| Final transcript | 32,768 UTF-8 bytes |
|
||||
| Error detail | 2,048 UTF-8 bytes |
|
||||
| Simultaneous client connections | 16 |
|
||||
| Pending output per client | 65,536 bytes |
|
||||
| Recorder stderr tail | 65,536 bytes |
|
||||
| Audio-level event rate | 10 per second |
|
||||
| Mono 16 kHz signed-16 PCM | 9,600,000 bytes, five minutes |
|
||||
| Local retained PCM | 9,600,000 bytes, five minutes at 16 kHz |
|
||||
| Cloud queued PCM | Fixed and bounded; the whole recording is never retained |
|
||||
|
||||
IDs are non-empty, opaque strings generated by the client. The service compares them only for request correlation, ownership, and stale-generation rejection. It does not interpret IDs as paths. Request objects reject unknown or missing fields.
|
||||
|
||||
Malformed JSON, a frame over the limit, or an unsupported version produces one bounded `error` frame and then closes the connection. The malformed-frame error uses an empty `requestId` when no client ID can be recovered. Schema, state, busy, and ownership errors leave the connection open.
|
||||
`backend` and `model` are optional diagnostic strings on `pong`. When present, they are non-empty, bounded, and safe for one-line display. They identify only the selected backend and model. They must not contain paths, query strings, key material, credential state, provider payloads, or endpoint details. Older services that omit these fields remain protocol-compatible; clients display `unknown` rather than inferring a backend.
|
||||
|
||||
Malformed JSON, invalid UTF-8, a frame over the limit, or an unsupported version produces one bounded `error` frame and then closes the connection. The malformed-frame error uses an empty `requestId` when no client ID can be recovered. Schema, state, busy, and ownership errors leave the connection open.
|
||||
|
||||
## Client requests
|
||||
|
||||
@@ -36,7 +42,7 @@ All examples are one line when transmitted.
|
||||
|
||||
Exact fields: `version`, `type`, `requestId`.
|
||||
|
||||
A successful response is `pong` with the same `requestId`. `ready` reports whether the model is warm.
|
||||
A successful response is `pong` with the same `requestId`. The response may include `service`, `ready`, `busy`, `backend`, and `model`. `ready` says whether the selected backend can begin immediately without preparation. The extension does not use `ready` for state transitions because a cold local model or a new cloud session may need preparation after a successful hello.
|
||||
|
||||
### `ping`
|
||||
|
||||
@@ -44,7 +50,9 @@ A successful response is `pong` with the same `requestId`. `ready` reports wheth
|
||||
{"version":2,"type":"ping","requestId":"ping-1"}
|
||||
```
|
||||
|
||||
Exact fields: `version`, `type`, `requestId`. `hello` is required first. The response is `pong` and may include `ready` and `busy`.
|
||||
Exact fields: `version`, `type`, `requestId`. `hello` is required first. The response is `pong` and may include `ready`, `busy`, `backend`, and `model`.
|
||||
|
||||
The `/dictate-status` extension command uses a short-lived connection: it sends `hello`, waits for its `pong`, sends `ping`, reports the bounded backend/model fields from the ping response, and closes the connection. It never starts a generation or reveals configuration and credential state.
|
||||
|
||||
### `start`
|
||||
|
||||
@@ -52,7 +60,13 @@ Exact fields: `version`, `type`, `requestId`. `hello` is required first. The res
|
||||
{"version":2,"type":"start","requestId":"start-1","generationId":"generation-1"}
|
||||
```
|
||||
|
||||
Exact fields: `version`, `type`, `requestId`, `generationId`. The service accepts only one global generation. It first emits `state: loading` for a cold model or `state: listening` for a warm model. Recording starts only after model loading succeeds.
|
||||
Exact fields: `version`, `type`, `requestId`, `generationId`. The service accepts only one global generation. `start` prepares the selected backend before capture:
|
||||
|
||||
- local: load or reuse the checksum-verified NeMo/Parakeet model;
|
||||
- Meta: connect to the fixed public endpoint, authenticate with the handshake frame, and wait for the session acknowledgement;
|
||||
- OpenAI: connect to the fixed Realtime transcription endpoint, wait for `session.created`, send session configuration, and wait for `session.updated`.
|
||||
|
||||
Recording begins only after preparation succeeds. The service declares the recorder rate internally: 16 kHz for local mode and 24 kHz for either cloud mode. A cold start emits `state: loading` while the extension displays `preparing transcription...`; it never displays `listening` before the recorder and selected backend are ready.
|
||||
|
||||
### `stop`
|
||||
|
||||
@@ -60,7 +74,7 @@ Exact fields: `version`, `type`, `requestId`, `generationId`. The service accept
|
||||
{"version":2,"type":"stop","requestId":"stop-1","generationId":"generation-1"}
|
||||
```
|
||||
|
||||
Exact fields: `version`, `type`, `requestId`, `generationId`. Only the owning connection may stop the active listening generation.
|
||||
Exact fields: `version`, `type`, `requestId`, `generationId`. Only the owning connection may stop the active listening generation. The service marks the generation stopping, signals `pw-record`, drains stdout to EOF, waits for child close, and then finalizes the selected backend. Local mode converts the complete PCM buffer and runs one inference. Cloud mode sends the provider end/commit marker only after every recorder chunk has been enqueued, then waits for one explicit provider final event.
|
||||
|
||||
### `cancel`
|
||||
|
||||
@@ -68,7 +82,7 @@ Exact fields: `version`, `type`, `requestId`, `generationId`. Only the owning co
|
||||
{"version":2,"type":"cancel","requestId":"cancel-1","generationId":"generation-1"}
|
||||
```
|
||||
|
||||
Exact fields: `version`, `type`, `requestId`, `generationId`. A valid cancellation immediately emits `cancelled`. Cancellation tombstones the generation, so it can never emit a later `result` or begin recording after a cancelled load.
|
||||
Exact fields: `version`, `type`, `requestId`, `generationId`. A valid cancellation immediately emits `cancelled`. Cancellation tombstones the generation, so it can never later emit a `result` or begin recording after cancelled preparation. The host closes a cloud provider turn, stops capture, joins workers with bounded timeouts, and discards retained or queued PCM as applicable.
|
||||
|
||||
## Service events
|
||||
|
||||
@@ -77,9 +91,11 @@ All service events contain `version`, `type`, and a bounded client-generated `re
|
||||
### `pong`
|
||||
|
||||
```json
|
||||
{"version":2,"type":"pong","requestId":"hello-1","service":"pi-dictate","ready":false}
|
||||
{"version":2,"type":"pong","requestId":"hello-1","service":"pi-dictate","ready":false,"busy":false,"backend":"local","model":"nemo-parakeet"}
|
||||
```
|
||||
|
||||
`service` is the fixed service identifier. `ready` and `busy` are bounded booleans. `backend` and `model` are optional bounded diagnostics. No event reports key presence, file paths, provider URLs, network state, or raw provider responses.
|
||||
|
||||
### `state`
|
||||
|
||||
```json
|
||||
@@ -88,13 +104,15 @@ All service events contain `version`, `type`, and a bounded client-generated `re
|
||||
|
||||
`state` is one of `loading`, `listening`, `stopping`, `transcribing`, or `idle`.
|
||||
|
||||
The wire name `loading` is retained for compatibility. It means backend preparation and includes local model load, cloud connection/authentication, or session setup. The extension renders it as `preparing transcription...`, not `loading model...`.
|
||||
|
||||
### `level`
|
||||
|
||||
```json
|
||||
{"version":2,"type":"level","requestId":"start-1","generationId":"generation-1","level":0.625}
|
||||
```
|
||||
|
||||
While listening, the service emits at most ten `level` events per second. `level` is a finite number from `0` through `1`, derived from the recorder's current 100 ms RMS window on a logarithmic -60 dBFS to 0 dBFS scale. The event carries no PCM samples and stops before recorder shutdown begins.
|
||||
While listening, the service emits at most ten `level` events per second. `level` is a finite number from `0` through `1`, derived from the recorder's current 100 ms RMS window on a logarithmic -60 dBFS to 0 dBFS scale. The event carries no PCM samples and stops before recorder shutdown begins. The recorder rate is selected by the host backend and is not exposed in the event.
|
||||
|
||||
### `result`
|
||||
|
||||
@@ -102,7 +120,7 @@ While listening, the service emits at most ten `level` events per second. `level
|
||||
{"version":2,"type":"result","requestId":"stop-1","generationId":"generation-1","transcript":"example text"}
|
||||
```
|
||||
|
||||
There is at most one result for a generation. An empty transcript is a successful result with `"transcript":""`.
|
||||
There is at most one result for a generation. An empty transcript is a successful result with `"transcript":""`. Only an explicit final provider event may produce this event. Provider partials, cumulative hypotheses, endpointing, diarization, stale completions, or cleanup never produce a result.
|
||||
|
||||
### `cancelled`
|
||||
|
||||
@@ -110,7 +128,7 @@ There is at most one result for a generation. An empty transcript is a successfu
|
||||
{"version":2,"type":"cancelled","requestId":"cancel-1","generationId":"generation-1"}
|
||||
```
|
||||
|
||||
This is a terminal delivery guarantee: the generation cannot later produce a result.
|
||||
This is a terminal delivery guarantee: the generation cannot later produce a result. The service may remain busy while a non-interruptible local inference or provider cleanup worker finishes.
|
||||
|
||||
### `error`
|
||||
|
||||
@@ -123,37 +141,65 @@ Stable codes include:
|
||||
- framing and schema: `malformed_json`, `frame_too_large`, `unsupported_version`, `invalid_request`;
|
||||
- session: `hello_required`, `busy`, `not_owner`, `generation_mismatch`, `invalid_state`;
|
||||
- recorder and audio: `recorder_spawn`, `recorder_exit`, `recorder_timeout`, `duration_limit`, `invalid_audio`;
|
||||
- model and output: `model_failure`, `transcript_too_large`.
|
||||
- local preparation and output: `config_invalid`, `credentials_invalid`, `model_failure`, `transcript_too_large`;
|
||||
- Meta provider failures: bounded `meta_auth`, `meta_rate_limit`, `meta_timeout`, `meta_protocol`, `meta_premature_close`, or another stable `meta_*` code;
|
||||
- OpenAI provider failures: bounded `openai_auth`, `openai_rate_limit`, `openai_timeout`, `openai_protocol`, `openai_premature_close`, or another stable `openai_*` code.
|
||||
|
||||
Error details are bounded and redacted. They never include API keys, authorization headers, credential paths, request bodies, raw provider payloads, or raw audio. A provider's malformed, oversized, invalid-UTF-8, partial-only, premature-close, authentication, rate-limit, duration, and network failures are terminal for the active generation. The service discards the generation rather than returning a plausible incomplete transcript.
|
||||
|
||||
## Provider finalization contract
|
||||
|
||||
The extension does not receive provider frames, but the host service must enforce these rules before emitting a protocol result.
|
||||
|
||||
### Meta Model API Muse Voice Transcribe
|
||||
|
||||
The service uses `wss://api.meta.ai/v1/asr/realtime` with the documented first-frame bearer handshake for `muse-voice-transcribe-1.0`, `PCM_24KHZ`, `PUSH_TO_TALK`, and cumulative partials. It waits for the `sessionId` acknowledgement before capture, sends ordered binary PCM16 frames through a bounded queue, sends `{ "type": "endStream" }` after recorder EOF, and requires a `transcript` event with `final: true`. It consumes the normal `1000` close and maps close codes `1008`, `1011`, and `1013` to stable failures.
|
||||
|
||||
The service must use only the public endpoint. It must not use the undocumented `shortwave.facebook.com` Muse Code endpoint. See the [Meta API references](https://ai.developer.meta.com/docs/api-reference/voice/realtime), [event schemas](https://ai.developer.meta.com/docs/api-reference/voice/schemas), and [authentication guide](https://ai.developer.meta.com/docs/getting-started/authentication).
|
||||
|
||||
### OpenAI Realtime transcription
|
||||
|
||||
The service uses `wss://api.openai.com/v1/realtime?intent=transcription` with `Authorization: Bearer <key>`. It waits for `session.created`, sends the GA nested `session.update` with `type: "transcription"`, the configured model, 24 kHz mono PCM16, and `turn_detection: null`, then waits for `session.updated`. Audio uses bounded base64 `input_audio_buffer.append` events. Stop commits only after recorder EOF and correlates `conversation.item.input_audio_transcription.completed` to the committed item. Delta and unmatched events are ignored.
|
||||
|
||||
The default is `gpt-live-transcribe`. `gpt-transcribe` is the documented commit-oriented alternative. `whisper-1` is not silently used as a Realtime model. A credentialed live check on 03SEP2026 confirmed that `gpt-live-transcribe` accepts transcription `delay: "low"`; configuration permits `minimal`, `low`, `medium`, `high`, and `xhigh`, while omission keeps the provider default. See the [Realtime transcription guide](https://developers.openai.com/api/docs/guides/realtime-transcription), [WebSocket guide](https://developers.openai.com/api/docs/guides/realtime-websocket), and [`gpt-live-transcribe` model page](https://developers.openai.com/api/docs/models/gpt-live-transcribe).
|
||||
|
||||
## Lifecycle and ownership
|
||||
|
||||
| Current state | Request or event | Outcome |
|
||||
| --- | --- | --- |
|
||||
| no generation | owner `start` | `loading`, then `listening`; warm models may emit `listening` directly |
|
||||
| no generation | owner `start` | `loading`, then `listening`; a prepared backend may emit `listening` directly |
|
||||
| any active generation | another `start` | `busy`; connection stays open |
|
||||
| `loading` | owner `cancel` | immediate `cancelled`; loading finishes in the worker but recording is suppressed |
|
||||
| `loading` | owner `cancel` | immediate `cancelled`; preparation finishes in the worker but recording is suppressed |
|
||||
| `listening` | recorder audio | rate-limited `level` events for the owning client |
|
||||
| `listening` | owner `stop` | `stopping`; signal recorder, drain stdout to EOF, await child close, convert PCM, then `transcribing` |
|
||||
| `listening` | owner `cancel` or disconnect | stop and drain recorder, discard audio, never infer |
|
||||
| `listening` | owner `stop` | `stopping`; stop and drain recorder fully, then `transcribing` and one local/provider final result |
|
||||
| `listening` | owner `cancel` or disconnect | stop and drain recorder, discard audio, never infer or finalize |
|
||||
| `stopping` | owner `cancel` | immediate `cancelled`; completed audio is discarded |
|
||||
| `transcribing` | owner `cancel` or disconnect | tombstone immediately; inference finishes but its result is discarded |
|
||||
| `transcribing` | successful model result | one `result`, then `idle` |
|
||||
| `transcribing` | owner `cancel` or disconnect | tombstone immediately; local inference/provider finalization finishes but its result is discarded |
|
||||
| `transcribing` | successful explicit final | one `result`, then `idle` |
|
||||
| any work state | terminal failure | one `error`, then `idle` |
|
||||
| active generation | non-owner `stop` or `cancel` | `not_owner`; connection stays open |
|
||||
| active generation | stale generation ID | `generation_mismatch`; connection stays open |
|
||||
|
||||
Cancelled model loading and inference are non-interruptible. The global service remains busy until the worker finishes even though `cancelled` was already emitted. Disconnecting a generation owner cancels capture or tombstones loading/inference under the same rules.
|
||||
The service is globally single-generation. A cloud queue overflow, provider stall, or worker failure is terminal. PCM is never silently dropped, retried to another provider, written to disk, or exposed to Pi.
|
||||
|
||||
The recorder command is exactly:
|
||||
The recorder command is selected by the host service and is otherwise fixed:
|
||||
|
||||
```text
|
||||
pw-record --raw --format s16 --rate 16000 --channels 1 -
|
||||
```
|
||||
|
||||
The service launches `pw-record` in its own process group and marks the generation stopping before SIGTERM. It continuously drains bounded stdout and stderr, waits a bounded time for pipe EOF and child close, and only then converts or transcribes. A shutdown timeout escalates to a process-group kill and fails the generation. PipeWire's exit code `1` is accepted only after this service intentionally signalled a still-running recorder and stderr remained empty; an already-exited process or stderr-bearing exit remains a failure. Raw audio and recorder logs are held only in bounded memory and are not written to disk.
|
||||
The local adapter uses 16 kHz. Meta and OpenAI use 24 kHz. The service launches `pw-record` in its own process group, marks the generation stopping before SIGTERM, continuously drains bounded stdout and stderr, waits for pipe EOF and child close, and only then converts or finalizes. A shutdown timeout escalates to a process-group kill and fails the generation. The service creates no audio file.
|
||||
|
||||
Cancellation and disconnect are delivery tombstones. They stop capture, close a provider turn where present, release application-owned buffers, and suppress every later final event. Cleanup is best effort and cannot authorize editor delivery.
|
||||
|
||||
## Idle exit
|
||||
|
||||
A successfully parsed protocol request resets the 15-minute monotonic idle activity timer, including requests that later produce state or ownership errors. Accepting a socket, receiving malformed data, and disconnecting do not reset activity.
|
||||
A successfully parsed protocol request resets the configured monotonic idle activity timer, including requests that later produce state or ownership errors. Accepting a socket, receiving malformed data, and disconnecting do not reset activity.
|
||||
|
||||
Idle exit is suspended while model loading, recording, recorder shutdown, PCM conversion, or inference is active. Once work becomes idle, the full timer starts again. Exiting closes accepted clients and the inherited listener but does not unlink the systemd-owned socket path.
|
||||
Idle exit is suspended while backend preparation, recording, recorder shutdown, PCM conversion, local inference, cloud streaming, provider finalization, or worker cleanup is active. Once work becomes idle, the configured timer starts again. It is 15 minutes by default. Exiting closes accepted clients and the inherited listener but does not unlink the systemd-owned socket path. The service's single owned worker is daemonized so bounded core shutdown permits normal interpreter exit even if native code is stuck. Explicit systemd shutdown also has a five-second stop deadline.
|
||||
|
||||
## Configuration and security boundary
|
||||
|
||||
The service reads strict versioned host configuration at startup from `~/.config/pi-dictate/config.json` and reads only the selected backend's regular, mode-`0600`, service-owned credential file. Keys never appear in command lines, systemd environment variables, URLs, redirects, protocol frames, diagnostics, or logs. Pi has no configuration writer and no access to this directory.
|
||||
|
||||
Local hardening denies IP networking and keeps the pinned runtime offline. Cloud hardening permits only `AF_UNIX`, `AF_INET`, and `AF_INET6` needed for provider DNS/TLS, while the base unit denies `bind()` and `listen()` so the service cannot create an inbound TCP listener. The service rejects a selected backend when the stamped network mode does not match the installed unit. Ambient `HTTP_PROXY`, `HTTPS_PROXY`, and `ALL_PROXY` variables are cleared unless separately reviewed. The exact host install and switch operations are documented in [`extensions/dictate/README.md`](../../extensions/dictate/README.md).
|
||||
|
||||
@@ -1,10 +1,10 @@
|
||||
# Dictate extension review and local Parakeet remediation plan
|
||||
|
||||
Status: implemented, with final host validation in progress. This document records the original review of `extensions/dictate/index.ts`, the agreed Linux implementation direction, and the acceptance criteria used for the replacement.
|
||||
Status: historical local-backend review; final host validation completed. This document records the original review of `extensions/dictate/index.ts`, the agreed Linux implementation direction, and the acceptance criteria used for the local replacement. The current configurable local, Meta, and OpenAI scope supersedes its local-only decisions; see [`configurable-asr-backends-plan.md`](configurable-asr-backends-plan.md) and [`protocol.md`](protocol.md).
|
||||
|
||||
## Decisions
|
||||
|
||||
The replacement will target this machine and workflow:
|
||||
The replacement will target this machine and workflow. The checked-in `profiles/nono/pi-local.json` is outside this change and remains byte-for-byte unchanged.
|
||||
|
||||
- **Platform:** Linux only, specifically CachyOS on x86-64 with niri and Ghostty.
|
||||
- **Transcription:** local NVIDIA NeMo with `nvidia/parakeet-tdt-0.6b-v3` on the machine's RTX 4090. Deepgram and runtime cloud transcription are removed.
|
||||
@@ -172,7 +172,7 @@ The service validates even-byte PCM length, converts little-endian signed 16-bit
|
||||
|
||||
Running capture and NeMo inside Pi would require PipeWire, model-directory, native-library, CUDA, and `/dev/nvidia*` access in the agent sandbox. Deepgram also requires cloud credentials and a network destination.
|
||||
|
||||
**Fix:** keep direct device and model capabilities in the host-side service. Add only `/run/user/1000/pi-dictate.sock` to `filesystem.unix_socket` in `profiles/nono/pi-local.json`. Do not add PipeWire configuration, `/run/user/1000/pipewire-0`, `/dev/snd`, `/dev/nvidia*`, model paths, Hugging Face cache paths, or `api.deepgram.com`.
|
||||
**Fix:** keep direct device and model capabilities in the host-side service. Preserve the existing `/run/user/1000/pi-dictate.sock` policy entry and make no profile edits. Do not add PipeWire configuration, `/run/user/1000/pipewire-0`, `/dev/snd`, `/dev/nvidia*`, model paths, Hugging Face cache paths, or `api.deepgram.com`.
|
||||
|
||||
The socket is deliberately a mediated microphone and transcription capability, not a harmless health endpoint. Any process running under `pi-local` or a descendant profile can request recording and receive the resulting text. The user accepts that broad profile scope; document it prominently and keep the service single-user with socket mode `0600`.
|
||||
|
||||
@@ -375,10 +375,10 @@ Build a throwaway prototype outside the production extension and nono sandbox:
|
||||
|
||||
### Phase 4: Add the minimum nono capability
|
||||
|
||||
1. Add `/run/user/1000/pi-dictate.sock` to `filesystem.unix_socket` in `profiles/nono/pi-local.json`.
|
||||
1. Preserve the existing `/run/user/1000/pi-dictate.sock` policy entry and make no profile edits.
|
||||
2. Do not add PipeWire, GPU, model, cache, Deepgram, or extra network permissions.
|
||||
3. Validate the draft and resolved policy according to the nono-profile workflow.
|
||||
4. Compare the resolved policy before and after and confirm this feature adds only the exact service socket.
|
||||
3. Validate the existing resolved policy separately.
|
||||
4. Confirm this feature does not change the resolved Pi policy.
|
||||
5. Restart Pi because an active nono sandbox cannot gain socket access in place.
|
||||
6. Verify the extension can activate and use the service but still cannot connect directly to PipeWire or initialize NVML.
|
||||
|
||||
|
||||
+167
-93
@@ -1,79 +1,152 @@
|
||||
# Pi Dictate
|
||||
|
||||
Linux-only push-to-dictate for Pi. Audio capture and NVIDIA Parakeet run in a socket-activated host service. The Pi extension receives state and finalized text through a Unix socket, then inserts successful text with `pasteToEditor()`.
|
||||
Pi Dictate is a Linux-only push-to-dictate extension. The extension has no microphone, model, GPU, cloud, or credential access. It is a bounded JSON client for the host-owned `pi-dictate-service` Unix socket and pastes one finalized result into Pi's main editor.
|
||||
|
||||
Tested on CachyOS with niri, Ghostty, PipeWire, an RTX 4090, CUDA 13.0, NeMo 3.0.0, and Parakeet-TDT 0.6B v3.
|
||||
The configured backend is local by default. Cloud backends are opt-in because their audio leaves the host and may incur provider charges.
|
||||
|
||||
## Backends
|
||||
|
||||
| Backend | Capture | Runtime | Default model and endpoint |
|
||||
| --- | --- | --- | --- |
|
||||
| `local` | 16 kHz mono PCM16 | Offline NeMo/Parakeet on the host GPU | `nemo-parakeet` and a pinned local archive |
|
||||
| `meta` | 24 kHz mono PCM16 | Meta Model API | `muse-voice-transcribe-1.0` at `api.meta.ai` |
|
||||
| `openai` | 24 kHz mono PCM16 | OpenAI Realtime transcription | `gpt-live-transcribe` at `api.openai.com` |
|
||||
|
||||
Only one backend is selected in host configuration. There is no per-session or per-generation picker, automatic failover, or submission of one recording to multiple providers. Switching backend is a privileged host operation and takes effect after configuration validation and a service restart.
|
||||
|
||||
OpenAI's `gpt-transcribe` is a supported commit-oriented alternative. `whisper-1` is not silently treated as a Realtime model and is not the default.
|
||||
|
||||
## Requirements
|
||||
|
||||
- Linux with PipeWire and `pw-record`
|
||||
- systemd user services
|
||||
- NVIDIA GPU with a working CUDA runtime
|
||||
- `uv`
|
||||
- Network access during initial dependency and model provisioning
|
||||
All backends require:
|
||||
|
||||
Runtime is offline. The systemd service restricts networking and loads an explicit local model archive.
|
||||
- Linux with PipeWire and `pw-record`;
|
||||
- a systemd user session;
|
||||
- `uv` for installation;
|
||||
- the Pi profile's exact `/run/user/1000/pi-dictate.sock` access, unless the socket path is changed for another UID.
|
||||
|
||||
## Install
|
||||
Local mode additionally requires an NVIDIA GPU with a working CUDA runtime and several gigabytes for the locked environment and model archive. Cloud-only mode does not install NeMo, CUDA, or the Parakeet archive, but it requires a provider key and outbound DNS/TLS access from the host service.
|
||||
|
||||
From the repository root:
|
||||
## Install and configure
|
||||
|
||||
Run these commands from the repository root. The installer owns the installed runtime under `~/.local/share/pi-dictate/` and host configuration under `~/.config/pi-dictate/`.
|
||||
|
||||
Install the default offline backend:
|
||||
|
||||
```bash
|
||||
./services/dictate/install.sh
|
||||
./services/dictate/install.sh install --backend local
|
||||
```
|
||||
|
||||
The installer creates `~/.local/share/pi-dictate/`, installs the locked service as a non-editable wheel, downloads and verifies the pinned model snapshot, seeds the coding dictionary, installs host-owned copies of the systemd user units, and enables `pi-dictate.socket`. The installed runtime does not depend on the repository remaining at its installation path.
|
||||
|
||||
To reuse an already downloaded model archive:
|
||||
Install a cloud backend by copying a key from a file. The key is never accepted as a command-line literal:
|
||||
|
||||
```bash
|
||||
./services/dictate/install.sh \
|
||||
--model-source /path/to/parakeet-tdt-0.6b-v3.nemo
|
||||
./services/dictate/install.sh install --backend meta --api-key-file /path/to/meta-key
|
||||
./services/dictate/install.sh install --backend openai --api-key-file /path/to/openai-key
|
||||
```
|
||||
|
||||
Preview without changing the host:
|
||||
Configure an installed service or inspect the effective setup:
|
||||
|
||||
```bash
|
||||
./services/dictate/install.sh --dry-run --model-source /path/to/model.nemo
|
||||
./services/dictate/install.sh configure --backend local
|
||||
./services/dictate/install.sh configure --backend meta --api-key-file /path/to/meta-key
|
||||
./services/dictate/install.sh configure --backend openai --api-key-file /path/to/openai-key
|
||||
./services/dictate/install.sh check
|
||||
```
|
||||
|
||||
The pinned model revision and SHA-256 are recorded in [`../../docs/dictate/parakeet-attribution.md`](../../docs/dictate/parakeet-attribution.md).
|
||||
`--api-key-file` must identify a regular file. The installer copies it into a service-owned mode-`0600` file and does not put the secret in systemd environment variables, process arguments, URLs, protocol frames, diagnostics, or logs. At startup the service reads only the selected backend's key. Inactive backend keys are neither required nor read.
|
||||
|
||||
## Sandbox access
|
||||
The installed authority is:
|
||||
|
||||
`profiles/nono/pi-local.json` grants exactly:
|
||||
```text
|
||||
~/.config/pi-dictate/
|
||||
config.json
|
||||
credentials/
|
||||
meta-api-key
|
||||
openai-api-key
|
||||
```
|
||||
|
||||
The service validates versioned JSON, selected-backend options, paths, numeric bounds, coding terms, and credential ownership and permissions before starting. Configuration changes require the configure operation and a validated service restart. Do not edit Pi extension source or try to write this directory from Pi.
|
||||
|
||||
Local model reuse remains checksum-gated. A custom archive must be accompanied by its explicit SHA-256 and a supported NeMo adapter:
|
||||
|
||||
```bash
|
||||
MODEL=/path/to/parakeet-tdt-0.6b-v3.nemo
|
||||
./services/dictate/install.sh install --backend local \
|
||||
--model-source "$MODEL" \
|
||||
--model-sha256 "$(sha256sum "$MODEL" | cut -d' ' -f1)"
|
||||
```
|
||||
|
||||
The installer preserves valid existing model archives, dictionaries, configuration, and credentials on reinstall. Local and cloud modes use different systemd network hardening. Local mode denies IP networking; cloud mode permits IPv4/IPv6 for provider DNS and TLS while the base unit denies `bind()` and `listen()`, preventing a TCP listener. A backend/hardening mismatch fails closed. Cloud mode does not provide a stable hostname allowlist because provider addresses can change.
|
||||
|
||||
## Socket and sandbox boundary
|
||||
|
||||
The Pi profile grants access only to the exact service socket:
|
||||
|
||||
```text
|
||||
/run/user/1000/pi-dictate.sock
|
||||
```
|
||||
|
||||
This is a mediated microphone and transcription capability. Any code running under `pi-local` or a descendant profile can ask the host service to record and can receive the transcript. Pi still receives no PipeWire, `/dev/snd`, CUDA device, model, cache, or dictionary access. An explicit filesystem deny protects `~/.local/share/pi-dictate` even if that directory is selected as `$WORKDIR`.
|
||||
This is a mediated microphone and transcription capability, not a health-only endpoint. Any code running under `pi-local` or a descendant profile can request a recording and receive its finalized transcript. Pi still has no PipeWire, `/dev/snd`, CUDA, model, dictionary, provider-key, or direct provider-network access. The checked-in profile policy is intentionally unchanged; review any local profile policy before exposing the service socket.
|
||||
|
||||
The active profile is repository-linked on the tested host. Start a new Pi sandbox after updating the profile because an existing nono process cannot gain the socket capability in place.
|
||||
Restart Pi after changing the nono profile or socket exposure. An existing nono process cannot gain a new socket capability in place.
|
||||
|
||||
## Controls
|
||||
## Controls and delivery
|
||||
|
||||
| Control | Action |
|
||||
| --- | --- |
|
||||
| `Alt+M` | Start recording when idle; stop and transcribe while listening |
|
||||
| `Alt+N` | Cancel the active generation |
|
||||
| `/dictate-cancel` | Cancel the active generation |
|
||||
| `/dictate-status` | Open a short read-only hello/ping connection and show backend/model diagnostics |
|
||||
|
||||
Shortcuts are handled by Pi's main editor. They do not claim to work while a selector, confirmation, or custom modal owns input. If a modal opens after recording starts, a successful result still goes only to the main editor.
|
||||
Shortcuts are registered through Pi's public main-editor APIs. They work while the main editor has focus, not while a selector, confirmation, or custom modal owns input. If a modal opens after recording starts, a successful result still goes only to the main editor.
|
||||
|
||||
The footer shows service state. While recording, a red circle and twelve-cell scrolling waveform make active capture visible and react to the microphone level:
|
||||
The footer shows backend-neutral service states:
|
||||
|
||||
```text
|
||||
preparing transcription...
|
||||
● listening ▁▂▄▇█▆▃▂▅▇▄▂
|
||||
stopping...
|
||||
transcribing...
|
||||
```
|
||||
|
||||
The circle uses Pi's semantic error color and the waveform uses its accent color. The other states remain `loading model...`, `stopping...`, and `transcribing...`. Audio levels are bounded scalar protocol events; raw audio remains inside the host service.
|
||||
The waveform contains only bounded scalar level events. Raw audio, provider events, partial hypotheses, and credentials never cross the socket. Preparation includes local model load or cloud connection and authentication, so recording does not begin until the selected backend is ready. `/dictate-status` reports only bounded backend and model names. It never reports paths, endpoints with query data, key state, or provider payloads.
|
||||
|
||||
This indicator requires protocol version 2. Run the installer again after updating the repository so the installed host service and repository-linked extension upgrade together.
|
||||
Only one explicit, bounded final transcript for the active generation can change the editor. Partial hypotheses are discarded at the service boundary. Empty speech reports `No speech recognized.` without changing the editor. Cancellation, stale generations, disconnects, provider failures, and cleanup never paste text.
|
||||
|
||||
Only a non-empty finalized result for the current generation changes the editor. Empty speech, cancellation, stale results, disconnects, and failures do not insert text.
|
||||
## Configuration and coding terms
|
||||
|
||||
## Coding dictionary
|
||||
The service reads host-owned configuration at process startup. A representative configuration is:
|
||||
|
||||
```json
|
||||
{
|
||||
"version": 1,
|
||||
"backend": "openai",
|
||||
"idleSeconds": 900,
|
||||
"local": {
|
||||
"adapter": "nemo-parakeet",
|
||||
"modelPath": "~/.local/share/pi-dictate/models/parakeet.nemo",
|
||||
"modelSha256": "...",
|
||||
"termsPath": "~/.local/share/pi-dictate/config/coding-terms.txt",
|
||||
"boostAlpha": 1.0
|
||||
},
|
||||
"meta": {
|
||||
"model": "muse-voice-transcribe-1.0",
|
||||
"apiKeyFile": "~/.config/pi-dictate/credentials/meta-api-key",
|
||||
"termsPath": "~/.local/share/pi-dictate/config/coding-terms.txt"
|
||||
},
|
||||
"openai": {
|
||||
"model": "gpt-live-transcribe",
|
||||
"apiKeyFile": "~/.config/pi-dictate/credentials/openai-api-key",
|
||||
"delay": "low",
|
||||
"prompt": "Coding dictation",
|
||||
"termsPath": "~/.local/share/pi-dictate/config/coding-terms.txt",
|
||||
"languages": ["en"]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The exact defaults are installer-generated. Unknown keys, malformed JSON, unsupported versions, invalid bounds, incompatible options, missing selected keys, symlinks, multi-line credentials, and insecure credential permissions are rejected. OpenAI `delay` is optional; a credentialed live check confirmed `low`, and accepted values are `minimal`, `low`, `medium`, `high`, and `xhigh`. Omitting it keeps the provider default. Provider endpoints are fixed in production and are not configurable through Pi. The service clears ambient proxy variables unless an explicitly reviewed proxy feature is added.
|
||||
|
||||
The installer seeds:
|
||||
|
||||
@@ -81,58 +154,94 @@ The installer seeds:
|
||||
~/.local/share/pi-dictate/config/coding-terms.txt
|
||||
```
|
||||
|
||||
The host service reads this file at model load and applies NeMo's native phrase boosting. Pi cannot read or modify it. Add one term or phrase per line, use `#` for comments, and restart the service after editing:
|
||||
Use one term or phrase per line and `#` for comments. In local mode, terms feed NeMo phrase boosting. In cloud mode, validated terms may be sent as provider keyword or language-bias context, so do not place secrets or sensitive phrases in the dictionary if they must remain on the host. The dictionary is bounded to 64 KiB, 512 distinct terms, and 128 UTF-8 bytes per term. Restart after an edit:
|
||||
|
||||
```bash
|
||||
systemctl --user restart pi-dictate.service
|
||||
```
|
||||
|
||||
Reinstalling preserves an existing dictionary. The file is bounded to 64 KiB, 512 distinct terms, and 128 UTF-8 bytes per term. A conservative boost of `1.0` is configured in `pi-dictate.service`; aggressive values can create false positives.
|
||||
## Privacy, network, and provider policies
|
||||
|
||||
## Model lifecycle and measured behavior
|
||||
Local mode captures and transcribes entirely on the host after installation. The runtime uses the pinned local model with offline settings and the systemd unit denies IP networking. No audio file is created. Application-owned buffers are bounded and released after transcription, cancellation, or failure. `LimitCORE=0` and `MemorySwapMax=0` reduce additional persistence where supported. This is best-effort non-persistence, not forensic zeroization: Python, NumPy, PyTorch, CUDA, drivers, host memory management, crash handling, and swap can retain copies outside the service's direct control.
|
||||
|
||||
The socket remains active while the Python service exits after 15 minutes of true inactivity. A later connection activates a fresh service. Loading, recording, recorder shutdown, and inference suspend idle exit.
|
||||
In Meta or OpenAI mode, the service connects and authenticates before recording. It then streams bounded 24 kHz PCM directly to the selected provider while recording. Stop drains `pw-record` completely before sending the provider end/commit marker, waits for one explicit final event, and returns only that bounded text. Audio is not written to disk and is not sent to Pi. Provider partials and wire events remain in the host service. The provider's current retention, terms, security, and billing policies apply to cloud audio and any context sent with the session. Review the links below before opting in, and monitor the provider account for usage charges.
|
||||
|
||||
Measured on the tested RTX 4090 using the pinned offline model:
|
||||
Provider references:
|
||||
|
||||
- Model archive: about 2.4 GB
|
||||
- Locked Python environment: about 5.6 GB in the Phase 0 prototype
|
||||
- Cold base-model load: about 6.4 seconds
|
||||
- Production-adapter load with phrase-boost setup: about 10 seconds, including the private Triton cache build
|
||||
- Ten-second microphone inference: about 0.29 seconds, 0.029 real-time factor
|
||||
- Sixty-second repeated-speech inference: about 0.43 seconds, 0.0072 real-time factor
|
||||
- Five-minute repeated-speech inference: about 0.82 seconds, 0.0027 real-time factor
|
||||
- Peak CUDA allocation: about 5.1 GB for short clips and 6.15 GB for five minutes
|
||||
- Five-minute raw PCM cap: 9.6 MB
|
||||
- Meta announcement: <https://research.meta.ai/blog/introducing-muse-voice-transcribe>
|
||||
- Meta speech-to-text guide: <https://ai.developer.meta.com/docs/speech-to-text>
|
||||
- Meta Realtime Voice API: <https://ai.developer.meta.com/docs/api-reference/voice/realtime>
|
||||
- Meta voice event schemas: <https://ai.developer.meta.com/docs/api-reference/voice/schemas>
|
||||
- Meta authentication: <https://ai.developer.meta.com/docs/getting-started/authentication>
|
||||
- Meta terms: <https://ai.meta.com/terms/>
|
||||
- OpenAI Realtime transcription: <https://developers.openai.com/api/docs/guides/realtime-transcription>
|
||||
- OpenAI Realtime WebSocket: <https://developers.openai.com/api/docs/guides/realtime-websocket>
|
||||
- OpenAI speech-to-text: <https://developers.openai.com/api/docs/guides/speech-to-text>
|
||||
- OpenAI `gpt-live-transcribe`: <https://developers.openai.com/api/docs/models/gpt-live-transcribe>
|
||||
- OpenAI terms: <https://openai.com/policies/terms-of-use/>
|
||||
- OpenAI privacy policy: <https://openai.com/policies/privacy-policy/>
|
||||
|
||||
The 60-second and five-minute stress inputs repeat a real speech clip in memory. They validate bounded duration, latency, and GPU allocation, not long-form accuracy.
|
||||
The service uses only the documented public endpoints. It does not reuse Pi's provider auth store, put keys in URLs, follow credential-bearing redirects, or use the undocumented `shortwave.facebook.com` Muse Code endpoint.
|
||||
|
||||
## Privacy
|
||||
## Local model, attribution, and measured behavior
|
||||
|
||||
The service passes this exact recorder command through pipes:
|
||||
Local mode uses NVIDIA's `parakeet-tdt-0.6b-v3` through NeMo. The model archive is about 2.4 GB, and the locked Python environment was about 5.6 GB in the Phase 0 prototype. On the tested RTX 4090, cold preparation took about 10 seconds with phrase-boost setup, warm inference was comfortably faster than real time, and peak CUDA allocation was about 5.1 GB for short clips and 6.15 GB for five minutes. The service caps a local recording at five minutes, approximately 9.6 MB of 16 kHz mono PCM16. Cloud sessions use the same five-minute Dictate cap even though provider sessions can be longer.
|
||||
|
||||
```text
|
||||
pw-record --raw --format s16 --rate 16000 --channels 1 -
|
||||
```
|
||||
See [`../../docs/dictate/parakeet-attribution.md`](../../docs/dictate/parakeet-attribution.md) for the pinned revision, SHA-256, CC BY 4.0 model terms, NeMo Apache 2.0 terms, and source links. The repository does not redistribute model weights.
|
||||
|
||||
It intentionally creates no audio files and releases application-owned buffers after transcription, cancellation, or failure. `LimitCORE=0` and `MemorySwapMax=0` reduce additional persistence. This is best-effort non-persistence, not forensic zeroization. Python, NumPy, PyTorch, CUDA, drivers, host memory management, and crash handling can retain copies outside the service's direct control.
|
||||
The service remains warm for the configured idle period, 15 minutes by default, while no recording, inference, preparation worker, or protocol activity is active. Idle exit releases local model/GPU resources. A later socket connection activates a fresh service.
|
||||
|
||||
The service does not log raw audio or transcripts by default. Recorder stderr and protocol output are bounded. The server accepts at most 16 clients and bounds each client's pending output to 64 KiB.
|
||||
|
||||
## Service operations
|
||||
## Service operations and troubleshooting
|
||||
|
||||
```bash
|
||||
systemctl --user status pi-dictate.socket pi-dictate.service
|
||||
journalctl --user -u pi-dictate.service
|
||||
journalctl --user -u pi-dictate.service -n 100
|
||||
systemctl --user restart pi-dictate.service
|
||||
systemctl --user stop pi-dictate.service
|
||||
./services/dictate/install.sh check
|
||||
```
|
||||
|
||||
Stopping only the service leaves the socket active for later reactivation. Stopping `pi-dictate.socket` removes the socket path.
|
||||
Stopping the service leaves the socket unit able to reactivate it. Stopping `pi-dictate.socket` removes the socket path. Service output is bounded and does not log keys, raw audio, transcripts, provider request bodies, or partial hypotheses by default.
|
||||
|
||||
### Socket or activation failure
|
||||
|
||||
```bash
|
||||
ls -l "$XDG_RUNTIME_DIR/pi-dictate.sock"
|
||||
systemctl --user status pi-dictate.socket pi-dictate.service
|
||||
journalctl --user -u pi-dictate.service -n 100
|
||||
```
|
||||
|
||||
The socket should be mode `0600` and owned by the current user. Run `/dictate-status` after a restart to confirm the selected backend and model. Restart Pi after a nono policy change.
|
||||
|
||||
### Local PipeWire, CUDA, or model failure
|
||||
|
||||
```bash
|
||||
wpctl status
|
||||
pw-record --raw --format s16 --rate 16000 --channels 1 - >/dev/null
|
||||
nvidia-smi
|
||||
```
|
||||
|
||||
Press `Ctrl+C` after checking capture. Confirm the model archive checksum and that the local systemd unit still denies IP networking. A local preparation failure must not start the recorder.
|
||||
|
||||
### Cloud key, DNS, TLS, or provider failure
|
||||
|
||||
Run `./services/dictate/install.sh check` and verify the selected credential file is a regular mode-`0600` file owned by the service user. Confirm the cloud drop-in is active, DNS works for the fixed provider endpoint, and the host can establish outbound TLS. Do not add a key to a command line or systemd environment. `meta_*` and `openai_*` service errors are intentionally bounded and redacted; provider request bodies are not a debugging log.
|
||||
|
||||
For an explicit live contract check, first select the cloud backend, then provide a non-sensitive raw 24 kHz mono PCM16 fixture of at most five seconds:
|
||||
|
||||
```bash
|
||||
~/.local/share/pi-dictate/venv/bin/python -m pi_dictate_service.live_smoke /path/to/sample.pcm
|
||||
```
|
||||
|
||||
This command immediately sends the sample to the configured provider and may incur charges. It reads only the managed selected credential, does not accept a literal key, rejects symlinked or oversized samples, and does not write audio.
|
||||
|
||||
### Busy, timeout, or duration errors
|
||||
|
||||
The service allows one generation globally. A cancelled preparation or inference worker can remain busy until its non-interruptible operation finishes. A stalled cloud stream, queue overflow, provider timeout, recorder failure, or five-minute limit discards the generation rather than returning a plausible incomplete transcript. Wait for it to settle or restart the service if the owning client crashed and the worker is stuck.
|
||||
|
||||
## niri and Ghostty
|
||||
|
||||
No compositor-global binding is required. Confirm the live niri config does not claim `Alt+M` or `Alt+N`, then validate it outside the Pi sandbox:
|
||||
No compositor-global binding is required. Confirm the live niri configuration does not claim `Alt+M` or `Alt+N`, then validate it outside the Pi sandbox:
|
||||
|
||||
```bash
|
||||
niri validate
|
||||
@@ -140,45 +249,10 @@ niri validate
|
||||
|
||||
The shortcuts should reach Pi only while its main editor has focus. No `wl-copy`, `wtype`, `ydotool`, X11 injection, or compositor spawn binding is used.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Socket or activation failure
|
||||
|
||||
```bash
|
||||
systemctl --user status pi-dictate.socket pi-dictate.service
|
||||
ls -l "$XDG_RUNTIME_DIR/pi-dictate.sock"
|
||||
journalctl --user -u pi-dictate.service -n 100
|
||||
```
|
||||
|
||||
The socket should be mode `0600` and owned by the current user. Restart Pi after changing nono policy.
|
||||
|
||||
### PipeWire or microphone failure
|
||||
|
||||
```bash
|
||||
wpctl status
|
||||
pw-record --raw --format s16 --rate 16000 --channels 1 - >/dev/null
|
||||
```
|
||||
|
||||
Press `Ctrl+C` after confirming capture. The service uses the WirePlumber default input. Selectable devices are not implemented.
|
||||
|
||||
### CUDA or model failure
|
||||
|
||||
```bash
|
||||
nvidia-smi
|
||||
systemctl --user restart pi-dictate.service
|
||||
journalctl --user -u pi-dictate.service -n 100
|
||||
```
|
||||
|
||||
Confirm the installed model checksum matches the attribution document and that runtime network access is not needed.
|
||||
|
||||
### Service reports busy
|
||||
|
||||
Only one recording, load worker, or inference worker is allowed globally. A cancelled load or inference remains busy until its non-interruptible worker finishes. Wait for it to settle or restart the service if the owning client crashed and the worker is stuck.
|
||||
|
||||
## Uninstall
|
||||
|
||||
```bash
|
||||
./services/dictate/install.sh uninstall
|
||||
```
|
||||
|
||||
The uninstaller disables the socket, removes only installer-managed unit copies, reloads systemd, and removes the fixed `~/.local/share/pi-dictate/` directory. It ignores install-root environment overrides and refuses to replace or remove unmanaged unit files.
|
||||
Uninstall disables the socket, removes installer-managed unit copies, reloads systemd, and removes the installed runtime. It preserves `~/.config/pi-dictate/` and credentials by default. Use the installer's explicit, confirmed `--purge-config` path only when those credentials and settings should be removed. Uninstall does not remove unmanaged unit files.
|
||||
|
||||
@@ -195,10 +195,66 @@ function serviceFrame(
|
||||
test("registers main-editor shortcuts, cancel command, and shutdown cleanup", () => {
|
||||
const harness = createHarness();
|
||||
assert.deepEqual([...harness.shortcuts.keys()], ["alt+m", "alt+n"]);
|
||||
assert.deepEqual([...harness.commands.keys()], ["dictate-cancel"]);
|
||||
assert.deepEqual([...harness.commands.keys()], ["dictate-cancel", "dictate-status"]);
|
||||
assert.ok(harness.events.has("session_shutdown"));
|
||||
});
|
||||
|
||||
test("status command negotiates a short read-only connection and reports backend diagnostics", async () => {
|
||||
const harness = createHarness();
|
||||
await harness.command("dictate-status")(
|
||||
"",
|
||||
harness.context as ExtensionCommandContext,
|
||||
);
|
||||
const socket = harness.sockets[0];
|
||||
assert.ok(socket);
|
||||
socket.connect();
|
||||
const hello = harness.sentFrames(socket)[0];
|
||||
assert.equal(hello?.type, "hello");
|
||||
socket.frame({
|
||||
version: PROTOCOL_VERSION,
|
||||
type: "pong",
|
||||
requestId: hello?.requestId,
|
||||
backend: "openai",
|
||||
model: "gpt-live-transcribe",
|
||||
});
|
||||
const ping = harness.sentFrames(socket)[1];
|
||||
assert.equal(ping?.type, "ping");
|
||||
socket.frame({
|
||||
version: PROTOCOL_VERSION,
|
||||
type: "pong",
|
||||
requestId: ping?.requestId,
|
||||
});
|
||||
|
||||
assert.deepEqual(harness.notifications, [
|
||||
{
|
||||
message: "Dictation backend: openai; model: gpt-live-transcribe",
|
||||
type: "info",
|
||||
},
|
||||
]);
|
||||
assert.equal(socket.endCount, 1);
|
||||
});
|
||||
|
||||
test("status diagnostics reject control characters without exposing them", async () => {
|
||||
const harness = createHarness();
|
||||
await harness.command("dictate-status")(
|
||||
"",
|
||||
harness.context as ExtensionCommandContext,
|
||||
);
|
||||
const socket = harness.sockets[0];
|
||||
assert.ok(socket);
|
||||
socket.connect();
|
||||
const hello = harness.sentFrames(socket)[0];
|
||||
socket.frame({
|
||||
version: PROTOCOL_VERSION,
|
||||
type: "pong",
|
||||
requestId: hello?.requestId,
|
||||
backend: "openai\nsecret",
|
||||
});
|
||||
|
||||
assert.match(harness.notifications[0]?.message ?? "", /diagnostic backend is invalid/i);
|
||||
assert.equal(socket.destroyCount, 1);
|
||||
});
|
||||
|
||||
test("connection errors return to idle and the next start uses a fresh socket", async () => {
|
||||
const harness = createHarness();
|
||||
await harness.shortcut("alt+m")(harness.context);
|
||||
@@ -277,7 +333,7 @@ test("renders authoritative service states and sends a bounded stop command", as
|
||||
serviceFrame(socket, generationId, "state", { state: "transcribing" });
|
||||
|
||||
assert.deepEqual(harness.statuses, [
|
||||
"loading model...",
|
||||
"preparing transcription...",
|
||||
"● listening ▁▁▁▁▁▁▁▁▁▁▁▁",
|
||||
"stopping...",
|
||||
"transcribing...",
|
||||
|
||||
+228
-19
@@ -9,6 +9,7 @@ export const MAX_ID_BYTES = 128;
|
||||
|
||||
const STATUS_KEY = "dictate";
|
||||
const MAX_ERROR_DETAIL_BYTES = 2 * 1024;
|
||||
const MAX_DIAGNOSTIC_BYTES = 128;
|
||||
const UTF8_DECODER = new TextDecoder("utf-8", { fatal: true });
|
||||
const WAVEFORM_BARS = ["▁", "▂", "▃", "▄", "▅", "▆", "▇", "█"] as const;
|
||||
const WAVEFORM_SAMPLES = 12;
|
||||
@@ -76,6 +77,15 @@ function isBoundedId(value: unknown): value is string {
|
||||
);
|
||||
}
|
||||
|
||||
function isBoundedDiagnostic(value: unknown): value is string {
|
||||
return (
|
||||
typeof value === "string" &&
|
||||
value.length > 0 &&
|
||||
utf8Bytes(value) <= MAX_DIAGNOSTIC_BYTES &&
|
||||
!/[\u0000-\u001f\u007f]/u.test(value)
|
||||
);
|
||||
}
|
||||
|
||||
function boundedText(value: string, maxBytes: number): string {
|
||||
if (utf8Bytes(value) <= maxBytes) return value;
|
||||
let text = value;
|
||||
@@ -92,14 +102,21 @@ export function createDictateExtension(options: DictateExtensionOptions = {}) {
|
||||
return function registerDictate(pi: ExtensionAPI) {
|
||||
let state: DictateState = "idle";
|
||||
let socket: DictateSocket | null = null;
|
||||
let statusSocket: DictateSocket | null = null;
|
||||
let activeContext: ExtensionContext | null = null;
|
||||
let statusContext: ExtensionContext | null = null;
|
||||
let activeGenerationId: string | null = null;
|
||||
let helloRequestId: string | null = null;
|
||||
let negotiated = false;
|
||||
let receiveBuffer = Buffer.alloc(0);
|
||||
let receiveBuffer: Buffer<ArrayBufferLike> = Buffer.alloc(0);
|
||||
let nextId = 0;
|
||||
let resultDelivered = false;
|
||||
let audioLevels: number[] = [];
|
||||
let statusRequest: "hello" | "ping" | null = null;
|
||||
let statusRequestId: string | null = null;
|
||||
let statusBackend: string | null = null;
|
||||
let statusModel: string | null = null;
|
||||
let statusReceiveBuffer: Buffer<ArrayBufferLike> = Buffer.alloc(0);
|
||||
|
||||
const newId = (kind: string) => `${kind}-${++nextId}`;
|
||||
|
||||
@@ -135,7 +152,20 @@ export function createDictateExtension(options: DictateExtensionOptions = {}) {
|
||||
}
|
||||
};
|
||||
|
||||
const closeStatusQuery = (forceClose = false) => {
|
||||
const current = statusSocket;
|
||||
statusSocket = null;
|
||||
statusContext = null;
|
||||
statusRequest = null;
|
||||
statusRequestId = null;
|
||||
statusBackend = null;
|
||||
statusModel = null;
|
||||
statusReceiveBuffer = Buffer.alloc(0);
|
||||
if (current) closeSocket(current, forceClose);
|
||||
};
|
||||
|
||||
const reset = (forceClose = false) => {
|
||||
closeStatusQuery(true);
|
||||
const current = socket;
|
||||
socket = null;
|
||||
activeGenerationId = null;
|
||||
@@ -162,6 +192,35 @@ export function createDictateExtension(options: DictateExtensionOptions = {}) {
|
||||
reset(true);
|
||||
};
|
||||
|
||||
const failStatus = (message: string) => {
|
||||
const context = statusContext;
|
||||
closeStatusQuery(true);
|
||||
context?.ui.notify(
|
||||
boundedText(message, MAX_ERROR_DETAIL_BYTES),
|
||||
"error",
|
||||
);
|
||||
};
|
||||
|
||||
const sendStatus = (
|
||||
current: DictateSocket,
|
||||
frame: Record<string, string | number>,
|
||||
): boolean => {
|
||||
if (statusSocket !== current) return false;
|
||||
const encoded = `${JSON.stringify(frame)}\n`;
|
||||
if (utf8Bytes(encoded) - 1 > MAX_FRAME_BYTES) {
|
||||
failStatus("Dictation protocol error: outgoing frame is too large.");
|
||||
return false;
|
||||
}
|
||||
try {
|
||||
current.write(encoded);
|
||||
return true;
|
||||
} catch (error) {
|
||||
const detail = error instanceof Error ? error.message : "write failed";
|
||||
failStatus(`Dictation status request failed: ${detail}`);
|
||||
return false;
|
||||
}
|
||||
};
|
||||
|
||||
const send = (frame: Record<string, string | number>): boolean => {
|
||||
if (!socket) return false;
|
||||
const encoded = `${JSON.stringify(frame)}\n`;
|
||||
@@ -201,7 +260,7 @@ export function createDictateExtension(options: DictateExtensionOptions = {}) {
|
||||
return true;
|
||||
}
|
||||
if (serviceState === "loading" && state === "connecting") {
|
||||
setStatus("loading model...");
|
||||
setStatus("preparing transcription...");
|
||||
return true;
|
||||
}
|
||||
if (serviceState === "listening" && state === "connecting") {
|
||||
@@ -246,6 +305,12 @@ export function createDictateExtension(options: DictateExtensionOptions = {}) {
|
||||
}
|
||||
|
||||
if (value.type === "pong") {
|
||||
for (const field of ["backend", "model"] as const) {
|
||||
if (field in value && !isBoundedDiagnostic(value[field])) {
|
||||
protocolError(`diagnostic ${field} is invalid.`);
|
||||
return;
|
||||
}
|
||||
}
|
||||
if (
|
||||
negotiated ||
|
||||
state !== "connecting" ||
|
||||
@@ -344,45 +409,184 @@ export function createDictateExtension(options: DictateExtensionOptions = {}) {
|
||||
protocolError("unknown frame type.");
|
||||
};
|
||||
|
||||
const consumeData = (chunk: Uint8Array, current: DictateSocket) => {
|
||||
if (socket !== current) return;
|
||||
receiveBuffer = Buffer.concat([receiveBuffer, Buffer.from(chunk)]);
|
||||
while (socket === current) {
|
||||
const newline = receiveBuffer.indexOf(0x0a);
|
||||
const handleStatusFrame = (value: unknown, current: DictateSocket) => {
|
||||
if (statusSocket !== current) return;
|
||||
if (!isRecord(value)) {
|
||||
failStatus("Dictation status protocol error: frame must be an object.");
|
||||
return;
|
||||
}
|
||||
if (value.version !== PROTOCOL_VERSION) {
|
||||
failStatus("Dictation status protocol version mismatch.");
|
||||
return;
|
||||
}
|
||||
if (typeof value.type !== "string" || !isBoundedId(value.requestId)) {
|
||||
failStatus("Dictation status protocol error: frame is invalid.");
|
||||
return;
|
||||
}
|
||||
|
||||
if (value.type === "error") {
|
||||
const code = typeof value.code === "string" ? value.code : "service_error";
|
||||
const detail = typeof value.detail === "string" ? value.detail : "Dictation status failed";
|
||||
failStatus(
|
||||
`Dictation service error (${boundedText(code, 64)}): ${boundedText(detail, 384)}`,
|
||||
);
|
||||
return;
|
||||
}
|
||||
if (value.type !== "pong" || !statusRequestId || value.requestId !== statusRequestId) {
|
||||
failStatus("Dictation status protocol error: unexpected response.");
|
||||
return;
|
||||
}
|
||||
for (const field of ["backend", "model"] as const) {
|
||||
if (field in value && !isBoundedDiagnostic(value[field])) {
|
||||
failStatus(`Dictation status protocol error: diagnostic ${field} is invalid.`);
|
||||
return;
|
||||
}
|
||||
}
|
||||
if (isBoundedDiagnostic(value.backend)) statusBackend = value.backend;
|
||||
if (isBoundedDiagnostic(value.model)) statusModel = value.model;
|
||||
|
||||
if (statusRequest === "hello") {
|
||||
statusRequest = "ping";
|
||||
statusRequestId = newId("status-ping");
|
||||
sendStatus(current, {
|
||||
version: PROTOCOL_VERSION,
|
||||
type: "ping",
|
||||
requestId: statusRequestId,
|
||||
});
|
||||
return;
|
||||
}
|
||||
if (statusRequest !== "ping") {
|
||||
failStatus("Dictation status protocol error: status request is invalid.");
|
||||
return;
|
||||
}
|
||||
|
||||
const backend = statusBackend ?? "unknown";
|
||||
const model = statusModel ?? "unknown";
|
||||
const context = statusContext;
|
||||
closeStatusQuery();
|
||||
context?.ui.notify(
|
||||
`Dictation backend: ${backend}; model: ${model}`,
|
||||
"info",
|
||||
);
|
||||
};
|
||||
|
||||
const consumeFramedData = (
|
||||
chunk: Uint8Array,
|
||||
current: DictateSocket,
|
||||
isCurrent: () => boolean,
|
||||
getBuffer: () => Buffer<ArrayBufferLike>,
|
||||
setBuffer: (value: Buffer<ArrayBufferLike>) => void,
|
||||
onFrame: (value: unknown, socket: DictateSocket) => void,
|
||||
onProtocolError: (detail: string) => void,
|
||||
) => {
|
||||
if (!isCurrent()) return;
|
||||
let buffer = Buffer.concat([getBuffer(), Buffer.from(chunk)]);
|
||||
while (isCurrent()) {
|
||||
const newline = buffer.indexOf(0x0a);
|
||||
if (newline < 0) {
|
||||
if (receiveBuffer.length > MAX_FRAME_BYTES) {
|
||||
protocolError("frame is too large.");
|
||||
setBuffer(buffer);
|
||||
if (buffer.length > MAX_FRAME_BYTES) {
|
||||
onProtocolError("frame is too large.");
|
||||
}
|
||||
return;
|
||||
}
|
||||
if (newline > MAX_FRAME_BYTES) {
|
||||
protocolError("frame is too large.");
|
||||
setBuffer(buffer);
|
||||
onProtocolError("frame is too large.");
|
||||
return;
|
||||
}
|
||||
const line = receiveBuffer.subarray(0, newline);
|
||||
receiveBuffer = receiveBuffer.subarray(newline + 1);
|
||||
const line = buffer.subarray(0, newline);
|
||||
buffer = buffer.subarray(newline + 1);
|
||||
setBuffer(buffer);
|
||||
if (line.length === 0) {
|
||||
protocolError("empty frame.");
|
||||
onProtocolError("empty frame.");
|
||||
return;
|
||||
}
|
||||
let decoded: string;
|
||||
try {
|
||||
decoded = UTF8_DECODER.decode(line);
|
||||
} catch {
|
||||
protocolError("frame contains invalid UTF-8.");
|
||||
onProtocolError("frame contains invalid UTF-8.");
|
||||
return;
|
||||
}
|
||||
try {
|
||||
handleFrame(JSON.parse(decoded), current);
|
||||
onFrame(JSON.parse(decoded), current);
|
||||
} catch {
|
||||
protocolError("malformed JSON frame.");
|
||||
onProtocolError("malformed JSON frame.");
|
||||
return;
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
const consumeStatusData = (chunk: Uint8Array, current: DictateSocket) =>
|
||||
consumeFramedData(
|
||||
chunk,
|
||||
current,
|
||||
() => statusSocket === current,
|
||||
() => statusReceiveBuffer,
|
||||
(value) => {
|
||||
statusReceiveBuffer = value;
|
||||
},
|
||||
handleStatusFrame,
|
||||
(detail) => failStatus(`Dictation status protocol error: ${detail}`),
|
||||
);
|
||||
|
||||
const consumeData = (chunk: Uint8Array, current: DictateSocket) =>
|
||||
consumeFramedData(
|
||||
chunk,
|
||||
current,
|
||||
() => socket === current,
|
||||
() => receiveBuffer,
|
||||
(value) => {
|
||||
receiveBuffer = value;
|
||||
},
|
||||
handleFrame,
|
||||
protocolError,
|
||||
);
|
||||
|
||||
const statusDictation = (ctx: ExtensionContext) => {
|
||||
if (state !== "idle") {
|
||||
ctx.ui.notify("Dictation is active; status is unavailable.", "info");
|
||||
return;
|
||||
}
|
||||
if (statusSocket) return;
|
||||
|
||||
statusContext = ctx;
|
||||
statusRequest = "hello";
|
||||
statusRequestId = newId("status-hello");
|
||||
statusReceiveBuffer = Buffer.alloc(0);
|
||||
|
||||
let current: DictateSocket;
|
||||
try {
|
||||
current = socketFactory(socketPath);
|
||||
} catch (error) {
|
||||
const detail = error instanceof Error ? error.message : "connection failed";
|
||||
failStatus(`Unable to query dictation service: ${detail}`);
|
||||
return;
|
||||
}
|
||||
statusSocket = current;
|
||||
current.onConnect(() => {
|
||||
if (statusSocket !== current || !statusRequestId) return;
|
||||
sendStatus(current, {
|
||||
version: PROTOCOL_VERSION,
|
||||
type: "hello",
|
||||
requestId: statusRequestId,
|
||||
});
|
||||
});
|
||||
current.onData((chunk) => consumeStatusData(chunk, current));
|
||||
current.onError((error) => {
|
||||
if (statusSocket !== current) return;
|
||||
failStatus(`Dictation status connection failed: ${error.message}`);
|
||||
});
|
||||
current.onClose(() => {
|
||||
if (statusSocket !== current) return;
|
||||
failStatus("Dictation status service disconnected.");
|
||||
});
|
||||
};
|
||||
|
||||
const startDictation = (ctx: ExtensionContext) => {
|
||||
if (state !== "idle") return;
|
||||
closeStatusQuery(true);
|
||||
state = "connecting";
|
||||
activeContext = ctx;
|
||||
activeGenerationId = newId("generation");
|
||||
@@ -438,18 +642,23 @@ export function createDictateExtension(options: DictateExtensionOptions = {}) {
|
||||
};
|
||||
|
||||
pi.registerShortcut("alt+m", {
|
||||
description: "Start or stop local voice dictation",
|
||||
description: "Start or stop voice dictation",
|
||||
handler: async (ctx) => toggleDictation(ctx),
|
||||
});
|
||||
pi.registerShortcut("alt+n", {
|
||||
description: "Cancel local voice dictation",
|
||||
description: "Cancel voice dictation",
|
||||
handler: async () => cancelDictation(),
|
||||
});
|
||||
pi.registerCommand("dictate-cancel", {
|
||||
description: "Cancel local voice dictation",
|
||||
description: "Cancel voice dictation",
|
||||
handler: async () => cancelDictation(),
|
||||
});
|
||||
pi.registerCommand("dictate-status", {
|
||||
description: "Show the selected dictation backend",
|
||||
handler: async (_args, ctx) => statusDictation(ctx),
|
||||
});
|
||||
pi.on("session_shutdown", async () => {
|
||||
closeStatusQuery(true);
|
||||
if (state === "idle") return;
|
||||
if (negotiated) sendGenerationCommand("cancel");
|
||||
reset();
|
||||
|
||||
+1081
-43
File diff suppressed because it is too large
Load Diff
@@ -1,13 +1,19 @@
|
||||
[project]
|
||||
name = "pi-dictate-service"
|
||||
version = "0.1.0"
|
||||
description = "Socket-activated local transcription service for the Pi dictate extension"
|
||||
description = "Socket-activated transcription service for the Pi dictate extension"
|
||||
requires-python = ">=3.12,<3.13"
|
||||
dependencies = [
|
||||
dependencies = []
|
||||
|
||||
[project.optional-dependencies]
|
||||
local = [
|
||||
"nemo-toolkit[asr]==3.0.0",
|
||||
"numpy==2.5.2",
|
||||
"torch==2.13.0",
|
||||
]
|
||||
cloud = [
|
||||
"websockets==15.0.1",
|
||||
]
|
||||
|
||||
[project.scripts]
|
||||
pi-dictate-service = "pi_dictate_service.server:main"
|
||||
|
||||
@@ -0,0 +1,530 @@
|
||||
"""Backend-neutral transcription and realtime turn abstractions."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
import json
|
||||
from queue import Empty, Full, Queue
|
||||
import threading
|
||||
import time
|
||||
from typing import Any, Protocol, runtime_checkable
|
||||
|
||||
from .recorder import Capture
|
||||
|
||||
MAX_PCM_QUEUE_BYTES = 240_000 # five seconds of 24 kHz mono PCM16
|
||||
MAX_PCM_QUEUE_CHUNKS = 64
|
||||
MAX_PROVIDER_EVENT_BYTES = 64 * 1024
|
||||
MAX_PROVIDER_EVENT_QUEUE_BYTES = 512 * 1024
|
||||
MAX_PROVIDER_EVENT_QUEUE_ITEMS = 128
|
||||
MAX_PROVIDER_TEXT_BYTES = 32 * 1024
|
||||
RECEIVE_TIMEOUT = object()
|
||||
|
||||
|
||||
class ProviderError(RuntimeError):
|
||||
"""A bounded, redacted provider failure suitable for the service boundary."""
|
||||
|
||||
def __init__(self, code: str, detail: str = "transcription provider failed") -> None:
|
||||
self.code = code
|
||||
try:
|
||||
self.detail = detail.encode("utf-8", errors="replace")[:2048].decode("utf-8", errors="ignore")
|
||||
except (AttributeError, UnicodeError):
|
||||
self.detail = "transcription provider failed"
|
||||
super().__init__(self.detail)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class BackendDescriptor:
|
||||
backend: str
|
||||
model: str
|
||||
|
||||
|
||||
@runtime_checkable
|
||||
class TranscriptionTurn(Protocol):
|
||||
sample_rate_hz: int
|
||||
retain_pcm: bool
|
||||
|
||||
def enqueue_pcm(self, chunk: bytes) -> None: ...
|
||||
def finish(self, capture: Capture | None = None) -> str: ...
|
||||
def poll_error(self) -> ProviderError | None: ...
|
||||
def cancel(self) -> None: ...
|
||||
def close(self) -> None: ...
|
||||
|
||||
@property
|
||||
def cleanup_complete(self) -> bool: ...
|
||||
|
||||
|
||||
@runtime_checkable
|
||||
class TranscriptionBackend(Protocol):
|
||||
descriptor: BackendDescriptor
|
||||
sample_rate_hz: int
|
||||
|
||||
def start_turn(self) -> TranscriptionTurn: ...
|
||||
|
||||
def cancel_prepare(self) -> None: ...
|
||||
|
||||
|
||||
class PcmQueue:
|
||||
"""A byte- and item-bounded nonblocking queue for recorder drain threads."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
*,
|
||||
max_bytes: int = MAX_PCM_QUEUE_BYTES,
|
||||
max_chunks: int = MAX_PCM_QUEUE_CHUNKS,
|
||||
) -> None:
|
||||
if max_bytes <= 0 or max_chunks <= 0:
|
||||
raise ValueError("PCM queue bounds must be positive")
|
||||
self._queue: Queue[bytes | None] = Queue(maxsize=max_chunks)
|
||||
self._max_bytes = max_bytes
|
||||
self._queued_bytes = 0
|
||||
self._lock = threading.Lock()
|
||||
self._close_lock = threading.Lock()
|
||||
self._closed = False
|
||||
self._end_queued = False
|
||||
|
||||
@property
|
||||
def queued_bytes(self) -> int:
|
||||
with self._lock:
|
||||
return self._queued_bytes
|
||||
|
||||
def put(self, chunk: bytes) -> None:
|
||||
if not isinstance(chunk, bytes) or not chunk:
|
||||
raise ProviderError("queue_invalid", "provider received invalid audio")
|
||||
with self._lock:
|
||||
if self._closed:
|
||||
raise ProviderError("queue_closed", "provider audio queue is closed")
|
||||
if len(chunk) > self._max_bytes or self._queued_bytes + len(chunk) > self._max_bytes:
|
||||
raise ProviderError("queue_overflow", "provider audio queue is full")
|
||||
self._queued_bytes += len(chunk)
|
||||
try:
|
||||
self._queue.put_nowait(chunk)
|
||||
except Full as error:
|
||||
with self._lock:
|
||||
self._queued_bytes -= len(chunk)
|
||||
raise ProviderError("queue_overflow", "provider audio queue is full") from error
|
||||
|
||||
def close(self, *, timeout: float = 15.0) -> None:
|
||||
if not self._close_lock.acquire(timeout=max(0.0, timeout)):
|
||||
raise ProviderError("queue_overflow", "provider audio queue could not be finalized")
|
||||
try:
|
||||
with self._lock:
|
||||
if self._end_queued:
|
||||
return
|
||||
self._closed = True
|
||||
# A full queue still has valid audio that must be drained. The
|
||||
# closed flag is a side-channel EOF marker when the sentinel cannot
|
||||
# be inserted yet, rather than dropping the queued tail.
|
||||
try:
|
||||
self._queue.put_nowait(None)
|
||||
except Full:
|
||||
pass
|
||||
with self._lock:
|
||||
self._end_queued = True
|
||||
finally:
|
||||
self._close_lock.release()
|
||||
|
||||
def get(self, timeout: float | None = None) -> bytes | None:
|
||||
deadline = None if timeout is None else time.monotonic() + timeout
|
||||
while True:
|
||||
poll_timeout = 0.1 if deadline is None else max(0.0, min(0.1, deadline - time.monotonic()))
|
||||
try:
|
||||
chunk = self._queue.get(timeout=poll_timeout)
|
||||
except Empty:
|
||||
with self._lock:
|
||||
closed = self._closed
|
||||
if closed:
|
||||
return None
|
||||
if deadline is not None and time.monotonic() >= deadline:
|
||||
raise
|
||||
continue
|
||||
if chunk is not None:
|
||||
with self._lock:
|
||||
self._queued_bytes -= len(chunk)
|
||||
return chunk
|
||||
|
||||
|
||||
class LocalNemoTurn:
|
||||
"""Post-stop local inference turn retaining audio only in the recorder."""
|
||||
|
||||
sample_rate_hz = 16_000
|
||||
retain_pcm = True
|
||||
|
||||
def __init__(self, model: Any) -> None:
|
||||
self._model = model
|
||||
self._cancelled = False
|
||||
self._closed = False
|
||||
self._finished = False
|
||||
|
||||
@property
|
||||
def cleanup_complete(self) -> bool:
|
||||
return self._closed
|
||||
|
||||
def enqueue_pcm(self, chunk: bytes) -> None:
|
||||
# Local turns use Capture.pcm so no second copy is made here.
|
||||
if not self.retain_pcm:
|
||||
raise ProviderError("local_invalid", "local turn cannot stream audio")
|
||||
|
||||
def finish(self, capture: Capture | None = None) -> str:
|
||||
if self._cancelled:
|
||||
raise ProviderError("local_cancelled", "local turn was cancelled")
|
||||
if self._closed or self._finished:
|
||||
raise ProviderError("local_invalid", "local turn was already finalized")
|
||||
self._finished = True
|
||||
if capture is None:
|
||||
raise ProviderError("local_invalid", "local capture is missing")
|
||||
if capture.sample_rate_hz != self.sample_rate_hz:
|
||||
raise ProviderError("local_invalid", "local capture has an unsupported sample rate")
|
||||
from .audio import pcm_s16le_to_floats
|
||||
|
||||
max_bytes = getattr(self._model, "max_pcm_bytes", 5 * 60 * 16_000 * 2)
|
||||
samples = pcm_s16le_to_floats(capture.pcm, max_bytes=max_bytes)
|
||||
result = self._model.transcribe(samples)
|
||||
if not isinstance(result, str):
|
||||
raise ProviderError("local_failure", "local model returned an invalid transcript")
|
||||
return result
|
||||
|
||||
def poll_error(self) -> ProviderError | None:
|
||||
return None
|
||||
|
||||
def cancel(self) -> None:
|
||||
self._cancelled = True
|
||||
|
||||
def close(self) -> None:
|
||||
self._closed = True
|
||||
|
||||
|
||||
class LocalNemoBackend:
|
||||
"""Backend wrapper preserving the lazy, warm NeMo model behavior."""
|
||||
|
||||
sample_rate_hz = 16_000
|
||||
|
||||
def __init__(self, model: Any) -> None:
|
||||
self._model = model
|
||||
self.descriptor = BackendDescriptor("local", "nemo-parakeet")
|
||||
|
||||
@property
|
||||
def ready(self) -> bool:
|
||||
return bool(getattr(self._model, "loaded", False))
|
||||
|
||||
def start_turn(self) -> LocalNemoTurn:
|
||||
self._model.load()
|
||||
return LocalNemoTurn(self._model)
|
||||
|
||||
def cancel_prepare(self) -> None:
|
||||
# NeMo model loading is not interruptible, but the service tombstone
|
||||
# prevents a cancelled preparation from starting capture.
|
||||
return None
|
||||
|
||||
|
||||
class QueuedCloudTurn:
|
||||
"""Common ordered sender/receiver for cloud adapters.
|
||||
|
||||
Provider handshakes happen before this object is created. The recorder only
|
||||
calls ``enqueue_pcm``; it never performs network I/O. A sender thread drains
|
||||
the bounded queue and sends the end marker after every queued PCM chunk. A
|
||||
receiver thread continuously consumes provider events during capture so a
|
||||
websocket's receive buffer cannot stall the sender.
|
||||
"""
|
||||
|
||||
sample_rate_hz = 24_000
|
||||
retain_pcm = False
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
transport: Any,
|
||||
*,
|
||||
timeout: float = 15.0,
|
||||
max_queue_bytes: int = MAX_PCM_QUEUE_BYTES,
|
||||
max_queue_chunks: int = MAX_PCM_QUEUE_CHUNKS,
|
||||
) -> None:
|
||||
if max_queue_bytes <= 0 or max_queue_bytes > MAX_PCM_QUEUE_BYTES:
|
||||
raise ValueError("cloud PCM backlog must not exceed five seconds")
|
||||
self._transport = transport
|
||||
self._timeout = timeout
|
||||
self._queue = PcmQueue(max_bytes=max_queue_bytes, max_chunks=max_queue_chunks)
|
||||
self._provider_events: Queue[tuple[dict[str, Any], int]] = Queue(maxsize=MAX_PROVIDER_EVENT_QUEUE_ITEMS)
|
||||
self._provider_event_bytes = 0
|
||||
self._provider_closed = False
|
||||
self._sender_error: ProviderError | None = None
|
||||
self._receiver_error: ProviderError | None = None
|
||||
self._event_lock = threading.Lock()
|
||||
self._cancel_event = threading.Event()
|
||||
self._transport_close_lock = threading.Lock()
|
||||
self._transport_close_thread: threading.Thread | None = None
|
||||
self._cancelled = False
|
||||
self._finished = False
|
||||
self._closed = False
|
||||
self._sent_bytes = 0
|
||||
self._stream_started_at = time.monotonic()
|
||||
self._sender = threading.Thread(target=self._send_loop, daemon=True, name="dictate-provider-send")
|
||||
self._receiver = threading.Thread(target=self._receive_loop, daemon=True, name="dictate-provider-recv")
|
||||
self._sender.start()
|
||||
self._receiver.start()
|
||||
|
||||
@property
|
||||
def cleanup_complete(self) -> bool:
|
||||
close_thread = self._transport_close_thread
|
||||
return (
|
||||
not self._sender.is_alive()
|
||||
and not self._receiver.is_alive()
|
||||
and (close_thread is None or not close_thread.is_alive())
|
||||
)
|
||||
|
||||
def enqueue_pcm(self, chunk: bytes) -> None:
|
||||
if self._cancelled or self._finished:
|
||||
raise ProviderError(f"{self._provider_prefix}_queue_closed", "provider turn is no longer accepting audio")
|
||||
try:
|
||||
self._queue.put(chunk)
|
||||
except ProviderError as error:
|
||||
raise ProviderError(f"{self._provider_prefix}_{error.code}", error.detail) from error
|
||||
|
||||
def close_input(self) -> None:
|
||||
"""Signal EOF after the recorder has drained stdout to EOF."""
|
||||
self._queue.close()
|
||||
|
||||
def poll_error(self) -> ProviderError | None:
|
||||
error = self._sender_error or self._receiver_error
|
||||
if error is not None:
|
||||
return error
|
||||
with self._event_lock:
|
||||
provider_closed = self._provider_closed
|
||||
if provider_closed and not self._finished and not self._cancelled:
|
||||
return ProviderError(
|
||||
self._provider_prefix + "_premature_close",
|
||||
"provider closed before recording was finalized",
|
||||
)
|
||||
return None
|
||||
|
||||
def finish(self, capture: Capture | None = None) -> str:
|
||||
if self._cancelled:
|
||||
raise ProviderError("cancelled", "provider turn was cancelled")
|
||||
if self._finished:
|
||||
raise ProviderError("invalid_state", "provider turn was already finalized")
|
||||
self._finished = True
|
||||
try:
|
||||
self._queue.close(timeout=self._timeout)
|
||||
except ProviderError:
|
||||
self.cancel()
|
||||
raise
|
||||
self._sender.join(timeout=self._timeout)
|
||||
if self._sender.is_alive():
|
||||
self.cancel()
|
||||
raise ProviderError(self._provider_prefix + "_timeout", "provider audio sender timed out")
|
||||
if self._sender_error is not None:
|
||||
error = self._sender_error
|
||||
self.close()
|
||||
raise error
|
||||
try:
|
||||
transcript = self._finish_provider()
|
||||
# A correlated final isn't enough to publish a result while the
|
||||
# receiver can still report a queued provider failure. Stop and
|
||||
# join it first, then make the receiver error check authoritative.
|
||||
self._cancel_event.set()
|
||||
self._request_transport_close()
|
||||
self._receiver.join(timeout=self._timeout)
|
||||
if self._receiver.is_alive():
|
||||
raise ProviderError(
|
||||
self._provider_prefix + "_timeout",
|
||||
"provider receiver did not stop after the final transcript",
|
||||
)
|
||||
if self._receiver_error is not None:
|
||||
raise self._receiver_error
|
||||
return transcript
|
||||
finally:
|
||||
self.close()
|
||||
|
||||
def cancel(self) -> None:
|
||||
if self._cancelled and self.cleanup_complete:
|
||||
return
|
||||
self._cancelled = True
|
||||
self._cancel_event.set()
|
||||
self._request_transport_close()
|
||||
try:
|
||||
self._queue.close(timeout=0.1)
|
||||
except ProviderError:
|
||||
pass
|
||||
# Closing the transport is the cancellation signal. Do not block the
|
||||
# protocol loop waiting for provider threads; ``cleanup_complete``
|
||||
# keeps the tombstoned generation busy until they actually exit.
|
||||
|
||||
def close(self) -> None:
|
||||
if self._closed and self.cleanup_complete:
|
||||
return
|
||||
self._cancel_event.set()
|
||||
self._request_transport_close()
|
||||
# Keep close bounded and nonblocking from the service loop. A later
|
||||
# poll retries the joins while cleanup_complete prevents a new turn.
|
||||
for thread in (self._sender, self._receiver, self._transport_close_thread):
|
||||
if thread is not None:
|
||||
thread.join(timeout=min(self._timeout, 0.05))
|
||||
self._closed = self.cleanup_complete
|
||||
|
||||
def _send_loop(self) -> None:
|
||||
try:
|
||||
while True:
|
||||
chunk = self._queue.get(timeout=self._timeout)
|
||||
if chunk is None:
|
||||
# Queue closure means commit only for a successful recorder
|
||||
# EOF. Cancellation closes the same bounded queue merely to
|
||||
# wake this worker and must never finalize partial audio.
|
||||
if not self._cancel_event.is_set():
|
||||
self._send_end()
|
||||
return
|
||||
# Pace the stream against the declared PCM rate. This prevents
|
||||
# a fast recorder from building more than five seconds of
|
||||
# provider-side backlog while preserving ordered sends.
|
||||
target_elapsed = self._sent_bytes / (self.sample_rate_hz * 2)
|
||||
elapsed = time.monotonic() - self._stream_started_at
|
||||
if target_elapsed > elapsed and self._cancel_event.wait(target_elapsed - elapsed):
|
||||
return
|
||||
if self._cancel_event.is_set():
|
||||
return
|
||||
self._send_audio(chunk)
|
||||
self._sent_bytes += len(chunk)
|
||||
except ProviderError as error:
|
||||
self._sender_error = error
|
||||
self._abort_transport()
|
||||
except Empty:
|
||||
self._sender_error = ProviderError(
|
||||
self._provider_prefix + "_timeout", "provider audio sender timed out"
|
||||
)
|
||||
self._abort_transport()
|
||||
except Exception:
|
||||
self._sender_error = ProviderError(
|
||||
self._provider_prefix + "_send", "provider audio upload failed"
|
||||
)
|
||||
self._abort_transport()
|
||||
|
||||
def _receive_loop(self) -> None:
|
||||
try:
|
||||
while not self._cancel_event.is_set():
|
||||
event = self._receive_provider_event(timeout=0.1)
|
||||
if event is RECEIVE_TIMEOUT:
|
||||
continue
|
||||
if event is None:
|
||||
with self._event_lock:
|
||||
self._provider_closed = True
|
||||
return
|
||||
if not isinstance(event, dict):
|
||||
raise ProviderError(self._provider_prefix + "_malformed_event", "provider event is invalid")
|
||||
try:
|
||||
size = len(json.dumps(event, ensure_ascii=False, separators=(",", ":")).encode("utf-8"))
|
||||
except (TypeError, UnicodeError, ValueError, RecursionError, OverflowError) as error:
|
||||
raise ProviderError(self._provider_prefix + "_malformed_event", "provider event is invalid") from error
|
||||
with self._event_lock:
|
||||
if self._provider_event_bytes + size > MAX_PROVIDER_EVENT_QUEUE_BYTES:
|
||||
raise ProviderError(self._provider_prefix + "_event_overflow", "provider event queue is full")
|
||||
self._provider_event_bytes += size
|
||||
try:
|
||||
self._provider_events.put_nowait((event, size))
|
||||
except Full as error:
|
||||
with self._event_lock:
|
||||
self._provider_event_bytes -= size
|
||||
raise ProviderError(self._provider_prefix + "_event_overflow", "provider event queue is full") from error
|
||||
except ProviderError as error:
|
||||
if not self._cancel_event.is_set():
|
||||
self._receiver_error = error
|
||||
self._abort_transport()
|
||||
except Exception:
|
||||
if not self._cancel_event.is_set():
|
||||
self._receiver_error = ProviderError(
|
||||
self._provider_prefix + "_receive", "provider event stream failed"
|
||||
)
|
||||
self._abort_transport()
|
||||
|
||||
def _next_provider_event(self, timeout: float) -> dict[str, Any] | None:
|
||||
try:
|
||||
event, size = self._provider_events.get(timeout=max(0.0, timeout))
|
||||
except Empty as error:
|
||||
error_from_receiver = self._receiver_error
|
||||
if error_from_receiver is not None:
|
||||
raise error_from_receiver
|
||||
with self._event_lock:
|
||||
closed = self._provider_closed
|
||||
if closed:
|
||||
return None
|
||||
raise ProviderError(self._provider_prefix + "_timeout", "timed out waiting for provider event") from error
|
||||
with self._event_lock:
|
||||
self._provider_event_bytes -= size
|
||||
return event
|
||||
|
||||
@property
|
||||
def _provider_prefix(self) -> str:
|
||||
return "provider"
|
||||
|
||||
def _send_audio(self, chunk: bytes) -> None:
|
||||
raise NotImplementedError
|
||||
|
||||
def _send_end(self) -> None:
|
||||
raise NotImplementedError
|
||||
|
||||
def _finish_provider(self) -> str:
|
||||
raise NotImplementedError
|
||||
|
||||
def _receive_provider_event(self, *, timeout: float) -> dict[str, Any] | None | object:
|
||||
raise NotImplementedError
|
||||
|
||||
def _request_transport_close(self) -> None:
|
||||
with self._transport_close_lock:
|
||||
if self._transport_close_thread is not None:
|
||||
return
|
||||
self._transport_close_thread = threading.Thread(
|
||||
target=self._abort_transport,
|
||||
daemon=True,
|
||||
name="dictate-provider-close",
|
||||
)
|
||||
self._transport_close_thread.start()
|
||||
|
||||
def _abort_transport(self) -> None:
|
||||
try:
|
||||
self._transport.close()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def bounded_provider_text(value: Any, *, field: str = "transcript") -> str:
|
||||
if not isinstance(value, str):
|
||||
raise ProviderError("provider_malformed", f"provider {field} is invalid")
|
||||
try:
|
||||
if len(value.encode("utf-8")) > MAX_PROVIDER_TEXT_BYTES:
|
||||
raise ProviderError("provider_malformed", f"provider {field} is too large")
|
||||
except (UnicodeEncodeError, AttributeError) as error:
|
||||
raise ProviderError("provider_malformed", f"provider {field} is invalid") from error
|
||||
return value
|
||||
|
||||
|
||||
def provider_event(value: Any) -> dict[str, Any]:
|
||||
"""Validate one already decoded event and apply a serialized-size bound."""
|
||||
|
||||
if not isinstance(value, dict) or any(not isinstance(key, str) for key in value):
|
||||
raise ProviderError("provider_malformed", "provider event is not an object")
|
||||
try:
|
||||
encoded = json.dumps(value, ensure_ascii=False, separators=(",", ":")).encode("utf-8")
|
||||
except (TypeError, UnicodeError, ValueError, RecursionError, OverflowError) as error:
|
||||
raise ProviderError("provider_malformed", "provider event is invalid") from error
|
||||
if len(encoded) > MAX_PROVIDER_EVENT_BYTES:
|
||||
raise ProviderError("provider_malformed", "provider event is too large")
|
||||
return value
|
||||
|
||||
|
||||
def backend_from_config(config: Any) -> TranscriptionBackend:
|
||||
"""Construct only the selected backend, keeping cloud imports lazy."""
|
||||
if config.backend == "local":
|
||||
from .backends.local import LocalNemoBackend
|
||||
return LocalNemoBackend(config.local)
|
||||
if config.backend == "meta":
|
||||
from .backends.meta import MetaBackend
|
||||
return MetaBackend(config.meta)
|
||||
if config.backend == "openai":
|
||||
from .backends.openai import OpenAIBackend
|
||||
return OpenAIBackend(config.openai)
|
||||
raise ProviderError("configuration", "unsupported transcription backend")
|
||||
|
||||
|
||||
__all__ = [
|
||||
"BackendDescriptor", "LocalNemoBackend", "LocalNemoTurn", "MAX_PCM_QUEUE_BYTES",
|
||||
"MAX_PROVIDER_EVENT_BYTES", "MAX_PROVIDER_EVENT_QUEUE_BYTES", "MAX_PROVIDER_TEXT_BYTES",
|
||||
"PcmQueue", "ProviderError", "QueuedCloudTurn", "RECEIVE_TIMEOUT",
|
||||
"TranscriptionBackend", "TranscriptionTurn", "backend_from_config",
|
||||
"bounded_provider_text", "provider_event",
|
||||
]
|
||||
@@ -0,0 +1,8 @@
|
||||
"""Optional cloud and local Dictate backend adapters."""
|
||||
|
||||
from ..backend import backend_from_config
|
||||
from .local import LocalNemoBackend
|
||||
from .meta import MetaBackend, MetaTurn
|
||||
from .openai import OpenAIBackend, OpenAITurn
|
||||
|
||||
__all__ = ["LocalNemoBackend", "MetaBackend", "MetaTurn", "OpenAIBackend", "OpenAITurn", "backend_from_config"]
|
||||
@@ -0,0 +1,32 @@
|
||||
"""Local NeMo backend factory."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
from ..backend import LocalNemoBackend as _LocalNemoBackend
|
||||
from ..config import LocalConfig
|
||||
from ..model import NemoParakeetModel
|
||||
|
||||
|
||||
class LocalNemoBackend(_LocalNemoBackend):
|
||||
"""Build the existing lazy NeMo adapter from typed host configuration."""
|
||||
|
||||
def __init__(self, config: LocalConfig | Any) -> None:
|
||||
if isinstance(config, LocalConfig):
|
||||
model = NemoParakeetModel(
|
||||
config.model_path,
|
||||
terms_path=config.terms_path,
|
||||
boost_alpha=config.boost_alpha,
|
||||
expected_sha256=config.model_sha256,
|
||||
)
|
||||
else:
|
||||
model = config
|
||||
super().__init__(model)
|
||||
|
||||
@property
|
||||
def model(self) -> Any:
|
||||
return self._model
|
||||
|
||||
|
||||
__all__ = ["LocalNemoBackend"]
|
||||
@@ -0,0 +1,405 @@
|
||||
"""Meta Muse Voice Transcribe realtime adapter.
|
||||
|
||||
The websocket client is imported only when cloud mode is selected. Tests inject
|
||||
an object implementing ``send``, ``recv`` and ``close``; no network dependency is
|
||||
needed to import or test the service in local mode.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
from queue import Empty
|
||||
import threading
|
||||
import time
|
||||
from typing import Any, Callable, Mapping
|
||||
|
||||
from ..backend import (
|
||||
BackendDescriptor,
|
||||
MAX_PROVIDER_EVENT_BYTES,
|
||||
ProviderError,
|
||||
QueuedCloudTurn,
|
||||
RECEIVE_TIMEOUT,
|
||||
bounded_provider_text,
|
||||
provider_event,
|
||||
)
|
||||
from ..config import CredentialError, MetaConfig, provider_keywords, read_credential
|
||||
|
||||
META_ENDPOINT = "wss://api.meta.ai/v1/asr/realtime"
|
||||
META_MODEL = "muse-voice-transcribe-1.0"
|
||||
META_SAMPLE_RATE_HZ = 24_000
|
||||
META_MAX_BACKLOG_BYTES = 240_000 # provider's documented five-second limit
|
||||
META_HANDSHAKE_TIMEOUT = 10.0
|
||||
META_EVENT_TIMEOUT = 45.0
|
||||
META_SOCKET_IO_TIMEOUT = 10.0
|
||||
META_URL = META_ENDPOINT
|
||||
|
||||
TransportFactory = Callable[..., Any]
|
||||
|
||||
|
||||
def _default_transport(endpoint: str, **_kwargs: Any) -> Any:
|
||||
try:
|
||||
from websockets.sync.client import connect
|
||||
except ImportError as error:
|
||||
raise ProviderError("meta_dependency", "Meta backend websocket dependency is unavailable") from error
|
||||
try:
|
||||
logger = logging.getLogger("pi_dictate_service.provider.meta")
|
||||
logger.handlers = [logging.NullHandler()]
|
||||
logger.propagate = False
|
||||
transport = connect(
|
||||
endpoint,
|
||||
open_timeout=META_HANDSHAKE_TIMEOUT,
|
||||
close_timeout=5,
|
||||
max_size=MAX_PROVIDER_EVENT_BYTES,
|
||||
proxy=None,
|
||||
logger=logger,
|
||||
)
|
||||
transport.socket.settimeout(META_SOCKET_IO_TIMEOUT)
|
||||
return transport
|
||||
except Exception as error:
|
||||
response = getattr(error, "response", None)
|
||||
status = getattr(response, "status_code", getattr(error, "status_code", getattr(error, "status", None)))
|
||||
if status in {401, 403}:
|
||||
raise ProviderError("meta_auth", "Meta authentication failed") from error
|
||||
if status == 429:
|
||||
raise ProviderError("meta_rate_limit", "Meta transcription rate limit reached") from error
|
||||
raise ProviderError("meta_connect", "could not connect to Meta transcription service") from error
|
||||
|
||||
|
||||
def _send_json(transport: Any, event: Mapping[str, Any]) -> None:
|
||||
try:
|
||||
payload = json.dumps(dict(event), separators=(",", ":"), ensure_ascii=False)
|
||||
if len(payload.encode("utf-8")) > MAX_PROVIDER_EVENT_BYTES:
|
||||
raise ProviderError("meta_event_too_large", "Meta transcription event is too large")
|
||||
transport.send(payload)
|
||||
except ProviderError:
|
||||
raise
|
||||
except Exception as error:
|
||||
raise ProviderError("meta_send", "could not send Meta transcription event") from error
|
||||
|
||||
|
||||
def _receive_event(
|
||||
transport: Any, *, timeout: float, tolerate_timeout: bool = False
|
||||
) -> dict[str, Any] | None | object:
|
||||
try:
|
||||
try:
|
||||
raw = transport.recv(timeout=timeout)
|
||||
except TypeError:
|
||||
raw = transport.recv()
|
||||
except (TimeoutError, Empty):
|
||||
if tolerate_timeout:
|
||||
return RECEIVE_TIMEOUT
|
||||
raise ProviderError("meta_timeout", "timed out waiting for Meta transcription service")
|
||||
except Exception as error:
|
||||
code = _close_code(error)
|
||||
if code == 1000:
|
||||
return None
|
||||
if code in {1008, 1011, 1013}:
|
||||
raise ProviderError(f"meta_close_{code}", "Meta transcription service closed the session") from error
|
||||
raise ProviderError("meta_premature_close", "Meta transcription service closed unexpectedly") from error
|
||||
if raw is None:
|
||||
return None
|
||||
if isinstance(raw, dict):
|
||||
try:
|
||||
return provider_event(raw)
|
||||
except ProviderError as error:
|
||||
raise ProviderError("meta_malformed_event", "Meta sent an invalid event") from error
|
||||
if isinstance(raw, bytes):
|
||||
if len(raw) > MAX_PROVIDER_EVENT_BYTES:
|
||||
raise ProviderError("meta_malformed_event", "Meta sent an oversized event")
|
||||
try:
|
||||
raw = raw.decode("utf-8")
|
||||
except UnicodeDecodeError as error:
|
||||
raise ProviderError("meta_malformed_event", "Meta sent an invalid event") from error
|
||||
if not isinstance(raw, str):
|
||||
raise ProviderError("meta_malformed_event", "Meta sent an invalid event")
|
||||
try:
|
||||
if len(raw.encode("utf-8")) > MAX_PROVIDER_EVENT_BYTES:
|
||||
raise ProviderError("meta_malformed_event", "Meta sent an oversized event")
|
||||
except UnicodeEncodeError as error:
|
||||
raise ProviderError("meta_malformed_event", "Meta sent an invalid event") from error
|
||||
try:
|
||||
value = json.loads(raw)
|
||||
except (UnicodeDecodeError, ValueError, RecursionError) as error:
|
||||
raise ProviderError("meta_malformed_event", "Meta sent malformed event JSON") from error
|
||||
try:
|
||||
return provider_event(value)
|
||||
except ProviderError as error:
|
||||
raise ProviderError("meta_malformed_event", "Meta sent an invalid event") from error
|
||||
|
||||
|
||||
def _close_transport(transport: Any) -> None:
|
||||
try:
|
||||
transport.close()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def _close_code(error: BaseException) -> int | None:
|
||||
code = getattr(error, "code", None)
|
||||
if isinstance(code, int):
|
||||
return code
|
||||
received = getattr(error, "rcvd", None)
|
||||
code = getattr(received, "code", None)
|
||||
return code if isinstance(code, int) else None
|
||||
|
||||
|
||||
class MetaTurn(QueuedCloudTurn):
|
||||
sample_rate_hz = META_SAMPLE_RATE_HZ
|
||||
retain_pcm = False
|
||||
|
||||
def _receive_provider_event(self, *, timeout: float) -> dict[str, Any] | None | object:
|
||||
event = _receive_event(self._transport, timeout=timeout, tolerate_timeout=True)
|
||||
if event is RECEIVE_TIMEOUT or event is None:
|
||||
return event
|
||||
_validate_meta_event(event)
|
||||
if event.get("type") == "error" or "error" in event:
|
||||
raise _meta_event_error(event)
|
||||
if event.get("type") == "transcript" and event.get("final") is True:
|
||||
return event
|
||||
# Partials, progress, endpointing, and diarization may be frequent.
|
||||
# Validate and discard them during capture instead of retaining an
|
||||
# unbounded session history for finish().
|
||||
return RECEIVE_TIMEOUT
|
||||
|
||||
@property
|
||||
def _provider_prefix(self) -> str:
|
||||
return "meta"
|
||||
|
||||
def _send_audio(self, chunk: bytes) -> None:
|
||||
try:
|
||||
self._transport.send(chunk)
|
||||
except Exception as error:
|
||||
raise ProviderError("meta_send", "could not send Meta audio") from error
|
||||
|
||||
def _send_end(self) -> None:
|
||||
_send_json(self._transport, {"type": "endStream"})
|
||||
|
||||
def _finish_provider(self) -> str:
|
||||
deadline = time.monotonic() + self._timeout
|
||||
transcript: str | None = None
|
||||
while time.monotonic() < deadline:
|
||||
event = self._next_provider_event(max(0.01, deadline - time.monotonic()))
|
||||
if event is None:
|
||||
if transcript is None:
|
||||
raise ProviderError("meta_premature_close", "Meta closed without a final transcript")
|
||||
return transcript
|
||||
_validate_meta_event(event)
|
||||
event_type = event["type"]
|
||||
if event_type == "error":
|
||||
raise _meta_event_error(event)
|
||||
if event_type != "transcript":
|
||||
# Partial, endpoint, and diarization events are intentionally
|
||||
# consumed but never exposed to the Pi protocol.
|
||||
continue
|
||||
if event.get("final") is not True:
|
||||
continue
|
||||
value = event.get("text", event.get("transcript"))
|
||||
if isinstance(value, Mapping):
|
||||
value = value.get("text")
|
||||
try:
|
||||
transcript = bounded_provider_text(value)
|
||||
except ProviderError as error:
|
||||
raise ProviderError("meta_malformed_event", "Meta sent an invalid transcript") from error
|
||||
# Keep reading until the provider's normal close. A subsequent
|
||||
# error or abnormal close therefore cannot be mistaken for success.
|
||||
raise ProviderError("meta_timeout", "timed out waiting for Meta final transcript")
|
||||
|
||||
|
||||
def _validate_meta_event(event: Mapping[str, Any]) -> None:
|
||||
event_type = event.get("type")
|
||||
if not isinstance(event_type, str) or not event_type or len(event_type.encode("utf-8")) > 128:
|
||||
raise ProviderError("meta_malformed_event", "Meta event type is invalid")
|
||||
if event_type == "transcript":
|
||||
if not isinstance(event.get("final"), bool):
|
||||
raise ProviderError("meta_malformed_event", "Meta transcript final flag is invalid")
|
||||
if "text" not in event and "transcript" not in event:
|
||||
raise ProviderError("meta_malformed_event", "Meta transcript text is missing")
|
||||
if event_type == "error" or "error" in event:
|
||||
nested = event.get("error")
|
||||
if nested is not None and not isinstance(nested, Mapping):
|
||||
raise ProviderError("meta_malformed_event", "Meta error event is invalid")
|
||||
value: Any = event.get("code", event.get("errorCode"))
|
||||
if isinstance(nested, Mapping):
|
||||
value = nested.get("code", nested.get("type", value))
|
||||
if not isinstance(value, (str, int)):
|
||||
raise ProviderError("meta_malformed_event", "Meta error code is invalid")
|
||||
|
||||
|
||||
def _meta_event_error(event: Mapping[str, Any]) -> ProviderError:
|
||||
value: Any = event.get("code", event.get("errorCode"))
|
||||
nested = event.get("error")
|
||||
if isinstance(nested, Mapping):
|
||||
value = nested.get("code", nested.get("type", value))
|
||||
normalized = str(value).lower() if isinstance(value, (str, int)) else ""
|
||||
if "auth" in normalized or "token" in normalized or normalized in {"401", "403"}:
|
||||
code = "meta_auth"
|
||||
elif "rate" in normalized or normalized in {"429", "1013"}:
|
||||
code = "meta_rate_limit"
|
||||
else:
|
||||
code = "meta_error"
|
||||
return ProviderError(code, "Meta transcription service reported an error")
|
||||
|
||||
|
||||
class MetaBackend:
|
||||
sample_rate_hz = META_SAMPLE_RATE_HZ
|
||||
descriptor = BackendDescriptor("meta", META_MODEL)
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
config: MetaConfig,
|
||||
*,
|
||||
transport_factory: TransportFactory = _default_transport,
|
||||
timeout: float = META_EVENT_TIMEOUT,
|
||||
max_queue_bytes: int = META_MAX_BACKLOG_BYTES,
|
||||
max_queue_chunks: int = 64,
|
||||
) -> None:
|
||||
self.config = config
|
||||
self._transport_factory = transport_factory
|
||||
self._timeout = timeout
|
||||
if max_queue_bytes > META_MAX_BACKLOG_BYTES:
|
||||
raise ValueError("Meta backlog must not exceed five seconds (240000 bytes)")
|
||||
self._max_queue_bytes = max_queue_bytes
|
||||
self._max_queue_chunks = max_queue_chunks
|
||||
self._prepare_lock = threading.Lock()
|
||||
self._preparing_transport: Any | None = None
|
||||
self._prepare_token = 0
|
||||
self._reserved_prepare_token: int | None = None
|
||||
|
||||
def begin_prepare(self) -> None:
|
||||
with self._prepare_lock:
|
||||
self._prepare_token += 1
|
||||
self._reserved_prepare_token = self._prepare_token
|
||||
|
||||
def cancel_prepare(self) -> None:
|
||||
with self._prepare_lock:
|
||||
self._prepare_token += 1
|
||||
transport = self._preparing_transport
|
||||
self._preparing_transport = None
|
||||
if transport is not None:
|
||||
threading.Thread(
|
||||
target=_close_transport,
|
||||
args=(transport,),
|
||||
daemon=True,
|
||||
name="dictate-meta-prepare-close",
|
||||
).start()
|
||||
|
||||
def start_turn(self) -> MetaTurn:
|
||||
transport: Any | None = None
|
||||
with self._prepare_lock:
|
||||
if self._reserved_prepare_token is None:
|
||||
self._prepare_token += 1
|
||||
prepare_token = self._prepare_token
|
||||
else:
|
||||
prepare_token = self._reserved_prepare_token
|
||||
self._reserved_prepare_token = None
|
||||
cancelled_before_start = prepare_token != self._prepare_token
|
||||
if cancelled_before_start:
|
||||
raise ProviderError("cancelled", "Meta preparation was cancelled")
|
||||
try:
|
||||
key = read_credential(self.config.api_key_file)
|
||||
keywords = provider_keywords(self.config.terms_path)
|
||||
transport = self._connect()
|
||||
with self._prepare_lock:
|
||||
if prepare_token != self._prepare_token:
|
||||
cancelled = True
|
||||
else:
|
||||
cancelled = False
|
||||
self._preparing_transport = transport
|
||||
if cancelled:
|
||||
threading.Thread(
|
||||
target=_close_transport,
|
||||
args=(transport,),
|
||||
daemon=True,
|
||||
name="dictate-meta-cancelled-connect-close",
|
||||
).start()
|
||||
raise ProviderError("cancelled", "Meta preparation was cancelled")
|
||||
handshake: dict[str, Any] = {
|
||||
"authorization": {"accessToken": f"Bearer {key}"},
|
||||
"audioEncoding": "PCM_24KHZ",
|
||||
"model": self.config.model,
|
||||
"mode": "PUSH_TO_TALK",
|
||||
"partialMode": "CUMULATIVE",
|
||||
"emitAudioProgress": False,
|
||||
}
|
||||
if keywords:
|
||||
handshake["keywords"] = keywords
|
||||
language_bias = getattr(self.config, "language_bias", ())
|
||||
if language_bias:
|
||||
handshake["languageBias"] = list(language_bias)
|
||||
_send_json(transport, handshake)
|
||||
deadline = time.monotonic() + self._timeout
|
||||
while time.monotonic() < deadline:
|
||||
event = _receive_event(transport, timeout=max(0.01, deadline - time.monotonic()))
|
||||
if event is None:
|
||||
raise ProviderError("meta_premature_close", "Meta closed before session acknowledgement")
|
||||
if not isinstance(event, dict):
|
||||
raise ProviderError("meta_malformed_event", "Meta session acknowledgement is invalid")
|
||||
if event.get("type") == "error" or "error" in event:
|
||||
raise _meta_event_error(event)
|
||||
if set(event) != {"sessionId"} or not isinstance(event.get("sessionId"), str) or not event["sessionId"]:
|
||||
raise ProviderError("meta_malformed_event", "Meta session acknowledgement is invalid")
|
||||
if len(event["sessionId"].encode("utf-8")) > 128:
|
||||
raise ProviderError("meta_malformed_event", "Meta session acknowledgement is too large")
|
||||
if isinstance(event.get("sessionId"), str) and event["sessionId"]:
|
||||
with self._prepare_lock:
|
||||
if prepare_token != self._prepare_token:
|
||||
cancelled = True
|
||||
else:
|
||||
cancelled = False
|
||||
self._preparing_transport = None
|
||||
if cancelled:
|
||||
raise ProviderError("cancelled", "Meta preparation was cancelled")
|
||||
return MetaTurn(
|
||||
transport,
|
||||
timeout=self._timeout,
|
||||
max_queue_bytes=self._max_queue_bytes,
|
||||
max_queue_chunks=self._max_queue_chunks,
|
||||
)
|
||||
raise ProviderError("meta_timeout", "timed out waiting for Meta session acknowledgement")
|
||||
except CredentialError as error:
|
||||
raise ProviderError("meta_credentials", "Meta credential is unavailable") from error
|
||||
except ProviderError:
|
||||
with self._prepare_lock:
|
||||
if prepare_token == self._prepare_token and self._preparing_transport is transport:
|
||||
self._preparing_transport = None
|
||||
if transport is not None:
|
||||
threading.Thread(
|
||||
target=_close_transport,
|
||||
args=(transport,),
|
||||
daemon=True,
|
||||
name="dictate-meta-prepare-error-close",
|
||||
).start()
|
||||
raise
|
||||
except Exception as error:
|
||||
with self._prepare_lock:
|
||||
if prepare_token == self._prepare_token and self._preparing_transport is transport:
|
||||
self._preparing_transport = None
|
||||
if transport is not None:
|
||||
threading.Thread(
|
||||
target=_close_transport,
|
||||
args=(transport,),
|
||||
daemon=True,
|
||||
name="dictate-meta-prepare-failure-close",
|
||||
).start()
|
||||
raise ProviderError("meta_connect", "could not prepare Meta transcription session") from error
|
||||
|
||||
def _connect(self) -> Any:
|
||||
try:
|
||||
return self._transport_factory(META_ENDPOINT)
|
||||
except TypeError:
|
||||
# Small test transports often accept no keyword arguments.
|
||||
return self._transport_factory(META_ENDPOINT, headers={})
|
||||
except ProviderError:
|
||||
raise
|
||||
except Exception as error:
|
||||
response = getattr(error, "response", None)
|
||||
status = getattr(response, "status_code", getattr(error, "status_code", getattr(error, "status", None)))
|
||||
if status in {401, 403}:
|
||||
raise ProviderError("meta_auth", "Meta authentication failed") from error
|
||||
if status == 429:
|
||||
raise ProviderError("meta_rate_limit", "Meta transcription rate limit reached") from error
|
||||
raise ProviderError("meta_connect", "could not connect to Meta transcription service") from error
|
||||
|
||||
|
||||
__all__ = ["META_ENDPOINT", "META_URL", "META_MODEL", "META_MAX_BACKLOG_BYTES", "META_HANDSHAKE_TIMEOUT", "META_EVENT_TIMEOUT", "META_SOCKET_IO_TIMEOUT", "MetaBackend", "MetaTurn"]
|
||||
@@ -0,0 +1,461 @@
|
||||
"""OpenAI GA Realtime transcription adapter."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import json
|
||||
import logging
|
||||
from queue import Empty
|
||||
import threading
|
||||
import time
|
||||
from typing import Any, Callable, Mapping
|
||||
|
||||
from ..backend import (
|
||||
BackendDescriptor,
|
||||
MAX_PROVIDER_EVENT_BYTES,
|
||||
ProviderError,
|
||||
QueuedCloudTurn,
|
||||
RECEIVE_TIMEOUT,
|
||||
bounded_provider_text,
|
||||
provider_event,
|
||||
)
|
||||
from ..config import OPENAI_DELAYS, CredentialError, OpenAIConfig, provider_keywords, read_credential
|
||||
|
||||
OPENAI_ENDPOINT = "wss://api.openai.com/v1/realtime?intent=transcription"
|
||||
OPENAI_SAMPLE_RATE_HZ = 24_000
|
||||
OPENAI_DEFAULT_MODEL = "gpt-live-transcribe"
|
||||
OPENAI_URL = OPENAI_ENDPOINT
|
||||
# Keep serialized append events comfortably below the provider limit and our
|
||||
# own provider-event bound. Recorder chunks are 4.8 KiB at 24 kHz.
|
||||
MAX_APPEND_BYTES = 32 * 1024
|
||||
OPENAI_SOCKET_IO_TIMEOUT = 10.0
|
||||
|
||||
TransportFactory = Callable[..., Any]
|
||||
|
||||
|
||||
def _default_transport(endpoint: str, *, headers: Mapping[str, str] | None = None, **_kwargs: Any) -> Any:
|
||||
try:
|
||||
from websockets.sync.client import connect
|
||||
except ImportError as error:
|
||||
raise ProviderError("openai_dependency", "OpenAI backend websocket dependency is unavailable") from error
|
||||
try:
|
||||
logger = logging.getLogger("pi_dictate_service.provider.openai")
|
||||
logger.handlers = [logging.NullHandler()]
|
||||
logger.propagate = False
|
||||
transport = connect(
|
||||
endpoint,
|
||||
additional_headers=dict(headers or {}),
|
||||
open_timeout=15,
|
||||
close_timeout=5,
|
||||
max_size=MAX_PROVIDER_EVENT_BYTES,
|
||||
proxy=None,
|
||||
logger=logger,
|
||||
)
|
||||
transport.socket.settimeout(OPENAI_SOCKET_IO_TIMEOUT)
|
||||
return transport
|
||||
except Exception as error:
|
||||
response = getattr(error, "response", None)
|
||||
status = getattr(response, "status_code", getattr(error, "status_code", getattr(error, "status", None)))
|
||||
if status in {401, 403}:
|
||||
raise ProviderError("openai_auth", "OpenAI authentication failed") from error
|
||||
if status == 429:
|
||||
raise ProviderError("openai_rate_limit", "OpenAI transcription rate limit reached") from error
|
||||
raise ProviderError("openai_connect", "could not connect to OpenAI transcription service") from error
|
||||
|
||||
|
||||
def _send_json(transport: Any, event: Mapping[str, Any]) -> None:
|
||||
try:
|
||||
payload = json.dumps(dict(event), separators=(",", ":"), ensure_ascii=False)
|
||||
if len(payload.encode("utf-8")) > MAX_PROVIDER_EVENT_BYTES:
|
||||
raise ProviderError("openai_event_too_large", "OpenAI transcription event is too large")
|
||||
transport.send(payload)
|
||||
except ProviderError:
|
||||
raise
|
||||
except Exception as error:
|
||||
raise ProviderError("openai_send", "could not send OpenAI transcription event") from error
|
||||
|
||||
|
||||
def _receive_event(
|
||||
transport: Any, *, timeout: float, tolerate_timeout: bool = False
|
||||
) -> dict[str, Any] | None | object:
|
||||
try:
|
||||
try:
|
||||
raw = transport.recv(timeout=timeout)
|
||||
except TypeError:
|
||||
raw = transport.recv()
|
||||
except (TimeoutError, Empty) as error:
|
||||
if tolerate_timeout:
|
||||
return RECEIVE_TIMEOUT
|
||||
raise ProviderError("openai_timeout", "timed out waiting for OpenAI transcription service") from error
|
||||
except Exception as error:
|
||||
code = _close_code(error)
|
||||
if code == 1000:
|
||||
return None
|
||||
if code in {1008, 1011, 1013}:
|
||||
raise ProviderError(f"openai_close_{code}", "OpenAI transcription service closed the session") from error
|
||||
raise ProviderError("openai_premature_close", "OpenAI transcription service closed unexpectedly") from error
|
||||
if raw is None:
|
||||
return None
|
||||
if isinstance(raw, dict):
|
||||
try:
|
||||
return provider_event(raw)
|
||||
except ProviderError as error:
|
||||
raise ProviderError("openai_malformed_event", "OpenAI sent an invalid event") from error
|
||||
if isinstance(raw, bytes):
|
||||
if len(raw) > MAX_PROVIDER_EVENT_BYTES:
|
||||
raise ProviderError("openai_malformed_event", "OpenAI sent an oversized event")
|
||||
try:
|
||||
raw = raw.decode("utf-8")
|
||||
except UnicodeDecodeError as error:
|
||||
raise ProviderError("openai_malformed_event", "OpenAI sent an invalid event") from error
|
||||
if not isinstance(raw, str):
|
||||
raise ProviderError("openai_malformed_event", "OpenAI sent an invalid event")
|
||||
try:
|
||||
if len(raw.encode("utf-8")) > MAX_PROVIDER_EVENT_BYTES:
|
||||
raise ProviderError("openai_malformed_event", "OpenAI sent an oversized event")
|
||||
except UnicodeEncodeError as error:
|
||||
raise ProviderError("openai_malformed_event", "OpenAI sent an invalid event") from error
|
||||
try:
|
||||
value = json.loads(raw)
|
||||
except (UnicodeDecodeError, ValueError, RecursionError) as error:
|
||||
raise ProviderError("openai_malformed_event", "OpenAI sent malformed event JSON") from error
|
||||
try:
|
||||
return provider_event(value)
|
||||
except ProviderError as error:
|
||||
raise ProviderError("openai_malformed_event", "OpenAI sent an invalid event") from error
|
||||
|
||||
|
||||
def _close_transport(transport: Any) -> None:
|
||||
try:
|
||||
transport.close()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def _close_code(error: BaseException) -> int | None:
|
||||
code = getattr(error, "code", None)
|
||||
if isinstance(code, int):
|
||||
return code
|
||||
received = getattr(error, "rcvd", None)
|
||||
code = getattr(received, "code", None)
|
||||
return code if isinstance(code, int) else None
|
||||
|
||||
|
||||
class OpenAITurn(QueuedCloudTurn):
|
||||
sample_rate_hz = OPENAI_SAMPLE_RATE_HZ
|
||||
retain_pcm = False
|
||||
|
||||
def __init__(self, transport: Any, *, timeout: float = 15.0, **kwargs: Any) -> None:
|
||||
self._committed_item_id: str | None = None
|
||||
super().__init__(transport, timeout=timeout, **kwargs)
|
||||
|
||||
def _receive_provider_event(self, *, timeout: float) -> dict[str, Any] | None | object:
|
||||
event = _receive_event(self._transport, timeout=timeout, tolerate_timeout=True)
|
||||
if event is RECEIVE_TIMEOUT or event is None:
|
||||
return event
|
||||
_validate_openai_event(event)
|
||||
if event.get("type") == "error" or event.get("type") == "conversation.item.input_audio_transcription.failed":
|
||||
raise _openai_event_error(event)
|
||||
if event.get("type") in {
|
||||
"input_audio_buffer.committed",
|
||||
"conversation.item.input_audio_transcription.completed",
|
||||
}:
|
||||
return event
|
||||
# Deltas and unrelated conversation events can be frequent. Validate
|
||||
# and discard them during capture rather than queueing session history.
|
||||
return RECEIVE_TIMEOUT
|
||||
|
||||
@property
|
||||
def _provider_prefix(self) -> str:
|
||||
return "openai"
|
||||
|
||||
def _send_audio(self, chunk: bytes) -> None:
|
||||
if len(chunk) > MAX_APPEND_BYTES:
|
||||
raise ProviderError("openai_audio_chunk", "OpenAI audio chunk is too large")
|
||||
event = {
|
||||
"type": "input_audio_buffer.append",
|
||||
"audio": base64.b64encode(chunk).decode("ascii"),
|
||||
}
|
||||
_send_json(self._transport, event)
|
||||
|
||||
def _send_end(self) -> None:
|
||||
# Commit starts transcription and deliberately does not create an
|
||||
# assistant response. Never send response.create here.
|
||||
_send_json(self._transport, {"type": "input_audio_buffer.commit"})
|
||||
|
||||
def _finish_provider(self) -> str:
|
||||
deadline = time.monotonic() + self._timeout
|
||||
transcript: str | None = None
|
||||
while time.monotonic() < deadline:
|
||||
event = self._next_provider_event(max(0.01, deadline - time.monotonic()))
|
||||
if event is None:
|
||||
if transcript is None:
|
||||
raise ProviderError("openai_premature_close", "OpenAI closed without a final transcript")
|
||||
return transcript
|
||||
_validate_openai_event(event)
|
||||
event_type = event["type"]
|
||||
if event_type == "error" or event_type == "conversation.item.input_audio_transcription.failed":
|
||||
raise _openai_event_error(event)
|
||||
if event_type == "input_audio_buffer.committed":
|
||||
item_id = event.get("item_id")
|
||||
if not isinstance(item_id, str) or not item_id:
|
||||
raise ProviderError("openai_malformed_event", "OpenAI commit acknowledgement is invalid")
|
||||
if len(item_id.encode("utf-8")) > 128:
|
||||
raise ProviderError("openai_malformed_event", "OpenAI commit acknowledgement is too large")
|
||||
if self._committed_item_id is not None and self._committed_item_id != item_id:
|
||||
raise ProviderError("openai_malformed_event", "OpenAI returned conflicting commit acknowledgements")
|
||||
self._committed_item_id = item_id
|
||||
continue
|
||||
if event_type == "conversation.item.input_audio_transcription.completed":
|
||||
item_id = event.get("item_id")
|
||||
if not isinstance(item_id, str) or not item_id:
|
||||
raise ProviderError("openai_malformed_event", "OpenAI completion correlation is invalid")
|
||||
if "transcript" not in event:
|
||||
raise ProviderError("openai_malformed_event", "OpenAI completion transcript is missing")
|
||||
try:
|
||||
text_value = bounded_provider_text(event["transcript"])
|
||||
except ProviderError as error:
|
||||
raise ProviderError("openai_malformed_event", "OpenAI sent an invalid transcript") from error
|
||||
if self._committed_item_id == item_id:
|
||||
# OpenAI keeps the realtime connection reusable after a
|
||||
# completed transcription. The turn is done as soon as
|
||||
# its correlated final arrives; finish() closes it.
|
||||
return text_value
|
||||
# Unmatched completions are stale events. They are validated
|
||||
# but never allowed to authorize an active generation.
|
||||
continue
|
||||
# Deltas and unrelated conversation events are intentionally ignored.
|
||||
raise ProviderError("openai_timeout", "timed out waiting for OpenAI final transcript")
|
||||
|
||||
|
||||
def _validate_openai_event(event: Mapping[str, Any]) -> None:
|
||||
event_type = event.get("type")
|
||||
if not isinstance(event_type, str) or not event_type or len(event_type.encode("utf-8")) > 128:
|
||||
raise ProviderError("openai_malformed_event", "OpenAI event type is invalid")
|
||||
if event_type == "error" or event_type == "conversation.item.input_audio_transcription.failed":
|
||||
error = event.get("error")
|
||||
if not isinstance(error, Mapping):
|
||||
raise ProviderError("openai_malformed_event", "OpenAI error event is invalid")
|
||||
code = error.get("code", error.get("type"))
|
||||
if not isinstance(code, (str, int)):
|
||||
raise ProviderError("openai_malformed_event", "OpenAI error event has an invalid code")
|
||||
if isinstance(code, str) and len(code.encode("utf-8")) > 128:
|
||||
raise ProviderError("openai_malformed_event", "OpenAI error event code is too large")
|
||||
|
||||
|
||||
def _openai_event_error(event: Mapping[str, Any]) -> ProviderError:
|
||||
error = event.get("error")
|
||||
if not isinstance(error, Mapping):
|
||||
raise ProviderError("openai_malformed_event", "OpenAI error event is invalid")
|
||||
code_value: Any = error.get("code", error.get("type", event.get("code")))
|
||||
normalized = str(code_value).lower() if isinstance(code_value, (str, int)) else ""
|
||||
if "auth" in normalized or "unauthor" in normalized or normalized in {"401", "403"}:
|
||||
code = "openai_auth"
|
||||
elif "rate" in normalized or normalized in {"429"}:
|
||||
code = "openai_rate_limit"
|
||||
else:
|
||||
code = "openai_error"
|
||||
return ProviderError(code, "OpenAI transcription service reported an error")
|
||||
|
||||
|
||||
class OpenAIBackend:
|
||||
sample_rate_hz = OPENAI_SAMPLE_RATE_HZ
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
config: OpenAIConfig,
|
||||
*,
|
||||
transport_factory: TransportFactory = _default_transport,
|
||||
timeout: float = 15.0,
|
||||
max_queue_bytes: int = 240_000,
|
||||
max_queue_chunks: int = 64,
|
||||
) -> None:
|
||||
if config.delay is not None and config.delay not in OPENAI_DELAYS:
|
||||
raise ValueError("unsupported OpenAI delay")
|
||||
self.config = config
|
||||
self.descriptor = BackendDescriptor("openai", config.model)
|
||||
self._transport_factory = transport_factory
|
||||
self._timeout = timeout
|
||||
if max_queue_bytes > 240_000:
|
||||
raise ValueError("OpenAI PCM backlog must not exceed five seconds (240000 bytes)")
|
||||
self._max_queue_bytes = max_queue_bytes
|
||||
self._max_queue_chunks = max_queue_chunks
|
||||
self._prepare_lock = threading.Lock()
|
||||
self._preparing_transport: Any | None = None
|
||||
self._prepare_token = 0
|
||||
self._reserved_prepare_token: int | None = None
|
||||
|
||||
def begin_prepare(self) -> None:
|
||||
with self._prepare_lock:
|
||||
self._prepare_token += 1
|
||||
self._reserved_prepare_token = self._prepare_token
|
||||
|
||||
def cancel_prepare(self) -> None:
|
||||
with self._prepare_lock:
|
||||
self._prepare_token += 1
|
||||
transport = self._preparing_transport
|
||||
self._preparing_transport = None
|
||||
if transport is not None:
|
||||
threading.Thread(
|
||||
target=_close_transport,
|
||||
args=(transport,),
|
||||
daemon=True,
|
||||
name="dictate-openai-prepare-close",
|
||||
).start()
|
||||
|
||||
def start_turn(self) -> OpenAITurn:
|
||||
transport: Any | None = None
|
||||
with self._prepare_lock:
|
||||
if self._reserved_prepare_token is None:
|
||||
self._prepare_token += 1
|
||||
prepare_token = self._prepare_token
|
||||
else:
|
||||
prepare_token = self._reserved_prepare_token
|
||||
self._reserved_prepare_token = None
|
||||
cancelled_before_start = prepare_token != self._prepare_token
|
||||
if cancelled_before_start:
|
||||
raise ProviderError("cancelled", "OpenAI preparation was cancelled")
|
||||
try:
|
||||
key = read_credential(self.config.api_key_file)
|
||||
keywords = provider_keywords(self.config.terms_path)
|
||||
transport = self._connect(key)
|
||||
with self._prepare_lock:
|
||||
if prepare_token != self._prepare_token:
|
||||
cancelled = True
|
||||
else:
|
||||
cancelled = False
|
||||
self._preparing_transport = transport
|
||||
if cancelled:
|
||||
threading.Thread(
|
||||
target=_close_transport,
|
||||
args=(transport,),
|
||||
daemon=True,
|
||||
name="dictate-openai-cancelled-connect-close",
|
||||
).start()
|
||||
raise ProviderError("cancelled", "OpenAI preparation was cancelled")
|
||||
self._wait_for_session_created(transport)
|
||||
_send_json(transport, self._session_update(keywords))
|
||||
self._wait_for_session_updated(transport)
|
||||
with self._prepare_lock:
|
||||
if prepare_token != self._prepare_token:
|
||||
cancelled = True
|
||||
else:
|
||||
cancelled = False
|
||||
self._preparing_transport = None
|
||||
if cancelled:
|
||||
raise ProviderError("cancelled", "OpenAI preparation was cancelled")
|
||||
return OpenAITurn(
|
||||
transport,
|
||||
timeout=self._timeout,
|
||||
max_queue_bytes=self._max_queue_bytes,
|
||||
max_queue_chunks=self._max_queue_chunks,
|
||||
)
|
||||
except CredentialError as error:
|
||||
raise ProviderError("openai_credentials", "OpenAI credential is unavailable") from error
|
||||
except ProviderError:
|
||||
with self._prepare_lock:
|
||||
if prepare_token == self._prepare_token and self._preparing_transport is transport:
|
||||
self._preparing_transport = None
|
||||
if transport is not None:
|
||||
threading.Thread(
|
||||
target=_close_transport,
|
||||
args=(transport,),
|
||||
daemon=True,
|
||||
name="dictate-openai-prepare-error-close",
|
||||
).start()
|
||||
raise
|
||||
except Exception as error:
|
||||
with self._prepare_lock:
|
||||
if prepare_token == self._prepare_token and self._preparing_transport is transport:
|
||||
self._preparing_transport = None
|
||||
if transport is not None:
|
||||
threading.Thread(
|
||||
target=_close_transport,
|
||||
args=(transport,),
|
||||
daemon=True,
|
||||
name="dictate-openai-prepare-failure-close",
|
||||
).start()
|
||||
raise ProviderError("openai_connect", "could not prepare OpenAI transcription session") from error
|
||||
|
||||
def _connect(self, key: str) -> Any:
|
||||
try:
|
||||
return self._transport_factory(
|
||||
OPENAI_ENDPOINT,
|
||||
headers={"Authorization": f"Bearer {key}"},
|
||||
)
|
||||
except TypeError as error:
|
||||
# Test transports may accept positional headers, but never fall
|
||||
# back to an unauthenticated connection.
|
||||
try:
|
||||
return self._transport_factory(OPENAI_ENDPOINT, {"Authorization": f"Bearer {key}"})
|
||||
except TypeError as positional_error:
|
||||
raise ProviderError("openai_connect", "OpenAI transport does not accept authenticated headers") from positional_error
|
||||
except ProviderError:
|
||||
raise
|
||||
except Exception as error:
|
||||
response = getattr(error, "response", None)
|
||||
status = getattr(response, "status_code", getattr(error, "status_code", getattr(error, "status", None)))
|
||||
if status in {401, 403}:
|
||||
raise ProviderError("openai_auth", "OpenAI authentication failed") from error
|
||||
if status == 429:
|
||||
raise ProviderError("openai_rate_limit", "OpenAI transcription rate limit reached") from error
|
||||
raise ProviderError("openai_connect", "could not connect to OpenAI transcription service") from error
|
||||
|
||||
def _wait_for_session_created(self, transport: Any) -> None:
|
||||
deadline = time.monotonic() + self._timeout
|
||||
while time.monotonic() < deadline:
|
||||
event = _receive_event(transport, timeout=max(0.01, deadline - time.monotonic()))
|
||||
if event is None:
|
||||
raise ProviderError("openai_premature_close", "OpenAI closed before session creation")
|
||||
_validate_openai_event(event)
|
||||
event_type = event["type"]
|
||||
if event_type == "error" or "error" in event:
|
||||
raise _openai_event_error(event)
|
||||
if event_type == "session.created":
|
||||
return
|
||||
raise ProviderError("openai_timeout", "timed out waiting for OpenAI session creation")
|
||||
|
||||
def _wait_for_session_updated(self, transport: Any) -> None:
|
||||
deadline = time.monotonic() + self._timeout
|
||||
while time.monotonic() < deadline:
|
||||
event = _receive_event(transport, timeout=max(0.01, deadline - time.monotonic()))
|
||||
if event is None:
|
||||
raise ProviderError("openai_premature_close", "OpenAI closed before session update")
|
||||
_validate_openai_event(event)
|
||||
event_type = event["type"]
|
||||
if event_type == "error" or "error" in event:
|
||||
raise _openai_event_error(event)
|
||||
if event_type == "session.updated":
|
||||
return
|
||||
raise ProviderError("openai_timeout", "timed out waiting for OpenAI session update")
|
||||
|
||||
def _session_update(self, keywords: list[str]) -> dict[str, Any]:
|
||||
transcription: dict[str, Any] = {"model": self.config.model}
|
||||
prompt = getattr(self.config, "prompt", None)
|
||||
languages = getattr(self.config, "languages", ())
|
||||
if prompt:
|
||||
transcription["prompt"] = prompt
|
||||
if keywords:
|
||||
transcription["keywords"] = keywords
|
||||
if languages:
|
||||
transcription["languages"] = list(languages)
|
||||
if self.config.delay:
|
||||
transcription["delay"] = self.config.delay
|
||||
return {
|
||||
"type": "session.update",
|
||||
"session": {
|
||||
"type": "transcription",
|
||||
"audio": {
|
||||
"input": {
|
||||
"format": {"type": "audio/pcm", "rate": OPENAI_SAMPLE_RATE_HZ},
|
||||
"transcription": transcription,
|
||||
"turn_detection": None,
|
||||
}
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
__all__ = [
|
||||
"OPENAI_DEFAULT_MODEL", "OPENAI_ENDPOINT", "OPENAI_URL", "OPENAI_SAMPLE_RATE_HZ", "OPENAI_SOCKET_IO_TIMEOUT", "OpenAIBackend", "OpenAITurn",
|
||||
]
|
||||
@@ -0,0 +1,388 @@
|
||||
"""Strict host-side configuration and credential handling for Dictate.
|
||||
|
||||
This module intentionally has no dependency on the provider SDKs. It is loaded by
|
||||
the socket service, outside Pi's sandbox, and never serializes credentials onto
|
||||
the protocol.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
import json
|
||||
import math
|
||||
import os
|
||||
from pathlib import Path
|
||||
import re
|
||||
import stat
|
||||
from typing import Any, Mapping
|
||||
import unicodedata
|
||||
|
||||
from .model import (
|
||||
MAX_CODING_TERMS,
|
||||
MAX_TERMS_FILE_BYTES,
|
||||
parse_coding_terms,
|
||||
)
|
||||
|
||||
CONFIG_VERSION = 1
|
||||
DEFAULT_IDLE_SECONDS = 15 * 60
|
||||
MIN_IDLE_SECONDS = 60
|
||||
MAX_IDLE_SECONDS = 24 * 60 * 60
|
||||
MAX_CONFIG_BYTES = 64 * 1024
|
||||
MAX_CREDENTIAL_BYTES = 4096
|
||||
MAX_PROMPT_BYTES = 4096
|
||||
MAX_LANGUAGE_BYTES = 32
|
||||
MAX_LANGUAGES = 16
|
||||
MAX_KEYWORDS = MAX_CODING_TERMS
|
||||
MAX_PROVIDER_KEYWORDS_BYTES = 16 * 1024
|
||||
MAX_MODEL_BYTES = 128
|
||||
|
||||
_CONFIG_FIELDS = frozenset({"version", "backend", "idleSeconds", "local", "meta", "openai"})
|
||||
_LOCAL_FIELDS = frozenset({"adapter", "modelPath", "modelSha256", "termsPath", "boostAlpha"})
|
||||
_META_FIELDS = frozenset({"model", "apiKeyFile", "termsPath", "languageBias"})
|
||||
# ``delay`` is intentionally not accepted until the conflicting OpenAI
|
||||
# documentation is resolved by a credentialed live spike. It must not be
|
||||
# silently accepted and then omitted from the wire request.
|
||||
_OPENAI_FIELDS = frozenset({"model", "apiKeyFile", "prompt", "termsPath", "languages", "delay"})
|
||||
OPENAI_DELAYS = frozenset({"minimal", "low", "medium", "high", "xhigh"})
|
||||
_HEX_SHA256 = re.compile(r"^[0-9a-fA-F]{64}$")
|
||||
|
||||
|
||||
class ConfigError(ValueError):
|
||||
"""A safe, user-facing configuration failure (never includes a key)."""
|
||||
|
||||
|
||||
class CredentialError(ConfigError):
|
||||
pass
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class LocalConfig:
|
||||
adapter: str
|
||||
model_path: Path
|
||||
model_sha256: str
|
||||
terms_path: Path
|
||||
boost_alpha: float
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class MetaConfig:
|
||||
model: str
|
||||
api_key_file: Path
|
||||
terms_path: Path
|
||||
language_bias: tuple[str, ...] = ()
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class OpenAIConfig:
|
||||
model: str
|
||||
api_key_file: Path
|
||||
terms_path: Path
|
||||
delay: str | None = None
|
||||
prompt: str | None = None
|
||||
languages: tuple[str, ...] = ()
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class DictateConfig:
|
||||
backend: str
|
||||
idle_seconds: float
|
||||
local: LocalConfig | None = None
|
||||
meta: MetaConfig | None = None
|
||||
openai: OpenAIConfig | None = None
|
||||
version: int = CONFIG_VERSION
|
||||
|
||||
@property
|
||||
def selected(self) -> LocalConfig | MetaConfig | OpenAIConfig:
|
||||
selected = getattr(self, self.backend)
|
||||
if selected is None:
|
||||
raise ConfigError("selected backend configuration is missing")
|
||||
return selected
|
||||
|
||||
|
||||
def _string(value: Any, field: str, *, nonempty: bool = True) -> str:
|
||||
if not isinstance(value, str) or (nonempty and not value):
|
||||
raise ConfigError(f"{field} must be a non-empty string")
|
||||
try:
|
||||
value.encode("utf-8")
|
||||
except UnicodeEncodeError as error:
|
||||
raise ConfigError(f"{field} must contain valid UTF-8") from error
|
||||
if "\x00" in value or "\r" in value or "\n" in value:
|
||||
raise ConfigError(f"{field} contains unsupported control characters")
|
||||
return value
|
||||
|
||||
|
||||
def _path(value: Any, field: str) -> Path:
|
||||
path = Path(_string(value, field)).expanduser()
|
||||
if not path.is_absolute():
|
||||
raise ConfigError(f"{field} must be an absolute path")
|
||||
return path
|
||||
|
||||
|
||||
def _section(value: Any, field: str, fields: frozenset[str]) -> Mapping[str, Any]:
|
||||
if not isinstance(value, Mapping):
|
||||
raise ConfigError(f"{field} configuration is required")
|
||||
unknown = set(value) - fields
|
||||
if unknown:
|
||||
raise ConfigError(f"unknown {field} configuration key")
|
||||
return value
|
||||
|
||||
|
||||
def _optional_string(section: Mapping[str, Any], field: str, default: str | None = None) -> str | None:
|
||||
value = section.get(field, default)
|
||||
if value is None:
|
||||
return None
|
||||
return _string(value, field)
|
||||
|
||||
|
||||
def parse_config(value: Any, *, config_path: str | os.PathLike[str] | None = None) -> DictateConfig:
|
||||
"""Parse a strict versioned configuration mapping without reading credentials."""
|
||||
|
||||
if not isinstance(value, Mapping):
|
||||
raise ConfigError("configuration root must be an object")
|
||||
unknown = set(value) - _CONFIG_FIELDS
|
||||
if unknown:
|
||||
raise ConfigError("unknown configuration key")
|
||||
version = value.get("version")
|
||||
if version != CONFIG_VERSION or isinstance(version, bool):
|
||||
raise ConfigError(f"unsupported configuration version (expected {CONFIG_VERSION})")
|
||||
backend = value.get("backend")
|
||||
if not isinstance(backend, str) or backend not in {"local", "meta", "openai"}:
|
||||
raise ConfigError("backend must be local, meta, or openai")
|
||||
idle = value.get("idleSeconds", DEFAULT_IDLE_SECONDS)
|
||||
if isinstance(idle, bool) or not isinstance(idle, int):
|
||||
raise ConfigError("idleSeconds must be an integer")
|
||||
if not MIN_IDLE_SECONDS <= float(idle) <= MAX_IDLE_SECONDS:
|
||||
raise ConfigError("idleSeconds is out of bounds")
|
||||
|
||||
local: LocalConfig | None = None
|
||||
meta: MetaConfig | None = None
|
||||
openai: OpenAIConfig | None = None
|
||||
if backend == "local":
|
||||
section = _section(value.get("local"), "local", _LOCAL_FIELDS)
|
||||
adapter = _string(section.get("adapter"), "local.adapter")
|
||||
if adapter != "nemo-parakeet":
|
||||
raise ConfigError("unsupported local adapter")
|
||||
model_path = _path(section.get("modelPath"), "local.modelPath")
|
||||
if model_path.suffix != ".nemo":
|
||||
raise ConfigError("local.modelPath must name a .nemo archive")
|
||||
model_sha = _string(section.get("modelSha256"), "local.modelSha256")
|
||||
if not _HEX_SHA256.fullmatch(model_sha):
|
||||
raise ConfigError("local.modelSha256 must be a SHA-256 digest")
|
||||
terms_path = _path(section.get("termsPath"), "local.termsPath")
|
||||
boost = section.get("boostAlpha")
|
||||
if isinstance(boost, bool) or not isinstance(boost, (int, float)) or not math.isfinite(float(boost)):
|
||||
raise ConfigError("local.boostAlpha must be a finite number")
|
||||
if not 0 < float(boost) <= 10:
|
||||
raise ConfigError("local.boostAlpha is out of bounds")
|
||||
local = LocalConfig(adapter, model_path, model_sha.lower(), terms_path, float(boost))
|
||||
elif backend == "meta":
|
||||
section = _section(value.get("meta"), "meta", _META_FIELDS)
|
||||
model = _string(section.get("model"), "meta.model")
|
||||
if model != "muse-voice-transcribe-1.0":
|
||||
raise ConfigError("unsupported Meta model")
|
||||
api_key_file = _path(section.get("apiKeyFile"), "meta.apiKeyFile")
|
||||
terms_path = _path(section.get("termsPath"), "meta.termsPath")
|
||||
language_bias_value = section.get("languageBias", ())
|
||||
if not isinstance(language_bias_value, (list, tuple)) or len(language_bias_value) > MAX_LANGUAGES:
|
||||
raise ConfigError("meta.languageBias must be a bounded array")
|
||||
language_bias: list[str] = []
|
||||
for language in language_bias_value:
|
||||
language_value = _string(language, "meta.languageBias entry")
|
||||
if len(language_value.encode("utf-8")) > MAX_LANGUAGE_BYTES:
|
||||
raise ConfigError("Meta language bias is too long")
|
||||
language_bias.append(language_value)
|
||||
meta = MetaConfig(model, api_key_file, terms_path, tuple(language_bias))
|
||||
else:
|
||||
section = _section(value.get("openai"), "openai", _OPENAI_FIELDS)
|
||||
model = _string(section.get("model"), "openai.model")
|
||||
if len(model.encode("utf-8")) > MAX_MODEL_BYTES:
|
||||
raise ConfigError("openai.model is too long")
|
||||
if model == "whisper-1":
|
||||
raise ConfigError("whisper-1 is not supported by the Realtime transcription backend")
|
||||
delay = _optional_string(section, "delay")
|
||||
if delay is not None and delay not in OPENAI_DELAYS:
|
||||
raise ConfigError("openai.delay is unsupported")
|
||||
prompt = _optional_string(section, "prompt")
|
||||
if prompt is not None and len(prompt.encode("utf-8")) > MAX_PROMPT_BYTES:
|
||||
raise ConfigError("openai.prompt is too large")
|
||||
languages_value = section.get("languages", ())
|
||||
if not isinstance(languages_value, (list, tuple)) or ("languages" in section and not languages_value):
|
||||
raise ConfigError("openai.languages must be a non-empty array when provided")
|
||||
if len(languages_value) > MAX_LANGUAGES:
|
||||
raise ConfigError("openai.languages has too many entries")
|
||||
languages: list[str] = []
|
||||
for language in languages_value:
|
||||
language_value = _string(language, "openai.languages entry")
|
||||
if len(language_value.encode("utf-8")) > MAX_LANGUAGE_BYTES:
|
||||
raise ConfigError("openai language is too long")
|
||||
languages.append(language_value)
|
||||
openai = OpenAIConfig(
|
||||
model=model,
|
||||
api_key_file=_path(section.get("apiKeyFile"), "openai.apiKeyFile"),
|
||||
terms_path=_path(section.get("termsPath"), "openai.termsPath"),
|
||||
delay=delay,
|
||||
prompt=prompt,
|
||||
languages=tuple(languages),
|
||||
)
|
||||
|
||||
config = DictateConfig(backend, float(idle), local, meta, openai)
|
||||
_validate_managed_credential_path(config)
|
||||
return config
|
||||
|
||||
|
||||
def _validate_managed_credential_path(config: DictateConfig) -> None:
|
||||
selected = config.selected
|
||||
if isinstance(selected, LocalConfig):
|
||||
return
|
||||
expected = Path("~/.config/pi-dictate/credentials").expanduser()
|
||||
# A test or service may use a different HOME. The fixed directory is
|
||||
# therefore checked after expansion, not by string matching the JSON.
|
||||
path = selected.api_key_file
|
||||
if path.parent != expected or path.name != f"{config.backend}-api-key":
|
||||
raise ConfigError("credential path must use the managed credential directory")
|
||||
|
||||
|
||||
def load_config(path: str | os.PathLike[str] | None = None) -> DictateConfig:
|
||||
"""Read and parse the service configuration, failing closed on file issues."""
|
||||
|
||||
config_path = Path(path or os.environ.get("PI_DICTATE_CONFIG_PATH", "~/.config/pi-dictate/config.json")).expanduser().absolute()
|
||||
try:
|
||||
info = config_path.lstat()
|
||||
if stat.S_ISLNK(info.st_mode) or not stat.S_ISREG(info.st_mode):
|
||||
raise ConfigError("configuration is not a regular file")
|
||||
if info.st_uid != os.getuid() or info.st_mode & 0o077:
|
||||
raise ConfigError("configuration ownership or permissions are insecure")
|
||||
if info.st_size > 64 * 1024:
|
||||
raise ConfigError("configuration is too large")
|
||||
raw = _read_bounded_descriptor(config_path, MAX_CONFIG_BYTES, require_private=True)
|
||||
value = json.loads(raw.decode("utf-8"))
|
||||
except ConfigError:
|
||||
raise
|
||||
except (OSError, UnicodeError, json.JSONDecodeError, ValueError) as error:
|
||||
raise ConfigError("could not read valid configuration") from error
|
||||
return parse_config(value, config_path=config_path)
|
||||
|
||||
|
||||
def _read_bounded_descriptor(
|
||||
path: Path,
|
||||
limit: int,
|
||||
*,
|
||||
require_private: bool = False,
|
||||
require_mode: int | None = None,
|
||||
) -> bytes:
|
||||
"""Read one regular file through an O_NOFOLLOW descriptor.
|
||||
|
||||
The descriptor is checked before reading, so a path replacement after the
|
||||
check cannot redirect the data consumed by the service.
|
||||
"""
|
||||
|
||||
descriptor: int | None = None
|
||||
try:
|
||||
flags = os.O_RDONLY | getattr(os, "O_CLOEXEC", 0) | getattr(os, "O_NOFOLLOW", 0)
|
||||
descriptor = os.open(path, flags)
|
||||
info = os.fstat(descriptor)
|
||||
if not stat.S_ISREG(info.st_mode):
|
||||
raise ConfigError("file is not a regular file")
|
||||
if info.st_uid != os.getuid():
|
||||
raise ConfigError("file ownership is insecure")
|
||||
if require_private and info.st_mode & 0o077:
|
||||
raise ConfigError("file permissions are insecure")
|
||||
if require_mode is not None and stat.S_IMODE(info.st_mode) != require_mode:
|
||||
raise ConfigError("file permissions are insecure")
|
||||
if info.st_size > limit:
|
||||
raise ConfigError("file is too large")
|
||||
chunks: list[bytes] = []
|
||||
remaining = limit + 1
|
||||
while remaining:
|
||||
chunk = os.read(descriptor, min(4096, remaining))
|
||||
if not chunk:
|
||||
break
|
||||
chunks.append(chunk)
|
||||
remaining -= len(chunk)
|
||||
raw = b"".join(chunks)
|
||||
if len(raw) > limit:
|
||||
raise ConfigError("file is too large")
|
||||
return raw
|
||||
finally:
|
||||
if descriptor is not None:
|
||||
try:
|
||||
os.close(descriptor)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
|
||||
def _reject_symlink_ancestry(path: Path) -> None:
|
||||
current = path.absolute()
|
||||
while current != current.parent:
|
||||
try:
|
||||
if current.is_symlink():
|
||||
raise CredentialError("credential path ancestry contains a symlink")
|
||||
except OSError as error:
|
||||
raise CredentialError("credential path ancestry could not be checked") from error
|
||||
current = current.parent
|
||||
|
||||
|
||||
def read_credential(path: str | os.PathLike[str]) -> str:
|
||||
"""Read one selected, mode-0600 regular credential without logging its value."""
|
||||
|
||||
credential_path = Path(path).expanduser()
|
||||
_reject_symlink_ancestry(credential_path.parent)
|
||||
try:
|
||||
raw = _read_bounded_descriptor(
|
||||
credential_path, MAX_CREDENTIAL_BYTES, require_mode=0o600
|
||||
)
|
||||
value = raw.decode("utf-8")
|
||||
except CredentialError:
|
||||
raise
|
||||
except (OSError, UnicodeError, ConfigError) as error:
|
||||
if isinstance(error, ConfigError):
|
||||
raise CredentialError(str(error)) from error
|
||||
raise CredentialError("could not read credential") from error
|
||||
if value.endswith("\n"):
|
||||
value = value[:-1]
|
||||
if not value or "\r" in value or "\n" in value or "\x00" in value:
|
||||
raise CredentialError("credential must be one non-empty line")
|
||||
return value
|
||||
|
||||
|
||||
def provider_keywords(path: str | os.PathLike[str]) -> list[str]:
|
||||
"""Load terms once through a bounded no-follow descriptor."""
|
||||
|
||||
try:
|
||||
raw = _read_bounded_descriptor(Path(path).expanduser(), MAX_TERMS_FILE_BYTES)
|
||||
except (OSError, ConfigError) as error:
|
||||
raise ConfigError("coding dictionary could not be read") from error
|
||||
# LF separates dictionary entries; CR is never accepted because it is
|
||||
# normalized by splitlines() and would otherwise silently alter a keyword.
|
||||
if b"\r" in raw:
|
||||
raise ConfigError("coding terms contain unsupported provider keyword characters")
|
||||
try:
|
||||
terms = parse_coding_terms(raw)
|
||||
except (OSError, UnicodeError, ValueError) as error:
|
||||
raise ConfigError("coding dictionary is invalid") from error
|
||||
if len(terms) > MAX_KEYWORDS:
|
||||
raise ConfigError("coding dictionary has too many provider keywords")
|
||||
if sum(len(term.encode("utf-8")) for term in terms) > MAX_PROVIDER_KEYWORDS_BYTES:
|
||||
raise ConfigError("coding dictionary provider keywords are too large")
|
||||
for term in terms:
|
||||
if (
|
||||
"<" in term
|
||||
or ">" in term
|
||||
or any(unicodedata.category(character) in {"Cc", "Cf"} for character in term)
|
||||
):
|
||||
raise ConfigError("coding terms contain unsupported provider keyword characters")
|
||||
return terms
|
||||
|
||||
|
||||
# Short aliases make the host-facing API convenient without exposing any raw
|
||||
# configuration dictionary to the service core.
|
||||
Config = DictateConfig
|
||||
read_api_key = read_credential
|
||||
|
||||
__all__ = [
|
||||
"CONFIG_VERSION", "Config", "ConfigError", "CredentialError", "DictateConfig",
|
||||
"MAX_CONFIG_BYTES",
|
||||
"LocalConfig", "MetaConfig", "OpenAIConfig", "parse_config", "load_config",
|
||||
"read_credential", "read_api_key", "provider_keywords", "MAX_PROVIDER_KEYWORDS_BYTES",
|
||||
"DEFAULT_IDLE_SECONDS",
|
||||
]
|
||||
@@ -0,0 +1,69 @@
|
||||
"""Opt-in live cloud contract smoke test using a short raw PCM fixture."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import os
|
||||
from pathlib import Path
|
||||
import stat
|
||||
|
||||
from .backend import backend_from_config
|
||||
from .config import load_config
|
||||
|
||||
MAX_SAMPLE_BYTES = 240_000 # five seconds of 24 kHz mono PCM16
|
||||
CHUNK_BYTES = 4_800
|
||||
|
||||
|
||||
def _read_sample(path: Path) -> bytes:
|
||||
descriptor = os.open(
|
||||
path,
|
||||
os.O_RDONLY | getattr(os, "O_CLOEXEC", 0) | getattr(os, "O_NOFOLLOW", 0),
|
||||
)
|
||||
try:
|
||||
info = os.fstat(descriptor)
|
||||
if not stat.S_ISREG(info.st_mode):
|
||||
raise ValueError("sample must be a regular file")
|
||||
if not 0 < info.st_size <= MAX_SAMPLE_BYTES or info.st_size % 2:
|
||||
raise ValueError("sample must contain at most five seconds of even-length PCM16")
|
||||
sample = bytearray()
|
||||
while len(sample) <= MAX_SAMPLE_BYTES:
|
||||
chunk = os.read(descriptor, min(64 * 1024, MAX_SAMPLE_BYTES + 1 - len(sample)))
|
||||
if not chunk:
|
||||
break
|
||||
sample.extend(chunk)
|
||||
if len(sample) != info.st_size:
|
||||
raise ValueError("sample changed while it was being read")
|
||||
return bytes(sample)
|
||||
finally:
|
||||
os.close(descriptor)
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Send one non-sensitive 24 kHz mono PCM16 sample to the configured cloud backend."
|
||||
)
|
||||
parser.add_argument("sample", type=Path, help="raw 24 kHz mono signed-16 little-endian PCM")
|
||||
parser.add_argument(
|
||||
"--config",
|
||||
type=Path,
|
||||
default=Path("~/.config/pi-dictate/config.json").expanduser(),
|
||||
)
|
||||
arguments = parser.parse_args()
|
||||
|
||||
config = load_config(arguments.config)
|
||||
if config.backend not in {"meta", "openai"}:
|
||||
parser.error("the configured backend must be meta or openai")
|
||||
backend = backend_from_config(config)
|
||||
sample = _read_sample(arguments.sample)
|
||||
turn = backend.start_turn()
|
||||
try:
|
||||
for offset in range(0, len(sample), CHUNK_BYTES):
|
||||
turn.enqueue_pcm(sample[offset : offset + CHUNK_BYTES])
|
||||
transcript = turn.finish()
|
||||
finally:
|
||||
turn.close()
|
||||
print(f"{backend.descriptor.backend}/{backend.descriptor.model}: {transcript!r}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -6,6 +6,7 @@ from copy import deepcopy
|
||||
import hashlib
|
||||
import os
|
||||
from pathlib import Path
|
||||
import stat
|
||||
from typing import Protocol, Sequence
|
||||
|
||||
MAX_TERMS_FILE_BYTES = 64 * 1024
|
||||
@@ -19,18 +20,19 @@ class TranscriptionModel(Protocol):
|
||||
def transcribe(self, samples: Sequence[float]) -> str: ...
|
||||
|
||||
|
||||
def load_coding_terms(path: str | os.PathLike[str]) -> list[str]:
|
||||
"""Load a bounded host-owned phrase list, ignoring comments and duplicates."""
|
||||
def parse_coding_terms(raw: bytes) -> list[str]:
|
||||
"""Parse one bounded, already-read host-owned phrase-list descriptor."""
|
||||
|
||||
terms_path = Path(path).expanduser()
|
||||
if not terms_path.is_file():
|
||||
raise ValueError("PI_DICTATE_TERMS_PATH must name an existing local dictionary")
|
||||
if terms_path.stat().st_size > MAX_TERMS_FILE_BYTES:
|
||||
if len(raw) > MAX_TERMS_FILE_BYTES:
|
||||
raise ValueError(f"coding dictionary exceeds {MAX_TERMS_FILE_BYTES} bytes")
|
||||
try:
|
||||
text = raw.decode("utf-8")
|
||||
except UnicodeDecodeError as error:
|
||||
raise ValueError("coding dictionary is not valid UTF-8") from error
|
||||
|
||||
terms: list[str] = []
|
||||
seen: set[str] = set()
|
||||
for raw_line in terms_path.read_text(encoding="utf-8").splitlines():
|
||||
for raw_line in text.splitlines():
|
||||
term = raw_line.strip()
|
||||
if not term or term.startswith("#"):
|
||||
continue
|
||||
@@ -44,9 +46,47 @@ def load_coding_terms(path: str | os.PathLike[str]) -> list[str]:
|
||||
return terms
|
||||
|
||||
|
||||
def load_coding_terms(path: str | os.PathLike[str]) -> list[str]:
|
||||
"""Load a bounded dictionary from a no-follow descriptor, avoiding TOCTOU."""
|
||||
|
||||
terms_path = Path(path).expanduser()
|
||||
descriptor: int | None = None
|
||||
try:
|
||||
flags = os.O_RDONLY | getattr(os, "O_CLOEXEC", 0) | getattr(os, "O_NOFOLLOW", 0)
|
||||
descriptor = os.open(terms_path, flags)
|
||||
info = os.fstat(descriptor)
|
||||
if not stat.S_ISREG(info.st_mode):
|
||||
raise ValueError("PI_DICTATE_TERMS_PATH must name a regular dictionary")
|
||||
if info.st_size > MAX_TERMS_FILE_BYTES:
|
||||
raise ValueError(f"coding dictionary exceeds {MAX_TERMS_FILE_BYTES} bytes")
|
||||
chunks: list[bytes] = []
|
||||
remaining = MAX_TERMS_FILE_BYTES + 1
|
||||
while remaining:
|
||||
chunk = os.read(descriptor, min(4096, remaining))
|
||||
if not chunk:
|
||||
break
|
||||
chunks.append(chunk)
|
||||
remaining -= len(chunk)
|
||||
raw = b"".join(chunks)
|
||||
if len(raw) > MAX_TERMS_FILE_BYTES:
|
||||
raise ValueError(f"coding dictionary exceeds {MAX_TERMS_FILE_BYTES} bytes")
|
||||
return parse_coding_terms(raw)
|
||||
except OSError as error:
|
||||
raise ValueError("PI_DICTATE_TERMS_PATH must name an existing local dictionary") from error
|
||||
finally:
|
||||
if descriptor is not None:
|
||||
try:
|
||||
os.close(descriptor)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
|
||||
class NemoParakeetModel:
|
||||
"""Load NeMo only when requested so core tests require neither CUDA nor NeMo."""
|
||||
|
||||
sample_rate_hz = 16_000
|
||||
max_pcm_bytes = 5 * 60 * sample_rate_hz * 2
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
model_path: str | os.PathLike[str],
|
||||
@@ -56,7 +96,11 @@ class NemoParakeetModel:
|
||||
expected_sha256: str = PINNED_MODEL_SHA256,
|
||||
) -> None:
|
||||
path = Path(model_path).expanduser()
|
||||
if path.suffix != ".nemo" or not path.is_file():
|
||||
try:
|
||||
info = path.lstat()
|
||||
except OSError as error:
|
||||
raise ValueError("PI_DICTATE_MODEL_PATH must name an existing local .nemo file") from error
|
||||
if path.suffix != ".nemo" or stat.S_ISLNK(info.st_mode) or not stat.S_ISREG(info.st_mode):
|
||||
raise ValueError("PI_DICTATE_MODEL_PATH must name an existing local .nemo file")
|
||||
if boost_alpha <= 0:
|
||||
raise ValueError("PI_DICTATE_BOOST_ALPHA must be positive")
|
||||
@@ -74,12 +118,21 @@ class NemoParakeetModel:
|
||||
def load(self) -> None:
|
||||
if self._model is not None:
|
||||
return
|
||||
with self._model_path.open("rb") as model_file:
|
||||
descriptor: int | None = None
|
||||
try:
|
||||
flags = os.O_RDONLY | getattr(os, "O_CLOEXEC", 0) | getattr(os, "O_NOFOLLOW", 0)
|
||||
descriptor = os.open(self._model_path, flags)
|
||||
info = os.fstat(descriptor)
|
||||
if not stat.S_ISREG(info.st_mode):
|
||||
raise ValueError("model archive is not a regular file")
|
||||
with os.fdopen(descriptor, "rb", closefd=False) as model_file:
|
||||
actual_sha256 = hashlib.file_digest(model_file, "sha256").hexdigest()
|
||||
if actual_sha256 != self._expected_sha256:
|
||||
raise ValueError(
|
||||
"model checksum mismatch: expected the pinned Parakeet revision"
|
||||
)
|
||||
os.lseek(descriptor, 0, os.SEEK_SET)
|
||||
|
||||
os.environ["HF_HUB_OFFLINE"] = "1"
|
||||
os.environ["TRANSFORMERS_OFFLINE"] = "1"
|
||||
os.environ["HF_DATASETS_OFFLINE"] = "1"
|
||||
@@ -90,8 +143,11 @@ class NemoParakeetModel:
|
||||
from nemo.collections.asr.models import EncDecRNNTBPEModel
|
||||
from omegaconf import open_dict
|
||||
|
||||
# Keep the verified inode open while NeMo restores it. Reopening
|
||||
# the configured pathname here would allow a replacement between
|
||||
# checksum verification and model loading.
|
||||
model = EncDecRNNTBPEModel.restore_from(
|
||||
restore_path=str(self._model_path),
|
||||
restore_path=f"/proc/self/fd/{descriptor}",
|
||||
map_location=torch.device("cuda"),
|
||||
)
|
||||
model.to("cuda")
|
||||
@@ -116,6 +172,14 @@ class NemoParakeetModel:
|
||||
model.change_decoding_strategy(decoding, verbose=False)
|
||||
self._numpy = numpy
|
||||
self._model = model
|
||||
except OSError as error:
|
||||
raise ValueError("model archive could not be read") from error
|
||||
finally:
|
||||
if descriptor is not None:
|
||||
try:
|
||||
os.close(descriptor)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
def transcribe(self, samples: Sequence[float]) -> str:
|
||||
if self._model is None or self._numpy is None:
|
||||
|
||||
@@ -51,9 +51,9 @@ _REQUEST_FIELDS = {
|
||||
|
||||
|
||||
def _bounded_utf8(value: str, limit: int) -> str:
|
||||
# Normalize replacement characters even when the original string contains
|
||||
# lone surrogates. This keeps error output serializable and bounded.
|
||||
raw = value.encode("utf-8", errors="replace")
|
||||
if len(raw) <= limit:
|
||||
return value
|
||||
return raw[:limit].decode("utf-8", errors="ignore")
|
||||
|
||||
|
||||
@@ -100,7 +100,11 @@ def parse_request(frame: Mapping[str, Any]) -> Request:
|
||||
raise ProtocolError("invalid_request", "unknown request type", request_id=request_id)
|
||||
|
||||
expected_fields = _REQUEST_FIELDS[request_type]
|
||||
if frozenset(frame) != expected_fields:
|
||||
try:
|
||||
actual_fields = frozenset(frame)
|
||||
except TypeError as error:
|
||||
raise ProtocolError("invalid_request", "request contains invalid field names", request_id=request_id) from error
|
||||
if actual_fields != expected_fields:
|
||||
raise ProtocolError(
|
||||
"invalid_request",
|
||||
f"{request_type} requires exactly: {', '.join(sorted(expected_fields))}",
|
||||
@@ -125,28 +129,38 @@ class FrameDecoder:
|
||||
def feed(self, data: bytes) -> list[dict[str, Any]]:
|
||||
if self._failed:
|
||||
raise ProtocolError("connection_closed", "decoder is closed", close_connection=True)
|
||||
self._buffer.extend(data)
|
||||
if not isinstance(data, bytes):
|
||||
self._fail("malformed_json", "frame data is invalid")
|
||||
frames: list[dict[str, Any]] = []
|
||||
remaining = data
|
||||
|
||||
while True:
|
||||
newline = self._buffer.find(b"\n")
|
||||
# Consume each newline-delimited frame before considering the next one.
|
||||
# The size limit applies to an individual frame, not to a recv batch:
|
||||
# clients may pipeline any number of bounded frames in one feed call.
|
||||
while remaining:
|
||||
newline = remaining.find(b"\n")
|
||||
if newline < 0:
|
||||
if len(self._buffer) > MAX_FRAME_BYTES:
|
||||
if len(self._buffer) + len(remaining) > MAX_FRAME_BYTES:
|
||||
self._fail("frame_too_large", f"frame exceeds {MAX_FRAME_BYTES} bytes")
|
||||
self._buffer.extend(remaining)
|
||||
return frames
|
||||
if newline > MAX_FRAME_BYTES:
|
||||
if len(self._buffer) + newline > MAX_FRAME_BYTES:
|
||||
self._fail("frame_too_large", f"frame exceeds {MAX_FRAME_BYTES} bytes")
|
||||
|
||||
raw = bytes(self._buffer[:newline])
|
||||
del self._buffer[: newline + 1]
|
||||
self._buffer.extend(remaining[:newline])
|
||||
raw = bytes(self._buffer)
|
||||
self._buffer.clear()
|
||||
remaining = remaining[newline + 1 :]
|
||||
try:
|
||||
value = json.loads(raw.decode("utf-8"))
|
||||
except (UnicodeDecodeError, ValueError):
|
||||
except (UnicodeDecodeError, ValueError, RecursionError):
|
||||
self._fail("malformed_json", "frame contains malformed JSON")
|
||||
if not isinstance(value, dict):
|
||||
self._fail("malformed_json", "frame must contain a JSON object")
|
||||
frames.append(value)
|
||||
|
||||
return frames
|
||||
|
||||
def _fail(self, code: str, detail: str) -> None:
|
||||
self._failed = True
|
||||
self._buffer.clear()
|
||||
@@ -161,8 +175,11 @@ def encode_frame(frame: Mapping[str, Any]) -> bytes:
|
||||
transcript = payload.get("transcript")
|
||||
if not isinstance(transcript, str):
|
||||
raise ProtocolError("invalid_event", "result transcript must be a string")
|
||||
try:
|
||||
if len(transcript.encode("utf-8")) > MAX_TRANSCRIPT_BYTES:
|
||||
raise ProtocolError("transcript_too_large", "transcript exceeds output limit")
|
||||
except UnicodeEncodeError as error:
|
||||
raise ProtocolError("invalid_event", "result transcript contains invalid Unicode") from error
|
||||
if payload.get("type") == "error":
|
||||
detail = payload.get("detail", "")
|
||||
if not isinstance(detail, str):
|
||||
@@ -175,7 +192,10 @@ def encode_frame(frame: Mapping[str, Any]) -> bytes:
|
||||
else:
|
||||
payload.pop("generationId", None)
|
||||
|
||||
try:
|
||||
encoded = json.dumps(payload, ensure_ascii=False, separators=(",", ":")).encode("utf-8")
|
||||
except (TypeError, ValueError, UnicodeError, RecursionError, OverflowError) as error:
|
||||
raise ProtocolError("invalid_event", "event contains unsupported values") from error
|
||||
if len(encoded) > MAX_FRAME_BYTES:
|
||||
raise ProtocolError("frame_too_large", "encoded frame exceeds output limit")
|
||||
return encoded + b"\n"
|
||||
@@ -186,8 +206,8 @@ def error_event(error: ProtocolError) -> dict[str, Any]:
|
||||
"version": PROTOCOL_VERSION,
|
||||
"type": "error",
|
||||
"requestId": _safe_id(error.request_id),
|
||||
"code": error.code,
|
||||
"detail": _bounded_utf8(error.detail, MAX_ERROR_DETAIL_BYTES),
|
||||
"code": _bounded_utf8(error.code if isinstance(error.code, str) else "service_error", MAX_ID_BYTES),
|
||||
"detail": _bounded_utf8(error.detail if isinstance(error.detail, str) else str(error.detail), MAX_ERROR_DETAIL_BYTES),
|
||||
}
|
||||
generation_id = _safe_id(error.generation_id)
|
||||
if generation_id:
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
"""PipeWire recorder adapter with bounded in-memory output."""
|
||||
"""PipeWire recorder adapter with bounded buffering and optional streaming."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
@@ -13,29 +13,42 @@ import threading
|
||||
import time
|
||||
from typing import BinaryIO, Callable, Protocol
|
||||
|
||||
from .audio import MAX_PCM_BYTES
|
||||
|
||||
PW_RECORD_COMMAND = [
|
||||
"pw-record",
|
||||
"--raw",
|
||||
"--format",
|
||||
"s16",
|
||||
"--rate",
|
||||
"16000",
|
||||
"--channels",
|
||||
"1",
|
||||
"-",
|
||||
"pw-record", "--raw", "--format", "s16", "--rate", "16000",
|
||||
"--channels", "1", "-",
|
||||
]
|
||||
MAX_STDERR_BYTES = 64 * 1024
|
||||
AUDIO_LEVEL_WINDOW_BYTES = 3_200 # 100 ms of mono 16 kHz signed-16 PCM
|
||||
|
||||
|
||||
def audio_level_window_bytes(sample_rate_hz: int) -> int:
|
||||
"""Return one 100 ms mono PCM16 meter window for a supported rate."""
|
||||
if sample_rate_hz not in {16_000, 24_000}:
|
||||
raise ValueError("unsupported recorder sample rate")
|
||||
return sample_rate_hz * 2 // 10
|
||||
|
||||
|
||||
def pw_record_command(sample_rate_hz: int) -> list[str]:
|
||||
"""Build the explicit mono PCM16 PipeWire command for a backend rate."""
|
||||
if sample_rate_hz not in {16_000, 24_000}:
|
||||
raise ValueError("unsupported recorder sample rate")
|
||||
command = list(PW_RECORD_COMMAND)
|
||||
command[command.index("16000")] = str(sample_rate_hz)
|
||||
return command
|
||||
|
||||
|
||||
class PcmSink(Protocol):
|
||||
"""Nonblocking sink used by realtime turns from the recorder drain thread."""
|
||||
def enqueue_pcm(self, chunk: bytes) -> None: ...
|
||||
def close_input(self) -> None: ...
|
||||
def cancel(self) -> None: ...
|
||||
|
||||
|
||||
class Process(Protocol):
|
||||
pid: int
|
||||
stdout: BinaryIO
|
||||
stderr: BinaryIO
|
||||
returncode: int | None
|
||||
|
||||
def poll(self) -> int | None: ...
|
||||
def terminate(self) -> None: ...
|
||||
def kill(self) -> None: ...
|
||||
@@ -52,17 +65,11 @@ class Capture:
|
||||
returncode: int
|
||||
signal: int | None
|
||||
stderr_tail: bytes
|
||||
sample_rate_hz: int = 16_000
|
||||
|
||||
|
||||
class RecorderError(RuntimeError):
|
||||
def __init__(
|
||||
self,
|
||||
code: str,
|
||||
detail: str,
|
||||
*,
|
||||
returncode: int | None = None,
|
||||
signal_number: int | None = None,
|
||||
) -> None:
|
||||
def __init__(self, code: str, detail: str, *, returncode: int | None = None, signal_number: int | None = None) -> None:
|
||||
super().__init__(detail)
|
||||
self.code = code
|
||||
self.detail = detail
|
||||
@@ -71,20 +78,40 @@ class RecorderError(RuntimeError):
|
||||
|
||||
|
||||
class PwRecorder:
|
||||
"""Capture raw PCM while continuously draining both child pipes."""
|
||||
"""Capture raw PCM while continuously draining both child pipes.
|
||||
|
||||
``pcm_sink`` receives each stdout chunk using a nonblocking call. The sink
|
||||
is closed only after stdout reaches EOF and the child has exited, providing
|
||||
the no-truncated-tail ordering required by realtime providers.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
*,
|
||||
sample_rate_hz: int = 16_000,
|
||||
pcm_sink: PcmSink | None = None,
|
||||
sample_rate: int | None = None,
|
||||
retain_pcm: bool = True,
|
||||
process_factory: ProcessFactory = subprocess.Popen,
|
||||
max_bytes: int = MAX_PCM_BYTES,
|
||||
max_bytes: int | None = None,
|
||||
stderr_limit: int = MAX_STDERR_BYTES,
|
||||
stop_timeout: float = 3.0,
|
||||
clock: Callable[[], float] = time.monotonic,
|
||||
process_group_signal: ProcessGroupSignal = os.killpg,
|
||||
) -> None:
|
||||
if sample_rate is not None:
|
||||
if sample_rate_hz != 16_000 and sample_rate_hz != sample_rate:
|
||||
raise ValueError("conflicting recorder sample rates")
|
||||
sample_rate_hz = sample_rate
|
||||
if sample_rate_hz not in {16_000, 24_000}:
|
||||
raise ValueError("unsupported recorder sample rate")
|
||||
self.sample_rate_hz = sample_rate_hz
|
||||
self._pcm_sink = pcm_sink
|
||||
self._retain_pcm = retain_pcm
|
||||
self._process_factory = process_factory
|
||||
self._max_bytes = max_bytes
|
||||
self._max_bytes = max_bytes if max_bytes is not None else 5 * 60 * sample_rate_hz * 2
|
||||
if self._max_bytes <= 0:
|
||||
raise ValueError("recorder byte limit must be positive")
|
||||
self._stderr_limit = stderr_limit
|
||||
self._stop_timeout = stop_timeout
|
||||
self._clock = clock
|
||||
@@ -100,6 +127,20 @@ class PwRecorder:
|
||||
self._duration_stop_at: float | None = None
|
||||
self._reported = False
|
||||
self._audio_level = 0.0
|
||||
self._sink_error: RecorderError | None = None
|
||||
self._drainer_error: RecorderError | None = None
|
||||
self._sink_closed = False
|
||||
self._audio_level_window_bytes = audio_level_window_bytes(sample_rate_hz)
|
||||
self._captured_bytes = 0
|
||||
self._cleanup_pending = False
|
||||
self._cleanup_complete = False
|
||||
self._cleanup_error: RecorderError | None = None
|
||||
|
||||
@property
|
||||
def cleanup_complete(self) -> bool:
|
||||
"""Whether the child process and both pipe drainers have been reaped."""
|
||||
with self._lock:
|
||||
return self._cleanup_complete
|
||||
|
||||
@property
|
||||
def buffered_bytes(self) -> int:
|
||||
@@ -116,17 +157,22 @@ class PwRecorder:
|
||||
raise RecorderError("invalid_state", "recorder is already started")
|
||||
try:
|
||||
process = self._process_factory(
|
||||
list(PW_RECORD_COMMAND),
|
||||
stdin=subprocess.DEVNULL,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.PIPE,
|
||||
start_new_session=True,
|
||||
pw_record_command(self.sample_rate_hz), stdin=subprocess.DEVNULL,
|
||||
stdout=subprocess.PIPE, stderr=subprocess.PIPE, start_new_session=True,
|
||||
)
|
||||
except OSError as error:
|
||||
self._abort_sink()
|
||||
self._mark_cleanup_complete()
|
||||
raise RecorderError("recorder_spawn", f"could not start pw-record: {error}") from error
|
||||
if process.stdout is None or process.stderr is None:
|
||||
raise RecorderError("recorder_spawn", "pw-record pipes were not created")
|
||||
self._process = process
|
||||
if process.stdout is None or process.stderr is None:
|
||||
try:
|
||||
self.cancel()
|
||||
except RecorderError:
|
||||
# A timed-out child remains cleanup-pending and is retried by
|
||||
# the service before another generation can begin.
|
||||
pass
|
||||
raise RecorderError("recorder_spawn", "pw-record pipes were not created")
|
||||
self._stdout_thread = threading.Thread(target=self._drain_stdout, daemon=True)
|
||||
self._stderr_thread = threading.Thread(target=self._drain_stderr, daemon=True)
|
||||
self._stdout_thread.start()
|
||||
@@ -141,119 +187,257 @@ class PwRecorder:
|
||||
finally:
|
||||
with self._lock:
|
||||
self._pcm.clear()
|
||||
self._abort_sink()
|
||||
|
||||
def poll_unexpected(self) -> RecorderError | None:
|
||||
process = self._require_process()
|
||||
if self._cleanup_pending:
|
||||
try:
|
||||
self._retry_pending_cleanup(process)
|
||||
except RecorderError:
|
||||
# The pending terminal error was already reported. Keep the
|
||||
# recorder tracked so a later poll can retry the reap.
|
||||
return None
|
||||
return None
|
||||
with self._lock:
|
||||
sink_error = self._sink_error
|
||||
drainer_error = self._drainer_error
|
||||
if sink_error is not None and not self._reported:
|
||||
self._reported = True
|
||||
if process.poll() is None:
|
||||
self._signal_group(process, signal.SIGTERM)
|
||||
return sink_error
|
||||
if drainer_error is not None and not self._reported:
|
||||
self._reported = True
|
||||
if process.poll() is None:
|
||||
self._signal_group(process, signal.SIGTERM)
|
||||
return drainer_error
|
||||
returncode = process.poll()
|
||||
if returncode is None:
|
||||
with self._lock:
|
||||
duration_stop_at = self._duration_stop_at
|
||||
if (
|
||||
not self._intentional
|
||||
and not self._reported
|
||||
and duration_stop_at is not None
|
||||
not self._intentional and not self._reported and duration_stop_at is not None
|
||||
and self._clock() - duration_stop_at >= self._stop_timeout
|
||||
):
|
||||
self._reported = True
|
||||
self._signal_group(process, signal.SIGKILL)
|
||||
try:
|
||||
process.wait(timeout=self._stop_timeout)
|
||||
self._join_drainers()
|
||||
except subprocess.TimeoutExpired as kill_error:
|
||||
error = RecorderError(
|
||||
"recorder_timeout",
|
||||
"pw-record did not exit after the duration-limit kill",
|
||||
returncode=process.returncode,
|
||||
signal_number=signal.SIGKILL,
|
||||
)
|
||||
self._mark_cleanup_pending(error)
|
||||
return error
|
||||
try:
|
||||
self._join_and_close_sink()
|
||||
except RecorderError as error:
|
||||
self._mark_cleanup_pending(error)
|
||||
raise
|
||||
self._abort_sink()
|
||||
self._mark_cleanup_complete()
|
||||
with self._lock:
|
||||
self._pcm.clear()
|
||||
return RecorderError(
|
||||
"duration_limit",
|
||||
"recording exceeded the five-minute duration limit; pw-record was killed after ignoring SIGTERM",
|
||||
returncode=process.returncode,
|
||||
signal_number=signal.SIGKILL,
|
||||
returncode=process.returncode, signal_number=signal.SIGKILL,
|
||||
)
|
||||
return None
|
||||
if self._intentional or self._reported:
|
||||
return None
|
||||
self._reported = True
|
||||
self._join_drainers()
|
||||
try:
|
||||
self._join_and_close_sink()
|
||||
except RecorderError as error:
|
||||
self._mark_cleanup_pending(error)
|
||||
raise
|
||||
self._abort_sink()
|
||||
self._mark_cleanup_complete()
|
||||
with self._lock:
|
||||
self._pcm.clear()
|
||||
if self._duration_limited:
|
||||
return RecorderError(
|
||||
"duration_limit",
|
||||
"recording exceeded the five-minute duration limit",
|
||||
returncode=returncode,
|
||||
signal_number=-returncode if returncode < 0 else None,
|
||||
"duration_limit", "recording exceeded the five-minute duration limit",
|
||||
returncode=returncode, signal_number=-returncode if returncode < 0 else None,
|
||||
)
|
||||
return self._exit_error(returncode, "pw-record exited unexpectedly")
|
||||
|
||||
def _finish(self, *, discard: bool) -> Capture:
|
||||
process = self._require_process()
|
||||
self._intentional = True
|
||||
if self._cleanup_pending:
|
||||
self._retry_pending_cleanup(process)
|
||||
if discard:
|
||||
return self._discarded_capture(process)
|
||||
with self._lock:
|
||||
cleanup_error = self._cleanup_error
|
||||
if cleanup_error is not None:
|
||||
raise cleanup_error
|
||||
raise RecorderError("recorder_timeout", "pw-record cleanup did not complete")
|
||||
|
||||
# Check the child before claiming the exit was caused by this stop.
|
||||
# A clean exit observed here is still an unexpected recorder exit: it
|
||||
# may have left only a partial clip in the bounded buffer.
|
||||
terminated_running_process = process.poll() is None
|
||||
if terminated_running_process:
|
||||
self._signal_group(process, signal.SIGTERM)
|
||||
if not terminated_running_process:
|
||||
self._reported = True
|
||||
self._intentional = terminated_running_process
|
||||
if terminated_running_process and not self._signal_group(process, signal.SIGTERM):
|
||||
if process.poll() is not None:
|
||||
terminated_running_process = False
|
||||
self._intentional = False
|
||||
self._reported = True
|
||||
try:
|
||||
returncode = process.wait(timeout=self._stop_timeout)
|
||||
except subprocess.TimeoutExpired as error:
|
||||
self._signal_group(process, signal.SIGKILL)
|
||||
try:
|
||||
process.wait(timeout=self._stop_timeout)
|
||||
self._join_drainers()
|
||||
except subprocess.TimeoutExpired as kill_error:
|
||||
timeout_error = RecorderError(
|
||||
"recorder_timeout",
|
||||
"pw-record did not exit after SIGKILL",
|
||||
returncode=process.returncode,
|
||||
signal_number=signal.SIGKILL,
|
||||
)
|
||||
self._mark_cleanup_pending(timeout_error)
|
||||
raise timeout_error from kill_error
|
||||
try:
|
||||
self._join_and_close_sink()
|
||||
except RecorderError as join_error:
|
||||
self._mark_cleanup_pending(join_error)
|
||||
raise
|
||||
self._abort_sink()
|
||||
self._mark_cleanup_complete()
|
||||
with self._lock:
|
||||
self._pcm.clear()
|
||||
raise RecorderError(
|
||||
"recorder_timeout",
|
||||
"pw-record did not stop after SIGTERM and was killed",
|
||||
returncode=process.returncode,
|
||||
signal_number=signal.SIGKILL,
|
||||
"recorder_timeout", "pw-record did not stop after SIGTERM and was killed",
|
||||
returncode=process.returncode, signal_number=signal.SIGKILL,
|
||||
) from error
|
||||
|
||||
self._join_drainers()
|
||||
# Joining both drainers before closing the sink is the critical
|
||||
# tail-ordering guarantee. Providers cannot be finalized before the
|
||||
# last PCM chunk has been enqueued.
|
||||
try:
|
||||
self._join_and_close_sink()
|
||||
except RecorderError as error:
|
||||
self._mark_cleanup_pending(error)
|
||||
raise
|
||||
self._mark_cleanup_complete()
|
||||
stderr_tail = self._stderr_bytes()
|
||||
with self._lock:
|
||||
sink_error = self._sink_error
|
||||
if sink_error is not None:
|
||||
with self._lock:
|
||||
self._pcm.clear()
|
||||
self._abort_sink()
|
||||
raise sink_error
|
||||
if self._duration_limited:
|
||||
with self._lock:
|
||||
self._pcm.clear()
|
||||
self._abort_sink()
|
||||
raise RecorderError(
|
||||
"duration_limit",
|
||||
"recording exceeded the five-minute duration limit",
|
||||
returncode=returncode,
|
||||
signal_number=-returncode if returncode < 0 else None,
|
||||
)
|
||||
clean_pipewire_sigterm = (
|
||||
terminated_running_process and returncode == 1 and not stderr_tail
|
||||
"duration_limit", "recording exceeded the five-minute duration limit",
|
||||
returncode=returncode, signal_number=-returncode if returncode < 0 else None,
|
||||
)
|
||||
if self._captured_bytes % 2:
|
||||
with self._lock:
|
||||
self._pcm.clear()
|
||||
self._abort_sink()
|
||||
raise RecorderError("invalid_audio", "PCM byte length must be even", returncode=returncode)
|
||||
if not terminated_running_process:
|
||||
with self._lock:
|
||||
self._pcm.clear()
|
||||
self._abort_sink()
|
||||
if discard:
|
||||
return self._discarded_capture(process)
|
||||
raise self._exit_error(returncode, "pw-record exited unexpectedly")
|
||||
clean_pipewire_sigterm = returncode == 1 and not stderr_tail
|
||||
if returncode not in (0, -signal.SIGTERM) and not clean_pipewire_sigterm:
|
||||
with self._lock:
|
||||
self._pcm.clear()
|
||||
self._abort_sink()
|
||||
raise self._exit_error(returncode, "pw-record failed while stopping")
|
||||
if discard:
|
||||
self._abort_sink()
|
||||
else:
|
||||
self._close_sink()
|
||||
with self._lock:
|
||||
pcm = b"" if discard else bytes(self._pcm)
|
||||
pcm = b"" if discard or not self._retain_pcm else bytes(self._pcm)
|
||||
self._pcm.clear()
|
||||
return Capture(
|
||||
pcm=pcm,
|
||||
returncode=returncode,
|
||||
pcm=pcm, returncode=returncode,
|
||||
signal=-returncode if returncode < 0 else None,
|
||||
stderr_tail=stderr_tail,
|
||||
stderr_tail=stderr_tail, sample_rate_hz=self.sample_rate_hz,
|
||||
)
|
||||
|
||||
def _drain_stdout(self) -> None:
|
||||
try:
|
||||
self._drain_stdout_inner()
|
||||
except Exception as error:
|
||||
self._record_drainer_error("stdout", error)
|
||||
|
||||
def _drain_stdout_inner(self) -> None:
|
||||
process = self._require_process()
|
||||
level_window = bytearray()
|
||||
remainder = b""
|
||||
while True:
|
||||
chunk = process.stdout.read(AUDIO_LEVEL_WINDOW_BYTES)
|
||||
if not chunk:
|
||||
raw_chunk = process.stdout.read(self._audio_level_window_bytes)
|
||||
if not raw_chunk:
|
||||
if remainder:
|
||||
self._record_drainer_error("stdout", ValueError("PCM byte length must be even"))
|
||||
return
|
||||
level_window.extend(chunk)
|
||||
while len(level_window) >= AUDIO_LEVEL_WINDOW_BYTES:
|
||||
self._update_audio_level(level_window[:AUDIO_LEVEL_WINDOW_BYTES])
|
||||
del level_window[:AUDIO_LEVEL_WINDOW_BYTES]
|
||||
with self._lock:
|
||||
if self._duration_limited:
|
||||
chunk = remainder + raw_chunk
|
||||
remainder = b""
|
||||
if len(chunk) % 2:
|
||||
remainder, chunk = chunk[-1:], chunk[:-1]
|
||||
if not chunk:
|
||||
continue
|
||||
if len(self._pcm) + len(chunk) > self._max_bytes:
|
||||
level_window.extend(chunk)
|
||||
while len(level_window) >= self._audio_level_window_bytes:
|
||||
self._update_audio_level(level_window[: self._audio_level_window_bytes])
|
||||
del level_window[: self._audio_level_window_bytes]
|
||||
with self._lock:
|
||||
if self._duration_limited or self._sink_error is not None:
|
||||
continue
|
||||
if self._captured_bytes + len(chunk) > self._max_bytes:
|
||||
self._duration_limited = True
|
||||
self._duration_stop_at = self._clock()
|
||||
self._pcm.clear()
|
||||
over_duration = True
|
||||
else:
|
||||
self._pcm.extend(chunk)
|
||||
continue
|
||||
self._captured_bytes += len(chunk)
|
||||
over_duration = False
|
||||
if over_duration:
|
||||
if process.poll() is None:
|
||||
self._signal_group(process, signal.SIGTERM)
|
||||
continue
|
||||
try:
|
||||
if self._pcm_sink is not None:
|
||||
self._pcm_sink.enqueue_pcm(chunk)
|
||||
except Exception as error:
|
||||
with self._lock:
|
||||
if self._sink_error is None:
|
||||
code = getattr(error, "code", "stream_failure")
|
||||
detail = getattr(error, "detail", "provider audio queue is full")
|
||||
if not isinstance(code, str) or not code.startswith(("meta_", "openai_")):
|
||||
code = f"provider_{code}"
|
||||
self._sink_error = RecorderError(code, str(detail)[:2048])
|
||||
self._pcm.clear()
|
||||
if process.poll() is None:
|
||||
self._signal_group(process, signal.SIGTERM)
|
||||
# Continue draining so a failed provider cannot deadlock pw-record.
|
||||
continue
|
||||
with self._lock:
|
||||
if self._duration_limited or self._sink_error is not None:
|
||||
continue
|
||||
if self._retain_pcm:
|
||||
self._pcm.extend(chunk)
|
||||
|
||||
def _update_audio_level(self, chunk: bytes) -> None:
|
||||
sample_bytes = len(chunk) - (len(chunk) % 2)
|
||||
@@ -269,6 +453,12 @@ class PwRecorder:
|
||||
self._audio_level = max(level, self._audio_level * 0.65)
|
||||
|
||||
def _drain_stderr(self) -> None:
|
||||
try:
|
||||
self._drain_stderr_inner()
|
||||
except Exception as error:
|
||||
self._record_drainer_error("stderr", error)
|
||||
|
||||
def _drain_stderr_inner(self) -> None:
|
||||
process = self._require_process()
|
||||
while True:
|
||||
chunk = process.stderr.read(16 * 1024)
|
||||
@@ -280,22 +470,130 @@ class PwRecorder:
|
||||
if overflow > 0:
|
||||
del self._stderr[:overflow]
|
||||
|
||||
def _join_drainers(self) -> None:
|
||||
def _retry_pending_cleanup(self, process: Process) -> None:
|
||||
"""Make another bounded attempt to reap a timed-out child and pipes."""
|
||||
if process.poll() is None:
|
||||
self._signal_group(process, signal.SIGKILL)
|
||||
try:
|
||||
process.wait(timeout=self._stop_timeout)
|
||||
except subprocess.TimeoutExpired as error:
|
||||
self._abort_sink()
|
||||
with self._lock:
|
||||
cleanup_error = self._cleanup_error
|
||||
self._pcm.clear()
|
||||
if cleanup_error is None:
|
||||
cleanup_error = RecorderError(
|
||||
"recorder_timeout", "pw-record cleanup did not complete",
|
||||
returncode=process.returncode, signal_number=signal.SIGKILL,
|
||||
)
|
||||
raise cleanup_error from error
|
||||
try:
|
||||
# A drainer error is terminal for the recording, but a completed
|
||||
# thread is no longer a resource leak and can be marked reaped.
|
||||
self._join_drainers(check_errors=False)
|
||||
except RecorderError:
|
||||
self._abort_sink()
|
||||
with self._lock:
|
||||
self._pcm.clear()
|
||||
raise
|
||||
self._abort_sink()
|
||||
self._mark_cleanup_complete()
|
||||
|
||||
def _mark_cleanup_pending(self, error: RecorderError) -> None:
|
||||
with self._lock:
|
||||
self._cleanup_pending = True
|
||||
self._cleanup_complete = False
|
||||
self._cleanup_error = error
|
||||
# Abort provider input before a later retry. A failed recording must
|
||||
# never send the normal end/commit marker.
|
||||
self._abort_sink()
|
||||
with self._lock:
|
||||
self._pcm.clear()
|
||||
|
||||
def _mark_cleanup_complete(self) -> None:
|
||||
with self._lock:
|
||||
self._cleanup_pending = False
|
||||
self._cleanup_complete = True
|
||||
|
||||
def _discarded_capture(self, process: Process) -> Capture:
|
||||
returncode = process.returncode
|
||||
if returncode is None:
|
||||
returncode = -signal.SIGKILL
|
||||
return Capture(
|
||||
pcm=b"", returncode=returncode,
|
||||
signal=-returncode if returncode < 0 else None,
|
||||
stderr_tail=self._stderr_bytes(), sample_rate_hz=self.sample_rate_hz,
|
||||
)
|
||||
|
||||
def _join_and_close_sink(self) -> None:
|
||||
try:
|
||||
self._join_drainers()
|
||||
except RecorderError:
|
||||
# A dead or stuck drainer must abort the streaming sink so its
|
||||
# worker cannot wait indefinitely or finalize partial audio.
|
||||
self._abort_sink()
|
||||
with self._lock:
|
||||
self._pcm.clear()
|
||||
raise
|
||||
|
||||
def _record_drainer_error(self, stream: str, _error: BaseException) -> None:
|
||||
with self._lock:
|
||||
if self._drainer_error is None:
|
||||
self._drainer_error = RecorderError(
|
||||
"recorder_pipe", f"pw-record {stream} pipe could not be drained"
|
||||
)
|
||||
process = self._process
|
||||
if process is not None and process.poll() is None:
|
||||
self._signal_group(process, signal.SIGTERM)
|
||||
|
||||
def _join_drainers(self, *, check_errors: bool = True) -> None:
|
||||
for thread in (self._stdout_thread, self._stderr_thread):
|
||||
if thread is None:
|
||||
continue
|
||||
thread.join(timeout=self._stop_timeout)
|
||||
if thread.is_alive():
|
||||
raise RecorderError(
|
||||
"recorder_timeout",
|
||||
"pw-record pipe did not close after the process group stopped",
|
||||
)
|
||||
raise RecorderError("recorder_timeout", "pw-record pipe did not close after the process group stopped")
|
||||
if check_errors:
|
||||
with self._lock:
|
||||
drainer_error = self._drainer_error
|
||||
if drainer_error is not None:
|
||||
raise drainer_error
|
||||
|
||||
def _signal_group(self, process: Process, signal_number: int) -> None:
|
||||
def _abort_sink(self) -> None:
|
||||
if self._pcm_sink is None or self._sink_closed:
|
||||
return
|
||||
self._sink_closed = True
|
||||
cancel = getattr(self._pcm_sink, "cancel", None)
|
||||
if callable(cancel):
|
||||
try:
|
||||
cancel()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
def _close_sink(self) -> None:
|
||||
if self._pcm_sink is None or self._sink_closed:
|
||||
return
|
||||
self._sink_closed = True
|
||||
close_input = getattr(self._pcm_sink, "close_input", None)
|
||||
if not callable(close_input):
|
||||
return
|
||||
try:
|
||||
close_input()
|
||||
except Exception as error:
|
||||
with self._lock:
|
||||
if self._sink_error is None:
|
||||
code = getattr(error, "code", "stream_failure")
|
||||
detail = getattr(error, "detail", "provider audio queue could not be finalized")
|
||||
if not isinstance(code, str) or not code.startswith(("meta_", "openai_")):
|
||||
code = f"provider_{code}"
|
||||
self._sink_error = RecorderError(code, str(detail)[:2048])
|
||||
|
||||
def _signal_group(self, process: Process, signal_number: int) -> bool:
|
||||
try:
|
||||
self._process_group_signal(process.pid, signal_number)
|
||||
except ProcessLookupError:
|
||||
pass
|
||||
return False
|
||||
return True
|
||||
|
||||
def _stderr_bytes(self) -> bytes:
|
||||
with self._lock:
|
||||
@@ -303,21 +601,14 @@ class PwRecorder:
|
||||
|
||||
def _exit_error(self, returncode: int, prefix: str) -> RecorderError:
|
||||
if returncode < 0:
|
||||
classification = f"signal {-returncode}"
|
||||
signal_number = -returncode
|
||||
classification, signal_number = f"signal {-returncode}", -returncode
|
||||
else:
|
||||
classification = f"code {returncode}"
|
||||
signal_number = None
|
||||
classification, signal_number = f"code {returncode}", None
|
||||
stderr = self._stderr_bytes().decode("utf-8", errors="replace").strip()
|
||||
detail = f"{prefix} with {classification}"
|
||||
if stderr:
|
||||
detail = f"{detail}: {stderr}"
|
||||
return RecorderError(
|
||||
"recorder_exit",
|
||||
detail,
|
||||
returncode=returncode,
|
||||
signal_number=signal_number,
|
||||
)
|
||||
return RecorderError("recorder_exit", detail, returncode=returncode, signal_number=signal_number)
|
||||
|
||||
def _require_process(self) -> Process:
|
||||
if self._process is None:
|
||||
|
||||
@@ -3,17 +3,20 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Sequence
|
||||
from concurrent.futures import Executor, Future, ThreadPoolExecutor
|
||||
from concurrent.futures import Executor, Future
|
||||
from dataclasses import dataclass
|
||||
import math
|
||||
import os
|
||||
from queue import Empty, SimpleQueue
|
||||
from queue import Empty, Queue, SimpleQueue
|
||||
import selectors
|
||||
import socket
|
||||
import threading
|
||||
import time
|
||||
from typing import Callable, Protocol
|
||||
|
||||
from .audio import AudioError, pcm_s16le_to_floats
|
||||
from .backend import ProviderError, TranscriptionBackend, TranscriptionTurn, backend_from_config
|
||||
from .config import ConfigError, load_config
|
||||
from .model import NemoParakeetModel, TranscriptionModel
|
||||
from .protocol import (
|
||||
MAX_FRAME_BYTES,
|
||||
@@ -34,6 +37,19 @@ MAX_CONNECTION_OUTPUT_BYTES = MAX_FRAME_BYTES
|
||||
LEVEL_EVENT_INTERVAL_SECONDS = 0.1
|
||||
|
||||
|
||||
def _looks_like_turn(value: object) -> bool:
|
||||
"""Keep injected test backends source-compatible while validating turns."""
|
||||
return all(callable(getattr(value, name, None)) for name in ("enqueue_pcm", "finish", "poll_error", "cancel", "close"))
|
||||
|
||||
|
||||
def _turn_cleanup_complete(turn: object) -> bool:
|
||||
return bool(getattr(turn, "cleanup_complete", True))
|
||||
|
||||
|
||||
def _recorder_cleanup_complete(recorder: object) -> bool:
|
||||
return bool(getattr(recorder, "cleanup_complete", True))
|
||||
|
||||
|
||||
class Recorder(Protocol):
|
||||
@property
|
||||
def audio_level(self) -> float: ...
|
||||
@@ -57,7 +73,12 @@ class _Generation:
|
||||
request_id: str
|
||||
state: str
|
||||
recorder: Recorder | None = None
|
||||
recorder_cleanup_started: bool = False
|
||||
turn: TranscriptionTurn | None = None
|
||||
tombstoned: bool = False
|
||||
failure_reported: bool = False
|
||||
cleanup_pending: bool = False
|
||||
cleanup_notify_idle: bool = False
|
||||
last_level_event_at: float | None = None
|
||||
|
||||
|
||||
@@ -68,28 +89,84 @@ class _Completion:
|
||||
future: Future[object]
|
||||
|
||||
|
||||
class _DaemonExecutor(Executor):
|
||||
"""Single-worker executor that cannot hold interpreter shutdown hostage."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self._queue: Queue[tuple[Future[object], Callable[..., object], tuple[object, ...], dict[str, object]] | None] = Queue()
|
||||
self._lock = threading.Lock()
|
||||
self._shutdown = False
|
||||
self._worker = threading.Thread(target=self._run, daemon=True, name="dictate-worker")
|
||||
self._worker.start()
|
||||
|
||||
def submit(self, fn: Callable[..., object], /, *args: object, **kwargs: object) -> Future[object]:
|
||||
future: Future[object] = Future()
|
||||
with self._lock:
|
||||
if self._shutdown:
|
||||
raise RuntimeError("cannot schedule new futures after shutdown")
|
||||
self._queue.put((future, fn, args, kwargs))
|
||||
return future
|
||||
|
||||
def shutdown(self, wait: bool = True, *, cancel_futures: bool = False) -> None:
|
||||
with self._lock:
|
||||
if not self._shutdown:
|
||||
self._shutdown = True
|
||||
if cancel_futures:
|
||||
while True:
|
||||
try:
|
||||
pending = self._queue.get_nowait()
|
||||
except Empty:
|
||||
break
|
||||
if pending is not None:
|
||||
pending[0].cancel()
|
||||
self._queue.put(None)
|
||||
if wait:
|
||||
self._worker.join()
|
||||
|
||||
def _run(self) -> None:
|
||||
while True:
|
||||
work = self._queue.get()
|
||||
if work is None:
|
||||
return
|
||||
future, fn, args, kwargs = work
|
||||
if not future.set_running_or_notify_cancel():
|
||||
continue
|
||||
try:
|
||||
result = fn(*args, **kwargs)
|
||||
except BaseException as error:
|
||||
future.set_exception(error)
|
||||
else:
|
||||
future.set_result(result)
|
||||
|
||||
|
||||
class DictateServiceCore:
|
||||
"""Single-generation protocol state machine with injected host boundaries."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
*,
|
||||
model: TranscriptionModel,
|
||||
recorder_factory: Callable[[], Recorder] = PwRecorder,
|
||||
model: TranscriptionModel | None = None,
|
||||
backend: TranscriptionBackend | None = None,
|
||||
recorder_factory: Callable[..., Recorder] = PwRecorder,
|
||||
executor: Executor | None = None,
|
||||
clock: Callable[[], float] = time.monotonic,
|
||||
idle_seconds: float = DEFAULT_IDLE_SECONDS,
|
||||
) -> None:
|
||||
if model is None and backend is None:
|
||||
raise ValueError("either model or backend is required")
|
||||
if model is not None and backend is not None:
|
||||
raise ValueError("model and backend are mutually exclusive")
|
||||
self._model = model
|
||||
self._backend = backend
|
||||
self._recorder_factory = recorder_factory
|
||||
self._executor = executor or ThreadPoolExecutor(max_workers=1, thread_name_prefix="dictate")
|
||||
self._executor = executor or _DaemonExecutor()
|
||||
self._owns_executor = executor is None
|
||||
self._clock = clock
|
||||
self._idle_seconds = idle_seconds
|
||||
self._last_activity = clock()
|
||||
self._clients: dict[str, bool] = {}
|
||||
self._active: _Generation | None = None
|
||||
self._model_loaded = bool(getattr(model, "loaded", False))
|
||||
self._model_loaded = bool(getattr(model, "loaded", False)) if model is not None else False
|
||||
self._worker_kind: str | None = None
|
||||
self._completions: SimpleQueue[_Completion] = SimpleQueue()
|
||||
self._events: list[Outbound] = []
|
||||
@@ -98,6 +175,19 @@ class DictateServiceCore:
|
||||
def active_generation_id(self) -> str | None:
|
||||
return self._active.generation_id if self._active else None
|
||||
|
||||
def _ready(self) -> bool:
|
||||
if self._backend is None:
|
||||
return self._model_loaded
|
||||
return bool(getattr(self._backend, "ready", False))
|
||||
|
||||
def _diagnostics(self) -> dict[str, str]:
|
||||
descriptor = getattr(self._backend, "descriptor", None)
|
||||
if descriptor is None:
|
||||
return {}
|
||||
# Descriptors are fixed, bounded values supplied by the host backend,
|
||||
# never paths, endpoint URLs, or credential state.
|
||||
return {"backend": str(descriptor.backend)[:64], "model": str(descriptor.model)[:128]}
|
||||
|
||||
def accepted(self, client_id: str) -> None:
|
||||
"""Track a socket without treating acceptance as protocol activity."""
|
||||
self._clients.setdefault(client_id, False)
|
||||
@@ -116,7 +206,8 @@ class DictateServiceCore:
|
||||
"type": "pong",
|
||||
"requestId": request.request_id,
|
||||
"service": "pi-dictate",
|
||||
"ready": self._model_loaded,
|
||||
"ready": self._ready(),
|
||||
**self._diagnostics(),
|
||||
},
|
||||
)
|
||||
return
|
||||
@@ -133,8 +224,9 @@ class DictateServiceCore:
|
||||
"type": "pong",
|
||||
"requestId": request.request_id,
|
||||
"service": "pi-dictate",
|
||||
"ready": self._model_loaded,
|
||||
"ready": self._ready(),
|
||||
"busy": self._active is not None,
|
||||
**self._diagnostics(),
|
||||
},
|
||||
)
|
||||
elif request.type == "start":
|
||||
@@ -154,14 +246,38 @@ class DictateServiceCore:
|
||||
self._apply_completion(completion)
|
||||
|
||||
active = self._active
|
||||
if active and active.cleanup_pending and self._worker_kind is None:
|
||||
if active.recorder is not None and not _recorder_cleanup_complete(active.recorder):
|
||||
self._submit("cleanup", active, active.recorder.poll_unexpected)
|
||||
return
|
||||
self._finish_generation(active, notify_idle=active.cleanup_notify_idle)
|
||||
if self._active is active:
|
||||
return
|
||||
active = self._active
|
||||
if active and active.tombstoned and self._worker_kind is None:
|
||||
if active.recorder is not None and not active.recorder_cleanup_started:
|
||||
active.recorder_cleanup_started = True
|
||||
active.state = "stopping"
|
||||
self._submit("cancel", active, active.recorder.cancel)
|
||||
else:
|
||||
self._finish_generation(active, notify_idle=False)
|
||||
return
|
||||
if active and active.state == "listening" and active.recorder and self._worker_kind is None:
|
||||
if active.turn is not None:
|
||||
failure_from_turn = active.turn.poll_error()
|
||||
if failure_from_turn is not None:
|
||||
active.failure_reported = True
|
||||
self._async_error(active, failure_from_turn.code, failure_from_turn.detail)
|
||||
self._abort_active(active)
|
||||
return
|
||||
try:
|
||||
failure = active.recorder.poll_unexpected()
|
||||
except RecorderError as error:
|
||||
failure = error
|
||||
if failure is not None:
|
||||
active.failure_reported = True
|
||||
self._async_error(active, failure.code, failure.detail)
|
||||
self._finish_generation(active, notify_idle=True)
|
||||
self._abort_active(active)
|
||||
return
|
||||
self._emit_audio_level(active)
|
||||
|
||||
@@ -175,9 +291,15 @@ class DictateServiceCore:
|
||||
if active is None or active.owner != client_id:
|
||||
return
|
||||
active.tombstoned = True
|
||||
if active.state == "listening" and active.recorder is not None:
|
||||
self._cancel_preparation(active)
|
||||
if active.turn is not None:
|
||||
self._cancel_turn(active)
|
||||
if active.recorder is not None and self._worker_kind is None and not active.recorder_cleanup_started:
|
||||
active.recorder_cleanup_started = True
|
||||
active.state = "stopping"
|
||||
self._submit("cancel", active, active.recorder.cancel)
|
||||
elif self._worker_kind is None:
|
||||
self._finish_generation(active, notify_idle=False)
|
||||
|
||||
def should_exit(self) -> bool:
|
||||
if self._active is not None or self._worker_kind is not None:
|
||||
@@ -186,14 +308,30 @@ class DictateServiceCore:
|
||||
|
||||
def close(self) -> None:
|
||||
active = self._active
|
||||
if active and active.recorder and active.state == "listening":
|
||||
if active:
|
||||
self._cancel_preparation(active)
|
||||
if active.turn is not None:
|
||||
self._cancel_turn(active)
|
||||
if active.recorder is not None:
|
||||
try:
|
||||
active.recorder.cancel()
|
||||
except RecorderError:
|
||||
except Exception:
|
||||
pass
|
||||
if active.turn is not None:
|
||||
deadline = time.monotonic() + 2.0
|
||||
while not _turn_cleanup_complete(active.turn) and time.monotonic() < deadline:
|
||||
try:
|
||||
active.turn.close()
|
||||
except Exception:
|
||||
break
|
||||
time.sleep(0.01)
|
||||
self._active = None
|
||||
if self._owns_executor:
|
||||
self._executor.shutdown(wait=True, cancel_futures=False)
|
||||
# A recorder/provider worker may be stuck in native or network I/O.
|
||||
# The owned worker is daemonized and isn't registered with Python's
|
||||
# ThreadPoolExecutor exit hook, so this bounded shutdown also permits
|
||||
# normal interpreter exit.
|
||||
self._executor.shutdown(wait=False, cancel_futures=True)
|
||||
|
||||
def _start(self, client_id: str, request: Request) -> None:
|
||||
assert request.generation_id is not None
|
||||
@@ -202,28 +340,70 @@ class DictateServiceCore:
|
||||
return
|
||||
active = _Generation(client_id, request.generation_id, request.request_id, "loading")
|
||||
self._active = active
|
||||
if self._backend is not None:
|
||||
begin_prepare = getattr(self._backend, "begin_prepare", None)
|
||||
if callable(begin_prepare):
|
||||
begin_prepare()
|
||||
self._state(active, "loading")
|
||||
self._submit("prepare", active, self._backend.start_turn)
|
||||
return
|
||||
assert self._model is not None
|
||||
if not self._model_loaded:
|
||||
self._state(active, "loading")
|
||||
self._submit("load", active, self._model.load)
|
||||
return
|
||||
self._begin_recording(active)
|
||||
|
||||
def _begin_recording(self, active: _Generation) -> None:
|
||||
def _begin_recording(self, active: _Generation, turn: TranscriptionTurn | None = None) -> None:
|
||||
if active.tombstoned:
|
||||
self._finish_generation(active, notify_idle=False)
|
||||
return
|
||||
try:
|
||||
active.turn = turn
|
||||
if turn is None:
|
||||
recorder = self._recorder_factory()
|
||||
else:
|
||||
# PwRecorder accepts these options. Keep injected legacy test
|
||||
# factories source-compatible while requiring a sink for cloud.
|
||||
sink = None if turn.retain_pcm else turn
|
||||
try:
|
||||
recorder = self._recorder_factory(
|
||||
sample_rate_hz=turn.sample_rate_hz,
|
||||
pcm_sink=sink,
|
||||
retain_pcm=turn.retain_pcm,
|
||||
)
|
||||
except TypeError:
|
||||
try:
|
||||
recorder = self._recorder_factory(
|
||||
sample_rate_hz=turn.sample_rate_hz, sink=sink
|
||||
)
|
||||
except TypeError:
|
||||
try:
|
||||
recorder = self._recorder_factory(turn.sample_rate_hz, sink)
|
||||
except TypeError:
|
||||
if not turn.retain_pcm:
|
||||
raise RecorderError(
|
||||
"recorder_config",
|
||||
"realtime recorder factory must accept a PCM sink",
|
||||
)
|
||||
recorder = self._recorder_factory()
|
||||
active.recorder = recorder
|
||||
recorder.start()
|
||||
except RecorderError as error:
|
||||
self._async_error(active, error.code, error.detail)
|
||||
self._finish_generation(active, notify_idle=True)
|
||||
active.failure_reported = True
|
||||
self._abort_active(active)
|
||||
return
|
||||
except OSError as error:
|
||||
self._async_error(active, "recorder_spawn", str(error))
|
||||
self._finish_generation(active, notify_idle=True)
|
||||
active.failure_reported = True
|
||||
self._abort_active(active)
|
||||
return
|
||||
except Exception as error:
|
||||
self._async_error(active, "recorder_failure", str(error))
|
||||
active.failure_reported = True
|
||||
self._abort_active(active)
|
||||
return
|
||||
active.recorder = recorder
|
||||
active.last_level_event_at = self._clock()
|
||||
self._state(active, "listening")
|
||||
|
||||
@@ -263,11 +443,15 @@ class DictateServiceCore:
|
||||
failure = error
|
||||
if failure is not None:
|
||||
self._async_error(active, failure.code, failure.detail)
|
||||
self._finish_generation(active, notify_idle=True)
|
||||
active.failure_reported = True
|
||||
self._abort_active(active)
|
||||
return
|
||||
active.state = "stopping"
|
||||
self._state(active, "stopping", assign=False)
|
||||
if active.turn is None:
|
||||
self._submit("stop", active, self._stop_and_convert, active.recorder)
|
||||
else:
|
||||
self._submit("stop", active, self._stop_capture, active.recorder)
|
||||
|
||||
def _cancel(self, client_id: str, request: Request) -> None:
|
||||
active = self._owned_generation(client_id, request)
|
||||
@@ -278,6 +462,9 @@ class DictateServiceCore:
|
||||
return
|
||||
active.tombstoned = True
|
||||
active.request_id = request.request_id
|
||||
# Emit the delivery tombstone before potentially blocking provider or
|
||||
# recorder cleanup. The client must learn cancellation promptly while
|
||||
# the host remains busy reaping the old generation.
|
||||
self._emit(
|
||||
client_id,
|
||||
{
|
||||
@@ -287,9 +474,49 @@ class DictateServiceCore:
|
||||
"generationId": active.generation_id,
|
||||
},
|
||||
)
|
||||
if active.state == "listening" and active.recorder is not None:
|
||||
self._cancel_preparation(active)
|
||||
if active.turn is not None:
|
||||
self._cancel_turn(active)
|
||||
if active.recorder is not None and self._worker_kind is None and not active.recorder_cleanup_started:
|
||||
active.recorder_cleanup_started = True
|
||||
active.state = "stopping"
|
||||
self._submit("cancel", active, active.recorder.cancel)
|
||||
elif active.recorder is None and self._worker_kind is None:
|
||||
self._finish_generation(active, notify_idle=False)
|
||||
|
||||
def _cancel_preparation(self, active: _Generation) -> None:
|
||||
if self._worker_kind != "prepare" or self._backend is None:
|
||||
return
|
||||
cancel_prepare = getattr(self._backend, "cancel_prepare", None)
|
||||
if callable(cancel_prepare):
|
||||
try:
|
||||
cancel_prepare()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
@staticmethod
|
||||
def _cancel_turn(active: _Generation) -> None:
|
||||
if active.turn is None:
|
||||
return
|
||||
try:
|
||||
active.turn.cancel()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
def _abort_active(self, active: _Generation) -> None:
|
||||
"""Tombstone a failed generation and always reap recorder/provider work."""
|
||||
active.tombstoned = True
|
||||
self._cancel_preparation(active)
|
||||
if active.turn is not None:
|
||||
self._cancel_turn(active)
|
||||
if active.recorder is not None and getattr(active.recorder, "cleanup_complete", False) is True:
|
||||
self._finish_generation(active, notify_idle=False)
|
||||
elif active.recorder is not None and self._worker_kind is None and not active.recorder_cleanup_started:
|
||||
active.recorder_cleanup_started = True
|
||||
active.state = "stopping"
|
||||
self._submit("cancel", active, active.recorder.cancel)
|
||||
elif active.recorder is None and self._worker_kind is None:
|
||||
self._finish_generation(active, notify_idle=False)
|
||||
|
||||
def _owned_generation(self, client_id: str, request: Request) -> _Generation | None:
|
||||
assert request.generation_id is not None
|
||||
@@ -315,6 +542,10 @@ class DictateServiceCore:
|
||||
capture = recorder.stop()
|
||||
return pcm_s16le_to_floats(capture.pcm)
|
||||
|
||||
@staticmethod
|
||||
def _stop_capture(recorder: Recorder) -> Capture:
|
||||
return recorder.stop()
|
||||
|
||||
def _submit(
|
||||
self,
|
||||
kind: str,
|
||||
@@ -339,23 +570,47 @@ class DictateServiceCore:
|
||||
try:
|
||||
result = completion.future.result()
|
||||
except RecorderError as error:
|
||||
if not active.tombstoned:
|
||||
if not active.tombstoned and not active.failure_reported:
|
||||
self._async_error(active, error.code, error.detail)
|
||||
self._finish_generation(active, notify_idle=not active.tombstoned)
|
||||
active.failure_reported = True
|
||||
self._abort_active(active)
|
||||
return
|
||||
except AudioError as error:
|
||||
if not active.tombstoned:
|
||||
if not active.tombstoned and not active.failure_reported:
|
||||
self._async_error(active, "invalid_audio", str(error))
|
||||
self._finish_generation(active, notify_idle=not active.tombstoned)
|
||||
active.failure_reported = True
|
||||
self._abort_active(active)
|
||||
return
|
||||
except ProviderError as error:
|
||||
if not active.tombstoned and not active.failure_reported:
|
||||
self._async_error(active, error.code, error.detail)
|
||||
active.failure_reported = True
|
||||
self._abort_active(active)
|
||||
return
|
||||
except BaseException as error:
|
||||
if not active.tombstoned:
|
||||
code = "model_failure" if completion.kind in ("load", "infer") else "recorder_failure"
|
||||
if not active.tombstoned and not active.failure_reported:
|
||||
code = "model_failure" if completion.kind in ("load", "prepare", "infer") else "recorder_failure"
|
||||
self._async_error(active, code, str(error))
|
||||
self._finish_generation(active, notify_idle=not active.tombstoned)
|
||||
active.failure_reported = True
|
||||
self._abort_active(active)
|
||||
return
|
||||
|
||||
if completion.kind == "load":
|
||||
if completion.kind == "prepare":
|
||||
if active.tombstoned:
|
||||
if _looks_like_turn(result):
|
||||
active.turn = result
|
||||
try:
|
||||
result.cancel()
|
||||
except Exception:
|
||||
pass
|
||||
self._finish_generation(active, notify_idle=False)
|
||||
elif _looks_like_turn(result):
|
||||
self._begin_recording(active, result)
|
||||
else:
|
||||
self._async_error(active, "backend_failure", "backend returned an invalid transcription turn")
|
||||
active.failure_reported = True
|
||||
self._abort_active(active)
|
||||
elif completion.kind == "load":
|
||||
self._model_loaded = True
|
||||
if active.tombstoned:
|
||||
self._finish_generation(active, notify_idle=False)
|
||||
@@ -363,14 +618,25 @@ class DictateServiceCore:
|
||||
self._begin_recording(active)
|
||||
elif completion.kind == "stop":
|
||||
if active.tombstoned:
|
||||
self._finish_generation(active, notify_idle=False)
|
||||
self._abort_active(active)
|
||||
return
|
||||
if active.turn is not None:
|
||||
if not isinstance(result, Capture):
|
||||
self._async_error(active, "invalid_audio", "recorder returned invalid capture")
|
||||
active.failure_reported = True
|
||||
self._abort_active(active)
|
||||
return
|
||||
self._state(active, "transcribing")
|
||||
self._submit("infer", active, active.turn.finish, result)
|
||||
return
|
||||
samples = result
|
||||
if not isinstance(samples, Sequence) or isinstance(samples, (str, bytes, bytearray)):
|
||||
self._async_error(active, "invalid_audio", "audio conversion returned invalid samples")
|
||||
self._finish_generation(active, notify_idle=True)
|
||||
active.failure_reported = True
|
||||
self._abort_active(active)
|
||||
return
|
||||
self._state(active, "transcribing")
|
||||
assert self._model is not None
|
||||
self._submit("infer", active, self._model.transcribe, samples)
|
||||
elif completion.kind == "infer":
|
||||
if active.tombstoned:
|
||||
@@ -378,7 +644,14 @@ class DictateServiceCore:
|
||||
return
|
||||
if not isinstance(result, str):
|
||||
self._async_error(active, "model_failure", "model returned a non-string transcript")
|
||||
elif len(result.encode("utf-8")) > MAX_TRANSCRIPT_BYTES:
|
||||
else:
|
||||
try:
|
||||
transcript_bytes = len(result.encode("utf-8"))
|
||||
except UnicodeEncodeError:
|
||||
self._async_error(active, "model_failure", "model returned invalid Unicode")
|
||||
self._finish_generation(active, notify_idle=True)
|
||||
return
|
||||
if transcript_bytes > MAX_TRANSCRIPT_BYTES:
|
||||
self._async_error(active, "transcript_too_large", "transcript exceeds output limit")
|
||||
else:
|
||||
self._emit(
|
||||
@@ -393,14 +666,33 @@ class DictateServiceCore:
|
||||
)
|
||||
self._finish_generation(active, notify_idle=True)
|
||||
elif completion.kind == "cancel":
|
||||
self._finish_generation(active, notify_idle=False)
|
||||
self._finish_generation(active, notify_idle=not active.tombstoned)
|
||||
elif completion.kind == "cleanup":
|
||||
self._finish_generation(active, notify_idle=active.cleanup_notify_idle)
|
||||
|
||||
def _finish_generation(self, active: _Generation, *, notify_idle: bool) -> None:
|
||||
if self._active is not active:
|
||||
return
|
||||
active.cleanup_pending = True
|
||||
active.cleanup_notify_idle = active.cleanup_notify_idle or notify_idle
|
||||
if active.recorder is not None and not _recorder_cleanup_complete(active.recorder):
|
||||
# Cleanup retries run through the executor so a stubborn child never
|
||||
# blocks the socket loop. The pending generation remains exclusive.
|
||||
return
|
||||
if active.turn is not None:
|
||||
try:
|
||||
active.turn.close()
|
||||
except Exception:
|
||||
pass
|
||||
# A provider close can be blocked by a wedged transport. Keep the
|
||||
# tombstoned generation busy until both provider workers have
|
||||
# actually exited, so no thread or transport crosses generations.
|
||||
if not _turn_cleanup_complete(active.turn):
|
||||
return
|
||||
active.cleanup_pending = False
|
||||
self._active = None
|
||||
self._last_activity = self._clock()
|
||||
if notify_idle and active.owner in self._clients:
|
||||
if (notify_idle or active.failure_reported) and active.owner in self._clients:
|
||||
self._state(active, "idle", assign=False)
|
||||
|
||||
def _state(self, active: _Generation, state: str, *, assign: bool = True) -> None:
|
||||
@@ -448,8 +740,12 @@ class _Connection:
|
||||
close_after_write: bool = False
|
||||
|
||||
|
||||
def inherited_listener(environ: dict[str, str] | None = None) -> socket.socket:
|
||||
"""Return systemd's single inherited listener at fd 3."""
|
||||
def inherited_listener(
|
||||
environ: dict[str, str] | None = None,
|
||||
*,
|
||||
expected_path: str | os.PathLike[str] | None = None,
|
||||
) -> socket.socket:
|
||||
"""Return systemd's fd 3 only when it is the expected Unix stream socket."""
|
||||
environment = os.environ if environ is None else environ
|
||||
try:
|
||||
listen_pid = int(environment.get("LISTEN_PID", "0"))
|
||||
@@ -458,9 +754,65 @@ def inherited_listener(environ: dict[str, str] | None = None) -> socket.socket:
|
||||
raise RuntimeError("invalid systemd socket activation environment") from error
|
||||
if listen_pid != os.getpid() or listen_fds != 1:
|
||||
raise RuntimeError("expected exactly one systemd socket-activated listener")
|
||||
listener: socket.socket | None = None
|
||||
try:
|
||||
listener = socket.socket(fileno=3)
|
||||
if listener.family != socket.AF_UNIX:
|
||||
raise RuntimeError("inherited listener must be AF_UNIX")
|
||||
socket_type = listener.type & getattr(socket, "SOCK_TYPE_MASK", 0xF)
|
||||
if socket_type != socket.SOCK_STREAM:
|
||||
raise RuntimeError("inherited listener must be SOCK_STREAM")
|
||||
try:
|
||||
accepting = listener.getsockopt(socket.SOL_SOCKET, socket.SO_ACCEPTCONN)
|
||||
except OSError as error:
|
||||
raise RuntimeError("could not validate inherited listener state") from error
|
||||
if accepting != 1:
|
||||
raise RuntimeError("inherited listener is not listening")
|
||||
actual = listener.getsockname()
|
||||
if isinstance(actual, bytes):
|
||||
actual = actual.decode("utf-8", errors="strict")
|
||||
if not isinstance(actual, str):
|
||||
raise RuntimeError("inherited listener has an invalid socket path")
|
||||
expected = expected_path
|
||||
if expected is None:
|
||||
runtime_dir = environment.get("XDG_RUNTIME_DIR", f"/run/user/{os.getuid()}")
|
||||
expected = os.path.join(runtime_dir, "pi-dictate.sock")
|
||||
if os.path.abspath(actual) != os.path.abspath(os.fspath(expected)):
|
||||
raise RuntimeError("inherited listener is not the expected pi-dictate socket")
|
||||
listener.setblocking(False)
|
||||
return listener
|
||||
except Exception:
|
||||
if listener is not None:
|
||||
listener.close()
|
||||
raise
|
||||
|
||||
|
||||
def _queue_connection_frame(
|
||||
selector: selectors.BaseSelector,
|
||||
connections: dict[str, _Connection],
|
||||
core: DictateServiceCore,
|
||||
client_id: str,
|
||||
frame: dict[str, object],
|
||||
) -> bool:
|
||||
"""Encode and append one frame without exceeding the per-client bound."""
|
||||
connection = connections.get(client_id)
|
||||
if connection is None:
|
||||
return False
|
||||
try:
|
||||
encoded = encode_frame(frame)
|
||||
except ProtocolError:
|
||||
_close_connection(selector, connections, core, client_id)
|
||||
return False
|
||||
if len(connection.output) + len(encoded) > MAX_CONNECTION_OUTPUT_BYTES:
|
||||
_close_connection(selector, connections, core, client_id)
|
||||
return False
|
||||
connection.output.extend(encoded)
|
||||
selector.modify(
|
||||
connection.socket,
|
||||
selectors.EVENT_READ | selectors.EVENT_WRITE,
|
||||
client_id,
|
||||
)
|
||||
return True
|
||||
|
||||
|
||||
def serve(listener: socket.socket, core: DictateServiceCore) -> None:
|
||||
@@ -477,19 +829,8 @@ def serve(listener: socket.socket, core: DictateServiceCore) -> None:
|
||||
connection = connections.get(outbound.client_id)
|
||||
if connection is None:
|
||||
continue
|
||||
try:
|
||||
encoded = encode_frame(outbound.frame)
|
||||
except ProtocolError:
|
||||
_close_connection(selector, connections, core, outbound.client_id)
|
||||
continue
|
||||
if len(connection.output) + len(encoded) > MAX_CONNECTION_OUTPUT_BYTES:
|
||||
_close_connection(selector, connections, core, outbound.client_id)
|
||||
continue
|
||||
connection.output.extend(encoded)
|
||||
selector.modify(
|
||||
connection.socket,
|
||||
selectors.EVENT_READ | selectors.EVENT_WRITE,
|
||||
outbound.client_id,
|
||||
_queue_connection_frame(
|
||||
selector, connections, core, outbound.client_id, outbound.frame
|
||||
)
|
||||
for key, mask in selector.select(timeout=0.1):
|
||||
if key.data is None:
|
||||
@@ -527,9 +868,19 @@ def serve(listener: socket.socket, core: DictateServiceCore) -> None:
|
||||
for frame in connection.decoder.feed(data):
|
||||
core.handle_request(client_id, parse_request(frame))
|
||||
except ProtocolError as error:
|
||||
connection.output.extend(encode_frame(error_event(error)))
|
||||
if error.close_connection:
|
||||
if _queue_connection_frame(
|
||||
selector,
|
||||
connections,
|
||||
core,
|
||||
client_id,
|
||||
error_event(error),
|
||||
) and error.close_connection:
|
||||
connection.close_after_write = True
|
||||
# Queueing a protocol error can close this connection when its
|
||||
# output bound is exceeded. The selector event was created for
|
||||
# the old socket, so do not send to or modify that stale socket.
|
||||
if connections.get(client_id) is not connection:
|
||||
continue
|
||||
if mask & selectors.EVENT_WRITE and connection.output:
|
||||
try:
|
||||
sent = connection.socket.send(connection.output)
|
||||
@@ -573,6 +924,23 @@ def _close_connection(
|
||||
|
||||
|
||||
def main() -> None:
|
||||
# Installed services use the strict host config. The environment fallback
|
||||
# keeps direct development invocations and existing local deployments
|
||||
# compatible until install.sh has written config.json.
|
||||
config_path = os.environ.get("PI_DICTATE_CONFIG_PATH")
|
||||
default_config = os.path.expanduser("~/.config/pi-dictate/config.json")
|
||||
if config_path or os.path.isfile(default_config):
|
||||
try:
|
||||
config = load_config(config_path)
|
||||
network_mode = os.environ.get("PI_DICTATE_NETWORK_MODE")
|
||||
expected_network_mode = "offline" if config.backend == "local" else "cloud"
|
||||
if network_mode is not None and network_mode != expected_network_mode:
|
||||
raise ConfigError("configured backend does not match service network mode")
|
||||
backend = backend_from_config(config)
|
||||
except (ConfigError, ValueError) as error:
|
||||
raise SystemExit(f"Invalid pi-dictate configuration: {error}") from error
|
||||
core = DictateServiceCore(backend=backend, idle_seconds=config.idle_seconds)
|
||||
else:
|
||||
model_path = os.environ.get("PI_DICTATE_MODEL_PATH")
|
||||
if not model_path:
|
||||
raise SystemExit("PI_DICTATE_MODEL_PATH must name an explicit local .nemo file")
|
||||
@@ -582,15 +950,16 @@ def main() -> None:
|
||||
idle_value = os.environ.get("PI_DICTATE_IDLE_SECONDS", str(DEFAULT_IDLE_SECONDS))
|
||||
boost_value = os.environ.get("PI_DICTATE_BOOST_ALPHA", "1.0")
|
||||
try:
|
||||
idle_seconds = float(idle_value)
|
||||
idle_seconds = int(idle_value)
|
||||
if str(idle_seconds) != idle_value:
|
||||
raise ValueError
|
||||
boost_alpha = float(boost_value)
|
||||
except ValueError as error:
|
||||
raise SystemExit("PI_DICTATE_IDLE_SECONDS and PI_DICTATE_BOOST_ALPHA must be numeric") from error
|
||||
if idle_seconds <= 0:
|
||||
raise SystemExit("PI_DICTATE_IDLE_SECONDS must be positive")
|
||||
raise SystemExit("PI_DICTATE_IDLE_SECONDS must be an integer; boost alpha must be numeric") from error
|
||||
if not 60 <= idle_seconds <= 86_400:
|
||||
raise SystemExit("PI_DICTATE_IDLE_SECONDS must be between 60 and 86400")
|
||||
if boost_alpha <= 0:
|
||||
raise SystemExit("PI_DICTATE_BOOST_ALPHA must be positive")
|
||||
|
||||
model = NemoParakeetModel(model_path, terms_path=terms_path, boost_alpha=boost_alpha)
|
||||
core = DictateServiceCore(model=model, idle_seconds=idle_seconds)
|
||||
listener = inherited_listener()
|
||||
|
||||
@@ -0,0 +1,170 @@
|
||||
"""Fail-closed checks for the installed systemd hardening contract."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import stat
|
||||
|
||||
from .config import ConfigError, DictateConfig, load_config
|
||||
|
||||
MAX_STARTUP_FILE_BYTES = 64 * 1024
|
||||
EXPECTED_BASE_UNIT_SHA256 = "345c79165029e52b446087e780c982961e9478da74b5ac5680ac27f5231ff4ce"
|
||||
EXPECTED_SOCKET_UNIT_SHA256 = "4868f54e2ec25fad132df858d5dfb9bf8337e9d5930998e7af83e58e57f737d0"
|
||||
|
||||
|
||||
def _read_regular(path: Path, limit: int, *, mode: int | None = None) -> bytes:
|
||||
descriptor: int | None = None
|
||||
try:
|
||||
descriptor = os.open(
|
||||
path,
|
||||
os.O_RDONLY | getattr(os, "O_CLOEXEC", 0) | getattr(os, "O_NOFOLLOW", 0),
|
||||
)
|
||||
info = os.fstat(descriptor)
|
||||
if not stat.S_ISREG(info.st_mode) or info.st_uid != os.getuid():
|
||||
raise ConfigError("managed systemd file is not a service-owned regular file")
|
||||
if mode is not None and stat.S_IMODE(info.st_mode) != mode:
|
||||
raise ConfigError("managed systemd file permissions are insecure")
|
||||
if info.st_size > limit:
|
||||
raise ConfigError("managed systemd file is too large")
|
||||
chunks: list[bytes] = []
|
||||
remaining = limit + 1
|
||||
while remaining:
|
||||
chunk = os.read(descriptor, min(4096, remaining))
|
||||
if not chunk:
|
||||
break
|
||||
chunks.append(chunk)
|
||||
remaining -= len(chunk)
|
||||
raw = b"".join(chunks)
|
||||
if len(raw) > limit:
|
||||
raise ConfigError("managed systemd file is too large")
|
||||
return raw
|
||||
finally:
|
||||
if descriptor is not None:
|
||||
try:
|
||||
os.close(descriptor)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
|
||||
def _reject_symlink_ancestry(path: Path) -> None:
|
||||
current = path.absolute()
|
||||
while True:
|
||||
try:
|
||||
info = current.lstat()
|
||||
except FileNotFoundError:
|
||||
current = current.parent
|
||||
if current == current.parent:
|
||||
return
|
||||
continue
|
||||
except OSError as error:
|
||||
raise ConfigError("systemd drop-in ancestry could not be checked") from error
|
||||
if stat.S_ISLNK(info.st_mode):
|
||||
raise ConfigError("systemd drop-in ancestry contains a symlink")
|
||||
if current == current.parent:
|
||||
return
|
||||
current = current.parent
|
||||
|
||||
|
||||
def _bind_line(path: Path) -> str:
|
||||
expanded = str(path.expanduser()).replace("%", "%%")
|
||||
return "BindReadOnlyPaths=" + json.dumps(expanded, ensure_ascii=False)
|
||||
|
||||
|
||||
def _expected_dropin(network_mode: str, config: DictateConfig) -> str:
|
||||
if network_mode == "offline":
|
||||
return "\n".join(
|
||||
[
|
||||
"# Managed by pi-dictate install.sh",
|
||||
"[Service]",
|
||||
"Environment=PI_DICTATE_NETWORK_MODE=offline",
|
||||
"Environment=HF_HUB_OFFLINE=1",
|
||||
"Environment=TRANSFORMERS_OFFLINE=1",
|
||||
"Environment=NEMO_HOME=/tmp/pi-dictate-nemo",
|
||||
"Environment=HF_HOME=/tmp/pi-dictate-huggingface",
|
||||
"Environment=TRITON_CACHE_DIR=/tmp/pi-dictate-triton",
|
||||
"Environment=NUMBA_CACHE_DIR=/tmp/pi-dictate-numba",
|
||||
"RestrictAddressFamilies=AF_UNIX",
|
||||
"IPAddressDeny=any",
|
||||
_bind_line(config.selected.model_path),
|
||||
_bind_line(config.selected.terms_path),
|
||||
]
|
||||
) + "\n"
|
||||
if network_mode == "cloud":
|
||||
return "\n".join(
|
||||
[
|
||||
"# Managed by pi-dictate install.sh",
|
||||
"[Service]",
|
||||
"Environment=PI_DICTATE_NETWORK_MODE=cloud",
|
||||
"# Cloud backends need outbound DNS and TLS, but never a listening IP socket.",
|
||||
"Environment=HF_HUB_OFFLINE=",
|
||||
"Environment=TRANSFORMERS_OFFLINE=",
|
||||
"Environment=NEMO_HOME=",
|
||||
"Environment=HF_HOME=",
|
||||
"Environment=TRITON_CACHE_DIR=",
|
||||
"Environment=NUMBA_CACHE_DIR=",
|
||||
"Environment=PI_DICTATE_MODEL_PATH=",
|
||||
"Environment=PI_DICTATE_BOOST_ALPHA=",
|
||||
"RestrictAddressFamilies=",
|
||||
"RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6",
|
||||
"IPAddressDeny=",
|
||||
_bind_line(config.selected.api_key_file),
|
||||
_bind_line(config.selected.terms_path),
|
||||
]
|
||||
) + "\n"
|
||||
raise ConfigError("invalid systemd network mode")
|
||||
|
||||
|
||||
def validate_startup() -> None:
|
||||
"""Validate config, marker, base unit, and effective generated drop-in."""
|
||||
|
||||
config_path = Path(os.environ.get("PI_DICTATE_CONFIG_PATH", "~/.config/pi-dictate/config.json")).expanduser()
|
||||
config = load_config(config_path)
|
||||
network_mode = os.environ.get("PI_DICTATE_NETWORK_MODE", "")
|
||||
expected_mode = "offline" if config.backend == "local" else "cloud"
|
||||
if network_mode != expected_mode:
|
||||
raise ConfigError("configured backend does not match systemd network mode")
|
||||
|
||||
marker = Path(os.environ.get("PI_DICTATE_DEPENDENCY_PROFILE_PATH", "~/.local/share/pi-dictate/.dependency-profile")).expanduser()
|
||||
marker_raw = _read_regular(marker, 32, mode=0o600)
|
||||
if marker_raw not in {b"local\n", b"cloud\n"} or marker_raw.rstrip(b"\n").decode("ascii") != expected_mode.replace("offline", "local"):
|
||||
raise ConfigError("installed dependency profile does not match configured backend")
|
||||
|
||||
unit_root = Path("~/.config/systemd/user").expanduser()
|
||||
_reject_symlink_ancestry(unit_root)
|
||||
dropin_dir = unit_root / "pi-dictate.service.d"
|
||||
dropin_path = dropin_dir / "10-pi-dictate-network.conf"
|
||||
_reject_symlink_ancestry(dropin_dir)
|
||||
if not dropin_dir.is_dir():
|
||||
raise ConfigError("managed systemd drop-in directory is missing")
|
||||
try:
|
||||
entries = list(dropin_dir.iterdir())
|
||||
except OSError as error:
|
||||
raise ConfigError("managed systemd drop-in directory cannot be read") from error
|
||||
if any(entry.name != dropin_path.name for entry in entries):
|
||||
raise ConfigError("unexpected systemd drop-in is present")
|
||||
if _read_regular(dropin_path, MAX_STARTUP_FILE_BYTES).decode("utf-8") != _expected_dropin(network_mode, config):
|
||||
raise ConfigError("effective systemd network hardening is not the expected managed policy")
|
||||
|
||||
unit_path = unit_root / "pi-dictate.service"
|
||||
unit_raw = _read_regular(unit_path, MAX_STARTUP_FILE_BYTES)
|
||||
if hashlib.sha256(unit_raw).hexdigest() != EXPECTED_BASE_UNIT_SHA256:
|
||||
raise ConfigError("base systemd hardening is incomplete or tampered")
|
||||
|
||||
socket_unit_path = unit_root / "pi-dictate.socket"
|
||||
socket_unit_raw = _read_regular(socket_unit_path, MAX_STARTUP_FILE_BYTES)
|
||||
if hashlib.sha256(socket_unit_raw).hexdigest() != EXPECTED_SOCKET_UNIT_SHA256:
|
||||
raise ConfigError("socket systemd policy is incomplete or tampered")
|
||||
|
||||
|
||||
def main() -> None:
|
||||
try:
|
||||
validate_startup()
|
||||
except (ConfigError, OSError, UnicodeError, ValueError) as error:
|
||||
raise SystemExit(f"Invalid pi-dictate startup policy: {error}") from error
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,2 @@
|
||||
{"type":"transcript","final":false,"text":"partial"}
|
||||
{"type":"transcript","final":true,"text":"final fixture transcript"}
|
||||
@@ -0,0 +1 @@
|
||||
{"sessionId":"fixture-session-1"}
|
||||
@@ -0,0 +1,4 @@
|
||||
{"type":"session.created"}
|
||||
{"type":"session.updated"}
|
||||
{"type":"input_audio_buffer.committed","event_id":"event-fixture-1","previous_item_id":null,"item_id":"fixture-item-1"}
|
||||
{"type":"conversation.item.input_audio_transcription.completed","item_id":"fixture-item-1","transcript":"final fixture transcript"}
|
||||
@@ -0,0 +1,507 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import json
|
||||
from pathlib import Path
|
||||
import tempfile
|
||||
import threading
|
||||
import time
|
||||
from types import SimpleNamespace
|
||||
import unittest
|
||||
from unittest.mock import Mock, patch
|
||||
|
||||
from pi_dictate_service.backend import PcmQueue, ProviderError, provider_event
|
||||
from pi_dictate_service.backends.meta import (
|
||||
META_ENDPOINT,
|
||||
META_MAX_BACKLOG_BYTES,
|
||||
META_SOCKET_IO_TIMEOUT,
|
||||
MetaBackend,
|
||||
_default_transport as default_meta_transport,
|
||||
)
|
||||
from pi_dictate_service.backends.openai import (
|
||||
OPENAI_ENDPOINT,
|
||||
OPENAI_SOCKET_IO_TIMEOUT,
|
||||
OpenAIBackend,
|
||||
_default_transport as default_openai_transport,
|
||||
)
|
||||
from pi_dictate_service.config import MetaConfig, OpenAIConfig
|
||||
|
||||
|
||||
FIXTURE_ROOT = Path(__file__).parent / "fixtures"
|
||||
|
||||
|
||||
def fixture_events(provider: str) -> list[str]:
|
||||
return (FIXTURE_ROOT / provider / "events.jsonl").read_text(encoding="utf-8").splitlines()
|
||||
|
||||
|
||||
def _capture_start_error(backend: object, errors: list[BaseException]) -> None:
|
||||
try:
|
||||
backend.start_turn()
|
||||
except BaseException as error:
|
||||
errors.append(error)
|
||||
|
||||
|
||||
class FakeTransport:
|
||||
def __init__(self, events: list[str | bytes | None]) -> None:
|
||||
self.events = list(events)
|
||||
self.sent: list[str | bytes] = []
|
||||
self.closed = False
|
||||
|
||||
def send(self, value: str | bytes) -> None:
|
||||
self.sent.append(value)
|
||||
|
||||
def recv(self, timeout: float | None = None) -> str | bytes | None:
|
||||
if not self.events:
|
||||
return None
|
||||
return self.events.pop(0)
|
||||
|
||||
def close(self) -> None:
|
||||
self.closed = True
|
||||
|
||||
|
||||
class BlockingCloseTransport(FakeTransport):
|
||||
def __init__(self, events: list[str | bytes | None]) -> None:
|
||||
super().__init__(events)
|
||||
self.close_started = threading.Event()
|
||||
self.allow_close = threading.Event()
|
||||
|
||||
def close(self) -> None:
|
||||
self.close_started.set()
|
||||
self.allow_close.wait(1)
|
||||
super().close()
|
||||
|
||||
|
||||
class HttpFailure(RuntimeError):
|
||||
def __init__(self, status_code: int) -> None:
|
||||
super().__init__(f"HTTP {status_code}")
|
||||
self.response = SimpleNamespace(status_code=status_code)
|
||||
|
||||
|
||||
class BackendTests(unittest.TestCase):
|
||||
def test_production_transports_apply_a_socket_io_deadline(self) -> None:
|
||||
for factory, timeout, arguments in (
|
||||
(default_meta_transport, META_SOCKET_IO_TIMEOUT, (META_ENDPOINT,)),
|
||||
(
|
||||
default_openai_transport,
|
||||
OPENAI_SOCKET_IO_TIMEOUT,
|
||||
(OPENAI_ENDPOINT,),
|
||||
),
|
||||
):
|
||||
with self.subTest(factory=factory.__module__):
|
||||
transport = SimpleNamespace(socket=Mock())
|
||||
with patch("websockets.sync.client.connect", return_value=transport):
|
||||
if factory is default_openai_transport:
|
||||
factory(*arguments, headers={"Authorization": "Bearer test"})
|
||||
else:
|
||||
factory(*arguments)
|
||||
transport.socket.settimeout.assert_called_once_with(timeout)
|
||||
|
||||
def make_files(self) -> tuple[Path, Path, tempfile.TemporaryDirectory[str]]:
|
||||
temporary = tempfile.TemporaryDirectory()
|
||||
root = Path(temporary.name)
|
||||
key = root / "key"
|
||||
key.write_text("secret", encoding="utf-8")
|
||||
key.chmod(0o600)
|
||||
terms = root / "terms"
|
||||
terms.write_text("JSON\n", encoding="utf-8")
|
||||
return key, terms, temporary
|
||||
|
||||
def test_meta_waits_for_ack_sends_documented_handshake_and_finalizes_after_tail(self) -> None:
|
||||
key, terms, temporary = self.make_files()
|
||||
self.addCleanup(temporary.cleanup)
|
||||
acknowledgement = (FIXTURE_ROOT / "meta" / "session_ack.json").read_text(encoding="utf-8").strip()
|
||||
transport = FakeTransport([acknowledgement, *fixture_events("meta"), None])
|
||||
connections: list[str] = []
|
||||
|
||||
def connect(endpoint: str) -> FakeTransport:
|
||||
connections.append(endpoint)
|
||||
return transport
|
||||
|
||||
backend = MetaBackend(
|
||||
MetaConfig("muse-voice-transcribe-1.0", key, terms, language_bias=("en",)),
|
||||
transport_factory=connect,
|
||||
timeout=1,
|
||||
)
|
||||
turn = backend.start_turn()
|
||||
turn.enqueue_pcm(b"pcm")
|
||||
self.assertEqual(turn.finish(), "final fixture transcript")
|
||||
|
||||
self.assertEqual(connections, [META_ENDPOINT])
|
||||
handshake = json.loads(transport.sent[0])
|
||||
self.assertEqual(handshake["authorization"]["accessToken"], "Bearer secret")
|
||||
self.assertEqual(handshake["audioEncoding"], "PCM_24KHZ")
|
||||
self.assertEqual(handshake["mode"], "PUSH_TO_TALK")
|
||||
self.assertFalse(handshake["emitAudioProgress"])
|
||||
self.assertEqual(handshake["languageBias"], ["en"])
|
||||
self.assertEqual(handshake["keywords"], ["JSON"])
|
||||
self.assertEqual(transport.sent[1], b"pcm")
|
||||
self.assertEqual(json.loads(transport.sent[2]), {"type": "endStream"})
|
||||
|
||||
def test_openai_uses_ga_session_shape_and_correlates_committed_item(self) -> None:
|
||||
key, terms, temporary = self.make_files()
|
||||
self.addCleanup(temporary.cleanup)
|
||||
transport = FakeTransport(fixture_events("openai"))
|
||||
connections: list[tuple[str, dict[str, str]]] = []
|
||||
|
||||
def connect(endpoint: str, *, headers: dict[str, str]) -> FakeTransport:
|
||||
connections.append((endpoint, headers))
|
||||
return transport
|
||||
|
||||
backend = OpenAIBackend(
|
||||
OpenAIConfig(
|
||||
"gpt-live-transcribe",
|
||||
key,
|
||||
terms,
|
||||
delay="low",
|
||||
prompt="Coding",
|
||||
languages=("en",),
|
||||
),
|
||||
transport_factory=connect,
|
||||
timeout=1,
|
||||
)
|
||||
turn = backend.start_turn()
|
||||
turn.enqueue_pcm(b"pcm")
|
||||
self.assertEqual(turn.finish(), "final fixture transcript")
|
||||
|
||||
self.assertEqual(
|
||||
connections,
|
||||
[(OPENAI_ENDPOINT, {"Authorization": "Bearer secret"})],
|
||||
)
|
||||
update = json.loads(transport.sent[0])
|
||||
session = update["session"]
|
||||
self.assertEqual(session["type"], "transcription")
|
||||
input_audio = session["audio"]["input"]
|
||||
self.assertEqual(input_audio["format"], {"type": "audio/pcm", "rate": 24000})
|
||||
self.assertEqual(input_audio["transcription"]["languages"], ["en"])
|
||||
self.assertEqual(input_audio["transcription"]["delay"], "low")
|
||||
self.assertEqual(input_audio["transcription"]["keywords"], ["JSON"])
|
||||
self.assertIsNone(input_audio["turn_detection"])
|
||||
self.assertEqual(json.loads(transport.sent[-1]), {"type": "input_audio_buffer.commit"})
|
||||
self.assertFalse(any(isinstance(item, str) and "response.create" in item for item in transport.sent))
|
||||
|
||||
def test_meta_rejects_malformed_ack_and_auth_error(self) -> None:
|
||||
key, terms, temporary = self.make_files()
|
||||
self.addCleanup(temporary.cleanup)
|
||||
malformed = MetaBackend(
|
||||
MetaConfig("muse-voice-transcribe-1.0", key, terms),
|
||||
transport_factory=lambda _endpoint: FakeTransport(['{"sessionId":"s","unexpected":1}']),
|
||||
timeout=0.1,
|
||||
)
|
||||
with self.assertRaisesRegex(ProviderError, "acknowledgement"):
|
||||
malformed.start_turn()
|
||||
|
||||
auth = MetaBackend(
|
||||
MetaConfig("muse-voice-transcribe-1.0", key, terms),
|
||||
transport_factory=lambda _endpoint: FakeTransport(['{"type":"error","code":"401"}']),
|
||||
timeout=0.1,
|
||||
)
|
||||
with self.assertRaisesRegex(ProviderError, "Meta transcription service reported") as error:
|
||||
auth.start_turn()
|
||||
self.assertEqual(error.exception.code, "meta_auth")
|
||||
with self.assertRaises(ValueError):
|
||||
MetaBackend(MetaConfig("muse-voice-transcribe-1.0", key, terms), max_queue_bytes=META_MAX_BACKLOG_BYTES + 1)
|
||||
|
||||
def test_cloud_turn_rejects_partial_only_close_and_queue_overflow(self) -> None:
|
||||
key, terms, temporary = self.make_files()
|
||||
self.addCleanup(temporary.cleanup)
|
||||
transport = FakeTransport(['{"sessionId":"s"}', '{"type":"transcript","final":false,"text":"partial"}', None])
|
||||
backend = MetaBackend(
|
||||
MetaConfig("muse-voice-transcribe-1.0", key, terms),
|
||||
transport_factory=lambda _endpoint: transport,
|
||||
timeout=0.1,
|
||||
)
|
||||
turn = backend.start_turn()
|
||||
with self.assertRaisesRegex(ProviderError, "final transcript"):
|
||||
turn.finish()
|
||||
|
||||
queue = PcmQueue(max_bytes=2, max_chunks=1)
|
||||
queue.put(b"12")
|
||||
with self.assertRaisesRegex(ProviderError, "full"):
|
||||
queue.put(b"3")
|
||||
|
||||
def test_cancel_before_prepare_worker_runs_prevents_connection(self) -> None:
|
||||
key, terms, temporary = self.make_files()
|
||||
self.addCleanup(temporary.cleanup)
|
||||
for backend in (
|
||||
MetaBackend(
|
||||
MetaConfig("muse-voice-transcribe-1.0", key, terms),
|
||||
transport_factory=lambda *_args, **_kwargs: self.fail("Meta connected"),
|
||||
),
|
||||
OpenAIBackend(
|
||||
OpenAIConfig("gpt-live-transcribe", key, terms),
|
||||
transport_factory=lambda *_args, **_kwargs: self.fail("OpenAI connected"),
|
||||
),
|
||||
):
|
||||
with self.subTest(backend=backend.descriptor.backend):
|
||||
backend.begin_prepare()
|
||||
backend.cancel_prepare()
|
||||
with self.assertRaises(ProviderError) as error:
|
||||
backend.start_turn()
|
||||
self.assertEqual(error.exception.code, "cancelled")
|
||||
|
||||
def test_cancel_during_connect_prevents_authenticated_handshake(self) -> None:
|
||||
key, terms, temporary = self.make_files()
|
||||
self.addCleanup(temporary.cleanup)
|
||||
for backend_name in ("meta", "openai"):
|
||||
with self.subTest(backend=backend_name):
|
||||
entered = threading.Event()
|
||||
release = threading.Event()
|
||||
transport = FakeTransport([])
|
||||
errors: list[BaseException] = []
|
||||
|
||||
def connect(*_args: object, **_kwargs: object) -> FakeTransport:
|
||||
entered.set()
|
||||
release.wait(1)
|
||||
return transport
|
||||
|
||||
if backend_name == "meta":
|
||||
backend = MetaBackend(
|
||||
MetaConfig("muse-voice-transcribe-1.0", key, terms),
|
||||
transport_factory=connect,
|
||||
timeout=0.1,
|
||||
)
|
||||
else:
|
||||
backend = OpenAIBackend(
|
||||
OpenAIConfig("gpt-live-transcribe", key, terms),
|
||||
transport_factory=connect,
|
||||
timeout=0.1,
|
||||
)
|
||||
worker = threading.Thread(
|
||||
target=lambda: _capture_start_error(backend, errors), daemon=True
|
||||
)
|
||||
worker.start()
|
||||
self.assertTrue(entered.wait(1))
|
||||
backend.cancel_prepare()
|
||||
release.set()
|
||||
worker.join(1)
|
||||
|
||||
self.assertFalse(worker.is_alive())
|
||||
self.assertEqual(getattr(errors[0], "code", None), "cancelled")
|
||||
self.assertEqual(transport.sent, [])
|
||||
|
||||
def test_meta_discards_high_volume_partials_instead_of_overflowing_event_queue(self) -> None:
|
||||
key, terms, temporary = self.make_files()
|
||||
self.addCleanup(temporary.cleanup)
|
||||
partial = '{"type":"transcript","final":false,"text":"partial"}'
|
||||
transport = FakeTransport([
|
||||
'{"sessionId":"s"}',
|
||||
*([partial] * 200),
|
||||
'{"type":"transcript","final":true,"text":"final"}',
|
||||
None,
|
||||
])
|
||||
turn = MetaBackend(
|
||||
MetaConfig("muse-voice-transcribe-1.0", key, terms),
|
||||
transport_factory=lambda _endpoint: transport,
|
||||
timeout=1,
|
||||
).start_turn()
|
||||
|
||||
self.assertEqual(turn.finish(), "final")
|
||||
|
||||
def test_cancel_never_sends_a_provider_end_marker(self) -> None:
|
||||
key, terms, temporary = self.make_files()
|
||||
self.addCleanup(temporary.cleanup)
|
||||
transports_and_turns = []
|
||||
meta_transport = FakeTransport(['{"sessionId":"s"}'])
|
||||
transports_and_turns.append((
|
||||
meta_transport,
|
||||
MetaBackend(
|
||||
MetaConfig("muse-voice-transcribe-1.0", key, terms),
|
||||
transport_factory=lambda _endpoint: meta_transport,
|
||||
timeout=1,
|
||||
).start_turn(),
|
||||
))
|
||||
openai_transport = FakeTransport([
|
||||
'{"type":"session.created"}', '{"type":"session.updated"}'
|
||||
])
|
||||
transports_and_turns.append((
|
||||
openai_transport,
|
||||
OpenAIBackend(
|
||||
OpenAIConfig("gpt-live-transcribe", key, terms),
|
||||
transport_factory=lambda _endpoint, **_kwargs: openai_transport,
|
||||
timeout=1,
|
||||
).start_turn(),
|
||||
))
|
||||
|
||||
for transport, turn in transports_and_turns:
|
||||
turn.cancel()
|
||||
deadline = time.monotonic() + 1
|
||||
while not turn.cleanup_complete and time.monotonic() < deadline:
|
||||
turn.close()
|
||||
sent_types = [
|
||||
value.get("type")
|
||||
for raw in transport.sent
|
||||
if isinstance(raw, str) and isinstance((value := json.loads(raw)), dict)
|
||||
]
|
||||
self.assertNotIn("endStream", sent_types)
|
||||
self.assertNotIn("input_audio_buffer.commit", sent_types)
|
||||
|
||||
def test_cancel_does_not_block_on_a_slow_transport_close(self) -> None:
|
||||
key, terms, temporary = self.make_files()
|
||||
self.addCleanup(temporary.cleanup)
|
||||
transport = BlockingCloseTransport(['{"sessionId":"s"}'])
|
||||
turn = MetaBackend(
|
||||
MetaConfig("muse-voice-transcribe-1.0", key, terms),
|
||||
transport_factory=lambda _endpoint: transport,
|
||||
timeout=0.1,
|
||||
).start_turn()
|
||||
|
||||
started = time.monotonic()
|
||||
turn.cancel()
|
||||
elapsed = time.monotonic() - started
|
||||
|
||||
self.assertLess(elapsed, 0.05)
|
||||
self.assertTrue(transport.close_started.wait(1))
|
||||
self.assertFalse(turn.cleanup_complete)
|
||||
transport.allow_close.set()
|
||||
deadline = time.monotonic() + 1
|
||||
while not turn.cleanup_complete and time.monotonic() < deadline:
|
||||
turn.close()
|
||||
self.assertTrue(turn.cleanup_complete)
|
||||
|
||||
def test_http_failures_use_websockets_response_status(self) -> None:
|
||||
key, terms, temporary = self.make_files()
|
||||
self.addCleanup(temporary.cleanup)
|
||||
meta = MetaBackend(
|
||||
MetaConfig("muse-voice-transcribe-1.0", key, terms),
|
||||
transport_factory=lambda _endpoint: (_ for _ in ()).throw(HttpFailure(401)),
|
||||
)
|
||||
with self.assertRaises(ProviderError) as meta_error:
|
||||
meta.start_turn()
|
||||
self.assertEqual(meta_error.exception.code, "meta_auth")
|
||||
|
||||
openai = OpenAIBackend(
|
||||
OpenAIConfig("gpt-live-transcribe", key, terms),
|
||||
transport_factory=lambda _endpoint, **_kwargs: (_ for _ in ()).throw(HttpFailure(429)),
|
||||
)
|
||||
with self.assertRaises(ProviderError) as openai_error:
|
||||
openai.start_turn()
|
||||
self.assertEqual(openai_error.exception.code, "openai_rate_limit")
|
||||
|
||||
def test_cloud_turn_reports_provider_close_during_recording(self) -> None:
|
||||
key, terms, temporary = self.make_files()
|
||||
self.addCleanup(temporary.cleanup)
|
||||
backend = MetaBackend(
|
||||
MetaConfig("muse-voice-transcribe-1.0", key, terms),
|
||||
transport_factory=lambda _endpoint: FakeTransport([
|
||||
'{"sessionId":"s"}',
|
||||
None,
|
||||
]),
|
||||
timeout=0.1,
|
||||
)
|
||||
turn = backend.start_turn()
|
||||
deadline = time.monotonic() + 1
|
||||
failure = turn.poll_error()
|
||||
while failure is None and time.monotonic() < deadline:
|
||||
time.sleep(0.001)
|
||||
failure = turn.poll_error()
|
||||
|
||||
self.assertIsNotNone(failure)
|
||||
self.assertEqual(failure.code, "meta_premature_close")
|
||||
|
||||
def test_provider_event_rejects_recursive_values(self) -> None:
|
||||
recursive: dict[str, object] = {}
|
||||
recursive["self"] = recursive
|
||||
with self.assertRaises(ProviderError):
|
||||
provider_event(recursive)
|
||||
|
||||
def test_openai_rejects_malformed_error_and_unmatched_completion(self) -> None:
|
||||
key, terms, temporary = self.make_files()
|
||||
self.addCleanup(temporary.cleanup)
|
||||
malformed = OpenAIBackend(
|
||||
OpenAIConfig("gpt-live-transcribe", key, terms),
|
||||
transport_factory=lambda _endpoint, **_kwargs: FakeTransport([
|
||||
'{"type":"session.created"}',
|
||||
'{"type":"error","error":{"message":"bad"}}',
|
||||
]),
|
||||
timeout=0.1,
|
||||
)
|
||||
with self.assertRaisesRegex(ProviderError, "error event"):
|
||||
malformed.start_turn()
|
||||
|
||||
transport = FakeTransport([
|
||||
'{"type":"session.created"}',
|
||||
'{"type":"session.updated"}',
|
||||
'{"type":"input_audio_buffer.committed","item_id":"new"}',
|
||||
'{"type":"conversation.item.input_audio_transcription.completed","item_id":"old","transcript":"stale"}',
|
||||
None,
|
||||
])
|
||||
backend = OpenAIBackend(
|
||||
OpenAIConfig("gpt-live-transcribe", key, terms),
|
||||
transport_factory=lambda _endpoint, **_kwargs: transport,
|
||||
timeout=0.1,
|
||||
)
|
||||
turn = backend.start_turn()
|
||||
with self.assertRaises(ProviderError) as error:
|
||||
turn.finish()
|
||||
self.assertIn(error.exception.code, {"openai_premature_close", "openai_timeout"})
|
||||
|
||||
def test_openai_commit_oriented_alternative_uses_explicit_commit(self) -> None:
|
||||
key, terms, temporary = self.make_files()
|
||||
self.addCleanup(temporary.cleanup)
|
||||
transport = FakeTransport(fixture_events("openai"))
|
||||
turn = OpenAIBackend(
|
||||
OpenAIConfig("gpt-transcribe", key, terms),
|
||||
transport_factory=lambda _endpoint, **_kwargs: transport,
|
||||
timeout=1,
|
||||
).start_turn()
|
||||
turn.enqueue_pcm(b"pcm")
|
||||
|
||||
self.assertEqual(turn.finish(), "final fixture transcript")
|
||||
update = json.loads(transport.sent[0])
|
||||
self.assertEqual(
|
||||
update["session"]["audio"]["input"]["transcription"]["model"],
|
||||
"gpt-transcribe",
|
||||
)
|
||||
self.assertEqual(json.loads(transport.sent[-1]), {"type": "input_audio_buffer.commit"})
|
||||
|
||||
def test_openai_rejects_unknown_delay_from_direct_callers(self) -> None:
|
||||
key, terms, temporary = self.make_files()
|
||||
self.addCleanup(temporary.cleanup)
|
||||
with self.assertRaisesRegex(ValueError, "delay"):
|
||||
OpenAIBackend(OpenAIConfig("gpt-live-transcribe", key, terms, delay="fast"))
|
||||
|
||||
def test_openai_receiver_failure_overrides_an_already_queued_completion(self) -> None:
|
||||
key, terms, temporary = self.make_files()
|
||||
self.addCleanup(temporary.cleanup)
|
||||
transport = FakeTransport([
|
||||
'{"type":"session.created"}',
|
||||
'{"type":"session.updated"}',
|
||||
'{"type":"input_audio_buffer.committed","item_id":"i"}',
|
||||
'{"type":"conversation.item.input_audio_transcription.completed","item_id":"i","transcript":"unsafe"}',
|
||||
'{}',
|
||||
])
|
||||
turn = OpenAIBackend(
|
||||
OpenAIConfig("gpt-live-transcribe", key, terms),
|
||||
transport_factory=lambda _endpoint, **_kwargs: transport,
|
||||
timeout=1,
|
||||
).start_turn()
|
||||
deadline = time.monotonic() + 1
|
||||
while turn._receiver_error is None and time.monotonic() < deadline:
|
||||
time.sleep(0.001)
|
||||
|
||||
with self.assertRaises(ProviderError) as error:
|
||||
turn.finish()
|
||||
self.assertEqual(error.exception.code, "openai_malformed_event")
|
||||
|
||||
def test_openai_audio_is_bounded_base64(self) -> None:
|
||||
key, terms, temporary = self.make_files()
|
||||
self.addCleanup(temporary.cleanup)
|
||||
transport = FakeTransport(
|
||||
['{"type":"session.created"}', '{"type":"session.updated"}',
|
||||
'{"type":"input_audio_buffer.committed","item_id":"i"}',
|
||||
'{"type":"conversation.item.input_audio_transcription.completed","item_id":"i","transcript":""}']
|
||||
)
|
||||
backend = OpenAIBackend(
|
||||
OpenAIConfig("gpt-live-transcribe", key, terms),
|
||||
transport_factory=lambda _endpoint, **_kwargs: transport,
|
||||
timeout=1,
|
||||
)
|
||||
turn = backend.start_turn()
|
||||
turn.enqueue_pcm(b"\x01\x02")
|
||||
self.assertEqual(turn.finish(), "")
|
||||
append = json.loads(transport.sent[1])
|
||||
self.assertEqual(base64.b64decode(append["audio"]), b"\x01\x02")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,120 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import tempfile
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
from pi_dictate_service.config import (
|
||||
ConfigError,
|
||||
CredentialError,
|
||||
load_config,
|
||||
parse_config,
|
||||
provider_keywords,
|
||||
read_credential,
|
||||
)
|
||||
from pi_dictate_service.recorder import audio_level_window_bytes, pw_record_command
|
||||
|
||||
|
||||
class ConfigTests(unittest.TestCase):
|
||||
def test_recorder_supports_both_backend_sample_rates(self) -> None:
|
||||
self.assertEqual(pw_record_command(16_000)[5], "16000")
|
||||
self.assertEqual(pw_record_command(24_000)[5], "24000")
|
||||
self.assertEqual(audio_level_window_bytes(24_000), 4_800)
|
||||
|
||||
def test_rejects_unknown_selected_fields_and_invalid_backend_values(self) -> None:
|
||||
for invalid_backend in ([], {}, 1, True, None):
|
||||
with self.subTest(backend=invalid_backend):
|
||||
with self.assertRaisesRegex(ConfigError, "backend must be"):
|
||||
parse_config({"version": 1, "backend": invalid_backend})
|
||||
|
||||
base = {
|
||||
"version": 1,
|
||||
"backend": "openai",
|
||||
"openai": {
|
||||
"model": "gpt-live-transcribe",
|
||||
"apiKeyFile": "~/.config/pi-dictate/credentials/openai-api-key",
|
||||
"termsPath": "/tmp/terms",
|
||||
},
|
||||
}
|
||||
with patch.dict(os.environ, {"HOME": "/tmp/pi-dictate-test"}):
|
||||
invalid = {**base, "openai": {**base["openai"], "endpoint": "https://bad"}}
|
||||
with self.assertRaises(ConfigError):
|
||||
parse_config(invalid)
|
||||
with self.assertRaises(ConfigError):
|
||||
parse_config({**base, "version": 2})
|
||||
configured = parse_config(
|
||||
{**base, "openai": {**base["openai"], "delay": "low"}}
|
||||
)
|
||||
self.assertEqual(configured.openai.delay, "low")
|
||||
with self.assertRaises(ConfigError):
|
||||
parse_config({**base, "openai": {**base["openai"], "delay": "fast"}})
|
||||
with self.assertRaises(ConfigError):
|
||||
parse_config({**base, "openai": {**base["openai"], "termsPath": "relative/terms"}})
|
||||
|
||||
def test_reads_only_one_trailing_newline_from_a_private_credential(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
path = Path(directory) / "key"
|
||||
path.write_text("secret\n", encoding="utf-8")
|
||||
path.chmod(0o600)
|
||||
self.assertEqual(read_credential(path), "secret")
|
||||
|
||||
path.chmod(0o640)
|
||||
with self.assertRaises(CredentialError):
|
||||
read_credential(path)
|
||||
|
||||
path.chmod(0o600)
|
||||
path.write_text("secret\nextra", encoding="utf-8")
|
||||
with self.assertRaises(CredentialError):
|
||||
read_credential(path)
|
||||
|
||||
def test_rejects_a_credential_reached_through_a_symlinked_directory(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
actual = root / "actual"
|
||||
actual.mkdir()
|
||||
key = actual / "key"
|
||||
key.write_text("secret", encoding="utf-8")
|
||||
key.chmod(0o600)
|
||||
linked = root / "credentials"
|
||||
linked.symlink_to(actual, target_is_directory=True)
|
||||
|
||||
with self.assertRaisesRegex(CredentialError, "ancestry"):
|
||||
read_credential(linked / "key")
|
||||
|
||||
def test_config_file_is_regular_private_and_provider_terms_reject_cr(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
config_path = root / "config.json"
|
||||
config_path.write_text(
|
||||
json.dumps(
|
||||
{
|
||||
"version": 1,
|
||||
"backend": "local",
|
||||
"local": {
|
||||
"adapter": "nemo-parakeet",
|
||||
"modelPath": "/tmp/model.nemo",
|
||||
"modelSha256": "0" * 64,
|
||||
"termsPath": "/tmp/terms",
|
||||
"boostAlpha": 1.0,
|
||||
},
|
||||
}
|
||||
),
|
||||
encoding="utf-8",
|
||||
)
|
||||
config_path.chmod(0o600)
|
||||
self.assertEqual(load_config(config_path).backend, "local")
|
||||
|
||||
terms = root / "terms"
|
||||
terms.write_bytes(b"bad\rterm")
|
||||
with self.assertRaises(ConfigError):
|
||||
provider_keywords(terms)
|
||||
terms.write_bytes(b"safe\nbad\x00term\n")
|
||||
with self.assertRaises(ConfigError):
|
||||
provider_keywords(terms)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,496 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import shutil
|
||||
import stat
|
||||
import subprocess
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
|
||||
SCRIPT = Path(__file__).resolve().parents[1] / "install.sh"
|
||||
SYSTEMD_ROOT = SCRIPT.parents[2] / "systemd/user"
|
||||
LEGACY_SERVICE = """# Managed by pi-dictate install.sh
|
||||
[Unit]
|
||||
Description=Pi local Parakeet dictation service
|
||||
Requires=pi-dictate.socket
|
||||
After=pipewire.service
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
ExecStart=%h/.local/share/pi-dictate/venv/bin/pi-dictate-service
|
||||
Environment=PI_DICTATE_MODEL_PATH=%h/.local/share/pi-dictate/models/parakeet-tdt-0.6b-v3/parakeet-tdt-0.6b-v3.nemo
|
||||
Environment=PI_DICTATE_TERMS_PATH=%h/.local/share/pi-dictate/config/coding-terms.txt
|
||||
Environment=PI_DICTATE_BOOST_ALPHA=1.0
|
||||
Environment=PI_DICTATE_IDLE_SECONDS=1800
|
||||
Environment=HF_HUB_OFFLINE=1
|
||||
Environment=TRANSFORMERS_OFFLINE=1
|
||||
Environment=NEMO_HOME=/tmp/pi-dictate-nemo
|
||||
Environment=HF_HOME=/tmp/pi-dictate-huggingface
|
||||
Environment=TRITON_CACHE_DIR=/tmp/pi-dictate-triton
|
||||
Environment=NUMBA_CACHE_DIR=/tmp/pi-dictate-numba
|
||||
Environment=PYTHONDONTWRITEBYTECODE=1
|
||||
UMask=0077
|
||||
LimitCORE=0
|
||||
MemorySwapMax=0
|
||||
NoNewPrivileges=yes
|
||||
PrivateTmp=yes
|
||||
ProtectClock=yes
|
||||
ProtectControlGroups=yes
|
||||
ProtectKernelLogs=yes
|
||||
ProtectKernelModules=yes
|
||||
ProtectKernelTunables=yes
|
||||
ProtectSystem=strict
|
||||
ProtectHome=read-only
|
||||
RestrictAddressFamilies=AF_UNIX
|
||||
IPAddressDeny=any
|
||||
LockPersonality=yes
|
||||
|
||||
# The socket unit owns %t/pi-dictate.sock. The service must not unlink it.
|
||||
"""
|
||||
|
||||
|
||||
class InstallFlowTests(unittest.TestCase):
|
||||
def setUp(self) -> None:
|
||||
self.directory = tempfile.TemporaryDirectory()
|
||||
self.root = Path(self.directory.name)
|
||||
self.home = self.root / "home"
|
||||
self.home.mkdir()
|
||||
self.bin = self.root / "bin"
|
||||
self.bin.mkdir()
|
||||
uv = self.bin / "uv"
|
||||
# A real uv sync creates the requested environment. Keep the fake
|
||||
# faithful to that contract so transactional environment swapping is
|
||||
# exercised without installing dependencies.
|
||||
uv.write_text("#!/bin/sh\nmkdir -p \"$UV_PROJECT_ENVIRONMENT\"\nexit 0\n", encoding="utf-8")
|
||||
uv.chmod(0o755)
|
||||
systemctl = self.bin / "systemctl"
|
||||
systemctl.write_text("#!/bin/sh\nexit 0\n", encoding="utf-8")
|
||||
systemctl.chmod(0o755)
|
||||
|
||||
def tearDown(self) -> None:
|
||||
self.directory.cleanup()
|
||||
|
||||
def run_installer(self, *arguments: str) -> subprocess.CompletedProcess[str]:
|
||||
environment = {
|
||||
**os.environ,
|
||||
"HOME": str(self.home),
|
||||
"XDG_CONFIG_HOME": str(self.home / ".config"),
|
||||
"PATH": f"{self.bin}:{os.environ['PATH']}",
|
||||
}
|
||||
return subprocess.run(
|
||||
[str(SCRIPT), *arguments],
|
||||
env=environment,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
|
||||
def test_real_staged_cloud_venv_python_runs_after_transactional_move(self) -> None:
|
||||
real_uv = shutil.which("uv")
|
||||
self.assertIsNotNone(real_uv)
|
||||
(self.bin / "uv").unlink()
|
||||
source = self.root / "openai-key"
|
||||
source.write_text("test-key", encoding="utf-8")
|
||||
|
||||
result = self.run_installer(
|
||||
"install", "--backend", "openai", "--api-key-file", str(source)
|
||||
)
|
||||
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
python = self.home / ".local/share/pi-dictate/venv/bin/python"
|
||||
executed = subprocess.run(
|
||||
[str(python), "-c", "import pi_dictate_service"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
self.assertEqual(executed.returncode, 0, executed.stderr)
|
||||
|
||||
def test_upgrades_the_previous_service_unit_and_preserves_its_idle_setting(self) -> None:
|
||||
unit_dir = self.home / ".config/systemd/user"
|
||||
unit_dir.mkdir(parents=True)
|
||||
(unit_dir / "pi-dictate.service").write_text(LEGACY_SERVICE, encoding="utf-8")
|
||||
(unit_dir / "pi-dictate.socket").write_bytes(
|
||||
(SYSTEMD_ROOT / "pi-dictate.socket").read_bytes()
|
||||
)
|
||||
source = self.root / "custom.nemo"
|
||||
source.write_bytes(b"custom archive")
|
||||
digest = hashlib.sha256(source.read_bytes()).hexdigest()
|
||||
|
||||
result = self.run_installer(
|
||||
"install",
|
||||
"--backend",
|
||||
"local",
|
||||
"--model-source",
|
||||
str(source),
|
||||
"--model-sha256",
|
||||
digest,
|
||||
)
|
||||
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
self.assertEqual(
|
||||
(unit_dir / "pi-dictate.service").read_bytes(),
|
||||
(SYSTEMD_ROOT / "pi-dictate.service").read_bytes(),
|
||||
)
|
||||
config = json.loads(
|
||||
(self.home / ".config/pi-dictate/config.json").read_text(encoding="utf-8")
|
||||
)
|
||||
self.assertEqual(config["idleSeconds"], 1800)
|
||||
|
||||
def test_rejects_symlinked_install_ancestry(self) -> None:
|
||||
outside = self.root / "outside"
|
||||
outside.mkdir()
|
||||
(self.home / ".local").symlink_to(outside, target_is_directory=True)
|
||||
source = self.root / "custom.nemo"
|
||||
source.write_bytes(b"custom archive")
|
||||
digest = hashlib.sha256(source.read_bytes()).hexdigest()
|
||||
|
||||
result = self.run_installer(
|
||||
"install",
|
||||
"--backend",
|
||||
"local",
|
||||
"--model-source",
|
||||
str(source),
|
||||
"--model-sha256",
|
||||
digest,
|
||||
)
|
||||
|
||||
self.assertNotEqual(result.returncode, 0)
|
||||
self.assertIn("symlinked path ancestry", result.stderr)
|
||||
self.assertEqual(list(outside.iterdir()), [])
|
||||
|
||||
def test_cloud_install_uses_only_cloud_profile_and_managed_key(self) -> None:
|
||||
source = self.root / "meta-key"
|
||||
source.write_text("test-key\n", encoding="utf-8")
|
||||
|
||||
result = self.run_installer(
|
||||
"install", "--backend", "meta", "--api-key-file", str(source)
|
||||
)
|
||||
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
self.assertNotIn("test-key", result.stdout + result.stderr)
|
||||
self.assertNotIn("nvidia", result.stdout)
|
||||
self.assertFalse(
|
||||
(self.home / ".local/share/pi-dictate/models").exists()
|
||||
)
|
||||
config_path = self.home / ".config/pi-dictate/config.json"
|
||||
config = json.loads(config_path.read_text(encoding="utf-8"))
|
||||
self.assertEqual(config["backend"], "meta")
|
||||
self.assertEqual(config["meta"]["model"], "muse-voice-transcribe-1.0")
|
||||
self.assertEqual(config["meta"]["languageBias"], [])
|
||||
self.assertEqual(
|
||||
(self.home / ".config/pi-dictate/credentials/meta-api-key").read_text(
|
||||
encoding="utf-8"
|
||||
),
|
||||
"test-key",
|
||||
)
|
||||
self.assertEqual(stat.S_IMODE(config_path.stat().st_mode), 0o600)
|
||||
self.assertEqual(
|
||||
stat.S_IMODE(
|
||||
(self.home / ".config/pi-dictate/credentials/meta-api-key").stat().st_mode
|
||||
),
|
||||
0o600,
|
||||
)
|
||||
self.assertEqual(
|
||||
(self.home / ".local/share/pi-dictate/.dependency-profile").read_text(
|
||||
encoding="utf-8"
|
||||
),
|
||||
"cloud\n",
|
||||
)
|
||||
|
||||
def test_check_rejects_values_that_runtime_configuration_rejects(self) -> None:
|
||||
source = self.root / "openai-key"
|
||||
source.write_text("test-key", encoding="utf-8")
|
||||
self.assertEqual(
|
||||
self.run_installer(
|
||||
"install", "--backend", "openai", "--api-key-file", str(source)
|
||||
).returncode,
|
||||
0,
|
||||
)
|
||||
config_path = self.home / ".config/pi-dictate/config.json"
|
||||
valid = json.loads(config_path.read_text(encoding="utf-8"))
|
||||
invalid_configs = []
|
||||
boolean_version = {**valid, "version": True}
|
||||
invalid_configs.append(boolean_version)
|
||||
unhashable_backend = {**valid, "backend": []}
|
||||
invalid_configs.append(unhashable_backend)
|
||||
long_model = json.loads(json.dumps(valid))
|
||||
long_model["openai"]["model"] = "m" * 129
|
||||
invalid_configs.append(long_model)
|
||||
unhashable_delay = json.loads(json.dumps(valid))
|
||||
unhashable_delay["openai"]["delay"] = []
|
||||
invalid_configs.append(unhashable_delay)
|
||||
relative_terms = json.loads(json.dumps(valid))
|
||||
relative_terms["openai"]["termsPath"] = "relative/terms.txt"
|
||||
invalid_configs.append(relative_terms)
|
||||
|
||||
for invalid in invalid_configs:
|
||||
with self.subTest(invalid=invalid):
|
||||
config_path.write_text(json.dumps(invalid), encoding="utf-8")
|
||||
config_path.chmod(0o600)
|
||||
result = self.run_installer("check")
|
||||
self.assertNotEqual(result.returncode, 0)
|
||||
self.assertNotIn("Traceback", result.stderr)
|
||||
|
||||
def test_check_rejects_extra_dependency_marker_newlines(self) -> None:
|
||||
source = self.root / "openai-key"
|
||||
source.write_text("test-key", encoding="utf-8")
|
||||
self.assertEqual(
|
||||
self.run_installer(
|
||||
"install", "--backend", "openai", "--api-key-file", str(source)
|
||||
).returncode,
|
||||
0,
|
||||
)
|
||||
marker = self.home / ".local/share/pi-dictate/.dependency-profile"
|
||||
marker.write_bytes(b"cloud\n\n")
|
||||
marker.chmod(0o600)
|
||||
|
||||
result = self.run_installer("check")
|
||||
|
||||
self.assertNotEqual(result.returncode, 0)
|
||||
self.assertIn("dependency profile", result.stderr)
|
||||
|
||||
def test_uninstall_removes_an_exact_openai_dropin(self) -> None:
|
||||
source = self.root / "openai-key"
|
||||
source.write_text("test-key", encoding="utf-8")
|
||||
self.assertEqual(
|
||||
self.run_installer(
|
||||
"install", "--backend", "openai", "--api-key-file", str(source)
|
||||
).returncode,
|
||||
0,
|
||||
)
|
||||
|
||||
result = self.run_installer("uninstall")
|
||||
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
self.assertFalse(
|
||||
(self.home / ".config/systemd/user/pi-dictate.service.d/10-pi-dictate-network.conf").exists()
|
||||
)
|
||||
|
||||
def test_openai_configure_preserves_live_verified_delay(self) -> None:
|
||||
source = self.root / "openai-key"
|
||||
source.write_text("test-key", encoding="utf-8")
|
||||
self.assertEqual(
|
||||
self.run_installer(
|
||||
"install", "--backend", "openai", "--api-key-file", str(source)
|
||||
).returncode,
|
||||
0,
|
||||
)
|
||||
config_path = self.home / ".config/pi-dictate/config.json"
|
||||
config = json.loads(config_path.read_text(encoding="utf-8"))
|
||||
config["openai"]["delay"] = "low"
|
||||
config_path.write_text(json.dumps(config), encoding="utf-8")
|
||||
config_path.chmod(0o600)
|
||||
|
||||
result = self.run_installer("configure", "--backend", "openai")
|
||||
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
configured = json.loads(config_path.read_text(encoding="utf-8"))
|
||||
self.assertEqual(configured["openai"]["delay"], "low")
|
||||
|
||||
def test_meta_configure_preserves_valid_language_bias(self) -> None:
|
||||
source = self.root / "meta-key"
|
||||
source.write_text("test-key", encoding="utf-8")
|
||||
self.assertEqual(
|
||||
self.run_installer(
|
||||
"install", "--backend", "meta", "--api-key-file", str(source)
|
||||
).returncode,
|
||||
0,
|
||||
)
|
||||
config_path = self.home / ".config/pi-dictate/config.json"
|
||||
config = json.loads(config_path.read_text(encoding="utf-8"))
|
||||
config["meta"]["languageBias"] = ["en", "fr"]
|
||||
config_path.write_text(json.dumps(config), encoding="utf-8")
|
||||
config_path.chmod(0o600)
|
||||
|
||||
result = self.run_installer("configure", "--backend", "meta")
|
||||
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
configured = json.loads(config_path.read_text(encoding="utf-8"))
|
||||
self.assertEqual(configured["meta"]["languageBias"], ["en", "fr"])
|
||||
|
||||
def test_local_custom_archive_requires_and_records_explicit_digest(self) -> None:
|
||||
source = self.root / "custom.nemo"
|
||||
source.write_bytes(b"custom archive")
|
||||
digest = hashlib.sha256(source.read_bytes()).hexdigest()
|
||||
|
||||
result = self.run_installer(
|
||||
"install",
|
||||
"--backend",
|
||||
"local",
|
||||
"--model-source",
|
||||
str(source),
|
||||
"--model-sha256",
|
||||
digest,
|
||||
)
|
||||
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
config = json.loads(
|
||||
(self.home / ".config/pi-dictate/config.json").read_text(encoding="utf-8")
|
||||
)
|
||||
self.assertEqual(config["local"]["modelSha256"], digest)
|
||||
installed = self.home / (
|
||||
".local/share/pi-dictate/models/parakeet-tdt-0.6b-v3/"
|
||||
"parakeet-tdt-0.6b-v3.nemo"
|
||||
)
|
||||
self.assertEqual(installed.read_bytes(), source.read_bytes())
|
||||
|
||||
def test_replacing_a_custom_local_model_preserves_its_configured_path(self) -> None:
|
||||
first = self.root / "first.nemo"
|
||||
first.write_bytes(b"first archive")
|
||||
first_digest = hashlib.sha256(first.read_bytes()).hexdigest()
|
||||
self.assertEqual(
|
||||
self.run_installer(
|
||||
"install", "--backend", "local", "--model-source", str(first),
|
||||
"--model-sha256", first_digest,
|
||||
).returncode,
|
||||
0,
|
||||
)
|
||||
config_path = self.home / ".config/pi-dictate/config.json"
|
||||
config = json.loads(config_path.read_text(encoding="utf-8"))
|
||||
custom_model = self.root / "custom-models/model.nemo"
|
||||
custom_model.parent.mkdir()
|
||||
custom_model.write_bytes(first.read_bytes())
|
||||
config["local"]["modelPath"] = str(custom_model)
|
||||
config_path.write_text(json.dumps(config), encoding="utf-8")
|
||||
config_path.chmod(0o600)
|
||||
second = self.root / "second.nemo"
|
||||
second.write_bytes(b"second archive")
|
||||
second_digest = hashlib.sha256(second.read_bytes()).hexdigest()
|
||||
|
||||
result = self.run_installer(
|
||||
"configure", "--backend", "local", "--model-source", str(second),
|
||||
"--model-sha256", second_digest,
|
||||
)
|
||||
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
configured = json.loads(config_path.read_text(encoding="utf-8"))
|
||||
self.assertEqual(configured["local"]["modelPath"], str(custom_model))
|
||||
self.assertEqual(custom_model.read_bytes(), second.read_bytes())
|
||||
self.assertFalse(list(custom_model.parent.glob("*.pi-dictate-rollback.*")))
|
||||
|
||||
def test_switching_from_cloud_to_local_accepts_a_local_archive(self) -> None:
|
||||
key = self.root / "meta-key"
|
||||
key.write_text("test-key", encoding="utf-8")
|
||||
self.assertEqual(
|
||||
self.run_installer(
|
||||
"install", "--backend", "meta", "--api-key-file", str(key)
|
||||
).returncode,
|
||||
0,
|
||||
)
|
||||
source = self.root / "custom.nemo"
|
||||
source.write_bytes(b"custom archive")
|
||||
digest = hashlib.sha256(source.read_bytes()).hexdigest()
|
||||
|
||||
result = self.run_installer(
|
||||
"configure",
|
||||
"--backend",
|
||||
"local",
|
||||
"--model-source",
|
||||
str(source),
|
||||
"--model-sha256",
|
||||
digest,
|
||||
)
|
||||
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
config = json.loads(
|
||||
(self.home / ".config/pi-dictate/config.json").read_text(encoding="utf-8")
|
||||
)
|
||||
self.assertEqual(config["backend"], "local")
|
||||
self.assertEqual(config["local"]["modelSha256"], digest)
|
||||
self.assertIn("meta", config)
|
||||
|
||||
def test_systemd_failure_rolls_back_staged_install(self) -> None:
|
||||
old_venv = self.home / ".local/share/pi-dictate/venv"
|
||||
old_venv.mkdir(parents=True)
|
||||
(old_venv / "previous-runtime").write_text("old", encoding="utf-8")
|
||||
source = self.root / "meta-key"
|
||||
source.write_text("test-key", encoding="utf-8")
|
||||
(self.bin / "systemctl").write_text(
|
||||
"#!/bin/sh\necho \"$*\" >> \"$HOME/systemctl.log\"\n"
|
||||
"case \"$*\" in *'is-enabled'*) exit 1;; "
|
||||
"*'stop pi-dictate.service'*) exit 9;; esac\nexit 0\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
(self.bin / "systemctl").chmod(0o755)
|
||||
|
||||
result = self.run_installer(
|
||||
"install", "--backend", "meta", "--api-key-file", str(source)
|
||||
)
|
||||
|
||||
self.assertNotEqual(result.returncode, 0)
|
||||
self.assertFalse((self.home / ".config/pi-dictate/config.json").exists())
|
||||
self.assertFalse((self.home / ".local/share/pi-dictate/.dependency-profile").exists())
|
||||
self.assertFalse((self.home / ".local/share/pi-dictate/config/coding-terms.txt").exists())
|
||||
self.assertEqual((old_venv / "previous-runtime").read_text(encoding="utf-8"), "old")
|
||||
systemctl_calls = (self.home / "systemctl.log").read_text(encoding="utf-8")
|
||||
self.assertGreaterEqual(systemctl_calls.count("daemon-reload"), 1)
|
||||
self.assertIn("disable pi-dictate.socket", systemctl_calls)
|
||||
|
||||
def test_invalid_cloud_key_does_not_commit_a_dependency_marker(self) -> None:
|
||||
source = self.root / "invalid-key"
|
||||
source.write_text("two\nlines\n", encoding="utf-8")
|
||||
|
||||
result = self.run_installer(
|
||||
"install", "--backend", "meta", "--api-key-file", str(source)
|
||||
)
|
||||
|
||||
self.assertNotEqual(result.returncode, 0)
|
||||
self.assertFalse(
|
||||
(self.home / ".local/share/pi-dictate/.dependency-profile").exists()
|
||||
)
|
||||
|
||||
def test_dictionary_symlink_is_rejected(self) -> None:
|
||||
dictionary = self.home / ".local/share/pi-dictate/config/coding-terms.txt"
|
||||
dictionary.parent.mkdir(parents=True)
|
||||
target = self.root / "unexpected-terms.txt"
|
||||
target.write_text("unexpected", encoding="utf-8")
|
||||
dictionary.symlink_to(target)
|
||||
source = self.root / "custom.nemo"
|
||||
source.write_bytes(b"custom archive")
|
||||
digest = hashlib.sha256(source.read_bytes()).hexdigest()
|
||||
|
||||
result = self.run_installer(
|
||||
"install",
|
||||
"--backend",
|
||||
"local",
|
||||
"--model-source",
|
||||
str(source),
|
||||
"--model-sha256",
|
||||
digest,
|
||||
)
|
||||
|
||||
self.assertNotEqual(result.returncode, 0)
|
||||
self.assertIn("non-regular coding dictionary", result.stderr)
|
||||
|
||||
def test_uninstall_preserves_config_unless_purge_is_requested(self) -> None:
|
||||
source = self.root / "meta-key"
|
||||
source.write_text("test-key", encoding="utf-8")
|
||||
self.assertEqual(
|
||||
self.run_installer(
|
||||
"install", "--backend", "meta", "--api-key-file", str(source)
|
||||
).returncode,
|
||||
0,
|
||||
)
|
||||
|
||||
result = self.run_installer("uninstall")
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
self.assertTrue((self.home / ".config/pi-dictate/config.json").exists())
|
||||
self.assertFalse((self.home / ".local/share/pi-dictate").exists())
|
||||
|
||||
# Reinstalling from the preserved key is possible without passing a
|
||||
# literal secret or a second key source.
|
||||
result = self.run_installer("install", "--backend", "meta")
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
self.assertEqual(
|
||||
self.run_installer("uninstall", "--purge-config").returncode, 0
|
||||
)
|
||||
self.assertFalse((self.home / ".config/pi-dictate").exists())
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -1,3 +1,4 @@
|
||||
import hashlib
|
||||
from pathlib import Path
|
||||
import os
|
||||
import subprocess
|
||||
@@ -24,8 +25,11 @@ class InstallerTests(unittest.TestCase):
|
||||
home / ".local/share/pi-dictate"
|
||||
),
|
||||
}
|
||||
command = [str(SCRIPT), *arguments, "--dry-run"]
|
||||
if not any(action in arguments for action in ("uninstall", "check")):
|
||||
command.extend(("--model-source", str(model), "--model-sha256", hashlib.sha256(model.read_bytes()).hexdigest()))
|
||||
return subprocess.run(
|
||||
[str(SCRIPT), *arguments, "--dry-run", "--model-source", str(model)],
|
||||
command,
|
||||
env=environment,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
@@ -47,7 +51,9 @@ class InstallerTests(unittest.TestCase):
|
||||
self.assertNotIn("ln -s", result.stdout)
|
||||
self.assertIn(".nemo.install.", result.stdout)
|
||||
self.assertIn("mv -f", result.stdout)
|
||||
self.assertIn("uv venv --relocatable", result.stdout)
|
||||
self.assertIn("uv sync --frozen --no-dev --no-editable", result.stdout)
|
||||
self.assertIn("--reinstall-package pi-dictate-service", result.stdout)
|
||||
self.assertIn("Would install pi-dictate", result.stdout)
|
||||
self.assertFalse((home / ".local/share/pi-dictate").exists())
|
||||
|
||||
@@ -87,6 +93,25 @@ class InstallerTests(unittest.TestCase):
|
||||
"[Socket]\nListenStream=/tmp/unrelated.sock\n",
|
||||
)
|
||||
|
||||
def test_uninstall_refuses_a_modified_managed_unit(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
home = Path(directory)
|
||||
model = home / "model.nemo"
|
||||
model.write_bytes(b"model")
|
||||
unit = home / ".config/systemd/user/pi-dictate.socket"
|
||||
unit.parent.mkdir(parents=True)
|
||||
unit.write_text(
|
||||
"# Managed by pi-dictate install.sh\n[Socket]\nListenStream=/tmp/tampered.sock\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
result = self.run_installer(home, model, "uninstall")
|
||||
|
||||
self.assertEqual(result.returncode, 1)
|
||||
self.assertIn("unmanaged systemd unit", result.stderr)
|
||||
self.assertNotIn("systemctl --user disable", result.stdout)
|
||||
self.assertTrue(unit.exists())
|
||||
|
||||
def test_reinstall_preserves_an_existing_host_dictionary(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
home = Path(directory)
|
||||
|
||||
@@ -0,0 +1,34 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
from pi_dictate_service.live_smoke import MAX_SAMPLE_BYTES, _read_sample
|
||||
|
||||
|
||||
class LiveSmokeTests(unittest.TestCase):
|
||||
def test_reads_one_bounded_even_pcm_fixture(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
sample = Path(directory) / "sample.pcm"
|
||||
sample.write_bytes(b"\x00\x01" * 100)
|
||||
self.assertEqual(_read_sample(sample), sample.read_bytes())
|
||||
|
||||
def test_rejects_symlink_odd_and_oversized_fixtures(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
valid = root / "valid.pcm"
|
||||
valid.write_bytes(b"\x00\x00")
|
||||
link = root / "link.pcm"
|
||||
link.symlink_to(valid)
|
||||
odd = root / "odd.pcm"
|
||||
odd.write_bytes(b"x")
|
||||
oversized = root / "oversized.pcm"
|
||||
oversized.write_bytes(b"x" * (MAX_SAMPLE_BYTES + 2))
|
||||
for path in (link, odd, oversized):
|
||||
with self.subTest(path=path), self.assertRaises((OSError, ValueError)):
|
||||
_read_sample(path)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -38,12 +38,20 @@ class FakeNemoModel:
|
||||
class FakeConcreteParakeetModel:
|
||||
restored_path: str | None = None
|
||||
restored_location: object | None = None
|
||||
restored_bytes: bytes | None = None
|
||||
replace_before_restore: tuple[Path, bytes] | None = None
|
||||
restored: FakeNemoModel | None = None
|
||||
|
||||
@classmethod
|
||||
def restore_from(cls, restore_path: str, map_location: object) -> FakeNemoModel:
|
||||
cls.restored_path = restore_path
|
||||
cls.restored_location = map_location
|
||||
if cls.replace_before_restore is not None:
|
||||
path, replacement = cls.replace_before_restore
|
||||
replacement_path = path.with_suffix(".replacement")
|
||||
replacement_path.write_bytes(replacement)
|
||||
replacement_path.replace(path)
|
||||
cls.restored_bytes = Path(restore_path).read_bytes()
|
||||
self = FakeNemoModel()
|
||||
cls.restored = self
|
||||
return self
|
||||
@@ -57,6 +65,13 @@ class FakeNumpy(ModuleType):
|
||||
|
||||
|
||||
class NemoParakeetModelTests(unittest.TestCase):
|
||||
def setUp(self) -> None:
|
||||
FakeConcreteParakeetModel.restored_path = None
|
||||
FakeConcreteParakeetModel.restored_location = None
|
||||
FakeConcreteParakeetModel.restored_bytes = None
|
||||
FakeConcreteParakeetModel.replace_before_restore = None
|
||||
FakeConcreteParakeetModel.restored = None
|
||||
|
||||
def test_load_is_lazy_offline_applies_coding_terms_and_transcribes_in_memory_audio(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
model_path = Path(directory) / "parakeet.nemo"
|
||||
@@ -78,7 +93,8 @@ class NemoParakeetModelTests(unittest.TestCase):
|
||||
|
||||
self.assertEqual(os.environ["HF_HUB_OFFLINE"], "1")
|
||||
self.assertEqual(os.environ["TRANSFORMERS_OFFLINE"], "1")
|
||||
self.assertEqual(FakeConcreteParakeetModel.restored_path, str(model_path))
|
||||
self.assertRegex(FakeConcreteParakeetModel.restored_path or "", r"^/proc/self/fd/\d+$")
|
||||
self.assertEqual(FakeConcreteParakeetModel.restored_bytes, b"model")
|
||||
self.assertEqual(FakeConcreteParakeetModel.restored_location, "device:cuda")
|
||||
self.assertTrue(FakeConcreteParakeetModel.restored.eval_called)
|
||||
self.assertEqual(FakeConcreteParakeetModel.restored.to_device, "cuda")
|
||||
@@ -107,6 +123,22 @@ class NemoParakeetModelTests(unittest.TestCase):
|
||||
with self.assertRaisesRegex(ValueError, "128 UTF-8 bytes"):
|
||||
load_coding_terms(terms_path)
|
||||
|
||||
def test_load_uses_the_verified_inode_if_the_configured_path_is_replaced(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
model_path = Path(directory) / "parakeet.nemo"
|
||||
model_path.write_bytes(b"verified model")
|
||||
FakeConcreteParakeetModel.replace_before_restore = (model_path, b"replacement")
|
||||
with patch.dict(sys.modules, self.fake_modules(), clear=False):
|
||||
adapter = NemoParakeetModel(
|
||||
model_path,
|
||||
expected_sha256=hashlib.sha256(b"verified model").hexdigest(),
|
||||
)
|
||||
|
||||
adapter.load()
|
||||
|
||||
self.assertEqual(model_path.read_bytes(), b"replacement")
|
||||
self.assertEqual(FakeConcreteParakeetModel.restored_bytes, b"verified model")
|
||||
|
||||
def test_refuses_a_model_that_no_longer_matches_the_pinned_hash(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
model_path = Path(directory) / "parakeet.nemo"
|
||||
|
||||
@@ -0,0 +1,20 @@
|
||||
import unittest
|
||||
|
||||
from pi_dictate_service.protocol import MAX_FRAME_BYTES, FrameDecoder
|
||||
|
||||
|
||||
class FrameDecoderRegressionTests(unittest.TestCase):
|
||||
def test_accepts_pipelined_bounded_frames_larger_than_aggregate_limit(self) -> None:
|
||||
decoder = FrameDecoder()
|
||||
frame = b'{"version":2,"type":"ping","requestId":"request"}\n'
|
||||
payload = frame * ((MAX_FRAME_BYTES // len(frame)) + 1)
|
||||
|
||||
self.assertGreater(len(payload), MAX_FRAME_BYTES + 1)
|
||||
decoded = decoder.feed(payload)
|
||||
|
||||
self.assertEqual(len(decoded), payload.count(b"\n"))
|
||||
self.assertEqual(decoded[0]["requestId"], "request")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -36,6 +36,18 @@ class TrackingStream(io.BytesIO):
|
||||
return value
|
||||
|
||||
|
||||
class BlockingTrackingStream(TrackingStream):
|
||||
def __init__(self, value: bytes, events: list[str], name: str) -> None:
|
||||
super().__init__(value, events, name)
|
||||
self.allow_eof = threading.Event()
|
||||
|
||||
def read(self, size: int = -1) -> bytes:
|
||||
value = super().read(size)
|
||||
if not value:
|
||||
self.allow_eof.wait(1)
|
||||
return value
|
||||
|
||||
|
||||
class FakeProcess:
|
||||
def __init__(
|
||||
self,
|
||||
@@ -45,6 +57,7 @@ class FakeProcess:
|
||||
returncode: int | None = None,
|
||||
timeout: bool = False,
|
||||
terminate_returncode: int = -15,
|
||||
kill_stalls: bool = False,
|
||||
) -> None:
|
||||
self.pid = 1234
|
||||
self.stdout = stdout
|
||||
@@ -52,6 +65,7 @@ class FakeProcess:
|
||||
self.returncode = returncode
|
||||
self._timeout = timeout
|
||||
self._terminate_returncode = terminate_returncode
|
||||
self._kill_stalls = kill_stalls
|
||||
self.events = stdout._events
|
||||
|
||||
def poll(self) -> int | None:
|
||||
@@ -64,6 +78,7 @@ class FakeProcess:
|
||||
|
||||
def kill(self) -> None:
|
||||
self.events.append("kill")
|
||||
if not self._kill_stalls:
|
||||
self.returncode = -9
|
||||
self._timeout = False
|
||||
|
||||
@@ -86,6 +101,22 @@ class ProcessFactory:
|
||||
return self.process
|
||||
|
||||
|
||||
class StreamingSink:
|
||||
def __init__(self) -> None:
|
||||
self.chunks: list[bytes] = []
|
||||
self.finalized = False
|
||||
self.cancelled = False
|
||||
|
||||
def enqueue_pcm(self, chunk: bytes) -> None:
|
||||
self.chunks.append(chunk)
|
||||
|
||||
def close_input(self) -> None:
|
||||
self.finalized = True
|
||||
|
||||
def cancel(self) -> None:
|
||||
self.cancelled = True
|
||||
|
||||
|
||||
class Clock:
|
||||
def __init__(self) -> None:
|
||||
self.now = 0.0
|
||||
@@ -95,6 +126,16 @@ class Clock:
|
||||
|
||||
|
||||
class PwRecorderTests(unittest.TestCase):
|
||||
def test_spawn_failure_is_already_cleanup_complete(self) -> None:
|
||||
def fail_spawn(*_args: object, **_kwargs: object) -> FakeProcess:
|
||||
raise OSError("spawn failed")
|
||||
|
||||
recorder = PwRecorder(process_factory=fail_spawn)
|
||||
|
||||
with self.assertRaisesRegex(RecorderError, "could not start"):
|
||||
recorder.start()
|
||||
self.assertTrue(recorder.cleanup_complete)
|
||||
|
||||
def make_recorder(
|
||||
self,
|
||||
pcm: bytes = b"\x00\x00\x01\x00",
|
||||
@@ -107,14 +148,23 @@ class PwRecorderTests(unittest.TestCase):
|
||||
clock: Clock | None = None,
|
||||
terminate_returncode: int = -15,
|
||||
stdout_read_size: int | None = None,
|
||||
kill_stalls: bool = False,
|
||||
block_stdout_at_eof: bool = False,
|
||||
pcm_sink: StreamingSink | None = None,
|
||||
) -> tuple[PwRecorder, FakeProcess, ProcessFactory, list[str]]:
|
||||
events: list[str] = []
|
||||
stdout = (
|
||||
BlockingTrackingStream(pcm, events, "stdout")
|
||||
if block_stdout_at_eof
|
||||
else TrackingStream(pcm, events, "stdout", stdout_read_size)
|
||||
)
|
||||
process = FakeProcess(
|
||||
TrackingStream(pcm, events, "stdout", stdout_read_size),
|
||||
stdout,
|
||||
TrackingStream(stderr, events, "stderr"),
|
||||
returncode=returncode,
|
||||
timeout=timeout,
|
||||
terminate_returncode=terminate_returncode,
|
||||
kill_stalls=kill_stalls,
|
||||
)
|
||||
factory = ProcessFactory(process)
|
||||
def signal_group(pid: int, signal_number: int) -> None:
|
||||
@@ -133,9 +183,44 @@ class PwRecorderTests(unittest.TestCase):
|
||||
stop_timeout=0.01,
|
||||
clock=clock or Clock(),
|
||||
process_group_signal=signal_group,
|
||||
pcm_sink=pcm_sink,
|
||||
)
|
||||
return recorder, process, factory, events
|
||||
|
||||
def test_successful_stop_finalizes_streaming_sink(self) -> None:
|
||||
sink = StreamingSink()
|
||||
recorder, process, _factory, _events = self.make_recorder(pcm_sink=sink)
|
||||
recorder.start()
|
||||
self.assertTrue(process.stdout.eof.wait(1))
|
||||
|
||||
recorder.stop()
|
||||
|
||||
self.assertTrue(sink.finalized)
|
||||
self.assertFalse(sink.cancelled)
|
||||
|
||||
def test_cancel_aborts_streaming_sink_without_finalizing(self) -> None:
|
||||
sink = StreamingSink()
|
||||
recorder, process, _factory, _events = self.make_recorder(pcm_sink=sink)
|
||||
recorder.start()
|
||||
self.assertTrue(process.stdout.eof.wait(1))
|
||||
|
||||
recorder.cancel()
|
||||
|
||||
self.assertTrue(sink.cancelled)
|
||||
self.assertFalse(sink.finalized)
|
||||
|
||||
def test_unexpected_exit_aborts_streaming_sink_without_finalizing(self) -> None:
|
||||
sink = StreamingSink()
|
||||
recorder, process, _factory, _events = self.make_recorder(returncode=3, pcm_sink=sink)
|
||||
recorder.start()
|
||||
self.assertTrue(process.stdout.eof.wait(1))
|
||||
|
||||
error = recorder.poll_unexpected()
|
||||
|
||||
self.assertIsNotNone(error)
|
||||
self.assertTrue(sink.cancelled)
|
||||
self.assertFalse(sink.finalized)
|
||||
|
||||
def test_spawns_exact_pw_record_command_and_drains_bounded_streams(self) -> None:
|
||||
recorder, process, factory, events = self.make_recorder(
|
||||
pcm=b"\x00\x00\x01\x00", stderr=b"0123456789", stderr_limit=8
|
||||
@@ -230,6 +315,39 @@ class PwRecorderTests(unittest.TestCase):
|
||||
with self.assertRaises(RecorderError):
|
||||
noisy.stop()
|
||||
|
||||
def test_shutdown_reports_a_bounded_error_when_sigkill_does_not_reap(self) -> None:
|
||||
recorder, process, _factory, _events = self.make_recorder(
|
||||
timeout=True,
|
||||
kill_stalls=True,
|
||||
)
|
||||
recorder.start()
|
||||
|
||||
with self.assertRaisesRegex(RecorderError, "after SIGKILL") as error:
|
||||
recorder.stop()
|
||||
|
||||
self.assertEqual(error.exception.code, "recorder_timeout")
|
||||
self.assertIn("kill", process.events)
|
||||
|
||||
def test_shutdown_retries_process_and_drainer_cleanup_after_double_timeout(self) -> None:
|
||||
recorder, process, _factory, _events = self.make_recorder(
|
||||
timeout=True,
|
||||
kill_stalls=True,
|
||||
block_stdout_at_eof=True,
|
||||
)
|
||||
recorder.start()
|
||||
self.assertTrue(process.stdout.eof.wait(1))
|
||||
|
||||
with self.assertRaises(RecorderError):
|
||||
recorder.stop()
|
||||
|
||||
self.assertFalse(recorder.cleanup_complete)
|
||||
process._kill_stalls = False
|
||||
process.stdout.allow_eof.set()
|
||||
|
||||
recorder.cancel()
|
||||
|
||||
self.assertTrue(recorder.cleanup_complete)
|
||||
|
||||
def test_shutdown_timeout_escalates_to_kill_and_fails(self) -> None:
|
||||
recorder, _process, _factory, events = self.make_recorder(timeout=True)
|
||||
recorder.start()
|
||||
@@ -240,6 +358,22 @@ class PwRecorderTests(unittest.TestCase):
|
||||
self.assertEqual(failed.exception.code, "recorder_timeout")
|
||||
self.assertLess(events.index("terminate"), events.index("kill"))
|
||||
|
||||
def test_stop_rejects_an_exit_observed_after_the_last_listening_poll(self) -> None:
|
||||
recorder, process, _factory, _events = self.make_recorder()
|
||||
recorder.start()
|
||||
self.assertTrue(process.stdout.eof.wait(1))
|
||||
|
||||
# The process was alive during the listening poll, then exited cleanly
|
||||
# before stop could signal it. A partial buffer is not a valid capture.
|
||||
self.assertIsNone(recorder.poll_unexpected())
|
||||
process.returncode = 0
|
||||
|
||||
with self.assertRaises(RecorderError) as failed:
|
||||
recorder.stop()
|
||||
|
||||
self.assertEqual(failed.exception.code, "recorder_exit")
|
||||
self.assertEqual(recorder.buffered_bytes, 0)
|
||||
|
||||
def test_classifies_unexpected_exit_code_and_signal(self) -> None:
|
||||
recorder, process, _factory, _events = self.make_recorder(
|
||||
stderr=b"pipewire failed", returncode=7
|
||||
@@ -288,6 +422,25 @@ class PwRecorderTests(unittest.TestCase):
|
||||
self.assertEqual(failed.exception.code, "duration_limit")
|
||||
self.assertEqual(recorder.buffered_bytes, 0)
|
||||
|
||||
def test_duration_limit_reports_a_bounded_error_when_sigkill_does_not_reap(self) -> None:
|
||||
clock = Clock()
|
||||
recorder, process, _factory, _events = self.make_recorder(
|
||||
pcm=b"\x00\x00" * 3,
|
||||
max_bytes=4,
|
||||
timeout=True,
|
||||
kill_stalls=True,
|
||||
clock=clock,
|
||||
)
|
||||
recorder.start()
|
||||
self.assertTrue(process.stdout.eof.wait(1))
|
||||
clock.now = 1
|
||||
|
||||
failure = recorder.poll_unexpected()
|
||||
|
||||
self.assertIsNotNone(failure)
|
||||
self.assertEqual(failure.code, "recorder_timeout")
|
||||
self.assertIn("kill", process.events)
|
||||
|
||||
def test_duration_limit_kills_a_recorder_that_ignores_sigterm(self) -> None:
|
||||
clock = Clock()
|
||||
recorder, process, _factory, events = self.make_recorder(
|
||||
|
||||
@@ -1,11 +1,22 @@
|
||||
from concurrent.futures import Future
|
||||
from collections.abc import Sequence
|
||||
from dataclasses import dataclass
|
||||
import errno
|
||||
import selectors
|
||||
import subprocess
|
||||
import sys
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
from pi_dictate_service.protocol import MAX_TRANSCRIPT_BYTES, PROTOCOL_VERSION, parse_request
|
||||
from pi_dictate_service.protocol import (
|
||||
MAX_FRAME_BYTES,
|
||||
MAX_TRANSCRIPT_BYTES,
|
||||
PROTOCOL_VERSION,
|
||||
encode_frame,
|
||||
parse_request,
|
||||
)
|
||||
from pi_dictate_service.recorder import Capture, RecorderError
|
||||
from pi_dictate_service.server import DictateServiceCore, Outbound
|
||||
from pi_dictate_service.server import DictateServiceCore, Outbound, serve
|
||||
|
||||
|
||||
class Clock:
|
||||
@@ -93,6 +104,154 @@ class FakeRecorder:
|
||||
return failure
|
||||
|
||||
|
||||
class SpawnFailRecorder(FakeRecorder):
|
||||
cleanup_complete = True
|
||||
|
||||
def start(self) -> None:
|
||||
raise RecorderError("recorder_spawn", "could not start pw-record")
|
||||
|
||||
|
||||
class SpawnFailRecorderFactory:
|
||||
def __call__(self) -> SpawnFailRecorder:
|
||||
return SpawnFailRecorder()
|
||||
|
||||
|
||||
class DelayedCleanupRecorder(FakeRecorder):
|
||||
def __init__(self) -> None:
|
||||
super().__init__()
|
||||
self.cleanup_complete = False
|
||||
self.cleanup_attempts = 0
|
||||
self.cleanup_started = False
|
||||
|
||||
def stop(self) -> Capture:
|
||||
self.cleanup_started = True
|
||||
raise RecorderError("recorder_timeout", "pw-record cleanup did not complete")
|
||||
|
||||
def poll_unexpected(self) -> RecorderError | None:
|
||||
if not self.cleanup_started:
|
||||
return None
|
||||
self.cleanup_attempts += 1
|
||||
if self.cleanup_attempts >= 3:
|
||||
self.cleanup_complete = True
|
||||
return None
|
||||
|
||||
|
||||
class DelayedCleanupRecorderFactory:
|
||||
def __init__(self) -> None:
|
||||
self.instance = DelayedCleanupRecorder()
|
||||
|
||||
def __call__(self) -> DelayedCleanupRecorder:
|
||||
return self.instance
|
||||
|
||||
|
||||
class DelayedCloseTurn:
|
||||
sample_rate_hz = 16_000
|
||||
retain_pcm = True
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.close_calls = 0
|
||||
|
||||
@property
|
||||
def cleanup_complete(self) -> bool:
|
||||
return self.close_calls >= 3
|
||||
|
||||
def enqueue_pcm(self, chunk: bytes) -> None:
|
||||
return None
|
||||
|
||||
def finish(self, capture: Capture | None = None) -> str:
|
||||
return "cloud result"
|
||||
|
||||
def poll_error(self) -> None:
|
||||
return None
|
||||
|
||||
def cancel(self) -> None:
|
||||
return None
|
||||
|
||||
def close(self) -> None:
|
||||
self.close_calls += 1
|
||||
|
||||
|
||||
class StreamingTurn:
|
||||
sample_rate_hz = 24_000
|
||||
retain_pcm = False
|
||||
|
||||
def __init__(self, events: list[str]) -> None:
|
||||
self.events = events
|
||||
self.closed = False
|
||||
|
||||
@property
|
||||
def cleanup_complete(self) -> bool:
|
||||
return self.closed
|
||||
|
||||
def enqueue_pcm(self, chunk: bytes) -> None:
|
||||
self.events.append(f"audio-{chunk.decode('ascii')}")
|
||||
|
||||
def finish(self, capture: Capture | None = None) -> str:
|
||||
self.events.append("finish")
|
||||
self.assert_capture = capture
|
||||
return "cloud result"
|
||||
|
||||
def poll_error(self) -> None:
|
||||
return None
|
||||
|
||||
def cancel(self) -> None:
|
||||
self.events.append("cancel")
|
||||
|
||||
def close(self) -> None:
|
||||
self.closed = True
|
||||
|
||||
|
||||
class StreamingRecorder(FakeRecorder):
|
||||
def __init__(self, sink: StreamingTurn, events: list[str]) -> None:
|
||||
super().__init__()
|
||||
self.sink = sink
|
||||
self.events = events
|
||||
self.capture = Capture(b"", -15, 15, b"")
|
||||
|
||||
def start(self) -> None:
|
||||
super().start()
|
||||
self.sink.enqueue_pcm(b"a")
|
||||
self.sink.enqueue_pcm(b"b")
|
||||
|
||||
def stop(self) -> Capture:
|
||||
self.events.append("recorder-stop")
|
||||
return super().stop()
|
||||
|
||||
|
||||
class StreamingRecorderFactory:
|
||||
def __init__(self, events: list[str]) -> None:
|
||||
self.events = events
|
||||
self.sample_rate_hz: int | None = None
|
||||
self.instance: StreamingRecorder | None = None
|
||||
|
||||
def __call__(
|
||||
self,
|
||||
*,
|
||||
sample_rate_hz: int,
|
||||
pcm_sink: StreamingTurn,
|
||||
retain_pcm: bool,
|
||||
) -> StreamingRecorder:
|
||||
self.sample_rate_hz = sample_rate_hz
|
||||
if retain_pcm:
|
||||
raise AssertionError("cloud capture must not retain PCM")
|
||||
self.instance = StreamingRecorder(pcm_sink, self.events)
|
||||
return self.instance
|
||||
|
||||
|
||||
class FakeBackend:
|
||||
sample_rate_hz = 16_000
|
||||
descriptor = type("Descriptor", (), {"backend": "fake", "model": "fake-model"})()
|
||||
|
||||
def __init__(self, turn: DelayedCloseTurn | StreamingTurn) -> None:
|
||||
self.turn = turn
|
||||
|
||||
def start_turn(self) -> DelayedCloseTurn:
|
||||
return self.turn
|
||||
|
||||
def cancel_prepare(self) -> None:
|
||||
return None
|
||||
|
||||
|
||||
class RecorderFactory:
|
||||
def __init__(self) -> None:
|
||||
self.instances: list[FakeRecorder] = []
|
||||
@@ -271,6 +430,121 @@ class StopAndInferenceTests(CoreTestCase):
|
||||
self.assertEqual(completed[0]["transcript"], "dictated text")
|
||||
self.assertEqual(completed[1]["state"], "idle")
|
||||
|
||||
def test_realtime_backend_streams_ordered_pcm_and_finalizes_after_recorder_stop(self) -> None:
|
||||
events: list[str] = []
|
||||
turn = StreamingTurn(events)
|
||||
recorders = StreamingRecorderFactory(events)
|
||||
core = DictateServiceCore(
|
||||
backend=FakeBackend(turn),
|
||||
recorder_factory=recorders,
|
||||
executor=self.executor,
|
||||
clock=self.clock,
|
||||
idle_seconds=900,
|
||||
)
|
||||
core.accepted("a")
|
||||
core.handle_request("a", parse_request({"version": 2, "type": "hello", "requestId": "hello"}))
|
||||
core.drain_events()
|
||||
core.handle_request("a", parse_request({
|
||||
"version": 2,
|
||||
"type": "start",
|
||||
"requestId": "start",
|
||||
"generationId": "g1",
|
||||
}))
|
||||
self.executor.run_next()
|
||||
core.poll()
|
||||
self.assertEqual(recorders.sample_rate_hz, 24_000)
|
||||
self.assertEqual(events, ["audio-a", "audio-b"])
|
||||
core.drain_events()
|
||||
|
||||
core.handle_request("a", parse_request({
|
||||
"version": 2,
|
||||
"type": "stop",
|
||||
"requestId": "stop",
|
||||
"generationId": "g1",
|
||||
}))
|
||||
self.executor.run_next()
|
||||
core.poll()
|
||||
self.executor.run_next()
|
||||
core.poll()
|
||||
|
||||
self.assertEqual(events, ["audio-a", "audio-b", "recorder-stop", "finish"])
|
||||
result = [event.frame for event in core.drain_events() if event.frame["type"] == "result"]
|
||||
self.assertEqual(result[0]["transcript"], "cloud result")
|
||||
self.assertIsNone(core.active_generation_id)
|
||||
|
||||
def test_cancel_after_prepare_completion_tracks_turn_cleanup_before_release(self) -> None:
|
||||
events: list[str] = []
|
||||
turn = StreamingTurn(events)
|
||||
core = DictateServiceCore(
|
||||
backend=FakeBackend(turn),
|
||||
recorder_factory=StreamingRecorderFactory(events),
|
||||
executor=self.executor,
|
||||
clock=self.clock,
|
||||
idle_seconds=900,
|
||||
)
|
||||
core.accepted("a")
|
||||
core.handle_request("a", parse_request({"version": 2, "type": "hello", "requestId": "hello"}))
|
||||
core.drain_events()
|
||||
core.handle_request("a", parse_request({
|
||||
"version": 2,
|
||||
"type": "start",
|
||||
"requestId": "start",
|
||||
"generationId": "g1",
|
||||
}))
|
||||
self.executor.run_next()
|
||||
# The future has completed, but the socket loop has not yet applied it.
|
||||
core.handle_request("a", parse_request({
|
||||
"version": 2,
|
||||
"type": "cancel",
|
||||
"requestId": "cancel",
|
||||
"generationId": "g1",
|
||||
}))
|
||||
|
||||
core.poll()
|
||||
|
||||
self.assertIn("cancel", events)
|
||||
self.assertTrue(turn.cleanup_complete)
|
||||
self.assertIsNone(core.active_generation_id)
|
||||
|
||||
def test_completed_turn_retries_bounded_cleanup_before_releasing_generation(self) -> None:
|
||||
turn = DelayedCloseTurn()
|
||||
core = DictateServiceCore(
|
||||
backend=FakeBackend(turn),
|
||||
recorder_factory=self.recorders,
|
||||
executor=self.executor,
|
||||
clock=self.clock,
|
||||
idle_seconds=900,
|
||||
)
|
||||
core.accepted("a")
|
||||
core.handle_request("a", parse_request({"version": 2, "type": "hello", "requestId": "hello"}))
|
||||
core.drain_events()
|
||||
core.handle_request("a", parse_request({
|
||||
"version": 2,
|
||||
"type": "start",
|
||||
"requestId": "start",
|
||||
"generationId": "g1",
|
||||
}))
|
||||
self.executor.run_next()
|
||||
core.poll()
|
||||
core.drain_events()
|
||||
core.handle_request("a", parse_request({
|
||||
"version": 2,
|
||||
"type": "stop",
|
||||
"requestId": "stop",
|
||||
"generationId": "g1",
|
||||
}))
|
||||
self.executor.run_next()
|
||||
core.poll()
|
||||
self.executor.run_next()
|
||||
|
||||
core.poll()
|
||||
self.assertEqual(core.active_generation_id, "g1")
|
||||
self.assertEqual(turn.close_calls, 2)
|
||||
|
||||
core.poll()
|
||||
self.assertIsNone(core.active_generation_id)
|
||||
self.assertEqual(turn.close_calls, 3)
|
||||
|
||||
def test_empty_transcript_is_a_result(self) -> None:
|
||||
self.model.transcript = ""
|
||||
self.load_to_listening()
|
||||
@@ -321,6 +595,8 @@ class StopAndInferenceTests(CoreTestCase):
|
||||
recorder = self.load_to_listening()
|
||||
recorder.unexpected = RecorderError("recorder_exit", "pw-record exited with code 3")
|
||||
|
||||
self.core.poll()
|
||||
self.executor.run_next()
|
||||
self.core.poll()
|
||||
failed = self.frames(self.core.drain_events(), "a")
|
||||
|
||||
@@ -332,12 +608,286 @@ class StopAndInferenceTests(CoreTestCase):
|
||||
recorder.unexpected = RecorderError("recorder_exit", "pw-record exited with code 0")
|
||||
|
||||
failed = self.request("a", "stop", "stop", "g1")
|
||||
self.executor.run_next()
|
||||
self.core.poll()
|
||||
failed.extend(self.frames(self.core.drain_events(), "a"))
|
||||
|
||||
self.assertEqual(failed[0]["code"], "recorder_exit")
|
||||
self.assertEqual(failed[-1]["state"], "idle")
|
||||
self.assertEqual(self.executor.pending, [])
|
||||
|
||||
|
||||
def test_recorder_spawn_failure_releases_generation_without_cleanup_loop(self) -> None:
|
||||
core = DictateServiceCore(
|
||||
model=self.model,
|
||||
recorder_factory=SpawnFailRecorderFactory(),
|
||||
executor=self.executor,
|
||||
clock=self.clock,
|
||||
idle_seconds=900,
|
||||
)
|
||||
core.accepted("a")
|
||||
core.handle_request("a", parse_request({
|
||||
"version": 2, "type": "hello", "requestId": "hello"
|
||||
}))
|
||||
core.drain_events()
|
||||
core.handle_request("a", parse_request({
|
||||
"version": 2,
|
||||
"type": "start",
|
||||
"requestId": "start",
|
||||
"generationId": "g1",
|
||||
}))
|
||||
self.executor.run_next()
|
||||
|
||||
core.poll()
|
||||
|
||||
events = self.frames(core.drain_events(), "a")
|
||||
self.assertEqual(events[-2]["code"], "recorder_spawn")
|
||||
self.assertEqual(events[-1]["state"], "idle")
|
||||
self.assertIsNone(core.active_generation_id)
|
||||
self.assertEqual(self.executor.pending, [])
|
||||
|
||||
def test_unreaped_recorder_keeps_generation_busy_until_cleanup_completes(self) -> None:
|
||||
recorders = DelayedCleanupRecorderFactory()
|
||||
core = DictateServiceCore(
|
||||
model=self.model,
|
||||
recorder_factory=recorders,
|
||||
executor=self.executor,
|
||||
clock=self.clock,
|
||||
idle_seconds=900,
|
||||
)
|
||||
core.accepted("a")
|
||||
core.handle_request("a", parse_request({
|
||||
"version": 2, "type": "hello", "requestId": "hello"
|
||||
}))
|
||||
core.drain_events()
|
||||
core.handle_request("a", parse_request({
|
||||
"version": 2,
|
||||
"type": "start",
|
||||
"requestId": "start",
|
||||
"generationId": "g1",
|
||||
}))
|
||||
self.executor.run_next()
|
||||
core.poll()
|
||||
core.drain_events()
|
||||
core.handle_request("a", parse_request({
|
||||
"version": 2,
|
||||
"type": "stop",
|
||||
"requestId": "stop",
|
||||
"generationId": "g1",
|
||||
}))
|
||||
self.executor.run_next()
|
||||
|
||||
core.poll()
|
||||
self.assertEqual(core.active_generation_id, "g1")
|
||||
# First reap the cancellation worker scheduled by the terminal stop.
|
||||
self.executor.run_next()
|
||||
core.poll()
|
||||
for attempt in range(1, 4):
|
||||
self.executor.run_next()
|
||||
core.poll()
|
||||
if attempt < 3:
|
||||
self.assertEqual(core.active_generation_id, "g1")
|
||||
self.assertIsNone(core.active_generation_id)
|
||||
self.assertEqual(recorders.instance.cleanup_attempts, 3)
|
||||
|
||||
|
||||
class SocketLoopRegressionTests(unittest.TestCase):
|
||||
def test_output_overflow_does_not_use_the_closed_socket_from_same_selector_event(self) -> None:
|
||||
frame = {"padding": "x" * (MAX_FRAME_BYTES - len(encode_frame({"padding": ""})))}
|
||||
self.assertEqual(len(encode_frame(frame)), MAX_FRAME_BYTES)
|
||||
client = _FakeClientSocket()
|
||||
listener = _FakeListener(client)
|
||||
selector = _FakeSelector(client)
|
||||
core = _FakeLoopCore(
|
||||
Outbound("client-1", frame),
|
||||
)
|
||||
|
||||
with patch("pi_dictate_service.server.selectors.DefaultSelector", return_value=selector):
|
||||
serve(listener, core)
|
||||
|
||||
self.assertEqual(client.send_calls, 0)
|
||||
self.assertTrue(client.closed)
|
||||
self.assertTrue(listener.closed)
|
||||
|
||||
def test_close_does_not_wait_for_owned_executor_workers(self) -> None:
|
||||
executor = _ShutdownRecordingExecutor()
|
||||
with patch("pi_dictate_service.server._DaemonExecutor", return_value=executor):
|
||||
core = DictateServiceCore(model=FakeModel())
|
||||
core.accepted("a")
|
||||
core.handle_request(
|
||||
"a", parse_request({"version": PROTOCOL_VERSION, "type": "hello", "requestId": "h"})
|
||||
)
|
||||
core.drain_events()
|
||||
core.handle_request(
|
||||
"a",
|
||||
parse_request(
|
||||
{
|
||||
"version": PROTOCOL_VERSION,
|
||||
"type": "start",
|
||||
"requestId": "s",
|
||||
"generationId": "g",
|
||||
}
|
||||
),
|
||||
)
|
||||
core.close()
|
||||
|
||||
self.assertEqual(executor.shutdown_args, (False, True))
|
||||
|
||||
def test_real_owned_executor_does_not_block_interpreter_exit(self) -> None:
|
||||
program = """
|
||||
import threading
|
||||
import time
|
||||
from pi_dictate_service.protocol import parse_request
|
||||
from pi_dictate_service.server import DictateServiceCore
|
||||
|
||||
class StuckModel:
|
||||
loaded = False
|
||||
def load(self):
|
||||
threading.Event().wait()
|
||||
def transcribe(self, _samples):
|
||||
return ''
|
||||
|
||||
core = DictateServiceCore(model=StuckModel())
|
||||
core.accepted('a')
|
||||
core.handle_request('a', parse_request({'version': 2, 'type': 'hello', 'requestId': 'h'}))
|
||||
core.drain_events()
|
||||
core.handle_request('a', parse_request({
|
||||
'version': 2, 'type': 'start', 'requestId': 's', 'generationId': 'g'
|
||||
}))
|
||||
time.sleep(0.05)
|
||||
core.close()
|
||||
print('closed')
|
||||
"""
|
||||
|
||||
result = subprocess.run(
|
||||
[sys.executable, "-c", program],
|
||||
text=True,
|
||||
capture_output=True,
|
||||
timeout=2,
|
||||
check=False,
|
||||
)
|
||||
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
self.assertEqual(result.stdout.strip(), "closed")
|
||||
|
||||
|
||||
@dataclass
|
||||
class _FakeSelectorKey:
|
||||
data: str | None
|
||||
|
||||
|
||||
class _FakeSelector:
|
||||
def __init__(self, client: "_FakeClientSocket") -> None:
|
||||
self._events = [
|
||||
[(_FakeSelectorKey(None), selectors.EVENT_READ)],
|
||||
[(_FakeSelectorKey("client-1"), selectors.EVENT_READ | selectors.EVENT_WRITE)],
|
||||
]
|
||||
self.client = client
|
||||
self.unregistered: list[object] = []
|
||||
self.closed = False
|
||||
|
||||
def register(self, fileobj: object, _events: int, _data: object = None) -> None:
|
||||
return None
|
||||
|
||||
def unregister(self, fileobj: object) -> None:
|
||||
self.unregistered.append(fileobj)
|
||||
|
||||
def modify(self, fileobj: object, _events: int, _data: object = None) -> None:
|
||||
if fileobj is self.client and self.client.closed:
|
||||
raise OSError(errno.EBADF, "bad file descriptor")
|
||||
|
||||
def select(self, timeout: float | None = None) -> list[tuple[_FakeSelectorKey, int]]:
|
||||
return self._events.pop(0)
|
||||
|
||||
def close(self) -> None:
|
||||
self.closed = True
|
||||
|
||||
|
||||
class _FakeClientSocket:
|
||||
def __init__(self) -> None:
|
||||
self.closed = False
|
||||
self.send_calls = 0
|
||||
self._read = True
|
||||
|
||||
def setblocking(self, _value: bool) -> None:
|
||||
return None
|
||||
|
||||
def recv(self, _size: int) -> bytes:
|
||||
if self._read:
|
||||
self._read = False
|
||||
return b"{broken}\n"
|
||||
return b""
|
||||
|
||||
def send(self, _data: bytearray) -> int:
|
||||
self.send_calls += 1
|
||||
raise OSError(errno.EBADF, "bad file descriptor")
|
||||
|
||||
def close(self) -> None:
|
||||
self.closed = True
|
||||
|
||||
|
||||
class _FakeListener:
|
||||
def __init__(self, client: _FakeClientSocket) -> None:
|
||||
self.client = client
|
||||
self.closed = False
|
||||
self._accepted = False
|
||||
|
||||
def setblocking(self, _value: bool) -> None:
|
||||
return None
|
||||
|
||||
def accept(self) -> tuple[_FakeClientSocket, object]:
|
||||
if self._accepted:
|
||||
raise BlockingIOError
|
||||
self._accepted = True
|
||||
return self.client, None
|
||||
|
||||
def close(self) -> None:
|
||||
self.closed = True
|
||||
|
||||
|
||||
class _FakeLoopCore:
|
||||
def __init__(self, outbound: Outbound) -> None:
|
||||
self.outbound = outbound
|
||||
self.accepted_client: str | None = None
|
||||
self._outbound_sent = False
|
||||
self._exit_checks = 0
|
||||
|
||||
def should_exit(self) -> bool:
|
||||
self._exit_checks += 1
|
||||
return self._exit_checks >= 3
|
||||
|
||||
def poll(self) -> None:
|
||||
return None
|
||||
|
||||
def drain_events(self) -> list[Outbound]:
|
||||
if self.accepted_client is not None and not self._outbound_sent:
|
||||
self._outbound_sent = True
|
||||
return [self.outbound]
|
||||
return []
|
||||
|
||||
def accepted(self, client_id: str) -> None:
|
||||
self.accepted_client = client_id
|
||||
|
||||
def disconnect(self, _client_id: str) -> None:
|
||||
return None
|
||||
|
||||
def close(self) -> None:
|
||||
return None
|
||||
|
||||
|
||||
class _ShutdownRecordingExecutor:
|
||||
def __init__(self) -> None:
|
||||
self.shutdown_args: tuple[bool, bool] | None = None
|
||||
|
||||
def submit(self, _function: object, *_args: object) -> Future[object]:
|
||||
return Future()
|
||||
|
||||
def shutdown(self, *, wait: bool = True, cancel_futures: bool = False) -> None:
|
||||
self.shutdown_args = (wait, cancel_futures)
|
||||
if wait:
|
||||
raise AssertionError("shutdown waited for a hung worker")
|
||||
|
||||
|
||||
class IdleTests(CoreTestCase):
|
||||
def test_only_valid_requests_reset_idle_and_workers_suspend_then_restart_timer(self) -> None:
|
||||
self.core.accepted("a")
|
||||
|
||||
@@ -0,0 +1,263 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import tempfile
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
from pi_dictate_service.config import ConfigError
|
||||
from pi_dictate_service.startup import validate_startup
|
||||
|
||||
|
||||
REPOSITORY_ROOT = Path(__file__).resolve().parents[3]
|
||||
SCRIPT = REPOSITORY_ROOT / "services/dictate/install.sh"
|
||||
|
||||
|
||||
class SystemdProfileTests(unittest.TestCase):
|
||||
def run_install(self, home: Path, fake_bin: Path, *arguments: str) -> subprocess.CompletedProcess[str]:
|
||||
environment = {
|
||||
**os.environ,
|
||||
"HOME": str(home),
|
||||
"XDG_CONFIG_HOME": str(home / ".config"),
|
||||
"PATH": f"{fake_bin}:{os.environ['PATH']}",
|
||||
}
|
||||
return subprocess.run(
|
||||
[str(SCRIPT), *arguments],
|
||||
env=environment,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
|
||||
def make_fake_bin(self, root: Path) -> Path:
|
||||
fake_bin = root / "bin"
|
||||
fake_bin.mkdir()
|
||||
uv = fake_bin / "uv"
|
||||
# Match uv sync's contract so transactional environment swapping is
|
||||
# exercised without installing dependencies.
|
||||
uv.write_text(
|
||||
"#!/bin/sh\nmkdir -p \"$UV_PROJECT_ENVIRONMENT\"\nexit 0\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
uv.chmod(0o755)
|
||||
systemctl = fake_bin / "systemctl"
|
||||
systemctl.write_text("#!/bin/sh\nexit 0\n", encoding="utf-8")
|
||||
systemctl.chmod(0o755)
|
||||
return fake_bin
|
||||
|
||||
def test_generated_cloud_dropin_replaces_offline_network_restrictions(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
home = root / "home"
|
||||
home.mkdir()
|
||||
fake_bin = self.make_fake_bin(root)
|
||||
key = root / "key"
|
||||
key.write_text("test-key", encoding="utf-8")
|
||||
|
||||
result = self.run_install(
|
||||
home, fake_bin, "install", "--backend", "openai", "--api-key-file", str(key)
|
||||
)
|
||||
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
dropin = (
|
||||
home / ".config/systemd/user/pi-dictate.service.d/"
|
||||
"10-pi-dictate-network.conf"
|
||||
).read_text(encoding="utf-8")
|
||||
self.assertIn("Environment=PI_DICTATE_NETWORK_MODE=cloud", dropin)
|
||||
self.assertIn(
|
||||
"RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6", dropin
|
||||
)
|
||||
self.assertIn("IPAddressDeny=", dropin)
|
||||
self.assertNotIn("IPAddressDeny=any", dropin)
|
||||
self.assertNotIn("HF_HUB_OFFLINE=1", dropin)
|
||||
self.assertIn(
|
||||
f'BindReadOnlyPaths="{home}/.config/pi-dictate/credentials/openai-api-key"',
|
||||
dropin,
|
||||
)
|
||||
self.assertIn(
|
||||
f'BindReadOnlyPaths="{home}/.local/share/pi-dictate/config/coding-terms.txt"',
|
||||
dropin,
|
||||
)
|
||||
|
||||
def test_check_rejects_tampered_network_dropin(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
home = root / "home"
|
||||
home.mkdir()
|
||||
fake_bin = self.make_fake_bin(root)
|
||||
key = root / "key"
|
||||
key.write_text("test-key", encoding="utf-8")
|
||||
result = self.run_install(
|
||||
home, fake_bin, "install", "--backend", "meta", "--api-key-file", str(key)
|
||||
)
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
dropin = (
|
||||
home / ".config/systemd/user/pi-dictate.service.d/"
|
||||
"10-pi-dictate-network.conf"
|
||||
)
|
||||
with dropin.open("a", encoding="utf-8") as stream:
|
||||
stream.write("IPAddressAllow=any\\n")
|
||||
|
||||
result = self.run_install(home, fake_bin, "check")
|
||||
|
||||
self.assertNotEqual(result.returncode, 0)
|
||||
self.assertIn("expected managed policy", result.stderr)
|
||||
|
||||
def test_startup_requires_the_exact_validation_command(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
home = root / "home"
|
||||
home.mkdir()
|
||||
fake_bin = self.make_fake_bin(root)
|
||||
key = root / "key"
|
||||
key.write_text("test-key", encoding="utf-8")
|
||||
result = self.run_install(
|
||||
home, fake_bin, "install", "--backend", "meta", "--api-key-file", str(key)
|
||||
)
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
environment = {
|
||||
"HOME": str(home),
|
||||
"XDG_CONFIG_HOME": str(home / ".config"),
|
||||
"PI_DICTATE_CONFIG_PATH": str(home / ".config/pi-dictate/config.json"),
|
||||
"PI_DICTATE_DEPENDENCY_PROFILE_PATH": str(
|
||||
home / ".local/share/pi-dictate/.dependency-profile"
|
||||
),
|
||||
"PI_DICTATE_NETWORK_MODE": "cloud",
|
||||
}
|
||||
with patch.dict(os.environ, environment, clear=False):
|
||||
validate_startup()
|
||||
service = home / ".config/systemd/user/pi-dictate.service"
|
||||
text = service.read_text(encoding="utf-8")
|
||||
service.write_text(
|
||||
"\n".join(
|
||||
"ExecStartPre=/bin/true" if line.startswith("ExecStartPre=") else line
|
||||
for line in text.splitlines()
|
||||
) + "\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
with self.assertRaisesRegex(ConfigError, "hardening"):
|
||||
validate_startup()
|
||||
|
||||
def test_startup_rejects_a_world_writable_socket_unit(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
home = root / "home"
|
||||
home.mkdir()
|
||||
fake_bin = self.make_fake_bin(root)
|
||||
key = root / "key"
|
||||
key.write_text("test-key", encoding="utf-8")
|
||||
result = self.run_install(
|
||||
home, fake_bin, "install", "--backend", "meta", "--api-key-file", str(key)
|
||||
)
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
environment = {
|
||||
"HOME": str(home),
|
||||
"XDG_CONFIG_HOME": str(home / ".config"),
|
||||
"PI_DICTATE_CONFIG_PATH": str(home / ".config/pi-dictate/config.json"),
|
||||
"PI_DICTATE_DEPENDENCY_PROFILE_PATH": str(
|
||||
home / ".local/share/pi-dictate/.dependency-profile"
|
||||
),
|
||||
"PI_DICTATE_NETWORK_MODE": "cloud",
|
||||
}
|
||||
socket_unit = home / ".config/systemd/user/pi-dictate.socket"
|
||||
socket_unit.write_text(
|
||||
socket_unit.read_text(encoding="utf-8").replace(
|
||||
"SocketMode=0600", "SocketMode=0666"
|
||||
),
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
with patch.dict(os.environ, environment, clear=False):
|
||||
with self.assertRaisesRegex(ConfigError, "socket systemd policy"):
|
||||
validate_startup()
|
||||
|
||||
def test_local_dropin_binds_custom_model_and_dictionary_paths(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
home = root / "home"
|
||||
home.mkdir()
|
||||
fake_bin = self.make_fake_bin(root)
|
||||
source = root / "source.nemo"
|
||||
source.write_bytes(b"model")
|
||||
digest = hashlib.sha256(source.read_bytes()).hexdigest()
|
||||
result = self.run_install(
|
||||
home,
|
||||
fake_bin,
|
||||
"install",
|
||||
"--backend",
|
||||
"local",
|
||||
"--model-source",
|
||||
str(source),
|
||||
"--model-sha256",
|
||||
digest,
|
||||
)
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
custom_model = home / "custom model.nemo"
|
||||
custom_terms = home / "custom terms.txt"
|
||||
custom_model.write_bytes(source.read_bytes())
|
||||
custom_terms.write_text("JSON\n", encoding="utf-8")
|
||||
config_path = home / ".config/pi-dictate/config.json"
|
||||
config = json.loads(config_path.read_text(encoding="utf-8"))
|
||||
config["local"]["modelPath"] = str(custom_model)
|
||||
config["local"]["termsPath"] = str(custom_terms)
|
||||
config_path.write_text(json.dumps(config), encoding="utf-8")
|
||||
config_path.chmod(0o600)
|
||||
|
||||
result = self.run_install(home, fake_bin, "configure", "--backend", "local")
|
||||
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
dropin = (home / ".config/systemd/user/pi-dictate.service.d/10-pi-dictate-network.conf").read_text(encoding="utf-8")
|
||||
self.assertIn(f'BindReadOnlyPaths="{custom_model}"', dropin)
|
||||
self.assertIn(f'BindReadOnlyPaths="{custom_terms}"', dropin)
|
||||
environment = {
|
||||
"HOME": str(home),
|
||||
"XDG_CONFIG_HOME": str(home / ".config"),
|
||||
"PI_DICTATE_CONFIG_PATH": str(config_path),
|
||||
"PI_DICTATE_DEPENDENCY_PROFILE_PATH": str(
|
||||
home / ".local/share/pi-dictate/.dependency-profile"
|
||||
),
|
||||
"PI_DICTATE_NETWORK_MODE": "offline",
|
||||
}
|
||||
with patch.dict(os.environ, environment, clear=False):
|
||||
validate_startup()
|
||||
|
||||
def test_local_dropin_and_profile_deny_are_restrictive(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
home = root / "home"
|
||||
home.mkdir()
|
||||
fake_bin = self.make_fake_bin(root)
|
||||
model = root / "model"
|
||||
model.write_bytes(b"model")
|
||||
# Dry-run keeps this test independent of the pinned model bytes.
|
||||
result = self.run_install(
|
||||
home,
|
||||
fake_bin,
|
||||
"install",
|
||||
"--backend",
|
||||
"local",
|
||||
"--model-source",
|
||||
str(model),
|
||||
"--model-sha256",
|
||||
hashlib.sha256(model.read_bytes()).hexdigest(),
|
||||
"--dry-run",
|
||||
)
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
self.assertIn("offline network hardening", result.stdout)
|
||||
|
||||
profile = json.loads(
|
||||
(REPOSITORY_ROOT / "profiles/nono/pi-local.json").read_text(
|
||||
encoding="utf-8"
|
||||
)
|
||||
)
|
||||
self.assertIn("$HOME/.local/share/pi-dictate", profile["filesystem"]["deny"])
|
||||
# User decision: the profile is intentionally unchanged from HEAD.
|
||||
self.assertNotIn("$HOME/.config/pi-dictate", profile["filesystem"]["deny"])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -1,12 +1,21 @@
|
||||
import hashlib
|
||||
from pathlib import Path
|
||||
import unittest
|
||||
|
||||
from pi_dictate_service.startup import EXPECTED_BASE_UNIT_SHA256
|
||||
|
||||
|
||||
REPOSITORY_ROOT = Path(__file__).resolve().parents[3]
|
||||
UNIT_DIR = REPOSITORY_ROOT / "systemd/user"
|
||||
|
||||
|
||||
class SystemdUnitTests(unittest.TestCase):
|
||||
def test_startup_hash_matches_the_checked_in_service_unit(self) -> None:
|
||||
self.assertEqual(
|
||||
hashlib.sha256((UNIT_DIR / "pi-dictate.service").read_bytes()).hexdigest(),
|
||||
EXPECTED_BASE_UNIT_SHA256,
|
||||
)
|
||||
|
||||
def test_socket_unit_owns_the_exact_private_runtime_path(self) -> None:
|
||||
socket_unit = (UNIT_DIR / "pi-dictate.socket").read_text(encoding="utf-8")
|
||||
self.assertIn("ListenStream=%t/pi-dictate.sock", socket_unit)
|
||||
@@ -24,11 +33,20 @@ class SystemdUnitTests(unittest.TestCase):
|
||||
"TRANSFORMERS_OFFLINE=1",
|
||||
"TRITON_CACHE_DIR=/tmp/pi-dictate-triton",
|
||||
"NUMBA_CACHE_DIR=/tmp/pi-dictate-numba",
|
||||
"TimeoutStopSec=5s",
|
||||
"LimitCORE=0",
|
||||
"MemorySwapMax=0",
|
||||
"ProtectSystem=strict",
|
||||
"ProtectHome=read-only",
|
||||
"ProtectHome=tmpfs",
|
||||
"BindReadOnlyPaths=%h/.local/share/pi-dictate",
|
||||
"BindReadOnlyPaths=%h/.local/share/uv/python",
|
||||
"BindReadOnlyPaths=%h/.config/pi-dictate/config.json",
|
||||
"BindReadOnlyPaths=%h/.config/systemd/user/pi-dictate.service",
|
||||
"BindReadOnlyPaths=%h/.config/systemd/user/pi-dictate.service.d",
|
||||
"BindReadOnlyPaths=%h/.config/systemd/user/pi-dictate.socket",
|
||||
"BindReadOnlyPaths=%t/pipewire-0",
|
||||
"RestrictAddressFamilies=AF_UNIX",
|
||||
"SystemCallFilter=~bind listen",
|
||||
"IPAddressDeny=any",
|
||||
):
|
||||
self.assertIn(expected, service_unit)
|
||||
|
||||
@@ -173,6 +173,10 @@ class UnixLoopTests(unittest.TestCase):
|
||||
class ActivationTests(unittest.TestCase):
|
||||
def test_inherited_listener_requires_one_fd_for_this_process_and_wraps_fd_three(self) -> None:
|
||||
listener = Mock()
|
||||
listener.family = socket.AF_UNIX
|
||||
listener.type = socket.SOCK_STREAM
|
||||
listener.getsockopt.return_value = 1
|
||||
listener.getsockname.return_value = "/run/user/1000/pi-dictate.sock"
|
||||
with patch("pi_dictate_service.server.os.getpid", return_value=42), patch(
|
||||
"pi_dictate_service.server.socket.socket", return_value=listener
|
||||
) as socket_factory:
|
||||
@@ -189,7 +193,9 @@ class ActivationTests(unittest.TestCase):
|
||||
inherited_listener({"LISTEN_PID": "41", "LISTEN_FDS": "1"})
|
||||
|
||||
def test_main_requires_local_model_path_and_reads_idle_override(self) -> None:
|
||||
with patch.dict(os.environ, {}, clear=True):
|
||||
with patch.dict(os.environ, {}, clear=True), patch(
|
||||
"pi_dictate_service.server.os.path.isfile", return_value=False
|
||||
):
|
||||
with self.assertRaisesRegex(SystemExit, "PI_DICTATE_MODEL_PATH"):
|
||||
main()
|
||||
|
||||
@@ -201,10 +207,12 @@ class ActivationTests(unittest.TestCase):
|
||||
"PI_DICTATE_MODEL_PATH": "/models/parakeet.nemo",
|
||||
"PI_DICTATE_TERMS_PATH": "/config/coding-terms.txt",
|
||||
"PI_DICTATE_BOOST_ALPHA": "0.5",
|
||||
"PI_DICTATE_IDLE_SECONDS": "12.5",
|
||||
"PI_DICTATE_IDLE_SECONDS": "120",
|
||||
},
|
||||
clear=True,
|
||||
), patch("pi_dictate_service.server.NemoParakeetModel", return_value=model) as adapter, patch(
|
||||
), patch("pi_dictate_service.server.os.path.isfile", return_value=False), patch(
|
||||
"pi_dictate_service.server.NemoParakeetModel", return_value=model
|
||||
) as adapter, patch(
|
||||
"pi_dictate_service.server.DictateServiceCore"
|
||||
) as core_factory, patch(
|
||||
"pi_dictate_service.server.inherited_listener", return_value=listener
|
||||
@@ -217,7 +225,7 @@ class ActivationTests(unittest.TestCase):
|
||||
terms_path="/config/coding-terms.txt",
|
||||
boost_alpha=0.5,
|
||||
)
|
||||
core_factory.assert_called_once_with(model=model, idle_seconds=12.5)
|
||||
core_factory.assert_called_once_with(model=model, idle_seconds=120)
|
||||
serve_loop.assert_called_once_with(listener, core)
|
||||
|
||||
|
||||
|
||||
Generated
+31
-4
@@ -1328,7 +1328,12 @@ wheels = [
|
||||
name = "pi-dictate-service"
|
||||
version = "0.1.0"
|
||||
source = { editable = "." }
|
||||
dependencies = [
|
||||
|
||||
[package.optional-dependencies]
|
||||
cloud = [
|
||||
{ name = "websockets" },
|
||||
]
|
||||
local = [
|
||||
{ name = "nemo-toolkit", extra = ["asr"] },
|
||||
{ name = "numpy" },
|
||||
{ name = "torch" },
|
||||
@@ -1336,10 +1341,12 @@ dependencies = [
|
||||
|
||||
[package.metadata]
|
||||
requires-dist = [
|
||||
{ name = "nemo-toolkit", extras = ["asr"], specifier = "==3.0.0" },
|
||||
{ name = "numpy", specifier = "==2.5.2" },
|
||||
{ name = "torch", specifier = "==2.13.0" },
|
||||
{ name = "nemo-toolkit", extras = ["asr"], marker = "extra == 'local'", specifier = "==3.0.0" },
|
||||
{ name = "numpy", marker = "extra == 'local'", specifier = "==2.5.2" },
|
||||
{ name = "torch", marker = "extra == 'local'", specifier = "==2.13.0" },
|
||||
{ name = "websockets", marker = "extra == 'cloud'", specifier = "==15.0.1" },
|
||||
]
|
||||
provides-extras = ["local", "cloud"]
|
||||
|
||||
[[package]]
|
||||
name = "pillow"
|
||||
@@ -2136,6 +2143,26 @@ wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/d9/00/aca6beb3658dab4ed3dbb41a78e6e7f31342e0b41d28088f205525751601/webdataset-1.0.2-py3-none-any.whl", hash = "sha256:3dbfced32b25c0d199c6b9787937b6f85742bc3c84f652c846893075c1c082d9", size = 74956, upload-time = "2025-06-19T23:26:20.354Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "websockets"
|
||||
version = "15.0.1"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/21/e6/26d09fab466b7ca9c7737474c52be4f76a40301b08362eb2dbc19dcc16c1/websockets-15.0.1.tar.gz", hash = "sha256:82544de02076bafba038ce055ee6412d68da13ab47f0c60cab827346de828dee", size = 177016, upload-time = "2025-03-05T20:03:41.606Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/51/6b/4545a0d843594f5d0771e86463606a3988b5a09ca5123136f8a76580dd63/websockets-15.0.1-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:3e90baa811a5d73f3ca0bcbf32064d663ed81318ab225ee4f427ad4e26e5aff3", size = 175437, upload-time = "2025-03-05T20:02:16.706Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/f4/71/809a0f5f6a06522af902e0f2ea2757f71ead94610010cf570ab5c98e99ed/websockets-15.0.1-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:592f1a9fe869c778694f0aa806ba0374e97648ab57936f092fd9d87f8bc03665", size = 173096, upload-time = "2025-03-05T20:02:18.832Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/3d/69/1a681dd6f02180916f116894181eab8b2e25b31e484c5d0eae637ec01f7c/websockets-15.0.1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:0701bc3cfcb9164d04a14b149fd74be7347a530ad3bbf15ab2c678a2cd3dd9a2", size = 173332, upload-time = "2025-03-05T20:02:20.187Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/a6/02/0073b3952f5bce97eafbb35757f8d0d54812b6174ed8dd952aa08429bcc3/websockets-15.0.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:e8b56bdcdb4505c8078cb6c7157d9811a85790f2f2b3632c7d1462ab5783d215", size = 183152, upload-time = "2025-03-05T20:02:22.286Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/74/45/c205c8480eafd114b428284840da0b1be9ffd0e4f87338dc95dc6ff961a1/websockets-15.0.1-cp312-cp312-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:0af68c55afbd5f07986df82831c7bff04846928ea8d1fd7f30052638788bc9b5", size = 182096, upload-time = "2025-03-05T20:02:24.368Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/14/8f/aa61f528fba38578ec553c145857a181384c72b98156f858ca5c8e82d9d3/websockets-15.0.1-cp312-cp312-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:64dee438fed052b52e4f98f76c5790513235efaa1ef7f3f2192c392cd7c91b65", size = 182523, upload-time = "2025-03-05T20:02:25.669Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/ec/6d/0267396610add5bc0d0d3e77f546d4cd287200804fe02323797de77dbce9/websockets-15.0.1-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:d5f6b181bb38171a8ad1d6aa58a67a6aa9d4b38d0f8c5f496b9e42561dfc62fe", size = 182790, upload-time = "2025-03-05T20:02:26.99Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/02/05/c68c5adbf679cf610ae2f74a9b871ae84564462955d991178f95a1ddb7dd/websockets-15.0.1-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:5d54b09eba2bada6011aea5375542a157637b91029687eb4fdb2dab11059c1b4", size = 182165, upload-time = "2025-03-05T20:02:30.291Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/29/93/bb672df7b2f5faac89761cb5fa34f5cec45a4026c383a4b5761c6cea5c16/websockets-15.0.1-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:3be571a8b5afed347da347bfcf27ba12b069d9d7f42cb8c7028b5e98bbb12597", size = 182160, upload-time = "2025-03-05T20:02:31.634Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/ff/83/de1f7709376dc3ca9b7eeb4b9a07b4526b14876b6d372a4dc62312bebee0/websockets-15.0.1-cp312-cp312-win32.whl", hash = "sha256:c338ffa0520bdb12fbc527265235639fb76e7bc7faafbb93f6ba80d9c06578a9", size = 176395, upload-time = "2025-03-05T20:02:33.017Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/7d/71/abf2ebc3bbfa40f391ce1428c7168fb20582d0ff57019b69ea20fa698043/websockets-15.0.1-cp312-cp312-win_amd64.whl", hash = "sha256:fcd5cf9e305d7b8338754470cf69cf81f420459dbae8a3b40cee57417f4614a7", size = 176841, upload-time = "2025-03-05T20:02:34.498Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/fa/a8/5b41e0da817d64113292ab1f8247140aac61cbf6cfd085d6a0fa77f4984f/websockets-15.0.1-py3-none-any.whl", hash = "sha256:f7a866fbc1e97b5c617ee4116daaa09b722101d4a3c170c787450ba409f9736f", size = 169743, upload-time = "2025-03-05T20:03:39.41Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "werkzeug"
|
||||
version = "3.1.8"
|
||||
|
||||
@@ -1,12 +1,19 @@
|
||||
# Managed by pi-dictate install.sh
|
||||
[Unit]
|
||||
Description=Pi local Parakeet dictation service
|
||||
Description=Pi dictation transcription service
|
||||
Requires=pi-dictate.socket
|
||||
After=pipewire.service
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
TimeoutStopSec=5s
|
||||
ExecStart=%h/.local/share/pi-dictate/venv/bin/pi-dictate-service
|
||||
Environment=PI_DICTATE_CONFIG_PATH=%h/.config/pi-dictate/config.json
|
||||
Environment=PI_DICTATE_DEPENDENCY_PROFILE_PATH=%h/.local/share/pi-dictate/.dependency-profile
|
||||
# local/offline is the fail-closed base. The installer-owned cloud drop-in
|
||||
# replaces these values only after a cloud configuration and dependency profile
|
||||
# have been installed.
|
||||
Environment=PI_DICTATE_NETWORK_MODE=offline
|
||||
Environment=PI_DICTATE_MODEL_PATH=%h/.local/share/pi-dictate/models/parakeet-tdt-0.6b-v3/parakeet-tdt-0.6b-v3.nemo
|
||||
Environment=PI_DICTATE_TERMS_PATH=%h/.local/share/pi-dictate/config/coding-terms.txt
|
||||
Environment=PI_DICTATE_BOOST_ALPHA=1.0
|
||||
@@ -18,6 +25,13 @@ Environment=HF_HOME=/tmp/pi-dictate-huggingface
|
||||
Environment=TRITON_CACHE_DIR=/tmp/pi-dictate-triton
|
||||
Environment=NUMBA_CACHE_DIR=/tmp/pi-dictate-numba
|
||||
Environment=PYTHONDONTWRITEBYTECODE=1
|
||||
# Never inherit ambient proxy credentials or route audio through an unreviewed proxy.
|
||||
Environment=HTTP_PROXY=
|
||||
Environment=HTTPS_PROXY=
|
||||
Environment=ALL_PROXY=
|
||||
Environment=http_proxy=
|
||||
Environment=https_proxy=
|
||||
Environment=all_proxy=
|
||||
UMask=0077
|
||||
LimitCORE=0
|
||||
MemorySwapMax=0
|
||||
@@ -29,9 +43,19 @@ ProtectKernelLogs=yes
|
||||
ProtectKernelModules=yes
|
||||
ProtectKernelTunables=yes
|
||||
ProtectSystem=strict
|
||||
ProtectHome=read-only
|
||||
ProtectHome=tmpfs
|
||||
BindReadOnlyPaths=%h/.local/share/pi-dictate
|
||||
BindReadOnlyPaths=%h/.local/share/uv/python
|
||||
BindReadOnlyPaths=%h/.config/pi-dictate/config.json
|
||||
BindReadOnlyPaths=%h/.config/systemd/user/pi-dictate.service
|
||||
BindReadOnlyPaths=%h/.config/systemd/user/pi-dictate.service.d
|
||||
BindReadOnlyPaths=%h/.config/systemd/user/pi-dictate.socket
|
||||
BindReadOnlyPaths=%t/pipewire-0
|
||||
RestrictAddressFamilies=AF_UNIX
|
||||
SystemCallFilter=~bind listen
|
||||
IPAddressDeny=any
|
||||
LockPersonality=yes
|
||||
# Refuse a service/config/dependency/network-mode or hardening mismatch before runtime code starts.
|
||||
ExecStartPre=%h/.local/share/pi-dictate/venv/bin/python -c 'from pi_dictate_service.startup import validate_startup; validate_startup()'
|
||||
|
||||
# The socket unit owns %t/pi-dictate.sock. The service must not unlink it.
|
||||
|
||||
Reference in New Issue
Block a user