5.5 KiB
Dictate Phase 0 results
Date: 26AUG2026
Pinned runtime
- CPython 3.12.13
- NVIDIA NeMo 3.0.0
- PyTorch 2.13.0 with CUDA 13.0
- NumPy 2.5.2
- Model revision
541d1f99c6b0c3cd0b11a95167540bb8edefd82b - Model SHA-256
3cbdc85877e668ca7b82d0d56770eb1fac76691f55d6b97545e8d61ca588d10d - NVIDIA GeForce RTX 4090
The complete dependency graph and artifact hashes are in services/dictate/uv.lock. Exact direct dependency sources were synchronized into /srv/data/opensrc.
Offline and in-memory validation
The model loaded from the explicit local .nemo archive with Hugging Face, Transformers, and NeMo offline settings enabled. Audio was passed to NeMo as an in-memory 16 kHz mono float array. The benchmark created no audio files.
| Input | Inference | Real-time factor | Peak CUDA allocation |
|---|---|---|---|
| Existing 2.78-second verification speech | 0.304 s | 0.109 | 5.07 GB |
| Live 9.96-second microphone speech | 0.285 s | 0.0286 | 5.07 GB |
| Repeated 60-second speech stress input | 0.431 s | 0.00718 | 5.18 GB |
| Repeated five-minute speech stress input | 0.820 s | 0.00273 | 6.15 GB |
Cold base-model loading was consistently about 6.4 seconds. The production adapter with its native phrase-boost tree loaded in about 10 seconds and transcribed the known verification fixture exactly. The model archive occupies about 2.4 GB. The prototype environment occupies about 5.6 GB.
The 60-second and five-minute inputs repeat real speech in memory. They validate latency and bounded GPU behavior, not natural long-form accuracy. Five-minute signed-16 PCM is exactly 9,600,000 bytes; the float32 inference representation is 19,200,000 bytes.
Accuracy observations
The verification sample transcribed exactly:
The quick brown fox jumps over the lazy dog.
A normal live dictation sample was coherent and punctuated, and the user accepted its accuracy.
The initial code-terminology sample missed JSON, npm, and git. The user failed the original Phase 0 accuracy gate and approved a conservative host-owned coding dictionary using NeMo's native phrase boosting.
A 37-term dictionary and boost strengths from 0.25 through 2.0 were tested against one in-memory capture. Alpha 1.0 changed the end of the captured phrase from run np to run npm while preserving the earlier TypeScript and JSON response terms. Alpha 1.0 was selected as the conservative default. The capture ended before git status, so that phrase still requires a focused validation sample.
Gate status
The packaging, latency, memory, offline, and normal-dictation gates pass. Alpha 1.0 produced the same expected clean transcript as baseline, the same degraded transcript on seeded noisy input, and an empty transcript for silence. No dictionary hallucination was observed.
The coding-term remediation passes the final focused live check through the installed socket-activated service. The spoken phrase “Run npm test, then git status.” produced exactly Run npm test, then git status. with native phrase boosting at alpha 1.0.
The upgraded configurable service was revalidated on 03SEP2026. Immediate cancellation during preparation was delivery-neutral, a second generation started successfully, and a six-second end-to-end capture returned Run npm test, then get status. The wording variation does not replace the earlier accuracy fixture; it confirms the current installer, systemd sandbox, socket protocol, PipeWire capture, in-memory model path, and final-only delivery operate together. Local systemd policy exposed only the PipeWire runtime socket while retaining the masked home and IP-network denial.
Configurable backend contract gate
Credentialed live contract checks passed on 03SEP2026 with a five-second, non-sensitive 24 kHz mono PCM microphone sample held only in memory. Dedicated mode-0600 provider key files were read by the host-side adapters; no credential entered Pi, a command line, a URL, or captured output.
| Provider | Model | Result |
|---|---|---|
| Meta | muse-voice-transcribe-1.0 |
The public api.meta.ai handshake accepted the session, streamed PCM, accepted endStream, and returned an explicit final transcript (run npm test). Total turn time was 30.5 seconds. |
| OpenAI | gpt-live-transcribe |
The GA transcription session accepted 24 kHz PCM, explicit commit, and returned a correlated completion event (Run npm test and). Total turn time was 7.1 seconds. |
The first OpenAI spike exposed forward-compatible fields (event_id and previous_item_id) on the commit acknowledgement. The adapter and sanitized fixture now accept those bounded fields while still requiring a non-empty item_id. A separate credentialed session update confirmed that gpt-live-transcribe accepts transcription delay: "low". Configuration therefore supports only the documented bounded values minimal, low, medium, high, and xhigh; omission keeps the provider default.
These checks validate the current wire contract and finalization path, not comparative transcription accuracy. The public contract references checked were the Meta Realtime Voice API, Meta event schemas, OpenAI Realtime transcription guide, OpenAI Realtime WebSocket guide, and gpt-live-transcribe model page.