49 lines
3.0 KiB
Markdown
49 lines
3.0 KiB
Markdown
# Dictate Phase 0 results
|
|
|
|
Date: 26AUG2026
|
|
|
|
## Pinned runtime
|
|
|
|
- CPython 3.12.13
|
|
- NVIDIA NeMo 3.0.0
|
|
- PyTorch 2.13.0 with CUDA 13.0
|
|
- NumPy 2.5.2
|
|
- Model revision `541d1f99c6b0c3cd0b11a95167540bb8edefd82b`
|
|
- Model SHA-256 `3cbdc85877e668ca7b82d0d56770eb1fac76691f55d6b97545e8d61ca588d10d`
|
|
- NVIDIA GeForce RTX 4090
|
|
|
|
The complete dependency graph and artifact hashes are in `services/dictate/uv.lock`. Exact direct dependency sources were synchronized into `/srv/data/opensrc`.
|
|
|
|
## Offline and in-memory validation
|
|
|
|
The model loaded from the explicit local `.nemo` archive with Hugging Face, Transformers, and NeMo offline settings enabled. Audio was passed to NeMo as an in-memory 16 kHz mono float array. The benchmark created no audio files.
|
|
|
|
| Input | Inference | Real-time factor | Peak CUDA allocation |
|
|
| --- | ---: | ---: | ---: |
|
|
| Existing 2.78-second verification speech | 0.304 s | 0.109 | 5.07 GB |
|
|
| Live 9.96-second microphone speech | 0.285 s | 0.0286 | 5.07 GB |
|
|
| Repeated 60-second speech stress input | 0.431 s | 0.00718 | 5.18 GB |
|
|
| Repeated five-minute speech stress input | 0.820 s | 0.00273 | 6.15 GB |
|
|
|
|
Cold base-model loading was consistently about 6.4 seconds. The production adapter with its native phrase-boost tree loaded in about 10 seconds and transcribed the known verification fixture exactly. The model archive occupies about 2.4 GB. The prototype environment occupies about 5.6 GB.
|
|
|
|
The 60-second and five-minute inputs repeat real speech in memory. They validate latency and bounded GPU behavior, not natural long-form accuracy. Five-minute signed-16 PCM is exactly 9,600,000 bytes; the float32 inference representation is 19,200,000 bytes.
|
|
|
|
## Accuracy observations
|
|
|
|
The verification sample transcribed exactly:
|
|
|
|
> The quick brown fox jumps over the lazy dog.
|
|
|
|
A normal live dictation sample was coherent and punctuated, and the user accepted its accuracy.
|
|
|
|
The initial code-terminology sample missed `JSON`, `npm`, and `git`. The user failed the original Phase 0 accuracy gate and approved a conservative host-owned coding dictionary using NeMo's native phrase boosting.
|
|
|
|
A 37-term dictionary and boost strengths from 0.25 through 2.0 were tested against one in-memory capture. Alpha `1.0` changed the end of the captured phrase from `run np` to `run npm` while preserving the earlier `TypeScript` and `JSON response` terms. Alpha `1.0` was selected as the conservative default. The capture ended before `git status`, so that phrase still requires a focused validation sample.
|
|
|
|
## Gate status
|
|
|
|
The packaging, latency, memory, offline, and normal-dictation gates pass. Alpha `1.0` produced the same expected clean transcript as baseline, the same degraded transcript on seeded noisy input, and an empty transcript for silence. No dictionary hallucination was observed.
|
|
|
|
The coding-term remediation passes the final focused live check through the installed socket-activated service. The spoken phrase “Run npm test, then git status.” produced exactly `Run npm test, then git status.` with native phrase boosting at alpha `1.0`.
|