185 lines
7.9 KiB
Markdown
185 lines
7.9 KiB
Markdown
# Pi Dictate
|
|
|
|
Linux-only push-to-dictate for Pi. Audio capture and NVIDIA Parakeet run in a socket-activated host service. The Pi extension receives state and finalized text through a Unix socket, then inserts successful text with `pasteToEditor()`.
|
|
|
|
Tested on CachyOS with niri, Ghostty, PipeWire, an RTX 4090, CUDA 13.0, NeMo 3.0.0, and Parakeet-TDT 0.6B v3.
|
|
|
|
## Requirements
|
|
|
|
- Linux with PipeWire and `pw-record`
|
|
- systemd user services
|
|
- NVIDIA GPU with a working CUDA runtime
|
|
- `uv`
|
|
- Network access during initial dependency and model provisioning
|
|
|
|
Runtime is offline. The systemd service restricts networking and loads an explicit local model archive.
|
|
|
|
## Install
|
|
|
|
From the repository root:
|
|
|
|
```bash
|
|
./services/dictate/install.sh
|
|
```
|
|
|
|
The installer creates `~/.local/share/pi-dictate/`, installs the locked service as a non-editable wheel, downloads and verifies the pinned model snapshot, seeds the coding dictionary, installs host-owned copies of the systemd user units, and enables `pi-dictate.socket`. The installed runtime does not depend on the repository remaining at its installation path.
|
|
|
|
To reuse an already downloaded model archive:
|
|
|
|
```bash
|
|
./services/dictate/install.sh \
|
|
--model-source /path/to/parakeet-tdt-0.6b-v3.nemo
|
|
```
|
|
|
|
Preview without changing the host:
|
|
|
|
```bash
|
|
./services/dictate/install.sh --dry-run --model-source /path/to/model.nemo
|
|
```
|
|
|
|
The pinned model revision and SHA-256 are recorded in [`../../docs/dictate/parakeet-attribution.md`](../../docs/dictate/parakeet-attribution.md).
|
|
|
|
## Sandbox access
|
|
|
|
`profiles/nono/pi-local.json` grants exactly:
|
|
|
|
```text
|
|
/run/user/1000/pi-dictate.sock
|
|
```
|
|
|
|
This is a mediated microphone and transcription capability. Any code running under `pi-local` or a descendant profile can ask the host service to record and can receive the transcript. Pi still receives no PipeWire, `/dev/snd`, CUDA device, model, cache, or dictionary access. An explicit filesystem deny protects `~/.local/share/pi-dictate` even if that directory is selected as `$WORKDIR`.
|
|
|
|
The active profile is repository-linked on the tested host. Start a new Pi sandbox after updating the profile because an existing nono process cannot gain the socket capability in place.
|
|
|
|
## Controls
|
|
|
|
| Control | Action |
|
|
| --- | --- |
|
|
| `Alt+M` | Start recording when idle; stop and transcribe while listening |
|
|
| `Alt+N` | Cancel the active generation |
|
|
| `/dictate-cancel` | Cancel the active generation |
|
|
|
|
Shortcuts are handled by Pi's main editor. They do not claim to work while a selector, confirmation, or custom modal owns input. If a modal opens after recording starts, a successful result still goes only to the main editor.
|
|
|
|
The footer shows service state. While recording, a red circle and twelve-cell scrolling waveform make active capture visible and react to the microphone level:
|
|
|
|
```text
|
|
● listening ▁▂▄▇█▆▃▂▅▇▄▂
|
|
```
|
|
|
|
The circle uses Pi's semantic error color and the waveform uses its accent color. The other states remain `loading model...`, `stopping...`, and `transcribing...`. Audio levels are bounded scalar protocol events; raw audio remains inside the host service.
|
|
|
|
This indicator requires protocol version 2. Run the installer again after updating the repository so the installed host service and repository-linked extension upgrade together.
|
|
|
|
Only a non-empty finalized result for the current generation changes the editor. Empty speech, cancellation, stale results, disconnects, and failures do not insert text.
|
|
|
|
## Coding dictionary
|
|
|
|
The installer seeds:
|
|
|
|
```text
|
|
~/.local/share/pi-dictate/config/coding-terms.txt
|
|
```
|
|
|
|
The host service reads this file at model load and applies NeMo's native phrase boosting. Pi cannot read or modify it. Add one term or phrase per line, use `#` for comments, and restart the service after editing:
|
|
|
|
```bash
|
|
systemctl --user restart pi-dictate.service
|
|
```
|
|
|
|
Reinstalling preserves an existing dictionary. The file is bounded to 64 KiB, 512 distinct terms, and 128 UTF-8 bytes per term. A conservative boost of `1.0` is configured in `pi-dictate.service`; aggressive values can create false positives.
|
|
|
|
## Model lifecycle and measured behavior
|
|
|
|
The socket remains active while the Python service exits after 15 minutes of true inactivity. A later connection activates a fresh service. Loading, recording, recorder shutdown, and inference suspend idle exit.
|
|
|
|
Measured on the tested RTX 4090 using the pinned offline model:
|
|
|
|
- Model archive: about 2.4 GB
|
|
- Locked Python environment: about 5.6 GB in the Phase 0 prototype
|
|
- Cold base-model load: about 6.4 seconds
|
|
- Production-adapter load with phrase-boost setup: about 10 seconds, including the private Triton cache build
|
|
- Ten-second microphone inference: about 0.29 seconds, 0.029 real-time factor
|
|
- Sixty-second repeated-speech inference: about 0.43 seconds, 0.0072 real-time factor
|
|
- Five-minute repeated-speech inference: about 0.82 seconds, 0.0027 real-time factor
|
|
- Peak CUDA allocation: about 5.1 GB for short clips and 6.15 GB for five minutes
|
|
- Five-minute raw PCM cap: 9.6 MB
|
|
|
|
The 60-second and five-minute stress inputs repeat a real speech clip in memory. They validate bounded duration, latency, and GPU allocation, not long-form accuracy.
|
|
|
|
## Privacy
|
|
|
|
The service passes this exact recorder command through pipes:
|
|
|
|
```text
|
|
pw-record --raw --format s16 --rate 16000 --channels 1 -
|
|
```
|
|
|
|
It intentionally creates no audio files and releases application-owned buffers after transcription, cancellation, or failure. `LimitCORE=0` and `MemorySwapMax=0` reduce additional persistence. This is best-effort non-persistence, not forensic zeroization. Python, NumPy, PyTorch, CUDA, drivers, host memory management, and crash handling can retain copies outside the service's direct control.
|
|
|
|
The service does not log raw audio or transcripts by default. Recorder stderr and protocol output are bounded. The server accepts at most 16 clients and bounds each client's pending output to 64 KiB.
|
|
|
|
## Service operations
|
|
|
|
```bash
|
|
systemctl --user status pi-dictate.socket pi-dictate.service
|
|
journalctl --user -u pi-dictate.service
|
|
systemctl --user restart pi-dictate.service
|
|
systemctl --user stop pi-dictate.service
|
|
```
|
|
|
|
Stopping only the service leaves the socket active for later reactivation. Stopping `pi-dictate.socket` removes the socket path.
|
|
|
|
## niri and Ghostty
|
|
|
|
No compositor-global binding is required. Confirm the live niri config does not claim `Alt+M` or `Alt+N`, then validate it outside the Pi sandbox:
|
|
|
|
```bash
|
|
niri validate
|
|
```
|
|
|
|
The shortcuts should reach Pi only while its main editor has focus. No `wl-copy`, `wtype`, `ydotool`, X11 injection, or compositor spawn binding is used.
|
|
|
|
## Troubleshooting
|
|
|
|
### Socket or activation failure
|
|
|
|
```bash
|
|
systemctl --user status pi-dictate.socket pi-dictate.service
|
|
ls -l "$XDG_RUNTIME_DIR/pi-dictate.sock"
|
|
journalctl --user -u pi-dictate.service -n 100
|
|
```
|
|
|
|
The socket should be mode `0600` and owned by the current user. Restart Pi after changing nono policy.
|
|
|
|
### PipeWire or microphone failure
|
|
|
|
```bash
|
|
wpctl status
|
|
pw-record --raw --format s16 --rate 16000 --channels 1 - >/dev/null
|
|
```
|
|
|
|
Press `Ctrl+C` after confirming capture. The service uses the WirePlumber default input. Selectable devices are not implemented.
|
|
|
|
### CUDA or model failure
|
|
|
|
```bash
|
|
nvidia-smi
|
|
systemctl --user restart pi-dictate.service
|
|
journalctl --user -u pi-dictate.service -n 100
|
|
```
|
|
|
|
Confirm the installed model checksum matches the attribution document and that runtime network access is not needed.
|
|
|
|
### Service reports busy
|
|
|
|
Only one recording, load worker, or inference worker is allowed globally. A cancelled load or inference remains busy until its non-interruptible worker finishes. Wait for it to settle or restart the service if the owning client crashed and the worker is stuck.
|
|
|
|
## Uninstall
|
|
|
|
```bash
|
|
./services/dictate/install.sh uninstall
|
|
```
|
|
|
|
The uninstaller disables the socket, removes only installer-managed unit copies, reloads systemd, and removes the fixed `~/.local/share/pi-dictate/` directory. It ignores install-root environment overrides and refuses to replace or remove unmanaged unit files.
|