feat(dictate): add voice-reactive listening waveform
This commit is contained in:
+21
-11
@@ -2,7 +2,7 @@
|
||||
|
||||
## Transport and version
|
||||
|
||||
Protocol version `1` uses UTF-8, newline-delimited JSON over one persistent Unix stream socket. Each line contains one JSON object. The systemd socket unit owns the socket path and permissions. The service accepts the inherited listening file descriptor and does not bind or unlink the path.
|
||||
Protocol version `2` uses UTF-8, newline-delimited JSON over one persistent Unix stream socket. Each line contains one JSON object. The systemd socket unit owns the socket path and permissions. The service accepts the inherited listening file descriptor and does not bind or unlink the path.
|
||||
|
||||
Clients must send `hello` before any other request. A connection negotiates one version only. Coding terms are host-owned service configuration and never cross this protocol.
|
||||
|
||||
@@ -17,6 +17,7 @@ Clients must send `hello` before any other request. A connection negotiates one
|
||||
| Simultaneous client connections | 16 |
|
||||
| Pending output per client | 65,536 bytes |
|
||||
| Recorder stderr tail | 65,536 bytes |
|
||||
| Audio-level event rate | 10 per second |
|
||||
| Mono 16 kHz signed-16 PCM | 9,600,000 bytes, five minutes |
|
||||
|
||||
IDs are non-empty, opaque strings generated by the client. The service compares them only for request correlation, ownership, and stale-generation rejection. It does not interpret IDs as paths. Request objects reject unknown or missing fields.
|
||||
@@ -30,7 +31,7 @@ All examples are one line when transmitted.
|
||||
### `hello`
|
||||
|
||||
```json
|
||||
{"version":1,"type":"hello","requestId":"hello-1"}
|
||||
{"version":2,"type":"hello","requestId":"hello-1"}
|
||||
```
|
||||
|
||||
Exact fields: `version`, `type`, `requestId`.
|
||||
@@ -40,7 +41,7 @@ A successful response is `pong` with the same `requestId`. `ready` reports wheth
|
||||
### `ping`
|
||||
|
||||
```json
|
||||
{"version":1,"type":"ping","requestId":"ping-1"}
|
||||
{"version":2,"type":"ping","requestId":"ping-1"}
|
||||
```
|
||||
|
||||
Exact fields: `version`, `type`, `requestId`. `hello` is required first. The response is `pong` and may include `ready` and `busy`.
|
||||
@@ -48,7 +49,7 @@ Exact fields: `version`, `type`, `requestId`. `hello` is required first. The res
|
||||
### `start`
|
||||
|
||||
```json
|
||||
{"version":1,"type":"start","requestId":"start-1","generationId":"generation-1"}
|
||||
{"version":2,"type":"start","requestId":"start-1","generationId":"generation-1"}
|
||||
```
|
||||
|
||||
Exact fields: `version`, `type`, `requestId`, `generationId`. The service accepts only one global generation. It first emits `state: loading` for a cold model or `state: listening` for a warm model. Recording starts only after model loading succeeds.
|
||||
@@ -56,7 +57,7 @@ Exact fields: `version`, `type`, `requestId`, `generationId`. The service accept
|
||||
### `stop`
|
||||
|
||||
```json
|
||||
{"version":1,"type":"stop","requestId":"stop-1","generationId":"generation-1"}
|
||||
{"version":2,"type":"stop","requestId":"stop-1","generationId":"generation-1"}
|
||||
```
|
||||
|
||||
Exact fields: `version`, `type`, `requestId`, `generationId`. Only the owning connection may stop the active listening generation.
|
||||
@@ -64,7 +65,7 @@ Exact fields: `version`, `type`, `requestId`, `generationId`. Only the owning co
|
||||
### `cancel`
|
||||
|
||||
```json
|
||||
{"version":1,"type":"cancel","requestId":"cancel-1","generationId":"generation-1"}
|
||||
{"version":2,"type":"cancel","requestId":"cancel-1","generationId":"generation-1"}
|
||||
```
|
||||
|
||||
Exact fields: `version`, `type`, `requestId`, `generationId`. A valid cancellation immediately emits `cancelled`. Cancellation tombstones the generation, so it can never emit a later `result` or begin recording after a cancelled load.
|
||||
@@ -76,21 +77,29 @@ All service events contain `version`, `type`, and a bounded client-generated `re
|
||||
### `pong`
|
||||
|
||||
```json
|
||||
{"version":1,"type":"pong","requestId":"hello-1","service":"pi-dictate","ready":false}
|
||||
{"version":2,"type":"pong","requestId":"hello-1","service":"pi-dictate","ready":false}
|
||||
```
|
||||
|
||||
### `state`
|
||||
|
||||
```json
|
||||
{"version":1,"type":"state","requestId":"start-1","generationId":"generation-1","state":"loading"}
|
||||
{"version":2,"type":"state","requestId":"start-1","generationId":"generation-1","state":"loading"}
|
||||
```
|
||||
|
||||
`state` is one of `loading`, `listening`, `stopping`, `transcribing`, or `idle`.
|
||||
|
||||
### `level`
|
||||
|
||||
```json
|
||||
{"version":2,"type":"level","requestId":"start-1","generationId":"generation-1","level":0.625}
|
||||
```
|
||||
|
||||
While listening, the service emits at most ten `level` events per second. `level` is a finite number from `0` through `1`, derived from the recorder's current 100 ms RMS window on a logarithmic -60 dBFS to 0 dBFS scale. The event carries no PCM samples and stops before recorder shutdown begins.
|
||||
|
||||
### `result`
|
||||
|
||||
```json
|
||||
{"version":1,"type":"result","requestId":"stop-1","generationId":"generation-1","transcript":"example text"}
|
||||
{"version":2,"type":"result","requestId":"stop-1","generationId":"generation-1","transcript":"example text"}
|
||||
```
|
||||
|
||||
There is at most one result for a generation. An empty transcript is a successful result with `"transcript":""`.
|
||||
@@ -98,7 +107,7 @@ There is at most one result for a generation. An empty transcript is a successfu
|
||||
### `cancelled`
|
||||
|
||||
```json
|
||||
{"version":1,"type":"cancelled","requestId":"cancel-1","generationId":"generation-1"}
|
||||
{"version":2,"type":"cancelled","requestId":"cancel-1","generationId":"generation-1"}
|
||||
```
|
||||
|
||||
This is a terminal delivery guarantee: the generation cannot later produce a result.
|
||||
@@ -106,7 +115,7 @@ This is a terminal delivery guarantee: the generation cannot later produce a res
|
||||
### `error`
|
||||
|
||||
```json
|
||||
{"version":1,"type":"error","requestId":"stop-1","generationId":"generation-1","code":"recorder_timeout","detail":"pw-record did not stop after SIGTERM and was killed"}
|
||||
{"version":2,"type":"error","requestId":"stop-1","generationId":"generation-1","code":"recorder_timeout","detail":"pw-record did not stop after SIGTERM and was killed"}
|
||||
```
|
||||
|
||||
Stable codes include:
|
||||
@@ -123,6 +132,7 @@ Stable codes include:
|
||||
| no generation | owner `start` | `loading`, then `listening`; warm models may emit `listening` directly |
|
||||
| any active generation | another `start` | `busy`; connection stays open |
|
||||
| `loading` | owner `cancel` | immediate `cancelled`; loading finishes in the worker but recording is suppressed |
|
||||
| `listening` | recorder audio | rate-limited `level` events for the owning client |
|
||||
| `listening` | owner `stop` | `stopping`; signal recorder, drain stdout to EOF, await child close, convert PCM, then `transcribing` |
|
||||
| `listening` | owner `cancel` or disconnect | stop and drain recorder, discard audio, never infer |
|
||||
| `stopping` | owner `cancel` | immediate `cancelled`; completed audio is discarded |
|
||||
|
||||
@@ -186,6 +186,8 @@ The current meter updates every 60 ms even when its rendered value has not chang
|
||||
|
||||
**Fix:** the extension renders state changes reported by the service and does not receive raw audio levels. Use static status text for `loading model...`, `listening...`, `stopping...`, and `transcribing...`. If a meter is wanted later, add a rate-limited level event only after measuring the need.
|
||||
|
||||
**Superseded 26AUG2026:** active-listening feedback now needs a voice-reactive waveform. [The accepted follow-up decision](../adr/dictate/listening-level-events-26AUG2026.md) adds a bounded 10 Hz scalar level event without exposing raw audio to Pi.
|
||||
|
||||
### 12. Tests, installation, attribution, and user documentation are missing
|
||||
|
||||
**Severity:** medium
|
||||
|
||||
Reference in New Issue
Block a user