fix(asr): bias Parakeet toward this machine's vocabulary #204

Merged

kblack0610 merged 1 commit from fix/asr-parakeet-hotword-biasing into master

2026-08-26 15:51:18 +00:00

kblack0610 commented

2026-08-26 15:50:53 +00:00

Owner

Enables contextual biasing: a hotword list under modified_beam_search (greedy ignores hotwords entirely, which is why the old config got nothing from them).

Measured on the same audio:

before	after
`the Fargo runner logs`	`the Forgejo runner logs`
`Run cubect to get pods`	`Run kubectl get pods`
`The Viconya Task Tracker`	`The Vikunja Task Tracker`
`SAPS encrypted secret`	`sops encrypted secret`
`in the image namespace`	`in the Immich namespace`

Cost: 0.17s -> 0.19s on a 7s clip.

Regression caught and fixed in the same change

Biasing re-introduces silence hallucination, the property Parakeet was chosen for. Digital silence decodes as "I" at score 2.0 and "I'm sorry." at 1.0, non-monotonically, so no score is safe on its own. Beam search alone is clean; biasing is what breaks it.

The shim now enforces the guarantee explicitly with a peak gate at one 16-bit quantization step, derived from pw-record's s16 capture format rather than picked. Verified it cannot swallow quiet dictation: speech attenuated 40dB still peaks 244x above the gate and transcribes.

Also

parakeet_precision is now a variable. fp16 was A/B'd and showed no measurable accuracy gain over int8 at ~2x disk and ~30% latency, so int8 stays default and the variable exists to re-test.
Threads 4 -> 8, the encoder's scaling knee. 8 -> 24 is noise.
hotwords_score 2.0 is tuned: below it Immich/sops fall back to phonetic, at 3.0 hotwords bleed into unrelated audio (Grafana -> Graf).

Verification

Applied through the role itself on cachyos-x8664-main and verified against the live daemon: /health reports hotwords:true, silence returns empty at 2s and 8s, jargon is biased, natural speech unchanged.

Not covered: the three privileged tasks (pacman, linger, whisper build-deps) did not run, sudo was gated. All were already satisfied.

Parakeet's transducer has almost no internal language model, so rare proper nouns decode phonetically with nothing to fall back on. That is the accuracy regression felt after moving off Whisper, whose seq2seq decoder was silently repairing them. Enables contextual biasing: a hotword list under `modified_beam_search` (greedy ignores hotwords entirely, which is why the old config got nothing from them). Measured on the same audio: | before | after | |---|---| | `the Fargo runner logs` | `the Forgejo runner logs` | | `Run cubect to get pods` | `Run kubectl get pods` | | `The Viconya Task Tracker` | `The Vikunja Task Tracker` | | `SAPS encrypted secret` | `sops encrypted secret` | | `in the image namespace` | `in the Immich namespace` | Cost: 0.17s -> 0.19s on a 7s clip. ### Regression caught and fixed in the same change Biasing re-introduces silence hallucination, the property Parakeet was chosen for. Digital silence decodes as `"I"` at score 2.0 and `"I'm sorry."` at 1.0, non-monotonically, so no score is safe on its own. Beam search alone is clean; biasing is what breaks it. The shim now enforces the guarantee explicitly with a peak gate at one 16-bit quantization step, derived from `pw-record`'s s16 capture format rather than picked. Verified it cannot swallow quiet dictation: speech attenuated 40dB still peaks 244x above the gate and transcribes. ### Also - `parakeet_precision` is now a variable. fp16 was A/B'd and showed no measurable accuracy gain over int8 at ~2x disk and ~30% latency, so int8 stays default and the variable exists to re-test. - Threads 4 -> 8, the encoder's scaling knee. 8 -> 24 is noise. - `hotwords_score` 2.0 is tuned: below it `Immich`/`sops` fall back to phonetic, at 3.0 hotwords bleed into unrelated audio (`Grafana` -> `Graf`). ### Verification Applied through the role itself on cachyos-x8664-main and verified against the live daemon: `/health` reports `hotwords:true`, silence returns empty at 2s and 8s, jargon is biased, natural speech unchanged. Not covered: the three privileged tasks (pacman, linger, whisper build-deps) did not run, sudo was gated. All were already satisfied.

kblack0610 added 1 commit

2026-08-26 15:50:53 +00:00

fix(asr): bias Parakeet toward this machine's vocabulary

checks / fleet-roster (pull_request) Successful in 6s

Details

checks / kustomize-builds (pull_request) Successful in 9s

Details

f7e21a2ced

Parakeet's transducer has almost no internal language model, so rare proper
nouns decode phonetically with nothing to fall back on. That is the accuracy
regression felt after moving off Whisper, whose seq2seq decoder was silently
repairing them.

Enable contextual biasing: a hotword list under modified_beam_search (greedy
ignores hotwords entirely). sherpa-onnx tokenizes hotwords with BPE and wants a
SentencePiece-style vocab, but the Parakeet release ships only tokens.txt, so
derive bpe.vocab by negating the id, preserving merge order.

Measured on the same audio: "Fargo" -> "Forgejo", "cubect" -> "kubectl",
"Viconya" -> "Vikunja", "SAPS" -> "sops". Costs ~30ms on a 7s clip.

Biasing re-introduces silence hallucination, which is the property Parakeet was
chosen for: digital silence decodes as "I" at score 2.0 and "I'm sorry." at 1.0,
non-monotonically, so no score is safe on its own. Enforce the guarantee in the
shim with a peak gate at one 16-bit quantization step, derived from pw-record's
s16 format. Speech attenuated 40dB still peaks 244x above it.

Also parametrize weight precision. fp16 showed no measurable accuracy gain over
int8 while costing ~2x disk and ~30% latency, so int8 stays the default and the
variable exists to re-test. Threads 4 -> 8, where the encoder's scaling knee is.