Commit Graph
4 Commits
Author SHA1 Message Date
Iulian ecd686073e wip: incremental ASR outputs in realtime, few bugs 2026-09-10 22:00:20 +03:00
Iulian 65855ace6c wip: incremental ASR 2026-09-09 20:31:03 +03:00
Iulian 0b44d7e1bf feat: live macOS dictation — push-to-talk mic into the realtime ASR session
- /dictate + ctrl+shift+d toggle one shared session: press to start, press again
  to stop, flush and paste into the prompt (a terminal gives no key-release event
  for hold-to-talk without the Kitty keyboard protocol)
- client.recordMic(): spawns ffmpeg AVFoundation capture and appends s16le frames
  as they arrive (no pacer — the mic is already 1x paced); stop() = commit + the
  same quiescence drain transcribe() uses, since is_last_result never flips true
- enumerates mics by slicing the "AVFoundation audio devices:" block off stderr
  (ffmpeg prints the table there, then fails opening the dummy input)
- --device takes an index or name, --lang / ASR_DICTATE_LANGUAGE, --submit sends
  instead of pasting; asr_health now reports the mic inputs it enumerated
- capture death surfaces via the emptyGraceMs path with the TCC hint (System
  Settings → Privacy & Security → Microphone) when ffmpeg exits non-zero at 0 bytes
- env.sh sources the gitignored .env so a shell gets ASR_NIM_URL / ASR_API_KEY
2026-09-09 19:06:06 +03:00
iulianp 19faa8b088 feat: speech-to-text for pi via NVIDIA Speech NIM realtime ASR
- asr_transcribe / asr_health tools + the /asr command, with live partials,
  per-word timings, transcript export and TUI rendering
- zero-dependency realtime client (https + Node 22 global WebSocket):
  mint transcription session -> subprotocol handshake -> event loop -> quiescence
- decodes anything ffmpeg supports (and native 16k mono/stereo pcm16/pcm_f32le wav)
  to the 16 kHz mono PCM16 the NIM mandates
- credentials via ASR_API_KEY / ASR_NIM_URL or ~/.pi/agent/asr.json; never in source
- README documents the six places the published API reference diverges from the
  deployed protocol (commit vs done, cumulative deltas, double .completed,
  HTTP 403 at upgrade, ?token= not rejected, model reset on session.update)
2026-09-09 18:05:31 +03:00